Here’s a math problem:
Medical researchers have developed a new cream fro treating skin rashes. New treatments often work but sometimes make rashes worse. Even when treatments don’t work, skin rashes sometimes get better and sometimes get worse on their own. As a result, it is necessary to test any new treatment in an experiment to see whether it makes the skin condition of those who use it better or worse than if they had not used it.
Researchers have conducted an experiment on patients with skin rashes. In the experiment, one group of patients used the new cream for two weeks, and a second group did not use the new cream.
In each group, the number of people whose skin condition got better and the number whose condition got worse are recorded in the table below. Because patients do not always complete studies, the total number of patients in each two groups is not the same, but this does not prevent assessment of the results.
Please indicate whether the experiment shows that using the new cream is likely to make the skin condition better or worse.
| Result | ||
|---|---|---|
| Rash Got Better | Rash Got Worse | |
| Patients who DID use the new skin cream | 223 | 75 |
| Patients who did NOT use the new skin cream | 107 | 21 |
What result does the study support?
[] People who used the skin cream were more likely to get better than those who didn’t.
[] People who used the skin cream were more likely to get worse than those who didn’t.
The solution to the problem
Wait a mo’. Stop reading now if you’re keen to solve it yourself.
Okay? We’re all set?
Okay then.
The solution to the problem is to convert the numbers into percentages before comparing them. 75% of the people who used the new skin cream got better, while 84% of those who didn’t use the skin cream got better. So the answer is that cream users were more likely to get worse.
I’m pretty sure the average “Alas” reader would be able to solve that math problem correctly. But what if the nouns were changed? Apparently, we’d do terribly. Chris Mooney reports:
The study, by Yale law professor Dan Kahan and his colleagues, has an ingenious design. At the outset, 1,111 study participants were asked about their political views and also asked a series of questions designed to gauge their “numeracy,” that is, their mathematical reasoning ability. Participants were then asked to solve a fairly difficult problem that involved interpreting the results of a (fake) scientific study. But here was the trick: While the fake study data that they were supposed to assess remained the same, sometimes the study was described as measuring the effectiveness of a “new cream for treating skin rashes.” But in other cases, the study was described as involving the effectiveness of “a law banning private citizens from carrying concealed handguns in public.” […]
So how did people fare on the handgun version of the problem? They performed quite differently than on the skin cream version, and strong political patterns emerged in the results—especially among people who are good at mathematical reasoning. Most strikingly, highly numerate liberal Democrats did almost perfectly when the right answer was that the concealed weapons ban does indeed work to decrease crime (version C of the experiment)—an outcome that favors their pro-gun-control predilections. But they did much worse when the correct answer was that crime increases in cities that enact the ban (version D of the experiment).
The opposite was true for highly numerate conservative Republicans: They did just great when the right answer was that the ban didn’t work (version D), but poorly when the right answer was that it did (version C). […]
For study author Kahan, these results are a fairly strong refutation of what is called the “deficit model” in the field of science and technology studies—the idea that if people just had more knowledge, or more reasoning ability, then they would be better able to come to consensus with scientists and experts on issues like climate change, evolution, the safety of vaccines, and pretty much anything else involving science or data (for instance, whether concealed weapons bans work). Kahan’s data suggest the opposite—that political biases skew our reasoning abilities, and this problem seems to be worse for people with advanced capacities like scientific literacy and numeracy.
A graph from Kevin Drum:
Looking at that graph, it does seem that a substantial minority – I’d eyeball it as 30 to 40 percent? – of highly numerate partisans were able to do the math correctly when the correct answer cut against their own biases. If I’m correct about that, then that’s a thirty to forty percent reason for hope.
This sort of thing makes me feel terribly bleak about the point of even arguing about politics, especially when I think about issues like climate change. It’s not that people never change their minds; it’s that most of us don’t change our minds in response to facts or logic.
This is another reason I find same-sex marriage fascinating: It’s an issue on which large numbers of Americans have changed their minds over a pretty short period of time. What makes SSM so different? Is there any way that the success of SSM can be applied to issues like climate change?
I got it right (naturally, he said with the modesty common to genius) though I eyeballed the ratio of better:worse rather than calculating out the percentages. 5 to 1 for the non-users, so the users would need 300 something to have the same ratio, they didn’t, the users did worse. Do I win a prize? There should be a prize.
I find that 30 to 40 percent of the population being able to actually reason an improbably high figure that, if it bears out, would cheer me up considerably. I’d have guessed the figure was 10% or less.
SSM directly affects human beings that I know here and now. Dennis and his lover are being discriminated against and I hear about their (real) emotional distress and (even more real) logistical and legal challenges and I think “well, fuck that.” Doomy hypotheticals about how David will drown 40 years from now (apparently either climate change or homosexuality makes your legs not work) when his city is submerged are a lot less immediate and infinitely less convincing, and have the further handicap of being arguable; there is no non-you’re-a-complete-asshole argument for the nonexistence of Dennis’ differential treatment in the law.
And this is among partisans who are actually willing to DO math. Most people aren’t going to even get that far.
I have suspected for a while that what people think is a “deficit” in their opponent’s thinking is really just a difference in interpretation or centering of values. I have issues with Jonathan Haidt, but his work articulates this very well.
If they had not bothered and just tossed a coin 50% would have got it right. Highly numerate people who had a wrong opinion and tried got it wrong more often than genuine, couldn’t read the question, idiots or rats or pigeons would have.
The solution to the problem is to convert the numbers into percentages before comparing them. 75% of the people who used the new skin cream got better, while 84% of those who didn’t use the skin cream got better.
That’s the first step, but shouldn’t you also evaluate for statistical significance? Those are fairly small numbers and the difference might be coincidence.
You’re right, but since “there isn’t enough information to say” wasn’t an option, in this context I think it makes sense to go with the answer we can deduce from the data the word problem gives us. (That’s often the case with word problems, I think.)
Some of it is false expert syndrome, and false correlation syndrome. The former goes like this, “The people we pay to create ‘studies’, for the purposes of pushing law, elections, etc., to our own benefit, have rarely come up with ideas that have failed us, or caused our businesses harm, therefor, they must be experts, who we can rely on for being right, not someone we are paying to specifically distort the facts, so that we get what we want.” Most of the time its true, so.. they forget all the times they screwed up.
The false correlation one you get, as one statistician, whose book I read a while back described, bad models, which seem to work. Pretty much every single economic model we have ever come up with, at least for specific markets, or even the “general” stock market, have blown up after 10-20 years, sometimes sooner. The reason? Because there are like a billion variables, and just pure accident can results in any two or three of them “trending” the same, for decades, in lock step, because some third variable, which you are either not aware of, or not watching, is pushing one, or both of them, to correlate. have something go wrong with the third variable, and suddenly “both” of your supposedly correlated values spin off on totally random paths from each other. The housing mess was, in part, caused by this – the misapplication of one model, to another entirely different market, over perceived similarities, and the near total lack of comprehension, or interest, in the differences, even when obvious, which should have clearly indicated that the model was a bad fit for that market. But, worse than that, when it blew up on them, it proved that the correlations, even when used in the original markets, “might not”, be anything other than a complete illusion. It won’t stop them from, again, picking a handful of seeming correlations, building a new model from them, and then concluding that its working, because their “data” seems to indicate that its done what they expect it to, for X number of years, even while they still don’t know all the variables, or how/if they really do correlate.
Unlike, say, weather, which we can predict with OK accuracy over 5-6 days, these people project 5-10 years, and they don’t even have the math to describe “one layer” of the economy with accuracy, never mind, say, 500, or how ever many “layers” a weather simulation uses to make predictions. They certainly are not doing what the weather models do, and running 10,000 different solutions through a super computer, then presenting a statistical projection of what percentage of those 10,000 runs gave rain, when small adjustments where made in the variables (to compensate for known levels of inaccuracy). Nope – one model, one run, using a few variables, and like a thousand unknowns, which are not even “in” the model. What can possibly go wrong with that? Sigh…
Actually, there is enough information to say. If you go to an online chi-square calculator (there are many excellent choices), you can plug in the numbers from that study and you’ll find that the p-value is 0.0472, which means it is significant at the p<0.05 level.
For this study, though, you can't expect study participants to calculate a chi-square on the fly and you also can't indicate a significant p-value without giving away to the (statistically informed) participants that there is a significant difference.
The study got around this by asking people to decide which option "was more likely"…which is quite the reasonable compromise. The point still stands, though, that many numerate people suddenly become innumerate when the numbers contradicted their political beliefs.
I’ve had that same thought about whether it would be possible to apply the apparent win of SSM to other areas, and decided it probably isn’t possible partly because, as Robert says in the first comment, there’s no real reason to oppose SSM. But it’s not just whether something is arguable; it’s also whether it will harm me to admit it. Even among the people who believe SSM hurts opposite-sex marriage, they don’t say, “On Monday I had a good marriage, on Tuesday New York started legalizing SSM, on Wednesday my marriage was clearly worse.” The arguments are all about long term effects, “on the margin,” the Megan McArdle BS about how *we just don’t know* what effect tearing down that wall will have. (https://www.evernote.com/shard/s1/sh/eac8d03a-cf5f-4761-8533-e41b8184caba/f0709171ee993d21a83370bd07f31df4)
In contrast, there are very good reasons not to want to think human-caused catastrophic climate change is occurring, i.e. every comfort we enjoy that contributes to said climate change. I can point immediately to everything I’d lose if we took climate change very seriously, with relatively-inexpensive fossil fuels for my home, work and vehicle at the top of the list. If we embedded the negative environmental externality costs of fossil fuels into their price, my life would get more uncomfortable and inconvenient immediately. No more driving to the subway station — I’d have to walk the half mile to the nearest bus stop every day, no matter the weather, no matter if I had to be dressed up for work.
The SSM-like shift might seem more possible for areas that really have no apparent effect on our current well-being, e.g. thinking evolution happened. But this can be differentiated from SSM by the arguability aspect Robert mentioned. There’s very little disagreement about past or present facts between pro-SSM folks and the otherwise-decent people who oppose SSM, as the latter faction generally *don’t* claim that homosexuality caused the fall of empires nor that present-day homosexuals are mostly pedophiles. The factual disagreement there is all about the future.
With evolution, we’re talking about past facts, and once someone has committed to the idea that the earth was created in 6 days and is only a few thousand years old, because a holy book can be read to say so, I don’t think there’s any possibility of a gradual move in that person’s thinking. Either you destroy his faith in the hyperliteral truth of the holy book, or he stays a creationist forever. There’s no halfway point like many people get to about domestic partnership and civil unions, where they still have an “ick” reaction to the idea of SSM but totally want their gay acquaintances to be able to inherit and visit each other in the hospital. (Indeed, the wide spectrum that exists between “gay people should be stoned” and “sex/gender should be socially and legally meaningless” may be a reason for the success of SSM — people could change their minds gradually instead of always as a revolution.)
I would be a lot less annoyed by some of the arguments given, since, yes, there is a practical issue, in terms of personal cost, if not for the fact that the people *most* impacted would be businesses, according to the businesses, but we hear, all the time, BS from the energy companies about how hard they are working at new sources (which never means new energy forms, just new sources for the stuff we want to use less of, oddly enough…), how “safe” their new wells, pipelines, etc. will be (never mind the glaring reality we have seen in the last decade, and keep seeing), or shennanegans like what Texas pulled on the Telsa auto maker, which was passing a law that requires all car purchases to be made from lots, and bans ‘direct sales’, of the sort where you, say, go in, tell them, “None o the ones on the lot are what I want, let me order a custom built.” This is a company that a) doesn’t have enough volume sales to “own” car lots yet, b) a lot of people have never the less, including in Texas, started buying, and c) with the S series managed to exceed all safety standards, with top ratings, oh.. and which at least one “magazine columnist” tried, and failed, to claim was useless, by messing with the test he drove, only to discover, after the fact, that the company had already figured out, from a prior test, that people would screw with them, and installed a black box, which showed every stop, distance driven, charge time, battery levels, etc., and thus proved that the columnist lied his ass off.
Yeah, there is always the argument that, “Well, you still need power from some place!”, and somehow that means you still need fossil fuels, etc.. Well, we came “close” to putting in solar at our house, but the company screwed up some deal on their end, for panels, so it never happened. The “cost” would have been the same, per month, as we already paid the regular power company, so… And then.. there is the push back against incentives to install such things, or buy vehicles, etc. Not just Texas, and their, “You need to buy it from a lot!”, BS, but Arizona talking about pulling the plug on a very successful, even looking at the parking lot where I work, program to encourage hybred/alternative fuel vehicles, etc.
There is no doubt “someone” is lying about how much of a disruption it would really cause, but the only people, interestingly enough, who seem to be disrupting anything are the ones apposed to change. No one ever asks, for example, “With the inability of energy companies to rob us with gas prices, will how long will they be able to get by with making those who still *require* fossil fuels, for the time being, pay an arm and a leg for it, like, shipping companies, for example?” Or, in other words – if most of the commuters are using something that doesn’t employ such fuels, why the f should this, at all, mean that shipping costs would go up, instead of down, and thus, if it does go down, lower the costs for every other product on the market? Its almost as though, as a result of losing their precious, subsidized at that, “fuel for America’s cars”, business, they plan to hike up prices, despite their being less demand, and more supply… Nah…