This post on Open Left has lots of tables and stuff showing that over 50% of Republicans believe Obama is a socialist Muslim who “wants to turn over the sovereignty of the United States to a one world government,” blah blah blah. But the page on Harris Interactive says that this is an online poll.
“Online poll,” if you rearrange the letters, spells “complete bullshit.” Is there any argument for taking a poll like this seriously?
Sure. It gauges the sentiment of a small group of loudmouths who bother with online polls. It’s data, of whatever limited value.
Beyond that, no.
On the other hand, its results aren’t far off from the results of the similar Research 2000 poll a while back, so while this particular poll has little value, its results aren’t anomalous enough to suggest that the flawed methodology completely skewed the results.
Additionally, the online polling done by Harris shouldn’t be confused with the completely useless, easily freepable polls done on news web sites and such. The group of people polled are people who have signed up to be polled by Harris, not people who were specifically interested in these questions, so while they are more likely to be (a) strongly opinionated and (b) politically interested, I would not necessarily expect them to hold the ridiculous views that this poll finds them to.
That the majority of strongly opinionated politically interested Republicans who signed up to be polled by Harris hold bizarre views does not seem like a result that would be expected (well, it would be, because the Republican Party and its media arm at Fox have gone completely over the edge into reality denying paranoia in the last year or so, but you wouldn’t expect to see it on the Democratic side: what percentage of Harris poll volunteering Dems do you think would say that George Bush was a fascist? 67%? I seriously doubt it.
Also, in a sane Republican Party, there would be plenty of Republicans who had strong views that the Birthers and their ilk were a dangerous and absurd blight upon the party and needed to be shouted down. Sadly, that isn’t the case.
Charles S is right. Just because a poll is online doesn’t mean it’s “complete bullshit.” I wouldn’t rely on this poll for decimal point precision, or even to be as reliable as your typical telephone poll … but that’s different than claiming the poll is wholly invalid.
Charles, I’d still expect a sample of people who have signed up to be polled to be unrepresentative. And I also want to know their refusal rate (even with a sample of people who signed up to be polled, there must be some refusals).
None of the things I want to see to make me trust a poll, are present in this poll.
Comparing the two polls, they found pretty similar results on the “socialist” question — but by and large, the Kos poll suggests that about a quarter of Republicans believe utterly ridiculous things (“Barack Obama wants the terrorists to win”), while the Harris poll finds that about half or more of Republicans believe such things (Obama wants to “turn over the sovereignty of the United States to a one world government”). That’s a substantial difference — although in either case, a frighteningly high number of Republicans have lost all ability to think coherently about politics.
If the Kos poll is reliable — and its methodology appears much better — than the Harris poll is biased in some way towards Republican extremists.
The problem with that poll is there’s no control, ie no comparison to what democrats believe. without the larger context, the poll is misleading. As Ilya Somin points out:
As he goes on to argue, while birtherism runs amok among the repubs, 911 conspiracy theories are equally prevalent among dems. unsurprisingly, polls indicate Dem’s are significantly more anti-semitic than repubs.
In general, I agree with you about parallel examples of extreme beliefs shown by polling. (I’d also argue that on both sides, you’d really need a more careful poll to find out how much people actually believe what they say).
Why do you say it’s not surprising that “Dems are significantly more anti-semitic than repubs”?
I find the poll you rely on for antisemitism not very convincing; it consisted of a single question. A more reasonable methodology would ask a range of questions and thus measure multiple kinds of antisemitism.
My guess is that the kinds of antisemitism exhibited on the left and the right differ. For instance, if you asked people if they believe that Jews that don’t convert are going to burn in hell, I expect you’d get many more “yes” answers from right-wingers than left-wingers.
Well, sort of.
If you look up “poll” in the dictionary it has two slightly different primary meanings: “a sampling or collection of opinions on a subject, taken from either a selected or a random group of persons, as for the purpose of analysis.”
(my emphasis)
So, if you are considering this a bit of data about a self-selected group, then no problem . . . it’s actually a census of that population (i.e., a census of the opinions of all the people who agreed to be a part of the poll). But what they seem to be doing here is the very, very biggest statistical no-no . . . taking data from a non-randomly selected group and then analyzing them as if they were from a random sample of a larger population (e.g., all conservatives in the US).
It’s not a matter of decimal points. It is “complete bullshit” to imply that this has anything to say about anybody except the people who agreed to take the poll.
There’s no reason to believe that these data are a good predictor of what a true random sample of “US conservatives” would look like.
I don’t know if this applies to Harris as well, but YouGov, who also do online polling, select most of their panelists themselves. They do allow people to sign up to be on their panel, but this is not the major source of their panelists (and the panelists they recruit by other means they aim to get a demographic balance on their sample). YouGov’s UK poltiical polling has been generally pretty accurate (Harris do far less UK polling, so I don’t know if they’re as good). There’s no reason an online sample has to be less random than a telephone or in-person sample, and in some ways it might be easier online (since you can send the survey to a roughly demographically-balanced sample in the first place, for instance, though you still have to weight for different response rates)
You could say the same about telephone polling. People have to have phones, and they have to be willing to spare the time to hear the survey (which the more politically interested might be more willing to do?) and they have to be in when you call. Provided the sample selection is done properly in both cases I see no reason why an online sample should be worse than a telephone one.
One might make four hypotheses about these kinds of polls, marmalade:
1. They’re a perfect representation of the underlying population.
2. They’re a very strong representation of the underlying population, with mathematically definable limits to their accuracy.
3. They’re a loose representation of the underlying population, and the extent of their accuracy can’t be mathematically defined.
4. There’s an extremely high probability that they have no relationship to the underlying population, and that they’re in fact a distinct subgroup.
I think #3 is the best way to see these kinds of online polls (given the methodology described at the Harris site). So the poll says the 1 out of every 2.5 Republicans surveyed think Obama is a Muslim … it wouldn’t surprise me to find that the real ratio is 1 out of every 1.5, or 1 out of every 5. It would surprise me to find that the real ratio is 9 out of 10, or 1 out of 20. I’ve learned something from this imperfect poll: that a significant portion of Republicans believe that Obama is a Muslim.
The assertion that this poll is “complete bullshit” suggests one believes in hypothesis #4: that there is a good likelihood that the poll respondents don’t merely represent a distorted sample of the underlying population, but in fact likely represent a completely distinct subset. There are situations where this would be the case. An online poll that found 99.9% of the respondents are familiar with the Internet is obviously not going to be terribly instructive about the larger population. But given the questions at hand and my understanding of the world, I simply don’t see why the methodology would create such a strong bifurcation between the respondents and the underlying population that the results have to be dismissed as completely void of any representative value.
At any rate, marmalade, though I think your stance is a bit doctrinaire and your (and Amp’s) vehemence somewhat puzzling, in general it’s very wise to be cautious about the interpretation of statistical data, and being thoroughly dismissive is probably better than taking polls such as these too literally. But I do disagree with you about them being “complete bullshit”; they’re used in marketing all the time and are better than nothing if you interpret them carefully.
I just want to let you know that I got the Onion reference immediately. Good one. ;)