Polls

March 22, 2002 2:37 PM

Polls Come Under Fire. Watchdog Group Issues Rebuke on Poll on Islamic Countries. Meanwhile, those bogus aggregates continue to circulate freely in this country and around the world.
posted by semmi (16 comments total)

According to a recent study 85% percent of all polls are flawed and when surveyed 76% of respondents say that studies should refine their methods. And in a recent vote 99% of Americans say that they are tired of surveys, studies and polls.
posted by jonmc at 3:10 PM on March 22, 2002

Links to background on Arab & Islamic public opinion and polls from a recent thread ; a related MetaTalk thread on Islam bashing.
posted by sheauga at 3:17 PM on March 22, 2002

Does this mean that everyone loves Amrica and supplrts what we are doing? Lordy, Bush is doing wonderful job and I am sorry I did not vote for him instead of Hubert Humphrey.
posted by Postroad at 3:22 PM on March 22, 2002

Gallup gets a big, deserved black eye for this one. Was it that this project exceeded anything they'd done previously? It seems inconceivable that a respected polling organization would make such gross, obvoius errors.
posted by dhartung at 3:24 PM on March 22, 2002

I knew something just didn't seem right with those poll results. The level of twisting of the data just begs, as the article points out, a comparison with Enron.
posted by laz-e-boy at 3:27 PM on March 22, 2002

Ditto on the Enron comparison. Cooking the numbers for a specific appearance seems to an accepted practice everywhere.

a related MetaTalk thread on Islam bashing.
America bashing = GOOD; Islam Bashing = BAD?
posted by HTuttle at 3:50 PM on March 22, 2002

Here's what absolutely bugs the hell out of me about polls, and I'll quote from a Washington Post report that's in front of me now (by the way, ignore the content, it's the numbers I take issue with): "The survey found that nine in 10 Americans continue to support the military action in Afghanistan, unchanged from November." And the small print reads: "This poll is based on telephone interviews with 1,008 randomly selected adults nationwide, conducted March 7-10." So in actuality, 907.2 people "continue to support," etc. I realize the Post/ABC can't ask everyone in America, but the sample is only 3% of the population, so how can it be meaningful? Yeah, I know, extrapolation and all that, but I find it terribly misleading, and that's why I pay no attention to polls, other than to get riled up about them.
posted by Dean King at 3:53 PM on March 22, 2002

Actually, if you choose them carefully, 1,000 people can in fact be representative.

I should note that 1,000 people is rather less than 3% of the population, however. ;)
posted by kindall at 3:58 PM on March 22, 2002

even worse, then!
sorry, math's not my best subject, and my kalakalator doesn't go up to 268 million...
posted by Dean King at 4:02 PM on March 22, 2002

Honestly, anyone that takes an elementary statistics course will agree that the size of a samle does not have to be very large to be representative. Some people, unfortunately, tend to hold the opinion that statistical methods overstate the census numbers in urban areas.

This is not to say that polls can be insanely biased for a multitude of other reasons. But the samle size isn't one of them.
posted by MarkO at 4:19 PM on March 22, 2002

Gallup actually had Bush winning by 19 percent over the summer of 2000, then Gore winning by 11 or so a coupla weeks later. There were 10 point or so shifts over a coupla days. Hard to conceive of Gallup weirdness? No attack on anyone here, because it's impossible to keep up with every poll these days, but . . . my ass!
posted by raysmj at 4:25 PM on March 22, 2002

For some reason I thought this was about the "Which Islamic Country Are You?" personality test. ("Congratulations, you're Yemen!")
posted by jjg at 4:55 PM on March 22, 2002

What bugs me about polls is the often blatant misreporting of them. For example, on March 8 USA Today ran an article with the headline "Bush approval rating dips below 80%." The relevant sentence: "The poll puts Bush's job approval at 77%, down from 81% a week ago..."

But here's what the LAST paragraph said: "The March 4-7 poll of 1,006 adults has an error margin of +/- 3 percentage points." That means that Bush's approval rating for the week was UNCHANGED from the week before, given the MOE. In fact, it could well have gone UP from the week before, but given the MOE the only honest way to report the results would be to say they were unchanged. The headline, the article, was just flat out an untrue piece of writing. (Gallup did the same thing on their press release though; it was hardly a USA Today internal conspiracy.) And sure enough, a week later Bush's rating was right back at 80%.

As for the pollsters only interviewing ~1000 people, it is true that you get greatly diminishing returns after a certain number. I'm making the following numbers up, but just to give you the general idea: Asking 1000 people gives you a +/- 3% MOE. Asking 1500 may only give you a +/- 2.7% MOE. Asking 2000 may only give you a +/- 2.6% MOE, etc. And since polling is very human-intensive and time-consuming, you don't want to spend twice the time and twice the money to get results that are only a few tenths of a percentage point more accurate.
posted by aaron at 5:46 PM on March 22, 2002

Interesting that from both the Washington Post's POV and from Aaron's there's only one truth - there's lies, there's damned lies, and then there's statistics.

And yes, I do mean that both points of view are equally valid - and that anyone can choose to present their statistics in whatever way best suits the results they want to give.

Knowing that fact is what enables us to understand the news, rather than just accept it.
posted by yhbc at 7:49 PM on March 22, 2002

You've got to love Kuwait. '36 percent of those interviewed in tiny Kuwait said the September 11 terrorist attacks were morally justifiable.' That's the sort of thanks normally only given by the French.

I heard on the news today that there is a town in Nigeria where 70% of male new born babies are presently being named Osama.
posted by RobertLoch at 9:25 PM on March 22, 2002

'36 percent of those interviewed in tiny Kuwait said the September 11 terrorist attacks were morally justifiable.' That's the sort of thanks normally only given by the French.

In light of the following excerpt from the linked article, the 36% of poll respondents in Kuwait who consider the September 11 terrorist attacks justified, could, perhaps, all be French expats...

In fact, you didn't need to be a citizen of the country where the interviews were conducted. For example, fewer than half of the individuals in the Kuwait sample were Kuwaiti citizens.

What disturbs me about stories like this is that, in my opinion, the vast majority of the masses (not just in America) consume their news uncritically, not stopping to question the validity of reported statistics, nor bothering to take those statistics with the prudent grain of salt. The mass news media outlets are looking for short, easily digestible and sensational blurbs, and pertinent and important details are most often either glossed over or emitted entirely from reporting (hurry, jump to the next item before everyone falls asleep). The problem arises when the media reports misleading or flat out incorrect news, and no one bothers to parse it for validity. And the problem is greatly exacerbated when advocacy groups latch on to and parrot statistics that they know are misleading and untrue, as such groups are often wont to do.

Frustrating, really.
posted by syzygy at 9:21 AM on March 23, 2002