Interesting item here that's been drawing some attention from the political blogs lately, especially Nate Silver's Fivethirtyeight.com. Apparently, there is quite a bit of evidence coming out that Atlanta-based pollster Strategic Vision, LLC (not to be confused with Strategic Vision, Inc., a well-established San Diego based research firm) has been publishing fraudulent poll results. A brief recap of the events so far:
Sept. 23: The American Association for Public Opinion Research reprimands SV LLC for failing to disclose basic information about their polls, including: the poll's sponsors, the sampling frame, "likely voter" methodologies, response rates, and weighting procedures.
Sept. 24: Silver reports on the AAPOR's action, noting its unusual nature and pointing out that SV has a history of obfuscating results - they never report demographic details, crosstabs, etc.
Sept. 25: Silver analyzes the results of all SV poll results since the beginning of 2005. He looks at the last digits of all their results in binary polls and catalogs how often each digit results (ex. a poll finding Obama 48 - Clinton 45 would result in a tally for 5 and for 8). Given the wide range of topics on which SV polls, one would expect this distribution to be roughly even. It wasn't. At all. Trailing 8's showed up 57 percent more often than trailing 1's, for instance. There were 599 trailing 5's and 639 trailing 7's, but only 533 trailing 6's. Something seems fishy about this. Humans, incidentally, are notoriously bad at selecting random numbers.
Sept. 26: Silver reports on an SV-run poll of 1,000 Oklahoma public high school students in which they were given 10 questions from the U.S. Citizenship Test. Again, remarkably fishy results arise. Only 2.8 percent of students scored a passing grade of 6 or higher. No student got 8, 9, or 10 answers correct - the odds of not hitting one nerd out of 1000 are astronomical. The poll claimed that only 23 percent of students identified Washington as our first president, and that only 43 percent could identify the 2 main political parties (a further 11 percent answered "Communist and Republican," and the rest responded "don't know"). It just doesn't make sense that Oklahoma high school students would be that stupid. Also, from a statistical point of view, the distribution is too tight - it's exactly what you'd get if you assumed no correlation between answering one question correctly and answering another one correctly. Clearly, that's false - a kid who knows one history fact is more likely to know others. That should increase the standard deviation of the scores.
This would probably be a good time to mention that the poll's sponsor was a conservative group that isn't terribly fond of the public school system to begin with.
Sept. 27: Turns out that SV's Atlanta address is a UPS store. They're really based out of a motel park in rural northern Georgia.
The CEO of SV LLC, incidentally, hasn't given any explanation for these strange results; he has, however, threatened to sue Silver.