Statistical analysis of 2004 presidential election exit polls

USVoteCounts, a "scientific research project whose mission is to objectively investigate the accuracy of elections in America" has published a paper reporting on the discrepancy between exit polling and vote counts in the 2004 presidential election. The paper is titled "National Election Data Archive Project Analysis of the 2004 Presidential Election Exit Poll Discrepancies" and is authored by a number of statisticians and mathematicians; it's available as a full-text PDF and an executive summary PDF.

The abstract is a good summary (as it should be):

"What is the Main Cause of the Discrepancies between the Official Election Results and the Exit Polls?

"The exit pollster of record for the 2004 election was the Edison/Mitofsky consortium. Their national poll results projected a Kerry victory by 3.0%, whereas the official count had Bush winning by 2.5%. Several methods have been used to estimate the probability that the national exit poll results would be as different as they were from the national popular vote by random chance. These estimates range from 1 in 959,000 to 1 in 1,240. No matter how one calculates it, the discrepancy cannot be attributed to chance.

"Edison/Mitofsky disavowed the results of their own poll, saying that the data cannot be construed as evidence that the official vote count was corrupted, and hypothesized that Kerry voters were more amenable to completing the poll questionnaire than Bush voters. However, Edison/Mitofsky's own exit poll data does not support their theory that a higher exit poll response rate by Kerry voters accounted for the discrepancies between the exit polls and the presidential election results. Using Edison/Mitofsky’s data tables we demonstrate that the “reluctant Bush responder” hypothesis is implausible because it is inconsistent with the combination of high response rates and high discrepancy rates among the precincts with the highest percentage for Bush.

"There are Three Primary Explanations for the Discrepancies:

"1. Statistical Sampling Error – or Chance
We agree with Edison/Mitofsky that the first possible cause, random statistical sampling error, can be ruled out.

"2. Inaccurate Exit Polls
This is the theory that Edison/Mitofsky put forth. They hypothesize that the reason the exit polls were so biased towards Kerry was because Bush voters were more reluctant to respond to exit polls than Kerry voters. Edison/Mitofsky did not come close to justifying this position, however, even though they have access to the raw, unadjusted, precinct-specific data set. The data that Edison/Mitofsky did offer in their report show how implausible this theory is.

"3. Inaccurate Election Results
Edison/Mitofsky did not even consider this hypothesis, and thus made no effort to contradict it. Some of Edison/Mitofsky's exit poll data may be construed as affirmative evidence for inaccurate election results. We conclude that the hypothesis that the voters’ intent was not accurately recorded or counted cannot be ruled out and needs further investigation."

Unfortunately, Edison/Mitofsky are not releasing the full precinct-by-precinct exit poll dataset for analysis, which is ridiculous considering the importance of the issue.

The closing summary of the article makes a number of very powerful points:

"There is already a strong case that there were significant irregularities in the presidential vote count from the 2004 election. Nevertheless, critics are asking for firmer proof before going forward with a thorough investigation. We feel strongly that this is the wrong standard. One cannot have proof before an investigation.

"In fact, the burden of proof should be to show that the election process is accurate and fair. The integrity of the American electoral system can and should be beyond reproach. Citizens in the world’s oldest and greatest democracy should be provided every assurance that the mechanisms they have put in place to count our votes are fair and accurate. The legitimacy of our elected leaders depends upon it.

"Well-documented security vulnerabilities and accuracy issues have affected voting equipment as far back as the late 1960s, and history shows that partisan election officials have long possessed the power to suppress and otherwise distort the vote counts. The recent and ongoing proliferation of sophisticated computerized vote recording and tallying equipment, much of it unverifiable and hence "faith-based", dramatically augments the opportunities for wholesale and outcome-determinative distortions of the vote counting process. That the lion's share of this equipment is developed, provided, and serviced by partisan private corporations only amplifies these serious concerns. The fact that, in the 2004 election, all voting equipment technologies except paper ballots were associated with large unexplained exit poll discrepancies all favoring the same party certainly warrants further inquiry.

"The absence of any statistically-plausible explanation for the discrepancy between Edison/Mitofsky’s exit poll data and the official presidential vote tally is an unanswered question of vital national importance that demands a thorough and unblinking investigation."