Question: How often do political reporters and commentators report poll results like
Margin of Error: + or – 2
(names courtesy of the cartoon South Park) as a “statistical dead heat” between the candidates?
Answer: All the time!
Problem: It’s wrong!
So, on the eve of the Arizona and Michigan primaries, here’s my attempt to get political reporters and commentators to get it right tomorrow…
What the “Margin of Error” Really Means: The “margin of error” is NOT a “window” within which the actual results are equally likely to fall. So, in the example above, it’s NOT equally likely that Douche and Turd truly both have 50% support.
The margin of error is what we call a “confidence interval,” and it’s generally a 95% confidence interval. In the example above, this means there’s a 95% chance that Douche’s true support within the polled population (e.g. “likely voters”) is between 50% and 54%, and there’s a 95% chance that Turd’s true support is between 46% and 50%.
So, in the example above, there’s really only a 5% chance that Douche’s true support is below 50%, and there’s really only a 5% chance that Turd’s true support is above 50%. Conversely, there’s really a 95% chance that Douche’s true support is above 50%, and there’s really a 95% chance that Turd’s support is below 50%.
Bottom line: In the example above, there’s actually a 95% chance that Douche is winning — it’s NOT a “statistical dead heat”!!!
(If you’re interested in reading more about how often statistics are misinterpreted and misreported, you might like Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart by Ian Ayres, 2007, which explains inaccurate opinion-poll interpretations better than I just did.)