Slippery polls

Why the media should quote values with uncertainties

Larger version: 1,280×960 px (304 KiB)

This graph shows opinion poll data from April for the UK General Election 2010, with the ‘statistical error’, a mathematical measure of uncertainty, shown as the variation in brightness of the colours.

Though news organisations do it all the time, quoting polls without the errors is meaningless. For example, if the uncertainty on a party’s score is larger than the difference between parties, it’s impossible to tell who is in the lead. Similarly, if a party’s polling score changes from one day to the next by an amount smaller than the uncertainty, then you can’t say whether their rating has really improved or not. This is particularly important in the run-up to this election as all three parties are polling very similar percentages of the vote.

Statistical errors are lower bounds on the uncertainty in poll results because there are also methodological difficulties with gathering data, such as choosing who is included in the sample, and whether they are honestly reporting their opinion.

Methodology

Poll data were obtained from ukpollingreport.co.uk. On days where multiple polls finished, the average value was computed. The errors are computed as N/N; N = 1,000n, where n is the number of polls taken that day (ie I have assumed that every poll has a sample size of 1,000 and taken the simplest estimate of the error, which is crude but probably not too far wrong).

Comments

  1. Patel says (13:25 02/05/2010)

    To be fair to the Beeb, most of the reporters I've seen on TV have made the point that "the differences between the parties are within the margin of error..."

  2. Statto says (13:29 02/05/2010)

    But look at the pixel-thin lines on their graphs! Makes it pretty easy to come away with a false sense of certainty.

Leave a comment

(please type ‘orange’ into this box)