Report of the Inquiry into the 2015 British general election opinion polls
Venue
National Centre for Research Methods (2016), pp. 115
Publication Year
2016
Authors
Patrick Sturgis, Nick Baker, Mario Callegaro, Stephen Fisher, Jane Green, Will Jennings, Jouni Kuha, Ben Lauderdale, Patten Smith
BibTeX
Abstract
Executive Summary The opinion polls in the weeks and months leading up to the 2015
General Election substantially underestimated the lead of the Conservatives over
Labour in the national vote share. This resulted in a strong belief amongst the
public and key stakeholders that the election would be a dead heat and that a
hung-parliament and coalition government would ensue. In historical terms, the 2015
polls were some of the most inaccurate since election polling first began in the UK
in 1945. However, the polls have been nearly as inaccurate in other elections but
have not attracted as much attention because they correctly ndicated the winning
party. The Inquiry considered eight different potential causes of the polling miss
and assessed the evidence in support of each of them. Our conclusion is that the
primary cause of the polling miss in 2015 was unrepresentative samples. The methods
the pollsters used to collect samples of voters systematically over-represented
Labour supporters and under-represented Conservative supporters. The statistical
adjustment procedures applied to the raw data did not mitigate this basic problem
to any notable degree. The other putative causes can have made, at most, only a
small contribution to the total error. We were able to replicate all published
estimates for the final polls using raw microdata, so we can exclude the
possibility that flawed analysis, or use of inaccurate weighting targets on the
part of the pollsters, contributed to the polling miss. The procedures used by the
pollsters to handle postal voters, overseas voters, and unregistered voters made no
detectable contribution to the polling errors. There may have been a very modest
‘late swing’ to the Conservatives between the final polls and Election Day,
although this can have contributed – at most – around one percentage point to the
error on the Conservative lead. We reject deliberate misreporting as a contributory
factor in the polling miss on the grounds that it cannot easily be reconciled with
the results of the re-contact surveys carried out by the pollsters and with two
random surveys undertaken after the election. Evidence from several different
sources does not support differential turnout misreporting making anything but, at
most, a very small contribution to the polling errors. There was no difference
between online and phone modes in the accuracy of the final polls. However, over
the 2010-2015 parliament and in much of the election campaign, phone polls produced
somewhat higher estimates of the Conservative vote share (1 to 2 percentage
points). It is not possible to say what caused this effect, given the many
confounded differences between the two modes. Neither is it possible to say which
was the more accurate mode on the basis of this evidence. The decrease in the
variance on the estimate of the Conservative lead in the final week of the campaign
is consistent with herding - where pollsters make design and reporting decisions
that cause published estimates to vary less than expected, given their sample
sizes. Our interpretation of the evidence is that this convergence was unlikely to
have been the result of deliberate collusion, or other forms of malpractice by the
pollsters.
