Another of Brexit’s big losers: the opinion polls

In the maelstrom of comment following Leave’s victory in the referendum there have been many groups blamed, criticised etc. Amongst the greatest of the losers, however, are once again the pollsters.

The pollsters have a singularly poor record in forecasting British national elections when they are remotely close.

The original failure of recent times was John Major’s victory in 1992. Then the pollsters all claimed Kinnock would win. During the Blair years the pollsters did better but that was as much as anything that everyone knew the outcome of Labour’s first two and realistically third triumph.

The last general election was a singular failure for opinion polling and a long inquest was held which, although it reached various conclusions, did not seem to change much in terms of methodology (at least no successful changes).

In this referendum campaign, the telephone polls consistently suggested an easy win for Remain. The internet based polls were closer but the modest lead for Brexit in the couple of weeks before polling day had evaporated in the last few days even on the internet.

The error continued even into polling day with Cameron apparently celebrating victory and his staff suggesting a 57:43 win for Remain. Even Nigel Farage was wrong, conceding defeat.

The book makers did no better, consistently forecasting a remain win.

There will no doubt be a further post-mortem on this failure: A few thoughts though.

The telephone pollsters are probably mainly using mobile phones. Whilst the majority of the population have mobile phones the mobile ownership rate is lowest amongst the older and poorer, who, despite the apparent desires of some Remainers, have an equal vote to everyone else.

Another problem is likely to be the one identified in 1992 about the Tories. That is that Brexit was the less PC position. If people are rung up or stopped in the street or whatever they are less likely to admit to being pro Leave. The more Leave was described as racist, stupid, ill educated etc. the more reticent its supporters will have become to admit to their true views.

In Northern Ireland we have had experience of this phenomenon. When the DUP and Sinn Fein were (rightly or wrongly) seen as extreme people never admitted to being about to vote for them: instead they often claimed allegiance to the likes of Alliance. This referendum has seen a close analogy.

I would not pretend to have seen Leave’s victory coming but the reality is (albeit post hoc) that we should have been suspicious that this is what would happen.

Furthermore if by chance anyone is foolish enough to listen to those demanding a second referendum or simply ignoring the vote they should note that any opinion poll, petition or whatever they point to is not worth the internet server space it exists on.

, ,

  • Actually, it is harder to get lists of mobile phone numbers, so a lot of telephone polling is to landlines.

    In 2015 the errors were caused by doing national polling, rather than constituency by constituency polling concentrating on 60 marginal constituencies (which is where the Conservatives campaigned). In the USA they have local polls. I didn’t see any referendum polls that tried to do geographical breakdowns finer than the 4 nations. There were no reports on the differences between northern England and London and Oxfordshire.

    Postal votes could be sent in up to 3 or 4 weeks before polling day, so any later polls ignored those votes.

    In any case, what we don’t know from polls (but might from academic research) is why people voted the way they did.

  • terence patrick hewett

    Bookies odds are estimates of implied probabilities and are only right about the balance of money bet. They have no view on the outcome, only that they will be still in profit whoever wins. So the odds have no value at all in predicting the outcome: bookies odds are not predictive they are priced. There’s an important difference, they are based on how people are betting as well as guestimates of what the result will be.

    In this case they have managed to sucker and scalp all the big players to the tune of £50+ million. Some estimates give a figure nearing £100 million.

    If you believe that bookies are predictive: take your house and all your money up to the very shirt on your back and put it on the party of your choice. Best of…..Luck?

  • mickfealty

    I suspect most of the problem with polls is our expectations of them. They cannot tell the future accurately, yet most of are still prepared to believe they can. John Currice was at pains to warn us right up to the night it was on a knife edge, 50/50 and that the betting markets were out. Freak weather conditions in London and poorer than expected results elsewhere just tripped it down rather than up.

    So how did the pollsters fail? One or two individuals are eating humble piece because they hardened their analysis of their own numbers in favour of the more comfortable direction for them. Matthew Paris on RTE News One last week pointed out there was no polling evidence saying remain, more that no one could bring themselves to believe the British people would be daft enough to vote for it.

  • Abucs

    I think two factors are the fear of participants to go against the politically correct religion and the fact that this event was a one off so the pollsters have not had the historical experience to ask the right questions, in the right way, to the correctly representative demographics.

  • kensei

    The polls had a definite shift back to remain in the days leading up, and I think people assumed the pattern of the Scottish referendum would hold.

    I think the lack of a true exit poll also told and that really caught the markets on the hop. I am at a loss as to why there wasn’t one on such a momentous vote.

  • Because such exit polls are based on the change from how the same areas voted in the previous election. In this case, the previous election was 41 years earlier and thus worthless.

  • Petronius

    The Southern Irish polls seem more accurate than those in the UK. The UK pollsters should talk to Millward Browne which was really close in the Irish General Election in 2016. I think the problem is how you apportion don’t knows. Overall the online polls were closer but only two of the last 8 polls said it would be Leave.

    I placed a bet on Leave at 11-8. A few hours later I heard Jo Cox died and the media seemed to believe it would affect the vote. I was sceptical because after Anna Lindhs death in the Swedish Euro referendum in 2003 they still voted No. I was also factoring in the “stiff upper lip” in contrast with our Southern Irish Catholic sentimentality, which told me the Brits would not link the Jo Cox issue with their voting intention. Also the western anti Establishment trend in the working class contributed to my thinking. I hope pollsters will allow for such societal trends in future.

    On the bookies odds, I was watching the Keiser Report on RT and someone said one person bet £85 million on Remain. Is it possible wealthy people were trying to rig the odds to influence the vote? I now wish I had waited for the odds on Leave to drift to the ridiculous 9-1.