Election 2016

Yep: The Polls Still Don't Work

The survey research industry is in crisis. Here are some of the reasons why.


dano / Dibujando

As the poll forecasters, the news websites, and the election betting markets all increasingly point to a win for Donald Trump tonight (Sweet Meteor of Death, help us), more than a few people are likely asking themselves, "How could this possibly be happening?"

Indeed, as recently as this morning FiveThirtyEight had Hillary Clinton with more than a 70 percent chance of winning; HuffPollster for their part had her above 98 percent. Of the last 10 polls in the RealClearPolitics four-way average, nine had the former secretary of state up by between two and six percentage points.

So what went wrong with the polls? It's too early to know, of course. Perhaps there really were "shy" Trump voters who were too embarrassed to admit as much to pollsters.

It seems more likely, however, that the problem is methodological. As I explained in a feature from the February issue of Reason ("Why Polls Don't Work"), modern pollsters face a variety of challenges, from the difficulty of identifying likely voters to the need to rely on guesstimations about how the final electorate will break down demographically. Plus, it's harder to get people on the phone than ever before, and you can't keep them talking to you for nearly as long as you once could.

The polls have understated conservative vote share repeatedly over the last four years. They underestimated the chances of a GOP landslide in the 2014 midterms. They underestimated both the Conservative Party's support in the last U.K. election and the level of support for Brexit. And those are just three of the highest-profile examples.

But the solution is not as simple as assuming the GOP will do better than the polls suggest. Recall that in 2012, many pollsters overestimated the vote share Romney would receive.

So yes, the polls are broken. That isn't a new development this year, though tonight's outcome—if indeed Trump holds on to win—will surely go down as one of the most stunning rebukes the public opinion industry has ever suffered. And what will it take to fix these issues? Cthulhu only knows.

NEXT: North Dakota Becomes the 27th Medical Marijuana State

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. I thought there might be a “shy Trumpkin” effect, but I couldn’t find anything to support my suspicions. Until tonight, that is.

    1. Speaking of stuff not working- Reason.com had serious comment posting issues for me for 3 hours.

      1. not a technical issue, they just HATE you!

  2. So Trump isn’t winning a single state just like all you dumb asses have been assuring me for the last year, right?

    1. Wisconsin has a lot singles in it? Interesting.

    2. We’ll be hearing a lot of mystical superstitious I-told-you-so’s if the farce makes it past the electoral college. But good luck convincing me that unverifiable secret ballot elections run by two looter parties are honest. I am glad Gary made the good showing I expected, surprised that the nationalsocialist looter drew to a fourflush and filled, and tickled that the soviet socialists got handed their ass after assimilating the entire Econazi platform following the defeat of 2000. The real loser (besides freedom) is the global warming religion.

  3. A Poll Apologist

  4. “the public opinion industry”. Their mission is to sell you an opinion, not gauge the one you have. Their “results” make more sense then.

    1. They are done to make people think the preferred policy is popular and thinking otherwise is hopeless

    2. Their mission is to manufacture consent. That is the phrase used by Lippmann (stolen by Chomsky) nearly 90 years ago in Public Opinion

      On the bright side, those who believe in that top-down approach to running the world using polls as one of their weapons were delivered a nick kick in the teeth. Your polls suck – and so does every other idea you have.

      I hate Trump – but I gotta admit I felt really good watching the visible discomfort among the political/media class yesterday. Even though they are still mostly clueless as to what hit them.

  5. So Trump hasn’t won a state just like all you dumb asses have been telling me for the last year right?

    1. I started thinking last week that Trump would win it but I have kept it to myself.

      1. Yeah, there was a lot of energy in Reddit’s The_Donald subreddit and lots of virtue signaling from my liberal FB friends.

      2. Not me. I pointed to the bookie odds and told people not to worry–that math was not backing der Fuehrer. Then again, I’ve won a lot of money filling fourflushes, and my party’s platform made out like gangbusters. What really surprised me was the FBI finding Gary’s emails over on The Hard Times: http://preview.tinyurl.com/junvw8c

    2. There you are! *bows*
      Your newsletter, I would like to subscribe!

      1. I won’t gloat too much. I don’t blame the people who thought he wouldn’t win. That wasn’t an unreasonable guess. The ones who claimed he had no chance, and you know who you are, deserve to be called out for making a dumb ass claim.

        1. He had no chance. And it’s not a good thing if he wins. Do you honestly think as scattered and stupid as Trump has been during the campaign, he’ll pull it together to make some assertive foreign and domestic policy push? There’s a probable recession on the horizon and the fallout from Obamacare looming over. Clinton and progressives will have taken it on the chin if she wins. Instead they’re left to rebuild with vengeance while Republicans scramble to keep up with this retard’s flailing about.

          1. bitter much?

          2. Do you think the alternative of what Clinton was offering, along with her open jabs at provoking Russia into an actual war, were somehow better idea’s than a blustering fool? Is calculated evil really better than incompetent bluster and foolhardiness?

            The voters don’t seem to think so by a thin margin. I’ll openly say that I didn’t realize how insane the entire Clinton machine was until I actually caught the nightly news for the first time in a decade and saw the blatant war posturing on Clinton’s part…against Russia. In a domestic election.

            No. Fucking. Thanks.

  6. the polls are broken.

    The polls work fine, everyone.

    Its just that the level of significance the media applies to them that’s completely retarded.

    The fact that the media completely blew the primaries should have been a warning which caused people to realize that they were relying on the wrong methods.

    Fool you once, shame on them?…

    Its not the polls that are the problem.

    1. The polls are fine with the wrong methods?

      1. If you can’t sanity-check/validate poll-results with proxy data of some sort, they’re bullshit.

        1. If there were any science in the social sciences, there’s probably a decent paper to be had on confirmation bias in polling. I won’t hold my breath.

          1. I’m sure if you tried you’d find one or 20.

  7. Is Cthulhu any relation to the Kochtopus?

  8. And a strong contender for winner of alt text emerges!

    1. Slade is good; she should write more frequently.

    1. There is not an i-dot of contemporary evidence that such a creature ever existed–and there are a number of surviving Pontius Pilate court records… (The book title is: Nailed) But I will grant that the reported outcome, as in 1928, and 2000 appears to be a victory for mystical superstition over a slightly more secular form of looter superstition. But I’m gonna haveta see evidence more convincing than the stupidity of the American public before I grant that some men aren’t mortal.

  9. I don’t like Trump but as I drink beer in Hawaii watching Hillary’s HQ I’m even more not-Hillary. I vote in TN, so Johnson and third party was my statement.

    Tom Brockaw (sp?) said it best: The media elites and political machine from the East Coast cities don’t represent a majority of America. The polls being wrong make sense in that context.

    I also want to here how Comey screwed Hillary until I’m old. That will give me years of smiles.


  11. The polls have consistently undersold support on the right for a couple of years now. And that’s a worldwide trend, not just in the US.

  12. HuffPo’s attack on Nate Silver for acknowledging the high degree of uncertainty in this race is looking so, so stupid

    1. Ryan Grim called Florida for Hillary 2 hours before it fell to Trump

      sorry, but you can’t even blame polling for that shit, its just incompetence.

      And that same guy apologized to Nate Silver here

  13. Perhaps there really were “shy” Trump voters who were too embarrassed to admit as much to pollsters.

    Or, just maybe, lots of people are really pissed off at the msm, and are hanging up instead of answering.

    So far, it’s legal to do that. It’s still a secret ballot.

    “Just say no.”

  14. So what went wrong with the polls?

    The question presumes that the purpose of the polls is to inform, rather than to influence the public. Pollsters are liars, who will say whatever the person hiring them wants them to say. We’ve seen this again and again. I remember polls that said Carter would squeak past Reagan.


    1. True dat. Nixon–coattailed in by the first Republican to be elected after Hoover’s prohibition enforcement wrecked the economy–debated JFK (ignoring 16 other contenders) and lost. The Tricky Solution was TITLE VII?TAX INCENTIVES FOR CONTRIBUTIONS TO CANDIDATES FOR PUBLIC OFFICE, TITLE VIII ? FINANCING OF PRESIDENTIAL ELECTION CAMPAIGNS?, SECTION 801. PRESIDENTIAL ELECTION CAMPAIGN FUND ACT. The Internal Revenue Code of 1954 is amended by adding at the end thereof the following new subtitle:
      “Subtitle H ? Financing of Presidential Election Campaigns (?)
      Translation: The IRS is instructed to make transfer payments for entrenched party candidates to hand to the media in exchange for their ignoring the libertarian party. If the GO-Pee urine testing party had lost on our spoiler votes, this coercive purchase of coverage might have been repealed.

  15. At least 538 always said that Trump could win, while HuffPo and WaPo were both 98% confident of a Hillary victory (oops!).

    1. What are you talking about? 538 had it as a Clinton victory 72-28.

      1. That’s exactly what he’s talking about…a 30% chance, which is pretty significant compared to a 2% chance.

  16. Maybe polling is bullshit, and you can’t accurately predict what individual people think based on their neighbors’ opinions, or their skin color, or whatever other collectivist criteria?

  17. Stephanie is right to be baffled. After filling over a thousand flushes by drawing a card the novelty of beating 4 to 1 odds goes away and the concern is that the pot contain at least four times the bet before the draw. But the delightful surprise of Brexit followed by this morning’s announcements of a new Chancellor of the Reich cause Lysander Spooner’s warning against secret ballots to echo out of the mists of Reconstruction:
    “But a secret government is little less than a government of assassins. Under it, a man knows not who his tyrants are, until they have struck, and perhaps not then. He may guess, beforehand, as to some of his immediate neighbors. But he really knows nothing. The man to whom he would most naturally fly for protection, may prove an enemy, when the time of trial comes. ”
    But there is hope. I recently signed for the feds a statement that read: “I understand that by faxing or emailing my voted ballot I am waiving my right to a secret ballot.” This phrase offers hope that forensic science and rules of evidence may someday have a say in how we are delivered into the hands of bloodthirsty looters.

    1. Actually, the odds of drawing a 5th for a flush are less than 1 in 5. There are only 9 cards of the suit you want left in the remaining 47 cards. 9/47 is .1915 or a hair over 19% odds of getting one of them. I hope you don’t try to make money with poker 🙂

  18. Does this mean we never again have to listen to this stupid shit about how Nate Silver is the smartest man on the face of the earth? Jesus Christ, I hope so.

    The man was almost completely wrong about everything this entire election cycle, despite “Calidissident’s” bullcrap about Silver the poll analyst and Silver the pundit were really two different people working at odds with each other.

  19. The polls aren’t ‘broken’. They’re not polls. They’re leftists in media lying to get their way. They’re like little kids telling their parents that ‘all the other kids have one’ or ‘X’s mom says it’s safe’.

    And lately, they use them to lie to themselves.

    “97% of ‘climate scientists agree…..” Gun control. Hillary’s popularity. BLM support.

    Now they’re telling themselves that their ideas are great because their own ‘polls’ say so.

    And it’s gotten so distant from reality that it’s finally starting to blow up in their faces.

    Will they fix it?

    No. Look at this article. They’ll make excuses and keep trying to go on.

  20. Cthulhu for president, 2020. You’d be insane to go against him.

  21. Well, if this was an error, wouldn’t we see the errors in predictions go both ways? All the examples underestimated conservative performance.

    1. Dammit–we TOLD you not to pay attention to the SJW behind the curtain!

  22. I think it’s less a question of “shy” and more a matter of, “None of your bees wax”. Try getting an accurate poll on gun ownership, while you’re at it.

  23. The funny thing to me is that Clinton clearly took internal polling seriously, because her final campaign stops were Michigan and Pennsylvania/Philadelphia with Obama. The Democrats went off cruise control and kicked into overdrive precisely where they lost. So what metrics where they using?

  24. Anyone know the link to where the image portraying this story is? If so, please post…

    If you buy the very plausible story that in order to shut out any opposition both the parties – Republicans and Democrats – agreed to fight it out between themselves a long long while ago, then the intense media opposition to Trump is understandable. As is the refusal of many Republican politicians to support him…

    So is the failure of the polls to forecast the correct result. Along with the media they were totally biased against Trump and so delivered the results they thought would help bias/swing all voters against him rather than true results…

    The drive towards a one world government, in which all but the oligarchy get screwed, has thankfully received a big setback. Let’s hope Trump doesn’t meet the same end as JFK…

  25. RE: Yep: The Polls Still Don’t Work
    The survey research industry is in crisis. Here are some of the reasons why.

    Here’s another reason why the survey industry is in crisis.
    People like to intentionally mislead pollsters to skew results.

  26. George Box once said: All models are wrong, but some are useful.

  27. Most pollsters are about implementing their desires but still getting paid..

  28. According to a new report, 3 out of every 4 Americans no longer trust polling data. When asked to comment on this, 70% of Americans said they’re not sure that the poll is accurate.

  29. According to a new report, 3 out of every 4 Americans no longer trust polling data. When asked to comment on this, 70% of Americans said they’re not sure that the poll is accurate.

  30. I am sick to death of this. Polls didn’t ‘miss’ anything. They purposefully ignored reality because they believed they could lie their way to victory.

  31. L.A. Times was the only poll predicting a Trump victory, and they were absolutely raked over the coals. HuffPo in particular mocked them pretty harshly. Funny how that turned out.

  32. until I looked at the paycheck saying $4730 , I did not believe that…my… brother woz like actualy bringing in money part time from there computar. . there friend brother started doing this for less than 7 months and resently paid for the morgage on there home and bought a new Cadillac …….

    …….. http://www.jobprofit9.com

  33. The pollsters also got it big time wrong for Bush Jr’s 2nd term where he got more votes than any prior President. Looks like Trump will be shattering that record.

  34. until I looked at the paycheck saying $4730 , I did not believe that…my… brother woz like actualy bringing in money part time from there computar. . there friend brother started doing this for less than 7 months and resently paid for the morgage on there home and bought a new Cadillac …….

    …….. http://www.jobprofit9.com

  35. until I looked at the paycheck saying $4730 , I did not believe that…my… brother woz like actualy bringing in money part time from there computar. . there friend brother started doing this for less than 7 months and resently paid for the morgage on there home and bought a new Cadillac …….

    …….. http://www.jobprofit9.com

  36. Maybe people should register to vote in polls.

  37. The polls over reached, and Nate Silver initially dismissed Trump as a “clown candidate”, BUT if you look at the data range predicted by Nate / 538, it wasn’t too bad.

    The probable range of electoral college votes, for the most part, gradually narrows as the results come in. In other words if 538 had refrained from calling likelihood on a winner, they could have been more realistic and called “don’t know yet” up until that last tranche of results removed the overlap in “is it Clinton, is it Trump”.

    False precision sunk their analysis. Unwise when so much voting was about “least loathed” rather than “preferred” candidate. (That’s just abnormal.)

Please to post comments

Comments are closed.