Election 2016

What Can We Learn From National Primary Polling? Virtually Nothing.

Who's up? Who's down? Who cares?


In the spring and early summer of 2007, the frontrunners for the Democratic and Republican nominations for president were—by healthy double-digit margins—New York Sen. Hillary Clinton and former New York City mayor Rudy Giuliani. Oops.

It's become a cliche to joke about these comically inaccurate results from exactly eight years ago, but they are far from the exception. National survey research, even right before primary elections and caucuses, is almost useless as a barometer of who is most likely to win a party's nomination—except in those circumstances when one candidate is so far ahead that no polling is necessary to know it.

This fact will not deter the press from spilling barrels of real and virtual ink writing about each new data point for the next 12 months. "A new poll of possible 2016 presidential contenders shows a crowded Republican field," begins a recent, and utterly typical, article from the International Business Times, "while former Secretary of State Hillary Clinton has a wide lead on the Democratic side."

But primary polling does not measure who is most likely to win a party's nomination. Betting markets, such as the now-defunct Intrade, which shuttered its operations after federal regulators brought suit against it in late 2012, come closer to doing that. They ask participants to look into the future and guess what public opinion will be like on the actual day of the election. Primary polling, on the other hand, measures whom people would support were the voting to happen right now.

The further out from Election Day a survey is conducted, the less likely the result is to hold. National polls taken months before the first primaries are often astonishingly bad at forecasting how the chips will have fallen by the end of a several-months-long series of nominating contests.

Consider the 2008 race. Summer 2007 saw Hillary Clinton leading all comers by an intimidating margin. National surveys conducted for ABC News and The Washington Post in June and July of that year put her 15 percentage points ahead of her main opponent, Illinois Sen. Barack Obama. In December, mere weeks before the Iowa caucuses, an NBC News/Wall Street Journal poll found her winning by 22 percentage points. Even heading into Super Tuesday (February 5), most national polls gave the former First Lady an edge. Nonetheless, Obama became the nominee.

Things were even more volatile on the Republican side of the aisle. Through the summer of 2007, Giuliani ran far ahead of the pack, while the eventual winner of the nomination, Arizona Sen. John McCain, was straggling somewhere between second and fourth place. When Giuliani started slipping in September, it wasn't because McCain was finally making a run for it; it was because Fred Thompson of Law & Order fame had jumped into the race and grabbed some of the mayor's supporters.

Thompson's moment proved short-lived. By November, Giuliani was back on top and, according to an NBC News/Wall Street Journal poll, capturing a third of the primary vote—a remarkable feat in a packed field of strong contenders. Onlookers might have reasonably concluded he was likeliest to win the nomination. After all, he'd held the top position for more than a year.

Then in December came a breathless memo from CBS News and The New York Times: "The race for the Republican nomination nationwide has undergone sweeping changes since October: former Arkansas Governor Mike Huckabee now challenges Rudy Giuliani for the lead, with Mitt Romney on their heels. Just six points separate the three candidates." Not mentioned there: McCain, whom the poll found tied for fourth and taking just 7 percent of the vote.

Anyone trying to judge the relative chances of the various candidates based on national primary polls conducted before voting began would have been hopelessly lost. There was little indication the Arizona senator was capable of coming back to win the January 8 New Hampshire primary, let alone the whole shebang. Yet somehow, McCain would go from capturing 12 percent of likely voters in a December Washington Post/ABC News poll to 48 percent in February en route to winning the nomination.

The 2012 race was even crazier. GOP voters flirted seriously with nearly every other candidate before finally settling on Romney. The chart below from HuffPost Pollster shows four different people—Texas Gov. Rick Perry, pizza mogul Herman Cain, former House Speaker Newt Gingrich, and former Pennsylvania Sen. Rick Santorum—beating the erstwhile Massachusetts governor at various moments. The shape of the graph mirrors the roller coaster ride taken by anyone trying to predict the GOP nominee solely from polling.

a chart of 2012 GOP candidates

Pollsters contend with an array of methodological obstacles. Response rates today are a fraction of what they were when everyone had landline telephones and no one had caller ID. And as we get closer to elections, pollsters try to limit their surveys to "likely" voters, but their processes for doing this are inexact.

The American system of holding many discrete statewide primaries also makes national polls questionably relevant. Because states don't all vote on the same day, the outcome of one election invariably alters expectations and thus choices in subsequent surveys. "You could measure voter preferences right now in Texas all you want," says Mark Blumenthal, a former pollster who now runs the HuffPost Pollster site, "but I guarantee you they're going to be different after Iowa, New Hampshire, and South Carolina than they are right now. And they may be different in the midst of those three primaries than they were before or after."

Giuliani famously discovered this in 2008, when his strategy of ignoring the early primary states and focusing on Florida and the Super Tuesday elections ended in spectacular failure. McCain and Romney, his top opponents, were able to pick up momentum with wins in Nevada, South Carolina, and Michigan. Meanwhile, Giuliani could not recover from the perception that his campaign had collapsed even before voters in the Sunshine State, where he'd concentrated his resources, went to the polls.

The biggest weakness of primary polls is evident from the very question they pose. Most surveys ask some variation of: "If the election were held today, for whom would you vote?" Respondents can only take into account the information they have at hand at the moment. The situation on Election Day, after months of news coverage and millions of dollars in advertising, is sure to be vastly different.

And while general election voters get to choose between the nominees of political parties they have long known and loved (or hated), primary voters have no equivalent default option to vote for or against a party. This means the answers they give pollsters are subject to violent swings.

"When you have a lot of candidates who are poorly known, there are events that can happen that might temporarily alter the snap preferences people give when interviewed," Blumenthal explains. "A lot of the people who will vote in the primaries haven't thought about this yet [and] don't know the names of all the candidates. Yet when intercepted in the midst of doing whatever they're doing and read a list of names, they will try to pick one…If there was a lot of news this week about a candidate jumping into the race, some people are going to pick that name just because they heard it recently."

Journalists are already showing they haven't internalized these quadrennial lessons. Since Barack Obama was re-elected, newspapers and political commentators have declared no fewer than five different Republicans as the man to beat in 2016.

Last September, The Washington Post was calling Kentucky Sen. Rand Paul the frontrunner for the Republican nomination. By the end of January, the same paper was describing former Florida Gov. Jeb Bush that way. In December, The Huffington Post ran this headline: "Congrats, America: Ben Carson Is Now Your 2016 GOP Frontrunner." But in February, as Wisconsin Gov. Scott Walker inched his way to the top of the polls, the BBC called him "a little-known 2016 frontrunner." A few weeks later, National Review titled a post: "No, Really, Scott Walker is the Frontrunner." Even Mike Huckabee, fresh off Fox News and boasting a huge popular following in Iowa, has been tagged as a frontrunner in some conservative outlets, including the Washington Examiner.

One of these predictions may end up looking prescient. But surveys conducted now, more than half a year before the primaries even start, do virtually nothing to tell us which one it will be.

NEXT: VID: Stonewall 2015: The Night Marriage Equality Became The Law of the Land

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. Polling only matters when you base your choice on who’s most popular. It’s a poor substitute for critical thought.

    1. Maybe you’re misconstruing polling, then. Election polling is about trying to predict elections, not about your own choice, except to rule out certain choices as ineffectual. Weather forecasting isn’t about helping you decide what weather you like, either.

    2. It’s not even popularity- it’s “name recognition”.

  2. OT and with the caveat that it’s the Daily Fail: Mayor of Charleston says Obama to propose legislation to unmask Klan members and other hate groups

    Among the ideas being looked at is legislation forcing extreme right wings groups and violent organizations such as the Ku Klux Klan to provide identities of supporters and members.

    Riley, in an exclusive interview with Daily Mail online, said: ‘I have spoken with the President and the Vice President about this. And I have talked to the White House too.
    ‘One of the things we need to do is for the national government to give resources to expose these hate groups.

    ‘In America we worship the first amendment and any body can say anything they want.

    ‘But we need to shine the spotlight on them (racist organizations), so at least we know where they are among the public.

    ‘Neighbors should be able to know that the person living next to them is an absolute bigot. So there is a lot of work to do.’

    He said many self styled racists such as alleged gunman Dylann Roof built their evil beliefs after ‘being fed by the internet’ and new legislation to marshal racial material on the web is set to be brought in.

    ‘We need a national council on these hate groups. The President is talking about that. We have just got a lot more work to do,’ he said.

    1. +10000 Subpoenas

    2. “One of the things we need to do is for the national government to give resources”

      That’s really what it is all about isn’t it. Moar. Moar. Moar.

    3. The thoughts that launched 1000 woodchippers…

    4. “‘Neighbors should be able to know that the person living next to them is an absolute bigot. So there is a lot of work to do.'”

      Maybe have them wear some identifying mark on their clothing…a huge red *B,* or some symbol, like a star or something….

      1. And why should Jeremiah Wright’s church in Chicago be forced to give its membership list to the government?

        Oh, they’re talking about *right-wing* racists, not the left-wing kind. Very well, carry on.

      2. We need to put more resources toward developing self-driving trains.

    5. Isn’t it funny how much of the Progressive Establishment’s agenda boils down to “We think the Citizen’s Rights embodied in the Constitution and Bill of Rights are swell in theory, but in practice we – of course – need to sodomize them brutally.”

    6. In hell Mayor Joe Riley needs to drug out into the street and woodchipped from orbit to be sure.

    7. “‘But we need to shine the spotlight on them (racist organizations), so at least we know where they are among the public.”

      So the Black Panthers, the Nation of Islam, NAACP, “historical” black colleges, etc.?

    8. At least one positive outcome could be people realizing that a tiny percentage of the country actually belongs to racially motivated hate groups and they’re not one of the pressing problems of our time? Not that that would outweigh the negatives and almost guaranteed scope creep.

      1. Ummm yeah, considering it would just about immediately move on to outing members of churches whose pastors refuse to perform gheymerge ceremonies…

    9. “At least we know where they are among the public.”

      So is the idea to create a sex offender-style registry, but for “bigots”?

      Or will it be more akin to a McCarthy-era blacklist?

      1. They’ll just pick the worst features of both.

    10. Oh, and why exactly do my neighbors deserve to know my political and moral beliefs? Since when is that a thing?

      1. Since you started blabbing them. Why else would you?

    11. “Neighbors should be able to know that the person living next to them is an absolute bigot.”
      So I have a legal right to know my neighbor’s beliefs, and his refusal to elucidate them violates my right to feel comfortable. And who ever said the idea of ‘positive rights’ was the gateway to totalitarianism?

      “new legislation to marshal racial material on the web is set to be brought in.”
      Oh boy. Government marshaling the web. That can’t go wrong.

  3. But can’t we rather certain that Secretary Clinton will be the Democratic nominee and alas it is looking as though Senator Rand is not going to be the GOP nominee.

  4. If national polling doesn’t mean much because voters opinions shift so wildly given the results of Iowa, New Hampshire (and perhaps South Carolina) does this prove how awful the system is that primary voters in those states are able to determine a race?

    1. Yep, just for one example that’s why the farm bill is as unkillable as a horde of zombies…

    2. I don’t know if it proves how awful the system is. Maybe it just highlights the fickle nature of voters or the changeability of politicians in election cycles, something which can never really be fixed unless you just throw out voting rights altogether.

      1. This is definitely at least partly the case. It may well not be that they are doing a poor job measuring public opinion; but rather that public opinion is so fickle that even if you measure it with 100% accuracy one moment, it will likely be drastically different the next moment, so there’s little point in trying at all.

    3. I’ve always found the fact that the states don’t hold their primary elections on the same day to be a crock of shit. All it takes is for one candidate to do well in the first few states, then a few of the other candidates drop out, candidates that might very well have performed well had every state voted on the same day.

      1. It saves a lot of political $ when people don’t have to waste it on candidates who early tests show not to have a chance.

  5. I thought polling meant getting the views of a sample of people who still have land lines.

    1. For a sec. I read that as “land mines”.

    2. Nowadays, good pollsters make sure between 20 percent and 50 percent of their interviews come from cell phones.

  6. Is anyone of an opinion that Secretary Clinton might actually come close to losing a primary or caucus vote?

    1. Besides Vermont’s?

    2. Bernie wins California, if he is still standing and has any momentum.

      You think Hillary is a sure bet because…why? Because she leads the pack now?

      One more scandal, which is sure to happen, and she is close to losing it all.

      I also don’t think her health is good, and I’m not certain she doesn’t collapse half-way through.

      1. The media and top power brokers in the Democratic Party are far too invested in Clinton for much too stand in her way (Assuming reasonable her reasonable health)

        Have any of the scandals so far _really_ out much dent in her popularity?

    3. My money’s on Bernie.

      1. I doubt it. I don’t think he’ll raise anywhere near the money Hillary will, and he’s definitively too old. He’ll be 2nd place and he’ll push her to the left.

    4. Until about a month ago, I thought she actually had no chance of winning. Then I realized a lot of her support probably comes from people who think they’re in effect going to elect Bill for another term or 2?& they’re probably right. Nobody thinks she’d make a good prez, but they’re counting on him to pull the strings.

  7. “except in those circumstances when one candidate is so far ahead that no polling is necessary to know it.”

    That is impossible and you are an idiot.

  8. We need a national council on these hate groups.

    This seems familiar, somehow…

    1. I was actually going to say, “we need a hate group for these national councils.”

  9. Punditry is a full time job. You can’t expect America’s Thought Leaders to just lie around on the beach collecting unemployment for three years at a stretch. There’s horse races to call.

  10. The worst part about these polls is that they are also used to keep candidates off the debate stage, which all but ensures their polling will not increase. This was an issue concerning Gary Johnson last election cycle. The FEC states that a candidate must receive a certain amount of responses in 5 different polls of the FEC’s choosing. Some of these polls did not even include Gary Johnson as a candidate, although he was on the ballot in all states (there may be 1 or 2 that he was excluded from, I do not recall). Some respondents in the polls that did not list Gary as a candidate claimed that the pollsters would not allow them to answer Gary Johnson, only to choose from the chooses they gave. This is why Gary Johnson has been attempting to sue the FEC. Not only are these polls a load of crock, they can also be used to unfairly influence an election.

    1. The federal bureaucracy people just cheered for as they granted themselves regulatory power over the internet? The FEC that once carried out the Fairness Doctrine, and which the government argued could ban books and pamphlets in the lead-up to an election? How dare you suggest they would do anything that would squash political speech and questioning of the two main parties!

      1. Youse guys is conflating the FEC w the CPD & the FCC.

  11. Speaking of pundits– let’s have a look inside the clammy, terrifying rat-infested dungeon which is Doktor Krugabe’s head.
    Was I on the edge of my seat, waiting for the Supreme Court decision on Obamacare subsidies? No ? I was pacing the room, too nervous to sit, worried that the court would use one sloppily worded sentence to deprive millions of health insurance, condemn tens of thousands to financial ruin, and send thousands to premature death.
    Awesome. ISIS ain’t shit. It’s the TEALIBAN we need to be askeered of.

  12. I refuse to believe that Ben Carson isn’t the man to beat in New Hampshire

  13. Start making cash right now… Get more time with your family by doing jobs that only require for you to have a computer and an internet access and you can have that at your home. Start bringing up to $8596 a month. I’ve started this job and I’ve never been happier and now I am sharing it with you, so you can try it too. You can check it out here…

  14. Google pay 97$ per hour my last pay check was $8500 working 1o hours a week online. My younger brother friend has been averaging 12k for months now and he works about 22 hours a week. I cant believe how easy it was once I tried it out.
    This is wha- I do…… ?????? http://www.netcash5.com

  15. Start making cash right now… Get more time with your family by doing jobs that only require for you to have a computer and an internet access and you can have that at your home. Start bringing up to $8596 a month. I’ve started this job and I’ve never been happier and now I am sharing it with you, so you can try it too. You can check it out here…

  16. Journalists are already showing they haven’t internalized these quadrennial lessons.

    Horse shit. They know them very well. They’re just counting on readers not having learned them. Or they realize the readers have learned them, but what else you gonna publicize when you’re a political editor that readers will want to read about?

    People’s att’n is never in a vacuum except maybe when they’re sleeping. It’s never about what’s noteworthy on a time-avg.d scale, it’s about how notable things are compared to each other at a given instant.

Please to post comments

Comments are closed.