Are Public Opinion Polls Exaggerating Obama’s Lead?

Public opinion polls have come under greater scrutiny this election season, with claims that several prominent polls have oversampled Democrats putting Obama in the lead. Frankly, I am skeptical of claims that contend Obama is leading in the polls due to oversampling of Democrats. Instead of focusing on the share of Democratic respondents in each poll, the primary issue is if pollsters’ likely voter models are correct. If pollsters are relying too heavily on 2008 turnout and overestimate Democratic turnout, they may overestimate Obama’s lead by about one and a half points.

NumbersMuncher, an anonymous poll analyst, tweeted the following graph, garnering Internet attention, here and here. Some conclude this is further evidence prominent public opinion polls have oversampled Democrats with the intent to bolster the bandwagon effect for the president.

 

Source: NumbersMuncher

When looking at this graph, one should not assume that each survey has oversampled Democrats, because typically more Americans do self-identify as Democratic than Republican, likewise more Americans self-identify as conservative than liberal. Moreover, these numbers do not sort Independent-leaning Republicans and Democrats. Yet, this graph’s horizontal access presumes each sample should have an equal number of Democrats and Republicans, since it’s titled “% of Dem Sample Advantage.” [emphasis added]

Moreover, this graph combines partisan breakdowns among both likely voters and all survey respondentswithout identifying each. If it did, one would quickly recognize that the polls with higher Democratic margins are also polls which reported partisan breakdowns among registered voters but reported election match-ups based on likely voters (Ipsos-Reuters, ABC/Washington Post). 

I have updated the above graph to reflect Real Clear Politics’ most recent poll compilation. This demonstrates how sometimes partisan breakdowns are based on registered voters when election match-ups are based on likely voters.

Source: Real Clear Politics, and author's calculations.

Typically, there is a wider partisan gap among registered voters than likely voters. Historical data from national exit polls reveals that actual voters tend to reflect more equal numbers of Democrats and Republicans, with a slight Democratic advantage before sorting Independents. Likely voter models should take this into account.

The following chart compiles actual voter data from national exit polls:


Source: Roper Center for Public Opinion Research

If pollsters rely too heavily on 2008’s partisan turnout, in which Democrats had a significant turnout advantage, this could skew the likely voter model to include more Democrats than would be historically advisable. There is little evidence thus far to conclude that 2012 will mobilize voters similar to 2008. Consequently, likely voter models should consider a broader historical trend.

It is hard to say whether pollsters are in fact relying too heavily on 2008 partisan turnout, because it is extraordinarily difficult to track down how these pollsters define likely voters. 

According to Chris Jackson at Ipsos-Reuters, “most research organizations use a combination of prior voting behavior, interest in the election and self-report likelihood to vote to categorize likely voters. ...Some pollsters also use ‘voter lists’ or commercial lists of people who voted in the last election instead of screening these individuals from the population.” 

Rasmussen gives a vague explanation here, “The questions involve voting history, interest in the current campaign, and likely voting intentions. Rasmussen Reports determines its partisan weighting targets through a dynamic weighting system that takes into account the state’s voting history, national trends, and recent polling in a particular state or geographic area.” 

ABC News explains, they “develop a range of ‘likely voter’ models, employing elements such as self-reported voter registration, intention to vote, attention to the race, past voting, age, respondents’ knowledge of their polling places, and political party identification.” 

As Huffington Post’s Mark Blumenthal reports, “CNN has published no explanation of how they select likely voters.”

Without seeing the demographic composition of the likely voters in each poll, it appears that several polls are extrapolating 2008 turnout beyond what will actually occur in 2012.  Consequently, these polls may overestimate Democratic turnout and thus Obama’s lead in the polls. Excluding Rasmussen, the average Democratic share lead among these polls is approximately 5 points, when historically it has been 3 points among actual voters (excluding partisan Independents). If likely voter models have in fact overestimated Democratic turnout, Obama’s lead could shrink about one and a half points.

Rather than protest the share of Democrats sampled, poll consumers should redirect their energies toward determining whether the likely voter models employed are in fact predictive of actual voter turnout.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • fried wylie||

    more Americans do self-identify as Democratic than Republican, likewise more Americans self-identify as conservative than liberal.

    *confused dog stare*

    I'm forced to conclude that more Americans have no fucking idea what they're talking about and should stop identifying with anything.

  • ||

    It would be about as productive to ask them whether they identify more with Rick or AJ from Simon and Simon.

  • Almanian's Evil Twin||

    +80's

  • Fist of Etiquette||

    Gerald McRaney wouldn't apologize for America.

  • ||

    Jameson Parker would be tempted to, but would come to his senses.

  • Paul.||

    Apropos of Epi's comment, I personally identify with Rick. Way Rick. Rick by two lengths. That's how much Rick.

  • Almanian's Evil Twin||

    So, basically by a Rick roll, then

  • ||

    Paul's mustache is THIS BIG.

  • tarran||

    The actor who played AJ is actually quite the badass.

    He broke his back as a teenager jumping out of a girls dorm window (on 2nd floor) when detection by the RA's seemed imminent, and pistol whipped his neighbor after the neighbor said mean things to his wife.

    There are a few other exploits I believe, but I can't remember them.

  • ||

    Well, he was shot by said neighbor but made a full recovery.

  • tarran||

    Damn, you are right.

    It doesn't make up for the fact that you are clueless about tachyons though.

  • ||

    Prove it, Einstein. I have Data and Geordi on my side. What do you have? CONSENSUS.

  • tarran||

    Really, a robot and a guy who wears a beret as if it were glasses?

    You really want to go there?

  • Pro Libertate||

    That's not a beret. That's an air filter.

  • ||

    Data is an android, not a robot, you idiot. Every time you open your mouth, you embarrass yourself further. And look, Geordi had to wear his punk glasses because...uh...he was, like, allergic to the miracle cures for blindness and stuff. You know, typical TNG plotting.

  • Tim||

    They haven't cured gay by 2400 AD?

  • nicole||

    Beret? Did you mean barrette? Also, everything Epi said.

  • GILMORE||

    Episiarch| 9.18.12 @ 1:00PM |#

    It would be about as productive to ask them whether they identify more with Rick or AJ from Simon and Simon.

    I agree, that is completely ridiculous.

    Because anyone in their right mind would completely identify with Cody Allen from Riptide

  • ||

    Perry King?!? What, are you insane? Joe Penny all the way.

  • GILMORE||

    Its people like you who destroyed Real America

  • EmmettCrunk||

    I for one fear the ascendency of these consrevative democrats.

  • Paul.||

    They are ascended, sir.

  • Untermensch||

    Look no further than Obama’s civil liberties record to know that he is Dick Cheney’s love child and that your worst fear is upon us.

  • Randian||

    more Americans do self-identify as Democratic than Republican, likewise more Americans self-identify as conservative than liberal.

    Let me help you out:

    Americans want to be seen as caring about the downtrodden (Democrats) and fiscally responsible at the same time (conservative).

    The fact that ne'er the twain shall meet never crosses their minds.

  • PapayaSF||

    Randian, surely you understand that those two positions aren't mutually exclusive? There are plenty of ways to help the downtrodden that don't involve the government spending money.

  • Paul.||

    I think Randian's point is the average voter doesn't care.

  • Randian||

    Surely I understand it.

    The point is most people don't.

    Voting for children / the elderly / education/ our troops is a sign of support. Voting against funding those things is bad.

    That's as far as it gets.

  • Robert||

    It has been true for a very long time that far more USAns identify as "conservative" than "liberal", and either enroll or otherwise identify as Democrats than Republicans. Keep in mind that this has been true since the parties were much less aligned along "liberal-conservative" lines than they are today. As the GOP has become more "conservative", many of their leadership y activists have looked at those polls in frustration, thinking the Democrats have some magic hold over what should be Republicans' constituency.

    Meanwhile, leftists in much of the rest of the world wonder why USAn workers and even organized labor are, by the standards of organized labor in most of the world, "conservative".

    I think what it largely comes to is that the Democrats have given what the workers' sector takes to be the best deal for them, although they hold "conservative" views on issues that don't affect their pocketbooks.

  • ||

    No, it's reasonable, because that's including independents. In other words, a larger number of independents self-identify as conservatives, which more than makes up the in-party difference.

  • robc||

    It makes perfect sense when you consider places like Kentucky, where the Democrats outside the major city (singular on purpose) are conservatives.

  • $park¥||

    WHO! FUCKING! CARES!

    Is there really nothing else going on that we have to give one single fuck about what party people identify themselves with?

  • $park¥||

    PS - Anyone who votes for a candidate because the majority of others are just so they will have voted for the winner should be subjected to the tender mercies of the Fullerton PD.

  • sloopyinca||

    Bigot!!!!!!!!

    You better watch out or you'll end up on the business end of a troll-o-meter.

  • Almanian's Evil Twin||

    That's about 0.000001874692 on the Trololololo meter.

  • Paul.||

    So you're saying if you had a pocket full of fucks, you wouldn't give one away?

  • BakedPenguin||

    Do you know how much they cost?

  • Almanian's Evil Twin||

    That's all inflation. As if anyone gave a flying fuck

  • GILMORE||

    $park¥| 9.18.12 @ 1:11PM |#

    WHO! FUCKING! CARES!

    Is there really nothing else going on that we have to give one single fuck about what party people identify themselves with?

    "Calling Dr. John, Emergency! We have a severe case of depleted partisanship!! Needs to inject 200cc's of TEAM HATE stat!"

  • Robert||

    Don't you think party self-identification to be a factor in figuring out how the world works?

  • R C Dean||

    Nicely done, Emily.

    Personally, I suspect that the polls oversample Dems because otherwise Obama is a lock and we are all doomed.

    Oh, and because the 2010 elections, as I don't think the issues or dynamics have really changed much since then. It might be interesting to look at what pollsters were doing in the runup to 2010, and how predictive their polling was then.

  • Auric Demonocles||

    I refuse to believe America is so racist as to elect Obama again.

  • Paul.||

    Personally, I suspect that the polls oversample Dems because otherwise Obama is a lock and we are all doomed.

    I still say he's a lock, except for this embassy thing... but then even I...even I underestimated how quickly the MSM and pundit class would come down on the filmmakers freedom of speech.

    This ain't the Amurrica I grew up in, so it looks like Obama may not lose any meaningful votes over his apology to the third world over our freedoms.

  • Killazontherun||

    They believe they are THE Press of which First Amendment speaks, and the Congress 'shall make no law' addresses a special privilege of their class. Only they have the professionalism that it takes to engage in speech responsibly. Otherwise, you wind up with bigots from the hoi polloi going around hurting people's feelings which leads to riots and murder.

  • ||

    Jay Cost over at Weekly Standard does some great internal analysis of polls and sampling rates. Other than his admitted Team Red affiliation he is probably one of the best poll analysts out there.

  • sloopyinca||

    First off, there is a bigger issue than whether the poll exaggerates Obama's lead, and it's this: The poll only gives two options when there are more than two people on the ballot. And while reasonably intelligent people like me (ha!) would tell the pollster to go fuck himself because there are more options than Obamatron and Romneybot, most of the drones out there will say one or the other's name.

    Sorry, but if the LP gets 3-4% and the Greens get 1%, which is entirely possible if not likely, the polls with just Team Blue and Team Red are irrelevant.

  • Auric Demonocles||

    Is Emily Ekins Discounting Alt-Text?

  • Heroic Mulatto||

    Have you seen the Electoral College "map" at the HoPuff?

    I didn't think it was possible for a candidate to get 600 electoral votes.

  • o3||

    which poll can be othered by the remainder?...oh yea, the same poll used by a network which can also be othered by the remainder!...plus kevin bacon acourse

  • The Late P Brooks||

    I certainly hope so.

  • PapayaSF||

    because typically more Americans do self-identify as Democratic than Republican

    Except that that is not the case at the moment.

  • PapayaSF||

    Some numbers from the link: in 2008, self-identification was over 41% D and 32% R. In August, it was 33.3% D and 37.6% R. And yet Obama supposedly has a lock on the election?

  • John||

    The numbers don't add up. Either the R versus D numbers are wrong or the Obama versus Romney numbers are wrong. They both can't be right. The WAPO poll last week had Romney up among independents by 14 points. But still had Obama with a small lead. That is fucking impossible. No major candidate can win independents by 14 points and still lose the election.

  • Auric Demonocles||

    Depends how many of those blacks that are unable to get IDs for some reason show up to the polls.

  • PapayaSF||

    Plus, the R over D advantage right now is greater than it was in November 2010, which was a catastrophe for Democrats.

  • R C Dean||

    Either the R versus D numbers are wrong or the Obama versus Romney numbers are wrong.

    As I understand it, the Obama v Romney numbers are generated by "correcting" the raw survey data to a pre-conceived ratio of Rs, Ds, and Is.

    The current polling on R v. D is way out of line with the pre-conceived ratio being used in these polls. The ratio has some historical justification, to be sure. But, if the current polling is even close to right, the story would be that Romney is heading for a blowout because the "corrected" poll results would show him with a 5 - 7% lead.

  • John||

    I suspect that the R v. D numbers are wrong. But then I can't believe all of the polling companies could possibly be that wrong. To believe that you either have to think they are completely stupid and are missing something that obvious or they are intentionally misleading the public.

  • Robert||

    Pollsters who identify likely voters don't want to tell you how, because that's their trade secret. If their methods turn out better predictions, they don't want their competitors to know how they did it.

  • John||

    The liberals I work with are dead certain Obama is going to win 360 electoral votes. I don't know who is going to win this election. But I can't figure out what reality you are living in if you think either guy is going to win that big.

  • ||

    Any attempt at using national numbers to predict a winner is retarded. This election will turn on maybe 5 to 10 swing states, and those swing states will be decided by turnout. It doesn't matter what people in Chicago or Manhattan or Oklahoma or Utah think, because those states are a lock.

    The reality is, no one knows what the turnout will be. If Obama gets about the same turnout in swing states that he got in 2008, he wins. At some lesser turnout, he loses. And no poll can capture that.

  • T o n y||

    It seems to be the case that polls have gotten more and more accurate over time, which makes sense. I do wonder, without sufficient data, about whether enthusiasm is being properly factored in, and the same goes for Republican voter suppression efforts.

    Nonetheless Obama gets around 300 electoral votes in nearly all the models out there. They could all be wrong, but it's clearly the case that even the Romney campaign doesn't think they are, seeing as how they're in panic mode already.

  • ||

    I suspect that the R v. D numbers are wrong. But then I can't believe all of the polling companies could possibly be that wrong. To believe that you either have to think they are completely stupid and are missing something that obvious or they are intentionally misleading the public.

    You left out the possibilities of groupthink, and of thinking "it happened that way last time, so it couldn't possibly be entirely different this time".

  • ||

    Nonetheless Obama gets around 300 electoral votes in nearly all the models out there. They could all be wrong, but it's clearly the case that even the Romney campaign doesn't think they are, seeing as how they're in panic mode already.

    Ummm, that's a 30 point margin. That's about one big state like Florida. So if the models are off in two to three swing states by overestimating Democratic turnout, that brings the race to about dead even.

    According to FiveThirtyEight.com, it will likely come down to Ohio, once again, followed by Florida, Virginia, Wisconsin, and Colorado (see tipping point states):

    http://fivethirtyeight.blogs.n.....more-34667

    If Romney wins Wisconsin, then Ryan will look like a very savvy pick.

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online

  • Video Game Nation: How gaming is making America freer – and more fun.
  • Matt Welch: How the left turned against free speech.
  • Nothing Left to Cut? Congress can’t live within their means.
  • And much more.

SUBSCRIBE

advertisement