OMG, No: Negative Opinion Polls Are Not Fake News
Even if pollsters have whiffed on a bunch of recent elections
According to the leader of the free world, any poll that finds less than a majority of Americans supporting him or his preferred policies is "fake news."
Any negative polls are fake news, just like the CNN, ABC, NBC polls in the election. Sorry, people want border security and extreme vetting.
— Donald J. Trump (@realDonaldTrump) February 6, 2017
He's right about one thing: Pollsters whiffed badly when it came to predicting the outcome of last November's election. In fact, the polls have been wrong a lot lately, and last year in Reason I reported on some of the reasons for the industry's sudden crisis. If the president had read my piece, however, he'd know that the main problem with "horserace polling" is how hard it is to predict ahead of time what an electorate will look like.
Because it's no longer possible to get a truly random (and therefore representative) sample, pollsters compensate by making complicated statistical adjustments to their data. If you know your poll reached too few young people, you might "weight up" the millennial results you did get. But this involves making some crucial assumptions about how many voters on Election Day will hail from different demographic groups.
We can't know ahead of time what the true electorate will look like, so pollsters have to guess. In 2012, some right-of-center polling outfits posited that young voters and African Americans would be less motivated to turn out than they had been four years earlier. That assumption turned out to be wrong, and their predictions about who would win were wrong along with it.
Pre-election polling has become very difficult to get right precisely because we can't know for sure the demographic makeup of the people who will actually show up to the polls until after the fact. If you weight your data on the assumption that a large number of African Americans will stay home, and that assumption turns out to be wrong, everything else will be, too. Meanwhile, if you assume that Democrats will turn out in the same numbers to support Hillary Clinton as they did to support Barack Obama, and that turns out to be wrong…well, you can end up expecting a Clinton victory and getting Donald Trump instead.
But public opinion polling of the kind Trump is talking about—gauging Americans' views on a policy issue or even on the president—doesn't suffer from this weakness. We know the demographic characteristics of the population as a whole, so pollsters can be confident when they adjust their data that they're doing it right. No predictions about some future state need to be made in order to measure what people believe right now.
So there's little justification for Trump's claim that the polls are methodologically flawed when, for example, they find that a plurality of Americans strongly disapprove of his job handling, a majority of Americans oppose his immigration order, and fully 6 in 10 Americans do not want a border wall.
Tweaks to the way a question is asked can alter the results you get, and it definitely still makes sense to focus on the overall picture painted by the totality of the research that's out there, as opposed to singling in on any one data point. For instance, it would be safer to say that the public is roughly split on the immigration order than to say the people have rejected it outright. On the other hand, it's indisputable at this point that Trump's approval rating is underwater.