Jonathan Chait is Wrong About Reason's Polling

New York Magazine's Jonathan Chait, in the course of disagreeing with the notion that we're living through a libertarian moment, has mischaracterized the Reason-Rupe public opinion research survey as an advocacy poll seeking to bend the truth:

Reason has invested a great deal of money and time in promoting its claim to represent America's youth, including regular polls that purport to bolster this claim. It's important to understand that, since small changes in the framing of a poll can produce dramatically different responses, it is possible to produce polling that seems to show agreement with any position the sponsor of the poll desires. Advocacy organizations routinely sponsor polls that show the public supportive of their own position. Reason is the only magazine I know of that uses this tactic.

Reason-Rupe has conducted 13 national or state polls to date, one of which was focused on millennials. The announcement of our millennials poll findings includes the following words:

Millennials Think Government Is Inefficient, Abuses Its Power, and Supports Cronyism

But young Americans also want government to guarantee health insurance and living wages; plan to vote for Democrats in 2014 and 2016….

[M]illennials also support more government action and higher spending in a number of key areas:

71 percent favor raising the federal minimum wage to $10.10 an hour

69 percent say it is government's responsibility to guarantee everyone access to health care and 51 percent have a favorable view of the Affordable Care Act

68 percent say government should ensure everyone makes a living wage

66 percent say raising taxes on the wealthy would help the economy

63 percent say spending more on job training would help the economy

58 percent say the government should spend more on assistance to the poor even it means higher taxes

57 percent favor spending more money on infrastructure

54 percent favor a larger government that provides more services, when taxes are not mentioned

54 percent want government to guarantee everyone a college education

Now, does this read like the outcome of a libertarian advocacy poll?

In Chait's view, Pew polls are reliable and Reason-Rupe polls are not. Most of the Reason-Rupe polls are conducted by Princeton Survey Research Associates International—the very same company that conducts the Pew telephone polls. Our recent poll of millennials was conducted by YouGov, using its web-enabled survey software, the same group that The New York Times, CBS, The Economist, and The Huffington Post trust to conduct their surveys.

Any serious pollster knows that fair and objective wording is crucial to the integrity of a poll, and it is a paramount consideration when we develop our surveys.

Chait offers zero specifics in his article to back up his claim. But in a Twitter exchange last night, he provides this example of what he calls "crafted" language that is not neutral:

Here are the questions to which he is referring:

Chait accepts our first question as neutral, but dismisses the second question as "crafted"—one of what he calls "obvious prompts for downsides of bigger government."

A key objective of the Reason-Rupe Poll is to ask tradeoff questions, such as this, which are often not asked by other major polls. This helps shed light on how opinions are affected when respondents are presented with realistic costs and tradeoffs associated with public policies, rather than simple questions that ignore such consequences. That's what we did here with half of our sample, and we reported both results. We believe this adds some depth and nuance to public opinion research.

Perhaps some are uncomfortable with poll questions that examine preferences when considering real world tradeoffs associated with an active government.  But readers can decide for themselves whether or not they think research questions such as these contribute to our understanding of public opinion.

As always, our survey questions, methodology, topline results, and complete crosstabs are all available online for review and discussion.