Measuring Up

Testing the pretensions of market research and polling.

Brian Doherty from the May 2003 issue

(Page 2 of 3)

Weissberg is a professor of political science at the University of Illinois-Urbana. He believes the basic techniques and goals of market research -- trying to discover what people want so that you can give it to them -- are pernicious when applied to politics. In proving this, he makes some more-daring claims about the unimportance of political democracy and the dangers of reducing civic virtue to political participation.

Does polling even work? Despite some delightful anecdotes of spectacular failure and amusing tales of prankster pollsters getting people to say they'd fight Canada over America's silverfish supply, for the most part the modern version does passing well. More often than not, the candidate whom pre-election polls indicate will win does in fact win. The Gallup Organization, for example, has correctly called every presidential election within three percentage points since 1952, after 1948's embarrassing, poll-driven "Dewey Beats Truman" gaffe. Polls in that year often used "quota sampling" techniques, trying to cover a certain number of people in specific gender, race, or income level categories. That method has since been replaced by the more accurate random sample.

But the 2002 elections saw some fresh poll missteps. Most major polling organizations, including Gallup, called at least one election result wrong, and Zogby International got five results wrong out of 17 polls performed. This led The Wall Street Journal to run a front-page feature casting doubts on polling's future in a world of cell phones and ever-easier ways for us to avoid talking to pollsters.

But Weissberg's critique is not aimed at the scientific art of polling per se. He has a narrower thesis to defend: that polls designed to ask voters if they want more government spending on any given item don't generate politically useful information. But he is also clearly fighting against what he sees as a growing belief within the political science profession: that poll-driven democracy, which permits citizens' voices to be heard (and heeded) at all times, is preferable to our traditional republican system. Weissberg defends periodic elections as all the democracy we require, thank you very much. He thinks most people don't have the slightest idea what they are talking about when it comes to public policy. That's why, he argues, we should be thankful for having elected representatives to make decisions for us.

He's tough about it too. Not for him the pat-on-the-head doctrine of "rational ignorance." That's the widely accepted notion from the public choice school of economics that says it makes perfect sense, given the high cost-benefit ratio of public policy savvy, for citizens to have only dim notions of what the hell is going on in government.

Economists have gone even further in explaining/excusing public sloth in regard to political beliefs and actions. George Mason University economist Bryan Caplan recently has posited the appropriateness of rational irrationality, whereby we choose an optimal amount of absurd and counterfactual things to believe based on what it costs us to hold these unrealistic beliefs.

Caplan's concept would have helped clarify Weissberg's findings, which show that people seem to credulously accept the endless possibilities of government goodies, believing they will all deliver exactly the benefits they promise. Weissberg argues that most polls are systematically biased toward manufacturing a vox populi that clamors for an ever-growing welfare state.

To test this thesis, he designed and executed a pair of surveys that he thinks provide a more sophisticated and accurate way of gauging an intelligent, informed decision -- not just an ignorant wish. He used these polls to retest public support for a couple of Clinton-era government expansions: shrinking public school class size by hiring tens of thousands of new teachers, and increasing government-supported day care.

Weissberg found exactly what he was looking for (and one wonders how often that happens in social science research -- there's a poll whose results I'd like to see). If you give longer, more detailed polls that demand citizens balance costs within a necessarily limited total budget, and inform them of both the possibilities of failure and the real dimensions of the problem allegedly being solved, previous apparent support for government action and spending quickly fades. For example, if respondents were told that the new teacher program could lead to cutbacks in other school programs, 71 percent of the support evaporated; when informed that an expenditure of $1.2 billion would lower average class size only from 17.8 to 17, 43 percent of supporters changed their minds.

Underlying Weissberg's argument is the dire hint that, to the extent that politicians speak and act in reaction to polls, we are in effect living in a plebiscitory democracy. His larger point is that the people are far too stupid to be heeded by politicians.

Weissberg is openly contemptuous of democracy, falling back on Bell Curve-type arguments about the invincible thickheadedness of many people. He forces the average would-be active citizen to choke down that sour persimmon, then feeds him the ripe plum that "surrendering one's voice is perhaps the surest path to advancement among those outgunned when politics is reduced to hyperindividualism....Powerful interest groups -- energetic pluralistic democracy, if you will -- offer one sensible alternative....The pretence of grassroots vitality and input would now be sacrificed to tangible accomplishment. Gone are the endless survey consultations....Ordinary citizens pay dues and obey orders." This is the way he thinks it should be.

However much anger that attitude might inspire, one can still agree with Weissberg's observation that most Americans aren't well-educated on political issues or well-trained in sophisticated cost-benefit analyses without embracing the conclusion that they are hopeless dunderheads begging for leaders. When it comes to their private lives, as Weissberg admits and as Bogart's book indicates, even the supposedly civically ignorant are perfectly able to make decisions for themselves. This is because of the vital and too often overlooked (especially by those who conflate marketing with oppression) difference between politics and markets.

In marketing research of Bogart's variety, people are discussing personal wishes and desires for themselves over which they have ultimate power, even if the people gathering the knowledge hope to use it to try to sell them something. A company may take the results of market research polls and craft its ad campaign to appeal to what it has deduced you want. But you decide whether you are going to buy or not, with your money and based on an intimate understanding of your own desires and circumstances that only you can possess.

In Weissberg's world of political polling, people express judgments about how much of other people's property should be used by still other people to pursue goals through methods the effectiveness of which the people being polled have no special ability to judge. While the adbusters of the world try to muddy the distinction, that's a clear and important ethical difference.

Seeking and quantifying the public's opinion is big business. Bogart reports that up to $10 billion annually is spent on commercial survey research. Government studies cost another $3 billion, and academic and nonprofit studies -- many also funded by government, of course -- add yet another $3 billion. The field of "social research" arose out of an intellectual ferment in the postwar years that is somewhat scary in retrospect: a governmental/academic/corporate movement encompassing Bogart's intellectual roots in the work of Paul Lazarsfeld and his Bureau of Applied Social Research at Columbia, along with CIA-sponsored psychology research and the early days of cybernetics.