How New York Times Got Swindled by Sociology Fraud
In September, social psychology professor Diederik A. Stapel was fired by Tilburg University after an investigation revealed he had falsified, lied and invented data in more than 30 experiments. Prior to Stapel's downfall, his work had attracted highly favorable international attention including multiple citations in the Times of New York (which happily repeated Stapel's bogus scientific evidence proving that advertising works by making "women feel worse about themselves" and that conservative politics causes hypocrisy) and the Times of Los Angeles (which parroted an absurd study on racism and tidiness, about which more in a moment).
True to form, Stapel made it clear that competitive pressure and lack of regulation were to blame for his fraud. "I did not withstand the pressure to score, to publish, the pressure to get better in time," Stapel told a Dutch paper. "I wanted too much, too fast. In a system where there are few checks and balances, where people work alone, I took the wrong turn. I want to emphasize that the mistakes that I made were not born out of selfish ends."
At the Weekly Standard, Andrew Ferguson looks at the "Chump Effect" that prompts reporters to write up dubious studies uncritically:
The silliness of social psychology doesn't lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior.
And when the research reaches beyond the classroom, it becomes sillier still.
Consider this recent study by Stapel, demonstrating the relationship between "disorder" and white racism and homophobia…
The experiment began after janitors at the Utrecht railroad station went on strike. Stapel and colleagues leapt into action. As the garbage in the station piled up, they cornered 40 white passengers. One by one the travelers were asked to take a seat in a row of folding chairs. They were given a questionnaire. If they filled it out, they were told, they would get a piece of chocolate or an apple as a reward.
The questionnaire asked to what degree the travelers agreed with stereotypes about certain types of people. (Are gays "creative and sweet" or "strange and feminine" or "impatient and intelligent"?) And then came the twist! Stapel had planted a person at the end of the row of chairs—sometimes a black person, sometimes a white. Researchers measured how far away from the person each respondent chose to sit. Meanwhile, thanks to the questionnaire, they could measure the degree of racism or homophobia each was feeling. On average, the travelers sat 25 percent closer to the white man than to the black man.
In time the janitors came back to work. The station was cleaned spick-and-span. Stapel and his gang returned and performed the experiment again, on another 40 white travelers. There in the tidy environment, their questionnaires showed they were less racist and homophobic than their counterparts from the earlier experiment. And on average, they sat the same distance from the white person as the black person. Hence, as the headline read: "Messy surroundings make people stereotype others."
But Stapel, as an internationally respected social psychologist, wasn't satisfied. So he designed another experiment to confirm his finding. The Stapel gang went to a wealthy neighborhood. They threw a bicycle on the ground, tore up paving stones, and, as the L.A. Times noted, parked Stapel's Subaru on the sidewalk. Chaos! Disorder! Forty-seven passersby were collared, given a new questionnaire to test their racism, and asked to donate money to (I'm not making this up) a charity called "Money for Minorities."
Then the bike was removed. The stones were replaced. Stapel moved his Subaru. Now it was just a nice, rich, tidy neighborhood. More passersby were collared, more questionnaires were filled out, and—here's the scientific finding—less racism and homophobia were revealed. And the passersby in the tidy neighborhood gave more money to minorities on average: to be precise, 0.65 euro more.
Social psychologists around the world gazed on these findings when they were published this spring. They gave their chins a good, firm tug. "This need for order matters a lot more than we might have thought," a Duke psychologist told the Times.
Did Stapel fake his research? Did he and his students really make all those people fill out forms for an apple? Did Stapel really cross-tabulate the data? Did he really park his car on the sidewalk?
Who cares? The experiments are preposterous. You'd have to be a highly trained social psychologist, or a journalist, to think otherwise. Just for starters, the experiments can never be repeated or their results tested under controlled conditions. The influence of a hundred different variables is impossible to record. The first group of passengers may have little in common with the second group. The groups were too small to yield statistically significant results. The questionnaire is hopelessly imprecise, and so are the measures of racism and homophobia. The notions of "disorder" and "stereotype" are arbitrary—and so on and so on.
Described in this way, it does seem like there could be real journalistic interest in this study – as a human interest story like the three-legged rooster or the world's largest rubber band collection. It just doesn't have any value as a study of abstract truths about human behavior. The telling thing is that the dullest part of Stapel's work – its ideologically motivated and false claims about sociology – got all the attention, while the spectacle of a lunatic digging up paving stones and giving apples to unlucky commuters at a trash-strewn train station was considered normal.