The Evidence Revolution: Why 'Take Nobody's Word for It' Really Matters
Beyond Belief explains how the "evidence revolution" is helping practitioners, policymakers, and the public understand what really works.
Beyond Belief: How Evidence Shows What Really Works, by Helen Pearson, Princeton University Press, 350 pages, $29.95
"Nullius in verba" is the official motto of the world's oldest national academy of sciences, the Royal Society of London. Usually translated as "Take nobody's word for it," the slogan represents a commitment to empirical evidence and experimental proof over reliance on authority, dogma, or tradition.
In Beyond Belief, the award-winning science journalist Helen Pearson writes an engrossing history of the modern "evidence revolution." That movement aims to draw on rigorous research to figure out what works in fields ranging from medicine to management to education to policing to conservation. As Pearson makes shockingly clear, many decisions in these fields are still based on anecdotes, the opinions of authority figures, and conventional wisdom.
Pearson illustrates the dangerous failures of conventional wisdom with a story about Benjamin Spock's vastly influential The Common Sense Book of Baby and Child Care. Apparently relying on the authority of the eminent pediatrician Paul Woolley, Jr., Spock revised his book in 1958 to say parents should place their infants face down to sleep to avoid choking on their vomit. Incidents of Sudden Infant Death Syndrome (SIDS) increased, even as evidence accumulated that face-down sleeping correlated with a much higher risk of SIDS. It was not until after a 1990 study showed that SIDS infants were nearly nine times more likely to have been sleeping face-down that a public health campaign advised parents to lay their sleeping infants on their backs. SIDS deaths dropped nearly 70 percent.
"The advocacy of front-sleeping by Spock and others is now understood to have been one of the most lethal pieces of unsubstantiated advice in the history of child health," Pearson writes. Authoritative conventional wisdom can be deadly.
"That medicine should be based on empirical evidence sounds glaringly obvious now, but few people aside from doctors realise that the term evidence-based medicine is barely 35 years old," Pearson notes. In that time, she shows, a few medical pioneers slowly began to insist on accumulating and systematizing data.
One central part of the rise of evidence-based medicine is to test treatments' efficacy by relying on randomized controlled trials (RCTs). In these trials, participants are randomly assigned to either an experimental group (receiving a new treatment) or a control group (receiving a placebo or standard care). This method compares outcomes between groups to determine an intervention's effectiveness while reducing bias.
Pearson cautions that too many RCTs have flaws, such as having too few participants to reliably detect an intervention's effect. A Lancet 2009 article argued that 85 percent of medical research is wasted, thanks to poorly designed studies, under-reporting of negative results, and inadequate information on how to implement proposed treatments. This mirrors the biostatistician John Ioannidis' 2005 investigation into why most published research findings are false.
The Cochrane Collaboration was founded in 1992 to address such shortcomings and to provide clinicians with the best available evidence for effective medical treatments. The nonprofit conducts systematic, standardized reviews of the data from research on various health questions; the goal is to provide clear, objective overviews of all the relevant evidence. Despite greater access to evidence-based medicine, Pearson points out, even now only "around 60% of healthcare in the United States, England, and Australia is in line with evidence-based clinical guidelines." Still, this is much improved over the situation a generation ago.
Often inspired by pioneers in evidence-based medicine, researchers in other fields are using RCTs and systematic reviews to try to determine the effectiveness of various economic and social policies. The majority of social RCTs, Pearson notes, "do not produce any meaningful effects." Overall, she notes, "roughly 80% of social programmes don't work, regardless of whether they aim to improve education, health, poverty, employment or something else."
Take microcredit programs, which were initially hailed as a successful intervention to lift poor people out of poverty. Follow-up research has knocked the shine off these efforts. "We found no changes in any of the development outcomes that are often believed to be affected by microfinance, including health, education, and women's empowerment," reported a team of economists in 2013.
These negative results are still highly valuable, because they can help officials avoid wasting money on useless programs. "The lesson is to beware politicians who produce, with a flourish, a brand-new programme," she explains. "If there is no evidence to show it's effective, it's probably prudent to assume that it won't work."
In education policy, Pearson points out, research shows that tutoring and fast, meaningful feedback from teachers boost students' educational performance. On the other hand, several popular panaceas, such as reducing class sizes and grouping children by attainment level, have little to no discernible effects on education outcomes.
The lack of evidence to support many business management practices is scandalous. Many decisions are still made based on HiPPO: the Highest Paid Person's Opinion. Supposedly cutting-edge management fads are contradictorily propounded by business gurus: In Search of Excellence vs. The Myth of Excellence, The Peaceable Kingdom vs. Capitalizing on Conflict, and Thinking Inside the Box vs. Out of the Box. Pearson cites studies showing that the hoary human resources practice of annual performance reviews harms morale while simultaneously being ineffective at improving employee performance.
In policing, RCTs have identified more effective strategies than the "three Rs": random patrol, rapid response, and reactive investigation. The Minneapolis police department conducted a random test doubling police patrols on half of the identified crime hotspots. The result was a cut of crime calls by 13 percent in areas with increased patrols. In another study in England, short-duration police foot patrols in violent crime hotspots caused crime to fall in those areas by 40 percent. Training police to use explanatory, courteous, and friendly procedural justice scripts when interacting with citizens in randomly chosen hotspots reduced both arrests and crimes. On the other hand, despite initial hopes, body-worn cameras do not consistently improve the behavior of either police or citizens.
As Pearson rightly observes, "evidence does not appeal to the emotions in the same way as personal stories do. We have to be trained to accept that it's more compelling and to think in analytical way." The development of projects like the Cochrane Collaboration does not guarantee that policymakers or practitioners will consult them.
But evidence-based practices do seem to be slowly taking hold. And that's a good thing. When trying to determine what's true or false, you should insist on empirical evidence. Take nobody's word for it.