The CDC Director Misrepresented the Study She Cited To Justify Her Misleading Estimate of Outdoor COVID-19 Risk
Rochelle Walensky's gloss is puzzling in light of the evidence presented in the systematic review on which she relied.
Rochelle Walensky, director of the Centers for Disease Control and Prevention (CDC), this week responded to criticism that she had grossly exaggerated the risk of outdoor COVID-19 transmission, citing a study she said supported her estimate. But Walensky mischaracterized the nature and scope of that study, and her gloss was highly misleading in light of the evidence the authors summarized.
When the CDC released new recommendations for vaccinated people last month, Walensky said "less than 10 percent of documented transmission[s], in many studies, have occurred outdoors." As critics such as New York Times columnist David Leonhardt and Reason science correspondent Ronald Bailey pointed out, that statement, which was widely echoed by the press, was true but deceptive, since it implied that outdoor transmission's share of infections is close to 10 percent—a figure that may be off by two orders of magnitude.
When Sen. Susan Collins (R–Maine) asked Walensky about her estimate during a hearing on Tuesday, Walensky said it came from a "meta-analysis" in "one of our top infectious disease journals." But the article to which she referred, which was published by The Journal of Infectious Diseases in February, describes a systematic review, which searches and summarizes the relevant scientific literature, rather than a meta-analysis, which pools data from several studies to generate an overall result. The distinction matters because "meta-analysis" implies that the "less than 10 percent" estimate was calculated based on the underlying data from multiple studies, when in fact it is a gloss that creates a misleading impression of the evidence.
Walensky also claimed that "over 19 studies were included" in the systematic review. The actual number was 12, only five of which looked at COVID-19 specifically. (Five "reported on influenza or influenza-like viruses," while two "reported on adenovirus transmission.") Of the five COVID-19 studies, according to the Journal of Infectious Diseases article, one found that outdoor settings accounted for 0.03 percent of infections; another put the share at less than 0.9 percent; and one found that "5% of work-related cases occurred outdoors."
Another COVID-19 study calculated the ratio of indoor to outdoor transmissions. It found that the "odds of transmission in closed environments was 18.7 times…greater than in open air." It also found that super-spreading, defined as transmission to three or more people, was 32.6 times more common "in closed environments" than outdoors.
The fifth study described a COVID-19 outbreak at an overnight camp in Georgia. The authors of the systematic review note that "the outbreak was clustered by cabin assignments, which suggests a high likelihood of transmission in indoor spaces during overnight cabin stays rather than during outdoor activities during the day."
None of these studies suggests that outdoor transmission accounts for anything like 10 percent of COVID-19 cases. Even the highest estimate—"5% of work-related cases"—is far lower than 10 percent, and that number is probably biased upward because of misclassification.
Leonhardt notes that "a very large share of supposed cases of outdoor transmission have occurred in a single setting: construction sites in Singapore." The likely explanation: Infections among construction workers were automatically treated as outdoor transmissions even though they may actually have happened indoors. "When academic researchers began collecting Covid data from around the world," Leonhardt writes, "many chose to define outdoors spaces very broadly. They deemed almost any setting that was a mix of outdoors and indoors to be outdoors."
That issue aside, the notion that outdoor settings account for close to 10 percent of cases is not consistent with any of the COVID-19 studies covered by the systematic review that Walensky cited. "Given that 90% of time is spent indoors in high- and middle-income countries," the authors note, "it would be expected that 90% of transmission occurs indoors, all else being equal." If outdoor transmission's share were in the neighborhood of 10 percent, in other words, the outdoor risk would be nearly as high as the indoor risk. Yet Walensky herself said there is "almost a 20-fold increased risk of transmission in the indoor setting [compared to] the outdoor setting."
To be fair, Walensky's "less than 10 percent" estimate is consistent with the language used in the abstract of the study on which she was relying. "Five identified studies found a low proportion of reported global SARS-CoV-2 infections occurred outdoors (<10%)," it says. On Tuesday, Walensky described that statement as "the top-line result." But anyone who reads the study can see that the "less than 10 percent" estimate is excessively conservative, and the abstract itself says the researchers considered five COVID-19 studies, not "over 19."
Two of the three studies that estimated the share of infections traceable to outdoor settings put that number at less than 1 percent. A more recent study that was not included in the systematic review found outdoor transmission accounted for 0.1 percent of infections in Ireland. "I'm sure it's possible for transmission to occur outdoors in the right circumstances," Aaron Richterman, an infectious disease expert at the University of Pennsylvania, told Leonhardt, "but if we had to put a number on it, I would say much less than 1 percent." There is obviously a huge difference between "less than 10 percent" and "much less than 1 percent."
Walensky's misrepresentation of the evidence matters because the general public depends on the CDC for authoritative health advice. It matters because Walensky is supposed to be a neutral expert who accurately summarizes the science that the Biden administration claims to be following. And it matters because the excessive conservatism of her estimate is reflected in the irrational caution of the CDC's COVID-19 guidelines.
If the CDC believes that outdoor transmission might be 10 or 100 times as common as the research suggests, it is easier to understand the agency's absurd, unworkable guidelines for summer camps, its recommendation that unvaccinated people routinely wear face masks in outdoor settings, and its advice that even vaccinated people should wear masks in some outdoor situations.* But the CDC should not believe that, because that is not what the evidence indicates.
"I always considered the CDC to be the gold standard," Collins said during Tuesday's hearing. "I don't anymore." Kavita Patel, the Obama administration's health policy director, expressed similar disappointment in a CNBC interview that night. "I think the CDC's credibility is eroding as quickly as our cases of coronavirus are eroding," Patel said. "That's not good news, because we do need workplace guidance; we need school guidance."
Walensky seems oblivious to the impact that careless statements and scientifically dubious advice have on the CDC's reputation. "My promise is that CDC will continue to follow the science as our guide," Walensky told senators on Tuesday. The problem is she thinks the CDC is already doing that.
*Update: Today the CDC issued revised guidelines that say fully vaccinated people do not need to wear face masks outdoors or indoors unless they are required to do so by businesses or the government.
