Prominent Researchers Say a Widely Cited Study on Wearing Masks Is Badly Flawed

"Masks matter. So does good science. Let's do both."

Outside researchers are calling for the retraction of a study published earlier this month in the Proceedings of the National Academy of Sciences (PNAS) that claimed to have discovered a strong correlation between public facemask-wearing and a subsequent decline in confirmed COVID-19 cases. In the challenged study, a team of atmospheric chemists led by Texas A&M chemist Renyi Zhang sought to compare how trends in confirmed diagnoses changed before and after mask-wearing had been mandated in Wuhan, China, Italy, and New York City.

The researchers calculated that mandated masking reduced the number of confirmed cases by more than 78,000 in Italy from April 6 to May 9, and by more than 66,000 in New York City from April 17 to May 9. In addition, the researchers argued that while recommendations like social distancing and frequent hand washing slowed the epidemic, the dramatic reductions in viral transmission in Italy and New York City occurred only after wearing masks in public was mandated. The reduction in confirmed cases occurred, they argued, because masking prevents the transmission of the disease by blocking the atomization of virus-containing respiratory droplets (coughing, sneezing, talking) and their subsequent inhalation by uninfected people. On that evidence, they concluded the airborne spread of the coronavirus that causes COVID-19 is the dominant route of infection.

Almost immediately, the study received pushback from outside statisticians and epidemiologists who argued that the study is severely flawed by sloppy statistical analyses. A group of critics has now sent a letter to the editors of the PNAS asking them to immediately retract the study. In addition, an evaluation of the study by researchers at the Johns Hopkins University's Bloomberg School of Public Health also urges the PNAS to strongly consider retraction.

According to the critics, the study's two most egregious errors were claims that other non-pharmaceutical interventions—e.g., social distancing, quarantine, and handwashing recommendations—had essentially no effect on pandemic trends until facemask-wearing was mandated, and the subsequent conclusion that because masks were so allegedly effective, "airborne transmission represents the only viable route for spreading the disease." The letter urging retraction observes, "While masks are almost certainly an effective public health measure for preventing and slowing the spread of SARS-CoV-2, the claims presented in this study are dangerously misleading and lack any basis in evidence."

In interviews with The New York Times, study co-author Mario Molina, who was a co-winner of the 1995 Nobel Prize in Chemistry for his work on identifying the man-made chemicals that were eroding the Earth's protective ozone layer, did not back away from the paper's conclusions. "We show in the paper itself that we know things are complicated, we know that there's social distancing, we know that it's sometimes perfect, sometimes not," he said. "Maybe we have some exaggerated sentences here—we're sorry. We should have been a bit more careful with the language."

In addition, the signers of the retraction letter are highly critical of the fact that the article was submitted to the PNAS via the "contributed track," which enables National Academy of Sciences members like Molina to submit both a manuscript and suggested peer reviewers. About 20 percent of PNAS papers are published under this procedure. The journal's editors are awaiting additional information before making a decision about the paper.

As PNAS retraction letter co-signer Harvard emergency medicine physician Jeremy Faust notes on Twitter: "Masks matter. So does good science. Let's do both."

What must be borne in mind is that this fight about the PNAS study's methodology is not over whether or not wearing facemasks is a useful tool in blunting the COVID-19 pandemic. Accumulating evidence shows that wearing facemasks in public does significantly contribute to reducing the spread of COVID-19. For example, a preprint article, "Face Masks Against COVID-19: An Evidence Review" that evaluates the effect on facemask wearing on disease spread notes,

Reducing disease spread requires two things: first, limit contacts of infected individuals via physical distancing and other measures, and second, reduce the transmission probability per contact. The preponderance of evidence indicates that mask wearing reduces the transmissibility per contact by reducing transmission of infected droplets in both laboratory and clinical contexts. Public mask wearing is most effective at reducing spread of the virus when compliance is high. The decreased transmissibility could substantially reduce the death toll and economic impact while the cost of the intervention is low.

A June 16 article in the journal Health Affairs attempts to analyze how facemask mandates in 15 states and the District of Columbia affected before and after COVID-19 confirmed case trends. While their statistical study is more carefully caveated than those in the PNAS article, two University of Iowa public health econometricians after examining the effects of mandates instituted between April 8 and May 5 do report:

Using an event study that examines daily changes in county-level COVID-19 growth rates, the study finds that mandating public use of face masks is associated with a reduction in the COVID-19 daily growth rate. Specifically, we find that the average daily county-level growth rate decreases by 0.9, 1.1, 1.4, 1.7, and 2.0 percentage-points in 1–5, 6–10, 11–15, 16–20, and 21+ days after signing [of such mandates], respectively.

These estimates are not small and represent nearly 16–19% of the effects of other social distancing measures (school closures, bans on large gatherings, shelter-in-place orders, and closures of restaurants, bars, and entertainment venues) after similar periods from their enactment. The estimates suggest increasing effectiveness and benefits from these mandates over time. By May 22, the estimates suggest that as many as 230,000–450,000 COVID-19 cases may have been averted based on when states passed these mandates. Again, the estimates of averted cases should be viewed cautiously as these are sensitive to assumptions and different approaches for transforming the changes in the daily growth rate estimates to cases.

The upshot is that however flawed the PNAS study is, the balance of the scientific evidence strongly suggests that widespread use of facemasks will significantly help reduce the spread of COVID-19.