Abortion Pill Studies Retracted: Politics or Science?

"Following the science" as the Supreme Court considers the safety and efficacy of medical abortions.

Our secular society has replaced the old arbiters of truth, priests and potentates, with science. One simple definition of the scientific method is that it is the self-correcting process of objectively establishing facts through testing and experimentation. Instead of appealing to the wisdom of divinely ordained scriptures or the pronouncements of princes, sophisticated moderns turn to the peer-reviewed scientific literature in search of reliable information on health, engineering, and, yes, public policy. Therefore, as we saw all too well during the late pandemic, politicians, public health practitioners, potion promoters, and pundits more or less all claim to "follow the science." (One notable recent exception is the ruling in a case involving in vitro fertilization by the chief judge of the Alabama Supreme Court.)

So is the recent retraction of three articles by the scientific journal Health Services Research and Managerial Epidemiology an example of the self-correcting processes of science or something less noble? After all, these articles were prominently cited as evidence in a federal court case that will now be heard by the U.S. Supreme Court later this month.

Sage Publications retracted two articles that suggested that the use of the abortion pill mifepristone significantly increased post-abortion emergency room use. Sage also retracted a third article from the same journal that reported that nearly half of Florida physicians the researchers identified as providing abortions had at least one malpractice claim, public complaint, disciplinary action, or criminal charge. (As background consider that a Florida law firm specializing in medical malpractice reports, "Physicians who provide care for women, particularly pregnant women, are the number one practice area for med mal lawsuits. Of OB-GYNs and related practitioners, 85 percent reported that they have been sued at some point during their career.")

In his April 7, 2023 decision, U.S. District Court for the Northern District of Texas judge Matthew Kacsmaryk "followed the science" by citing the now retracted articles when he overturned the U.S Food and Drug Administration's (FDA) approval back in 2000 of the drug as safe and efficacious. Some of the studies have now been cited and their retraction decried in amicus briefs filed with the Supreme Court.

So why were the articles retracted? And why now? The Sage retraction notice states that "we made this decision with the journal's editor because of undeclared conflicts of interest and after expert reviewers found that the studies demonstrate a lack of scientific rigor that invalidates or renders unreliable the authors' conclusions."

With respect to conflicts of interest, the Sage note observes that all but one of the authors of the studies were affiliated with various "pro-life organizations that explicitly support judicial action to restrict access to mifepristone." This is entirely true. What is puzzling is that these affiliations are acknowledged in all of the articles as well as included in the fairly extensive professional biographies at the conclusions of each article. The authors did however declare for each article that there were "no potential conflicts of interest with respect to the research, authorship, and/or publication of this article."

On the other hand, the authors did disclose in two articles the "receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Charlotte Lozier Institute." According to the mission statement of the Lozier Institute, it "advises and leads the pro-life movement with groundbreaking scientific, statistical, and medical research. We leverage this research to educate policymakers, the media, and the public on the value of life from fertilization to natural death."

In its retraction note Sage argues that the authors should have explicitly declared a conflict of interest—not just mentioned the Lozier Institute's funding—had they properly followed the relevant guidelines issued by the International Committee of Medical Journal Editors (ICMJE). The ICMJE disclosure form does ask all authors "to disclose all relationships/activities/interests listed below that are related to the content of your manuscript. 'Related' means any relation with for-profit or not-for-profit third parties whose interests may be affected by the content of the manuscript."

Given that two of these articles challenged the safety of FDA-approved abortion pills, it's pretty clear that the interests of the anti-abortion Lozier Institute would be affected by their content. It's also pretty clear that the editor of the journal could not have been deceived about the institutional affiliations, financial support, and interests of the authors.

The Sage retraction note also reports that "all three articles were originally reviewed by a researcher who was also affiliated with the Charlotte Lozier Institute at the time of the review." It is worth noting that the Sage guidelines ask reviewers to "carefully consider whether you have any potential conflicts of interest [emphasis in original] relating to the paper before undertaking the review. As an example, you should not be reviewing the paper of anyone you have worked with, taught, and/or published work with in the past." Sage points to the Committee on Publication Ethics guidelines that advise reviewers to "declare all potential competing, or conflicting, interests" specifically noting that reviewers should not agree to review if they are employed by the same institution, or have been recent mentors, mentees, close collaborators or joint grant holders.

The authors' response over at the Lozier Institute counters that Sage uses "'double-anonymized' review, meaning neither the author nor the reviewer knows each other's identities." But just how credible is it that the Lozier Institute-affiliated peer reviewer would not have recognized the provenance of the three articles?

What about the "lack of scientific rigor" that post-publication peer review of the articles found? The Sage retraction reports that two independent subject matter experts "identified fundamental problems with the study design and methodology, unjustified or incorrect factual assumptions, material errors in the authors' analysis of the data, and misleading presentations of the data." The retraction does not detail the experts' findings and their reasoning.

Citing a private letter from the Sage journal to the authors, Science reports:

Sage specifies that they "artificially inflat[ed] the number of adverse events" by counting multiple visits by the same patient; that "conflating" ER visits with adverse events without examining diagnoses or treatments "may not be a valid or rigorous approach"; and that one paper's conclusion that the miscoding of incomplete abortions as miscarriages caused serious adverse events was "inaccurate and unsupported by the data."

Some insight as to the specific concerns of the experts engaged by Sage can be discerned from the articles authors' response to an earlier expression of concern last year. It is noteworthy that the expressions of concern appeared only after the research was cited by Judge Kacsmaryk as evidence for overturning the FDA's approval of the abortifacient.

Will the retractions affect the Supreme Court case? "The whole basis of claims of danger from mifepristone to women sits on these papers. There's nothing else in the literature," says New York University bioethicist Arthur Caplan in Science. "If these papers fall, then the argument that upper courts are reviewing falls apart."

Is Caplan correct that there is nothing else in the literature supporting the epidemiological claims about emergency room visits resulting from the use of mifepristone made by the Lozier Institute affiliated researchers? Basically, yes.

A quick check of Google Scholar finds that the retracted articles are thinly cited and mostly by other researchers who are associated with pro-life organizations. One exception citing their articles is a 2023 Canadian study evaluating the adverse events from using a combination of abortion medications. But even that study reported: "Although rare, short-term adverse events are more likely after mifepristone–misoprostol IA [induced abortion] than procedural IA [induced abortion], especially for less serious adverse outcomes."

When comparing women using abortion pills versus outpatient procedural abortions performed at nine weeks of pregnancy or before, the relative risks of serious adverse events was essentially the same (3.4 versus 3.3 per 1,000). In contrast, an earlier Canadian study published in the New England Journal of Medicine in 2022 evaluating the safety of mifepristone found that the incidence of abortion-related adverse events and complications remained stable as the proportion of abortions provided by medication increased rapidly.

Overall, most studies find that both medication and procedural abortions are safe for the women choosing to end their pregnancies. For example, a 2018 study in BMC Medicine estimating the major incident rate related to abortion care found: "The major incident rate for abortion (0.1%) is lower than the published rates for pregnancy (1.4%), as well as other common procedures such as colonoscopy (0.2%), wisdom tooth removal (1.0%), and tonsillectomy (1.4%). Abortion care is, thus, safer than many other unregulated outpatient procedures."

A 2015 study in Obstetrics & Gynecology reported: "The major complication rate was 0.23%: 0.31% for medication abortion, 0.16% for first-trimester aspiration abortion, and 0.41% for second-trimester or later procedures." And a 2024 study in Nature Medicine evaluating the safety and effectiveness of telehealth medication abortions found that "in total, 0.25% of patients experienced a serious abortion-related adverse event."

In its 2022 evidence-based evaluation of 99 medical abortion studies, the non-profit Cochrane Collaboration concluded, "Medical abortion is a safe and effective way to terminate pregnancy in the first three months." The Cochrane analysis added, "Mifepristone combined with misoprostol is more effective than using these medications on their own."

Toting up all of the cases reported to the FDA since 2000 finds that around 0.07 percent of the 5.9 million American women who have used medications to terminate their pregnancies have experienced any adverse events from taking them. Not surprisingly, researchers associated with various pro-life organizations including the Lozier Institute challenged the FDA figures in a 2021 article in, where else?, Health Services Research and Managerial Epidemiology.

So how could the Lozier Institute researchers come to conclusions contrary to the results found by so many other researchers with respect to the safety of medication abortions? The broken science of epidemiology is perhaps to blame. Epidemiologists anxious to make a significant finding can unconsciously or consciously torture nearly any set of observational data into confessing to whatever correlations that just happen to confirm the researchers' hypotheses.

Stanford statistician John Ioannidis surveys the dire state of epidemiology in his seminal 2005 PLoS Medicine article, "Why Most Published Research Findings Are False." "Any claim coming from an observational study is most likely to be wrong," asserted National Institute of Statistical Sciences researchers Stanley Young and Alan Karr in their 2011 article in the journal Significance. Young has estimated that only 5 to 10 percent of observational studies can be replicated. Of course, all of the researchers seeking to analyze the side effects of mifepristone are parsing observational data with respect to their prevalence and severity. The upshot is that the epidemiological literature is so cluttered with flawed studies that anyone can find some that confirm what they already believe and so assert that they are just "following the science."

The disclosure justifications cited by Sage for retracting the articles are largely spurious since the researchers behind the three retracted studies clearly did not hide their pro-life institutional affiliations. More problematically, the conflicted peer reviewer should have declined to evaluate the studies. And whatever their methodological failings, the three outlier articles from an obscure journal would most likely never have attracted extra scrutiny except for being cited as "follow the science" evidence to challenge the FDA's approval of a widely used abortifacient.

At the center of the case before the Supreme Court later this month is the proposition that any odd federal judge who claims to be "following the science" can overrule the decisions of the FDA that also claims to be "following the science." Will the Supreme Court now "follow the science" and ignore the retracted articles when it rules on that issue later this year?