Corpus Linguistics in Court?

Two Sixth Circuit judges debate the issue, in an opinion filed today.

See the concurring opinions of Judges Jane B. Stranch and Amul Thapar, in Wilson v. Safelite Group, Inc. (For an earlier example of such a debate in a judicial decision, see here; for guest posts by Justice Thomas Lee and Stephen Mouritsen explaining and defending the use of corpus linguistics in law, see here.)

From Judge Thapar:

I agree with the majority's textual analysis and concur to the extent the majority rests on that analysis. Because the text of 29 U.S.C. § 1002(2)(A)(ii) is clear, we should go no further. And the text is clear, as many tried-and-true tools of interpretation confirm. But so does one more: corpus linguistics. Courts should consider adding this tool to their belts….

[Corpus linguistics] draws on the common knowledge of the lay person by showing us the ordinary uses of words in our common language. How does it work? Corpus linguistics allows lawyers to use a searchable database to find specific examples of how a word was used at any given time. State v. Rasabout, 356 P.3d 1258, 1275–76, 1289 (Utah 2015) (Lee, A.C.J., concurring in part and concurring in the judgment). These databases, available mostly online, contain millions of examples of everyday word usage (taken from spoken words, works of fiction, magazines, newspapers, and academic works). See, e.g., Corpus of Contemporary American English, BYU, https://corpus.byu.edu/coca/help/texts.asp (listing types of sources); Corpus of Historical American English, BYU, http://www.english-corpora.org/coha/. Lawyers can search these databases for the ordinary meaning of statutory language like "results in." The corresponding search results will yield a broader and more empirically-based understanding of the ordinary meaning of a word or phrase by giving us different situations in which the word or phrase was used across a wide variety of common usages.  See Lee & Mouritsen, supra, at  831 ("Linguistic corpora can perform a variety of tasks that cannot be performed by human linguistic intuition alone."). In short, corpus linguistics is a powerful tool for discerning how the public would have understood a statute's text at the time it was enacted.

Of course, corpus linguistics is one tool—new to lawyers and continuing to develop—but not the whole toolbox. Its foremost value may come in those difficult cases where statutes split and dictionaries diverge. In those cases, corpus linguistics can serve as a cross-check on established methods of interpretation (and vice versa). See Lawrence B. Solum, Triangulating Public Meaning: Corpus Linguistics, Immersion, and the Constitutional Record, 2017 BYU L. Rev. 1621, 1669–70 (2017) ("[C]orpus linguistics allows for rigorous intersubjective validation of individual subjective judgments about word meaning."); see also Clark D. Cunningham et al., Plain Meaning and Hard Cases, 103 Yale L.J. 1561, 1566 (1994) (book review) (arguing that empirical methods may assist judges in hard cases of ordinary meaning). This cross-check can provide both judges and parties with greater certainty about the meaning of words in a statute….

Judge Stranch's response:

I write separately to express some concerns about the concurrence's endorsement of "corpus linguistics," a proposed method of statutory interpretation described in a handful of recent state court opinions [mostly from Utah]. This tool invites judges to perform the same kind of subjective decision making that the concurrence otherwise cautions us to avoid. There are  several reasons why we should decline this invitation.

The first is a practical problem. A keyword search using a corpus linguistics database  will likely result in dozens, if not hundreds or thousands, of examples of a term's usage. How should courts make sense of all this information? First, we could count the number of times a term is used in the database (assuming appropriately selected parameters) and then decide that the most frequently used meaning is the ordinary meaning. But that approach would risk privileging the most newsworthy connotations of a term over its ordinary meaning. See, e.g., Carissa Byrne Hessick, Corpus Linguistics and the Criminal Law, 2017 B.Y.U. L. Rev. 1503, 1509 ("[A] corpus analysis may demonstrate that seventy percent of all mentions of the word 'flood' occur in the context of [] superstorms. But that does not tell us whether the average American would understand the statutory term 'flood' to include three inches of water in a homeowner's basement after a neighboring water main burst."). It would also fail to cull irrelevant results. If we use the database to determine the meaning of "results in" during the 1960s and 1970s (the era closest to when ERISA was drafted), we find examples of this term's usage in contexts that bear no relationship to our own. Does it matter, for purposes of deciding whether ERISA applies to Wilson's deferred compensation plan, how "results in" was used in a book about farm animal management in 1976, or in an article from Sports Illustrated about New York's cool spring weather in 1964? I think it does not. And even if consulting this overinclusive data set might help judges "to avoid basing conclusions on a few speakers' idiosyncrasies," it is "the 'idiosyncrasies' of [Congress that] constitute the rule of law in this [country]. And the only way to identify those idiosyncrasies is through the text of the [U.S.] Code, which is wholly absent from [the corpus linguistics] data set." State v. Rasabout, 356 P.3d 1258, 1266 (Utah 2015) (call numbers and internal quotation marks omitted)). This suggests to me a disconnect between corpus linguistics and the judicial work of statutory interpretation.

Another approach would require the court to perform this culling process itself. For example, we could assume that the drafters employed popular, as opposed to technical or legal, language and decide which uses of "results in" during the 1960s and 1970s should be included in our analysis and which should not. But by what metric would we make that choice? Perhaps most could agree that a book about farm animal management is not relevant here. But what  about an article reporting a union strike? Or one about federal tax penalties?3 Such choices  would require highly subjective, case-by-case determinations about the import and relevance of a given source. Textualists have long advised us to forgo that interpretive method. See, e.g., Conroy v. Aniskoff, 507 U.S. 511, 519 (1993) (Scalia, J., concurring in the judgment) ("The legislative history of [this] Act contains a variety of diverse personages, a selected few of whom—its 'friends'—the Court has introduced to us in support of its result. But there are many other faces in the crowd, most of which, I think, are set against today's result."). Legislative history tells us, at a minimum, how some of the statute's authors understood a term; corpus linguistics does not offer even that insight.

In part because of these practical problems, the use of corpus linguistics is a difficult and complex exercise. That is why, for centuries, we have left this task to the trained lexicographers who author the tool we already employ—a dictionary. [Judge Thapar's] concurrence describes much of what lexicographers do every day.

Lexicographers engage in "research, but also decisionmaking: the primary job of the lexicographer in creating a dictionary is to determine meanings of words, and to determine what different meanings a word might have." Rubin, supra at 181. And because "[t]he line between one meaning and another is seldom clear," this process "leaves much of the final determination to the experienced judgment of the editorial staff." Id. The other concurrence argues that, notwithstanding their training, these experts sometimes select outdated or otherwise unreliable meanings for disputed terms. But I would not substitute the ad hoc selection process of individual judges for the "experienced judgment" of these trained scholars. Doing so would convert judges into armchair lexicographers, attempting the same work that dictionary authors have been performing for centuries. But unlike those experts, judges would shoulder this task without the specialized training necessary to make a reliable and neutral judgment call. Encouraging litigants to take on that same role would make the problem worse, not better.

I do not suggest that corpus linguistics can never assist judges in the difficult project of statutory interpretation. But, in the unusual case where such a tool might prove useful, I would leave this task to qualified experts, not to untrained judges and lawyers. See, e.g., Brief for Professor Clark D. Cunningham, et al. as Amicus Curiae on Behalf of Neither Party, In Re: Donald J. Trump, President of the United States of America, No. 18-2486 (4th Cir. Jan. 29, 2019) (discussing use of corpus linguistics by professor of applied linguistics to help determine the meaning of "emoluments" during the founding era). And before we add corpus linguistics to our judicial toolkit, we should first remind ourselves what our toolkit is for. I agree with the concurrence that statutes ought to give "ordinary people fair warning about what the law demands of them." United States v. Davis, 139 S. Ct. 2319, 2323 (2019). But that is the responsibility of legislators, not judges. Once the torch passes from Congress to the courts, our duty as judges is simply to determine what our elected members of Congress meant when they passed the statute—even  if that is  not  the meaning we or the public might  routinely employ.  In most cases, adding corpus linguistics to our judicial toolkit would make it harder to focus on that narrow duty. This case underscores why.  Our task here is to decide what Congress meant  by "results in" and "extending to" when it defined the universe of employee pension benefit plans covered by ERISA. The other concurrence proposes that we divine that meaning not by considering ERISA's statutory framework or legislative history, but by looking to the language of an article from Sports Illustrated and a book about farm animal management. I struggle to see why those sources would tell us as much as, for example, what the legislature told us about the structure and purpose of ERISA when it drafted the statute.

Underlying these practical usage issues is my concern with the implicit suggestion that corpus linguistics is a simple, objective tool capable of providing answers to the puzzle of statutory interpretation. The use of corpus linguistics brings us no closer to an objective method of statutory interpretation. Instead, it encourages judges to stray even further from our historic and common-sense considerations—including the "text, structure, history, and purpose" of a statute, Maracich v. Spears, 570 U.S. 48, 76 (2013) (citation omitted)—that ought to guide our analysis.

Judge Thapar's reply:

The other concurring opinion argues that we should not add corpus linguistics to the judicial toolkit for several reasons. The first is methodological—corpora are not representative because of their sources. For instance, a corpus search for "flood" may lead to an overinclusion of newspaper articles talking about giant flood waters rather than basements flooding. But the entire practice of law—and certainly the practice of interpretation—involves judgment calls about whether a particular source is relevant. And, at least with corpus linguistics, those calls  can be vetted by the public in a more transparent way. Cf. Muscarello v. United States, 524 U.S. 125, 142–43 (1998) (Ginsburg, J., dissenting) (criticizing the majority opinion for selective and non-transparent examples of word use). That is more than can be said of the alternative, which, as Justice Lee has thoughtfully noted, is for a judge to use his or her intuition—something far less representative and frankly far less "democratic." See Rasabout, 356 P.3d at 1274–75 (Lee, A.C.J., concurring in part and concurring in the judgment). Plus, the danger of judges relying upon their own intuition is that we introduce other risks, like confirmation bias. Id. at 1274. Judges may unintentionally give greater weight to those definitions that match up with their preconceived notions of a word's meaning. We cannot get away from confirmation bias altogether, but we can surely check our intuition against additional sources of a word's meaning. The corpus allows us to do this.

Second, the other concurring opinion argues that the use of corpus linguistics will descend into mere rote frequency analysis; judges will simply pick the use of the word that shows up the most. Yet judges who use corpora do not become automatons of algorithms. They will still need to exercise judgment consistent with the use of the other tools of statutory interpretation. Sometimes the most frequent use of a word will line up with its ordinary meaning as used in a statute. Sometimes it will not. The data from the corpus will provide a helpful set of information in making that interpretive decision. But the judge must make the ultimate decision after considering multiple tools.

Third, the other concurring opinion suggests that corpus linguistics is redundant when compared with another tool—dictionaries. Expert lexicographers already do corpus linguistics when compiling dictionaries, so, the argument goes, when judges use corpus linguistics, they become unnecessary and unhelpful armchair lexicographers. But the use of corpus linguistics improves upon dictionaries by helping pinpoint the ordinary uses of a word at the time a statute was enacted. For example, when a court considers a dictionary definition, it looks at a dictionary from that time period. See New Prime, 139 S. Ct. at 539–40 & n.1. But the usage examples in those dictionaries  often  come  from  a  time  before  the  dictionary  was  published.  See  Lee  & Mouritsen, supra, at 808–09; Henry M. Hart, Jr. & Albert M. Sacks, The Legal Process: Basic Problems in the Making and Application of Law 1190 (William N. Eskridge, Jr. & Phillip P. Frickey eds., 1994) ("An unabridged dictionary is simply a[] historical record, not necessarily all-inclusive, of the meanings which words in fact have borne, in the judgment of the editors, in the writings of reputable authors."); see also id. at 1375–76. So the dictionary definition may actually tell us the ordinary meaning at a time long before Congress enacted the statute. See Lee & Mouritsen, supra, at 809; Scalia & Garner, supra, Appendix A at 419 (noting that dictionaries lag behind the times). And in many cases (like the ones discussed below), both the majority and dissent will point to dictionaries without any clear resolution. Instead of relying on just a few sample sentences in the dictionary, the corpus develops a broader picture of how words were actually used when Congress passed the statute.

Plus, "[w]e judges are experts on one thing—interpreting the law." Rasabout, 356 P.3d  at 1285 (Lee, A.C.J., concurring in part and concurring in the judgment) (emphasis omitted). Corpus linguistics is just one variation on a very old theme in this field of expertise. Judges  often consider the context of words—both within and beyond the text of the statute in dispute. See Scalia & Garner, supra, § 31 (detailing the "noscitur a sociis" canon of interpretation); see also Caleb Nelson, What is Textualism?, 91 Va. L. Rev. 347, 355 (2005). Judges look to contemporaneous judicial decisions. See, e.g., New Prime, 139 S. Ct. at 540. They look to seemingly common phrases. See, e.g., FCC v. AT&T, Inc., 562 U.S. 397, 403–04 (2011) (considering how the word "personal" is used in "personal life" and "personal expenses"). And, for constitutional cases, they look to word use in the Anti-Federalist and Federalist Papers. See, e.g., United States v. Lopez, 514 U.S. 549, 586 (1995) (Thomas, J., concurring). While sometimes this "enterprise may implicate disciplines or fields of study on which we lack expertise, [it] is no reason to raise the white flag" and forgo considering corpus linguistics. Rasabout, 356 P.3d at 1285 (Lee, A.C.J., concurring in part and concurring in the judgment).

Instead, judges should do what they have always done—"summon all our faculties as best we can" and take advantage of adversarial briefing. See id.

In sum, I agree that corpus linguistics is not the only tool we should use, but it is an important tool that can assist us in figuring out the meaning of a term.

Thanks to Howard Bashman (How Appealing) for the pointer.