The Volokh Conspiracy
Mostly law professors | Sometimes contrarian | Often libertarian | Always independent
Judge Strikes Part of Anthropic (Claude.AI) Expert's Declaration, Because of Uncaught AI Hallucination in Part of Citation
From Friday's order by Magistrate Judge Susan van Keulen in Concord Music Group, Inc. v. Anthropic PBC (N.D. Cal.)
At the outset, the Court notes that during the hearing, Publishers asked this Court to examine Anthropic's expert, Ms. Chen and strike her declaration because at least one of the citations therein appeared to have been an "AI hallucination": a citation to an article that did not exist and whose purported authors had never worked together. The Court gave Anthropic time to investigate the circumstances surrounding the challenged citation. Having considered the declaration of Anthropic's counsel and Publishers' response, the Court finds this issue is a serious one—if not quite so grave as it at first appeared.
Anthropic's counsel protests that this was "an honest citation mistake" but admits that Claude.ai was used to "properly format" at least three citations and, in doing so, generated a fictitious article name with inaccurate authors (who have never worked together) for the citation at issue. That is a plain and simple AI hallucination. Yet the underlying article exists, was properly linked to and was located by a human being using Google search; so, this is not a case where "attorneys and experts [have] abdicate[d] their independent judgment and critical thinking skills in favor of ready-made, AI-generated answers…."
A remaining serious concern, however, is Anthropic's attestation that a "manual citation check" was performed but "did not catch th[e] error." It is not clear how such an error—including a complete change in article title—could have escaped correction during manual cite-check by a human being. Furthermore, although the undersigned's [i.e., the Magistrate Judge's] standing order does not expressly address the use of AI by parties or counsel, Section VIII.G of [District] Judge Lee's Civil Standing Order requires a certification "that lead trial counsel has personally verified the content's accuracy." Neither the certification nor verification has occurred here. In sum, the Court STRIKES-IN-PART Ms. Chen's declaration, striking paragraph 9 [which contains the footnote that contains the citation with the hallucination], and notes for the record that this issue undermines the overall credibility of Ms. Chen's written declaration, a factor in the Court's conclusion.
Thanks to ChatGPT Is Eating the World for the pointer; it also discusses more about the substantive role of paragraph 9 in the declaration. Here's more backstory (from an earlier post):
The Declaration filed by a "Data Scientist at Anthropic" in Concord Music Group, Inc. v. Anthropic PBC includes this citation:
But the cited article doesn't seem to exist at that citation or at that URL, and Google found no other references to any article by that title….
Here's the explanation, from one of Anthropic's lawyers (emphasis added):
Our investigation of the matter confirms that this was an honest citation mistake and not a fabrication of authority. The first citation in footnote 3 of Dkts. 340-3 (sealed) and 341-2 (public) includes an erroneous author and title, while providing a correct link to, and correctly identifying the publication, volume, page numbers, and year of publication of, the article referenced by Ms. Chen as part of the basis for her statement in paragraph 9. We apologize for the inaccuracy and any confusion this error caused.
The American Statistician article reviewed and relied upon by Ms. Chen [the Anthropic expert], and accessible at the first link provided in footnote 3 of Dkts. 340-3 and 341-2, is titled Binomial Confidence Intervals for Rare Events: Importance of Defining Margin of Error Relative to Magnitude of Proportion, by Owen McGrath and Kevin Burke. A Latham & Watkins associate located that article as potential additional support for Ms. Chen's testimony using a Google search. The article exists and supports Ms. Chen's testimony in her declaration and at the May 13, 2025 hearing, which she proffered based on her pre-existing knowledge regarding the appropriate relative margin of error for rare events. A copy of the complete article is attached as Exhibit A.
Specifically, "in the context of small or rare-event success probabilities," the authors "suggest restricting the range of values to εR ∈ [0.1, 0.5]"—meaning, a relative margin of error between 10% to 50%—"as higher values lead to imprecision and poor interval coverage, whereas lower values lead to sample sizes that are likely to be impractically large for many studies." See Exhibit A, at 446. This recommendation is entirely consistent with Ms. Chen's testimony, which proposes using a 25% relative margin of error based on her expertise.
After the Latham & Watkins team identified the source as potential additional support for Ms. Chen's testimony, I asked Claude.ai to provide a properly formatted legal citation for that source using the link to the correct article. Unfortunately, although providing the correct publication title, publication year, and link to the provided source, the returned citation included an inaccurate title and incorrect authors. Our manual citation check did not catch that error. Our citation check also missed additional wording errors introduced in the citations during the formatting process using Claude.ai. These wording errors are: (1) that the correct title of the source in footnote 2 of Ms. Chen's declaration is Computing Necessary Sample Size, not, as listed in footnote 2, Sample Size Estimation, and (2) the author/preparer of the third source cited in footnote 3 is "Windward Environmental LLC", not "Lower Windward Environmental LLC." Again, we apologize for these citation errors.
Ms. Chen, as well as counsel, reviewed the complete text of Ms. Chen's testimony and also reviewed each of the cited references prior to submitting Ms. Chen's declaration to the Court. In reviewing her declaration both prior to submission and in preparation for the hearing on May 13, 2025, Ms. Chen reviewed the actual article available at the first link in footnote 3 of her declaration and attached hereto as Exhibit A, and the article supports the proposition expressed in her declaration with respect to the appropriate margin of error.
During the production and cite-checking process for Ms. Chen's declaration, the Latham & Watkins team reviewing and editing the declaration checked that the substance of the cited document supported the proposition in the declaration, and also corrected the volume and page numbers in the citation, but did not notice the incorrect title and authors, despite clicking on the link provided in the footnote and reviewing the article. The Latham & Watkins team also did not notice the additional wording errors in footnotes 2 and 3 of Ms. Chen's declaration, as described above in paragraph 6.
This was an embarrassing and unintentional mistake. The article in question genuinely exists, was reviewed by Ms. Chen and supports her opinion on the proper margin of error to use for sampling. The insinuation that Ms. Chen's opinion was influenced by false or fabricated information is thus incorrect. As is the insinuation that Ms. Chen lacks support for her opinion. Moreover, the link provided both to this Court and to Plaintiffs was accurate and, when pasted into a browser, calls up the correct article upon which Ms. Chen had relied. Had Plaintiffs' counsel raised the citation issue when they first discovered it, we could and would have confirmed that the article cited was the one upon which Ms. Chen relied and corrected the citation mistake.
We have implemented procedures, including multiple levels of additional review, to work to ensure that this does not occur again and have preserved, at the Court's direction, all information related to Ms. Chen's declaration. I understand that Anthropic has also preserved all information related to Ms. Chen's declaration as well….
