Seemingly Nonexistent Citation in Anthropic Expert's Declaration [UPDATE: Apparently Caused by Lawyer's Misuse of Claude to Format Citations]

UPDATE 5/15/2025 (post moved up): Anthropic's lawyers filed a declaration stating that the error was not the expert's, but stemmed from the (unwise) use of Claude AI to format citations.

The Declaration filed by a "Data Scientist at Anthropic" in Concord Music Group, Inc. v. Anthropic PBC includes this citation:

But the cited article doesn't seem to exist at that citation or at that URL, and Google found no other references to any article by that title.

Bloomberg (Annelise Levy) has a story about this, under the title "Anthropic Expert Accused of Citing Fake Article in AI Lawsuit" (Chat GPT Is Eating the World links to that). Magistrate Judge Susan van Keulen ordered the parties, apparently (according to Bloomberg) referring to this problem, to explain matters:

[N]o later than May 15, 2025, Defendant shall file a Statement addressing the issue raised by Plaintiffs' counsel at the outset of the hearing ….

I'll report what the Statement asserts once it is filed. Thanks to Prof. Edward Lee for the pointer.

UPDATE 5/15/25, 4:17 pm: Here's the explanation, from one of Anthropic's lawyers (emphasis added):

Our investigation of the matter confirms that this was an honest citation mistake and not a fabrication of authority. The first citation in footnote 3 of Dkts. 340-3 (sealed) and 341-2 (public) includes an erroneous author and title, while providing a correct link to, and correctly identifying the publication, volume, page numbers, and year of publication of, the article referenced by Ms. Chen as part of the basis for her statement in paragraph 9. We apologize for the inaccuracy and any confusion this error caused.

The American Statistician article reviewed and relied upon by Ms. Chen [the Anthropic expert], and accessible at the first link provided in footnote 3 of Dkts. 340-3 and 341-2, is titled Binomial Confidence Intervals for Rare Events: Importance of Defining Margin of Error Relative to Magnitude of Proportion, by Owen McGrath and Kevin Burke. A Latham & Watkins associate located that article as potential additional support for Ms. Chen's testimony using a Google search. The article exists and supports Ms. Chen's testimony in her declaration and at the May 13, 2025 hearing, which she proffered based on her pre-existing knowledge regarding the appropriate relative margin of error for rare events. A copy of the complete article is attached as Exhibit A.

Specifically, "in the context of small or rare-event success probabilities," the authors "suggest restricting the range of values to εR ∈ [0.1, 0.5]"—meaning, a relative margin of error between 10% to 50%—"as higher values lead to imprecision and poor interval coverage, whereas lower values lead to sample sizes that are likely to be impractically large for many studies." See Exhibit A, at 446. This recommendation is entirely consistent with Ms. Chen's testimony, which proposes using a 25% relative margin of error based on her expertise.

After the Latham & Watkins team identified the source as potential additional support for Ms. Chen's testimony, I asked Claude.ai to provide a properly formatted legal citation for that source using the link to the correct article. Unfortunately, although providing the correct publication title, publication year, and link to the provided source, the returned citation included an inaccurate title and incorrect authors. Our manual citation check did not catch that error. Our citation check also missed additional wording errors introduced in the citations during the formatting process using Claude.ai. These wording errors are: (1) that the correct title of the source in footnote 2 of Ms. Chen's declaration is Computing Necessary Sample Size, not, as listed in footnote 2, Sample Size Estimation, and (2) the author/preparer of the third source cited in footnote 3 is "Windward Environmental LLC", not "Lower Windward Environmental LLC." Again, we apologize for these citation errors.

Ms. Chen, as well as counsel, reviewed the complete text of Ms. Chen's testimony and also reviewed each of the cited references prior to submitting Ms. Chen's declaration to the Court. In reviewing her declaration both prior to submission and in preparation for the hearing on May 13, 2025, Ms. Chen reviewed the actual article available at the first link in footnote 3 of her declaration and attached hereto as Exhibit A, and the article supports the proposition expressed in her declaration with respect to the appropriate margin of error.

During the production and cite-checking process for Ms. Chen's declaration, the Latham & Watkins team reviewing and editing the declaration checked that the substance of the cited document supported the proposition in the declaration, and also corrected the volume and page numbers in the citation, but did not notice the incorrect title and authors, despite clicking on the link provided in the footnote and reviewing the article. The Latham & Watkins team also did not notice the additional wording errors in footnotes 2 and 3 of Ms. Chen's declaration, as described above in paragraph 6.

This was an embarrassing and unintentional mistake. The article in question genuinely exists, was reviewed by Ms. Chen and supports her opinion on the proper margin of error to use for sampling. The insinuation that Ms. Chen's opinion was influenced by false or fabricated information is thus incorrect. As is the insinuation that Ms. Chen lacks support for her opinion. Moreover, the link provided both to this Court and to Plaintiffs was accurate and, when pasted into a browser, calls up the correct article upon which Ms. Chen had relied. Had Plaintiffs' counsel raised the citation issue when they first discovered it, we could and would have confirmed that the article cited was the one upon which Ms. Chen relied and corrected the citation mistake.

We have implemented procedures, including multiple levels of additional review, to work to ensure that this does not occur again and have preserved, at the Court's direction, all information related to Ms. Chen's declaration. I understand that Anthropic has also preserved all information related to Ms. Chen's declaration as well….