The Volokh Conspiracy

Mostly law professors | Sometimes contrarian | Often libertarian | Always independent

AI in Court

Federal Public Defender Submits Brief with Nonexistent Citation, Apparently Refuses to Admit This to the Judge at a Hearing

It's not the hallucination, it's the coverup.

|

A short excerpt from the nearly 10,000-word opinion in U.S. v. Hayes, decided last week by Magistrate Judge Chi Soo Kim (E.D. Cal.):

The Court finds that Mr. Francisco submitted a fictitious or non-existent case and quotation in his written motion to unseal filed on November 21, 2024; knowingly made inaccurate and misleading statements in his written reply filed on December 5, 2024 to the government's opposition that expressly raised the fictitious case and quotation; and knowingly made inaccurate and misleading statements at the December 9, 2024 hearing.

The Court further finds that Mr. Francisco's inaccurate and misleading statements were not inadvertent as claimed, but knowing and made in bad faith. Despite being provided multiple opportunities to candidly acknowledge and correct his errors as required under his duty of candor to the court, Mr. Francisco unfortunately failed to do so….

In his motion to unseal, Mr. Francisco cited eight cases, six of which were cited in near string citation form. The primary case upon which Mr. Francisco relied—analyzing, quoting, describing the district court's analysis, and addressing in two different places in the motion—was "United States v. Harris, 761 F. Supp. 409, 414 (D.D.C. 1991)."

Unfortunately, "United States v. Harris, 761 F. Supp. 409, 414 (D.D.C. 1991)" is not a real case. The citation has all the markings of a hallucinated case created by generative artificial intelligence (AI) tools such as ChatGPT and Google Bard that have been widely discussed by courts grappling with fictitious legal citations and reported by national news outlets. Similar to other fictitious case citations created by generative AI tools, the fictitious "United States v. Harris" case looks like a real case with a case name; a citation to the Federal Supplement, which is the reporter that publishes opinions from federal district courts; identification of a district court; and the year for the decision….

Unlike other cases where counsel and litigants have admitted, sometimes reluctantly, that the fictitious citations and quotations were created by generative AI, Mr. Francisco states that he "did not use and ha[s] never used AI (artificial intelligence) to draft any of my motions." Instead, Mr. Francisco states that he "hastily" drafted the portion of his motion discussing the legislative history of 18 U.S.C. § 3509(d), which includes the fictitious case and quotation. Regarding his citation to the fictitious United States v. Harris case, he states: "I am baffled to say where it came from, first in a case name which does not exist, second from a district and year which does not match the citation volume and page, and third, which never even addresses the Motion's issues. It was an inadvertent drafting error I cannot begin to explain."

The Court finds this response inadequate and not credible. Though Mr. Francisco admits that he drafted his filing with the fictitious case and quotation, he fails to explain where or how he found or created the fictitious case and quotation. Regardless of the source of the fictitious case and quotation, an attorney with almost six years of experience such that case citations are "second nature" to him, knows how to conduct legal research, retrieve cases, and accurately cite those cases. The Court need not make any finding as to whether Mr. Francisco actually used generative AI to draft any portion of his motion and reply, including the fictitious case and quotation. See Grant v. City of Long Beach (9th Cir. 2024) (without determining source of two non-existent case citations and misrepresentation of thirteen cases, striking opening brief and dismissing appeal where opening brief "represents a material failure to comply with our rules")….

The Court ultimately finds that Mr. Francisco made knowing and willful misrepresentations with the intent to mislead the Court, which violated the standards of professional conduct, including the duty of candor to the court, and demonstrates bad faith. Mr. Francisco's first opportunity to candidly acknowledge and correct the fictitious case and quotation was in his written reply filed in response to the government's opposition that directly raised the fictitious case and quotation. But instead of candidly acknowledging and correcting his errors, Mr. Francisco filed with the Court the following two-sentence response in a footnote in his written reply: "Defense counsel acknowledges the government's observation that United States v. Harris does not discuss the legislative history of 18 U.S.C. § 3509(d) and clarifies that United States v. Broussard, 767 F. Supp. 1536, 1542 (D. Or. 1991), is the district court case from which this material is quoted. This inadvertent citation error does not affect the substance of the defense's position that the legislative history supports a far narrower interpretation of § 3509(d) than the government asserts."

This response was not accurate and was misleading. It failed to acknowledge that the United States v. Harris case is fictitious, failed to acknowledge that the actual case is the Harris v. Murray case [the case that appears at the cited volume and page number -EV], and failed to acknowledge that the quotation is not from any case. Instead, Mr. Francisco stated in his reply that an "inadvertent citation error" was made and the quotation is from a different case, United States v. Broussard, 767 F. Supp. 1536, 1542 (D. Or. 1991). This, too, was misleading, as the language Mr. Francisco quoted does not appear anywhere in Broussard.

Though Mr. Francisco reviewed Broussard multiple times before filing his motion to unseal, including on October 26, November 13, and November 14, 2024, he did not refer to or cite Broussard in his motion to unseal. That was likely with good reason. Broussard does not support Mr. Francisco's motion to unseal a total of eight (8) words in his motion to compel that described Child Victim 1's relationship to Defendant Hayes and Child Victim 1's gender, which had been properly redacted by Judge Claire. [Details of Broussard omitted. -EV] In Mr. Francisco's response to the Order to Show Cause, he acknowledges that Broussard did not support his motion to unseal arguments. This undermines his other post-hoc statements that he actually intended to cite and rely on United States v. Broussard, not the fictitious United States v. Harris case. Therefore, these statements are not credited.

The court also concluded that a later-filed Errata was inadequate, and Francisco's later statements about what he knew and when he knew it weren't candid. The discussion is long and quite detailed, but here's an excerpt that should offer its flavor:

At the December 9, 2024 hearing, Mr. Francisco had a second opportunity to candidly acknowledge and correct his errors. Unfortunately, Mr. Francisco persisted in his misrepresentations and made matters worse. In response to the Court's questions, Mr. Francisco continued to assert that the quotation was from Broussard. After the Court provided Mr. Francisco with a hard copy of Broussard and Mr. Francisco was not able to locate the quotation, he suggested that the quotation was from a second Broussard case. Unfortunately for Mr. Francisco, the Court had also reviewed this second Broussard case before the hearing, United States v. Broussard, 767 F. Supp. 1545 (D. Or. 1991) ("Broussard II"), and informed Mr. Francisco that Broussard II did not contain the quotation….

At the hearing, the Court then directly asked Mr. Francisco how his statements regarding "United States v. Harris" could be an "inadvertent citation error" …. Instead of candidly acknowledging that United States v. Harris was a non-existent case, Mr. Francisco continued to suggest that United States v. Harris was a real case and argued (inaccurately) that the legislative history "absolutely supports the Defense position." …

Not only did Mr. Francisco fail to acknowledge and correct his errors at the hearing, he made matters worse by repeatedly refusing to admit that United States v. Harris was non-existent despite knowing that it was non-existent. The Court gave Mr. Francisco several chances to tell the truth at the hearing, providing five distinct opportunities to respond to the Court's direct questions, and each time he knowingly persisted in his misrepresentations to the Court.

In his OSC response, Mr. Francisco admits that he reviewed the real Harris v. Murray case on December 3 after reviewing the government's opposition raising the non-existent United States v. Harris case and quotation, and that at the time of the December 9 hearing, he "was aware at that time that [United States v.] Harris was an incorrect citation and did not support my argument."

The Court can only conclude that Mr. Francisco's statements at the December 9, 2024 hearing were made knowingly with the intent to mislead the Court, and they constitute evidence of bad faith….

The court ordered Francisco to pay $1,500, and also ordered the Clerk to serve a copy of the order on the California and D.C. Bars, and "on all the district judges and magistrate judges in this district."