11 Court Opinions in the Last 30 Days Mention AI-Hallucinated Material, and …

that's likely just the tip of the iceberg.

I did a Westlaw search, and found that 11 court opinions in the last 30 days mention that a party had likely included AI-hallucinated case citations or (in one instance) AI-hallucinated quotes from real cases; ten involved court filings, and one involved a party's communication with opponents. That's a rate of over 100 per year.

And that's likely just the tip of the iceberg, since the overwhelming majority of all court cases in the U.S. are state trial court cases, and opinions in those cases only rarely make it onto Westlaw. Eight of the cases I found were federal trial court cases, two were state appellate cases, and one was a state trial court case. This makes me think there are many more state trial court cases in which such hallucinations were noticed and mentioned but which aren't on Westlaw, and still more in which such hallucinations weren't mentioned or weren't even noticed.

Six of the cases involved pro se litigants, but five involved lawyers.

Comments

[Commenter_XY]

Aren't there services to check citations in a brief, Professor Volokh? Seems like a no-brainer if you use AI.

Are those services (if they exist) expensive? If so, I see a potential business opportunity. 😉

[neurodoc]

"Seems like a no-brainer if you use AI."

Did you mean for that to be understood literally or figuratively?

A when the day comes (how far off?) that only are the pleadings authored by AI, but so to are thy "judged by AI"?

[Life of Brian]

There are a number of services from the usual suspects (Lexis/Westlaw/Bloomberg/etc.) that cost about what you might imagine. But those just check to make sure the citation is to a case that exists, and perhaps that the actual words supposedly quoted from the case are indeed in it.

Verifying non-quoted characterizations of what a case says/stands for is a much harder problem, and I doubt it's possible at all to reliably catch the classic issue of an eager young associate or an LLM citing a case and quoting a super-favorable word snippet, but the case itself is squarely against your overall position. Particularly when the stakes are even vaguely high, IMO there's just no reasonable substitute for pulling the cases and reading what they actually say.

[John F. Carr]

This reminds me that one of the Midwestern circuits had a standard fine for making tax protester arguments. They came up so often in that part of the country that the response was routine.

[neurodoc]

"This reminds me that one of the Midwestern circuits had a standard fine for making tax protester arguments. They came up so often in that part of the country that the response was routine."

Judicial economy, no?

[TwelveInchPianist]

So the judges are doing it too? Or at least not checking.

[Roger S]

Hallucination might not be mentioned, as it is not really the correct term. A hallucination is an apparent sensory perception.

[neurodoc]

"Hallucination might not be mentioned, as it is not really the correct term. A hallucination is an apparent sensory perception."

Your point is taken, but not sure it is an entirely valid one. However these fictive cites come to be, they presumably reflect a belief, albeit wrong, that they in fact represent reality. phenom

What do you think this AI phenomena should be called?

[Roger S]

The AI chats can be inaccurate in many ways. The bogus cites have gotten attention on this blog, but other things can go wrong also.

[David Nieporent]

AI is also not really the correct term, since there's no intelligence involved in these services; they're just generating text based on predictive algorithms. Nonetheless, "hallucination" is the term that these companies themselves use.

[neurodoc]

Are you saying that unless what is expressed is of living origin, that is not machine generated, it cannot be called "intelligence," no matter that said expression might be beyond human capacity? I don't think that is correct.

[Bwaaah]

I think the "artificial" modifier is sufficient to distinguish it from human intelligence. And anyway, human intelligence is often unworthy of being described as such.

[neurodoc]

When someone experiencing a toxic delerium talks back to imagined voices, we say they are "hallucinating." Would you say they weren't because there was nothing going on outside their own cranium for them to perceive? If you would, then you would be disagreeing with every neurologist, psychiatrist and other mental health professional alive.

[William Cheswick]

AI is a very big deal, a coming singularity. For me, it is already vastly helpful in my programming efforts.

But at this stage of the technology, everyone should consider every answer to be a rough draft.

[Bwaaah]

everyone should consider every answer to be a rough draft wrong

[NvEric]

It's a bug in the programming.

[Scott Fruehwald]

Professor Moppett has written an article suggesting that law schools teach critical thinking to help students and lawyers deal with the problems of AI:
Preparing Students for the Artificial Intelligence Era: The Crucial Role of Critical Thinking Skills by Samantha A. Moppett.
[https://lwic.mobilize.io/links?url=https%3A%2F%2Fpapers.ssrn.com%2Fsol3%2Fpapers.cfm%3Fabstract_id%3D5193298]
"In a nutshell, in the era of generative AI critical thinking skills are more important than ever for the practice of law."
"Critical thinking in legal practice is an intellectual necessity."
"Critical thinking is key to both the work of lawyers that AI cannot facilitate as well as in the assessment of the output that AI produces."
"The critical thinking and problem-solving skills of today’s entering law students are significantly lower than those of students in the past."
"A wealth of materials exists that address how to teach critical thinking skills and develop critical thinking dispositions."
I have written a book on critical thinking for lawyers and law students.
Critical Thinking: An Essential Skill for Law Students, Lawyers, Law Professors, and Judges
[https://www.amazon.com/exec/obidos/ASIN/B0B6L8121Z/reasonmagazinea-20/.E6YHsZF_NWRGiCbHhh9-PTg04075ZW8Wc5-d373XeqC2DbJAPLFBlhQYbxDl89DhrA9P_hOWpZp0i1e9ZWW1A-f8WzeHPCVwZrG12d9ViQM16f4yxCW22FPzBSS5jPF0-VoF-nyufvV0fy7nx5WUTnjis1GBW4CTROAb0PBL9ka-hLxmrN9TXaZbBSmG76NKSaXTPdimVi7jfz8I8d8zVYVZYzXZqBaSjJ3tAqlF5Ic.4Qn-wGogvwf_3tBN6TSCCWiWfyWxldFs9bVjGup4e_Y&dib_tag=se&qid=1743210849&refinements=p_27%3AE.+Scott+Fruehwald&s=books&sr=1-7&text=E.+Scott+Fruehwald]

[Scott Fruehwald]

Professor Moppett has written an article suggesting that law schools teach critical thinking to help students and lawyers deal with the problems of AI:

Preparing Students for the Artificial Intelligence Era: The Crucial Role of Critical Thinking Skills by Samantha A. Moppett.

[https://lwic.mobilize.io/links?url=https%3A%2F%2Fpapers.ssrn.com%2Fsol3%2Fpapers.cfm%3Fabstract_id%3D5193298]

"In a nutshell, in the era of generative AI critical thinking skills are more important than ever for the practice of law."

"Critical thinking in legal practice is an intellectual necessity."

"Critical thinking is key to both the work of lawyers that AI cannot facilitate as well as in the assessment of the output that AI produces."

"The critical thinking and problem-solving skills of today’s entering law students are significantly lower than those of students in the past."

"A wealth of materials exists that address how to teach critical thinking skills and develop critical thinking dispositions."

I have written a book on critical thinking for lawyers and law students.

Critical Thinking: An Essential Skill for Law Students, Lawyers, Law Professors, and Judges

[https://www.amazon.com/exec/obidos/ASIN/B0B6L8121Z/reasonmagazinea-20/.E6YHsZF_NWRGiCbHhh9-PTg04075ZW8Wc5-d373XeqC2DbJAPLFBlhQYbxDl89DhrA9P_hOWpZp0i1e9ZWW1A-f8WzeHPCVwZrG12d9ViQM16f4yxCW22FPzBSS5jPF0-VoF-nyufvV0fy7nx5WUTnjis1GBW4CTROAb0PBL9ka-hLxmrN9TXaZbBSmG76NKSaXTPdimVi7jfz8I8d8zVYVZYzXZqBaSjJ3tAqlF5Ic.4Qn-wGogvwf_3tBN6TSCCWiWfyWxldFs9bVjGup4e_Y&dib_tag=se&qid=1743210849&refinements=p_27%3AE.+Scott+Fruehwald&s=books&sr=1-7&text=E.+Scott+Fruehwald]