The Volokh Conspiracy
Mostly law professors | Sometimes contrarian | Often libertarian | Always independent
Hallucinations in the District of New Jersey
Is it an abuse of judicial power for a judge to issue an opinion with AI hallucinations?
Eugene blogged about Judge Julien Xavier Neals of the U.S. District Court for the District of New Jersey. Judge Neals issued an opinion that included errors, including made-up quotes and incorrect case outcomes. One of the parties submitted a letter pointing out these errors. It seems pretty clear that the Judge, or at least someone in his chambers, used generative AI to create the opinion. (Judge Neals is sixty years old, so I would suspect a law clerk made this error.) The judge promptly withdrew the opinion.
I suppose this particular case is settled, but I would wager there are more orders on Judge Neals' dockets that have hallucinations. Indeed, I suspect there are many judges throughout the country that have issued opinions with hallucinations. Savvy litigators should start combing through all adverse orders, and try to determine if there are obvious indicia of hallucinations. This will make excellent grounds for reversal on appeal.
But let's take a step back. What do we make of a judge who issues an opinion based on made-up cases? To be sure, judges makes mistakes all the time. Moreover, clerks make mistakes that their judges do not catch. When I was clerking, I made a particularly egregious error that my judge did not catch. The non-prevailing party promptly filed a motion for reconsideration. The opinion was withdrawn, and a new opinion was issued. The outcome of the case was not altered by that error, but at least the opinion was corrected.
Still, one might ask how closely Judge Neals, and other judges, review the work of their law clerks. Do the judges actually check the citations to see if they are hallucinated? I would suspect most judges do not check citations. District Court dockets are very busy, and it is not realistic to expect judges to so closely scrutinize their clerks' work. (Later in Justice Blackmun's career, he apparently limited his review of draft opinions to checking citations.)
I think the more useful frame is to ask whether the judge has failed to adequately supervise his law clerks. Judges invariably have to delegate authority to law clerks, even if the judge ultimately signs all of the orders. That delegation must include what is effectively a duty of care. In other words, the judge should tell the clerk how to go about doing the job, and in particular, how not to go about doing the job. In 2025, I think all judges should tell their clerks to either not use AI at all (my advice), or to use AI responsibly and triple-check any cited cases. The failure to give this advice would be an abuse of discretion.
But is it more than just an abuse of discretion? Does an Article III judge abuse his power when he issues an opinion based on hallucinated cases that no one in his chambers bothered to check? Judges have the awesome power to affect a person's life, liberty, or property, merely by signing their names to a piece of paper. It is the order, and not the opinion, that has the legal effect.
I think we would all agree that a judge would abuse his power by deciding a case by flipping a coin or rolling a dice. I suppose using AI is a bit less reckless than a game of chance, but not by much. Relatedly, is it an abuse of power when a judge grants an ex parte TRO without evening reading the brief? I think we are starting to see some of the boundaries of the judicial power.
I don't know that Judge Neals will receive a misconduct complaint, as he promptly withdrew his opinion. But an enterprising sleuth could do a close analysis of all opinions from Judge Neals, and judges nationwide, and perhaps find a pattern of misconduct. Would that record show a judge cannot be trusted to exercise the judicial power?
And speaking of the District of New Jersey, can we be certain that Judge Neals voted to appoint Desiree Grace as the United States Attorney for the District of New Jersey? Or maybe it was a Chatbot?
I'll leave you with one anecdote I read in a recent article about what AI is doing to students:
My unease about ChatGPT's impact on writing turns out to be not just a Luddite worry of poet-professors. Early research suggests reasons for concern. A recent M.I.T. Media Lab study monitored 54 participants writing essays, with and without A.I., in order to assess what it called "the cognitive cost of using an L.L.M. in the educational context of writing an essay." The authors used EEG testing to measure brain activity and understand "neural activations" that took place while using L.L.M.s. The participants relying on ChatGPT to write demonstrated weaker brain connectivity, poorer memory recall of the essay they had just written, and less ownership over their writing, than the people who did not use L.L.M.s. The study calls this "cognitive debt" and concludes that the "results raise concerns about the long-term educational implications of L.L.M. reliance."
I still refuse to use AI. I may be the last man standing.
