"A Major Law Firm's ChatGPT Fail"

"The problem with these cases is that [only one] exist[s]."

David Lat (Original Jurisdiction) reports, in his characteristically readable and interesting style, about the latest incident (you can also read Judge Kelly Rankin's short order laying out the problem, in Wadsworth v. Walmart (D. Wyo.)). Folks, just cite-check.

Comments

[Robert Efroymson]

Missing Lat Link

[Eugene Volokh]

Whoops, fixed, thanks!

[Mark Regan]

All of the hallucinated cases were offered simply with a Westlaw cite. Shouldn't a law firm with PACER access cite cases using the file number, e.g., 2:22-cv-223 (N.D. Tex.)? That kind of citation should be a lot harder for anyone to hallucinate and a lot easier for anyone to cite check.

[Chacho]

Not necessarily. Adding PACER file numbers clutters up the citation, and frankly the Westlaw citation is more useful for the court (read: the law clerk slaving away checking all the cases) anyway. Some courts actually have citation guides that address this - Delaware Chancery, for example, calls for the Westlaw cite only without any case file number.

[Kenneth Almquist]

From David Lat’s post:

“As they explained to Judge Castel at the sanctions hearing, at the time their firm did not have access to Westlaw or LexisNexis—which are, as we all know, extremely expensive...”

It seems to me that a Court that insists that filings use a citation system tied to a product that is out of the reach of many lawyers, much less the average layman or concerned citizen, the Court isn’t taking its obligations to make its proceedings accessible to the public very seriously.

[Chacho]

Fair, but we’re talking about unpublished district court opinions. Circuit opinions and published district court opinions, which are what you should mostly rely on anyway, have reporter citations.

[Noscitur a sociis]

West stoppped publishing the Federal Appendix several years ago in favor of a “WL” citation.

[Chacho]

Oof, that sucks. Not sure how I missed that.

[Riva]

And Westlaw is probably using AI to write its headnotes. They’ve created their own stupid legal feedback.

[tuphat]

Morgan & Morgan

[Sam Bankman-Fried]

Is this like “I’m a cat”?

[TextFirst]

I'm absolutely not surprised this happened, I guessed it would be Morgan & Morgan or a similar mill firm when I saw the headline. The entire business model is to advertise a ton, farm out the cases won to local firms, and collect a percentage for the referral. The lawyers at Morgan & Morgan may have never read this pleading. As for the local attorney, strong chance they put too much trust in somebody junior that they're overworking. It would be genuinely shocking if that happened at, say, Skadden. Calling Morgan & Morgan a major law firm is a bit like saying McDonald's is a famous restaurant. It's true, kinda, but misleading.

[Bubba Jones]

I think the point was that Morgan and Morgan can afford database access.

Not that they are particularly good.

[Roger S]

The federal rules do not actually say that lawyers must check the citations. They say that the factual contentions must have evidentiary support, and that attorneys must not multiply the proceedings unreasonably, and some related things. But I do not see anything that says a judge can sanction a lawyer for not checking the citations.

This mistake may be embarrassing, but the judge is the one who is taking a frivolous action here.

[TextFirst]

? Parties certify that the legal claims, defenses, and other contentions in the filing are warranted by existing law or by a nonfrivolous argument for extending, modifying, or reversing existing law or for establishing new law. How exactly do you make a nonfrivolous argument with fictitious cases? But beyond that, courts have inherent authority to issue sanctions and manage their courtroom that go beyond the rules. Good luck persuading the appellate court that the court abused its discretion. Perhaps some fictitious cases would help?

[Roger S]

You are right that the bigoted judges probably prefer humans over AI bots, and may get upset at someone using new technology. I am just saying that the rules do not back up what this judge is doing.

[Bubba Jones]

The lawyers literally fabricated their pleading.

Failing to fact check the AI isn’t an excuse for filing a falsified document.

[David Nieporent]

The federal rules do not actually say that lawyers must check the citations.

Demonstrating the insight that leads you to think that libertarianism and marxism are the same thing, you make up law as well. The federal rules do in fact say that. Fed. R. Civ. P. 11(b) says,

"By presenting to the court a pleading, written motion, or other paper … an attorney or unrepresented party certifies that to the best of the person's knowledge, information, and belief, formed after an inquiry reasonable under the circumstances:

[...]

(2) the claims, defenses, and other legal contentions are warranted by existing law or by a nonfrivolous argument for extending, modifying, or reversing existing law or for establishing new law;

Obviously nonexistent cases do not constitute "existing law." Nor do cases that have been overruled (which is what cite checking usually checks for). There is no way to make the required certification w/o cite checking.

[Roger S]

The lawyer made a reasonable inquiry to ChatGPT, and formed an opinion that his contentions were warranted under existing law. What's the problem? The rule could say that citations must be checked, if that were intended.

ChatGPT has a better understanding of the law than I do. There is nothing inherently wrong with me relying on ChatGPT. If the feds want to ban it, then they need to adopt a rule against it.

[Noscitur a sociis]

If ChatGPT regularly responds to inquiries with nonexistent cases, as it does, it is not, in fact, reasonable to rely on it without verifying the cases it provides.

And, with all due respect, I have trouble believing that even you are dumb enough to not realize that. And frankly, I’m a little offended that you think we’re dumb enough to think you might be.

[Roger S]

What you think is reasonable is different from what I think is reasonable. I think that the court should check the cases itself.

[Stupid Government Tricks]

How do you think they found the lies?

[Noscitur a sociis]

I am certainly not disputing that you genuinely don’t know what “reasonable” means.

[Roger S]

Reasonable is a subjective term.

[mike petrik]

It’s actually an objective legal standard. But keep digging. It’s amusing.

[Stupid Government Tricks]

I'm with you on that. Roger S comes across in this little thread like a bratty kid arguing that eating cookies in bed is OK because he was only told to not eat crackers in bed. I despise our legal system, but I'm never going to argue that lying to a court is OK if there's no specific rule against it.

[David Nieporent]

What's the problem?

That asking ChatGPT is not making a "reasonable inquiry." It wouldn’t have been reasonable before Mata, and it's certainly not reasonable 2 years later after it was well publicized that ChatGPT makes stuff up.

ChatGPT has a better understanding of the law than I do.

So does the average toddler. Or chipmunk.

[David Nieporent]

ChatGPT has a better understanding of the law than I do.

To be clear, not only does this evince your lack of understanding of the law, but also a lack of understanding of ChatGPT. ChatGPT has no understanding of the law. It has no understanding of anything. It is just a predictive model that generates words based on statistical use of those words in other sources.
​
​
But I think the focus on ChatGPT making up sources kind of misses the point, too. Even if ChatGPT only cited cases that actually existed, relying on it would be a breach of professional obligations. A lawyer has to read the cases he's citing, and Shepardize them; he needs to see what those cases actually say, and that they're still good law. The mere fact that the cases exist does not allow one to ethically cite them.

[Don Nico]

Even outside the law, in the world of academic publishing it is considered unethical to cite sources that have not actually been used by the author in the preparation of the manuscript.

[Roger S]

You think your legal skills are valuable because you know how to Shepardize cases. Whoop-dee-doo. AI will soon be doing that, if it is not already.

Lawyers do not need to understand the law. They just need to present arguments that persuade the court and make their clients happy. When AI does that, lawyers will be obsolete.

[MaddogEngineer]

What's the problem? You literally just described it. I can't tell from your other posts here whether you're being sincere or playing a troll character. Your "judge bigoted against AI" makes me wonder.

Whether an AI generated false citations, or another human made one up, they are still false. I don't understand the need to make any distinction about that. No good faith AI exception. The lawyer submitting anything to a court is ultimately responsible for its contents, no matter how it was "generated". If AI is otherwise going to save you time writing, it shouldn't be too burdensome to verify its citations.

[Roger S]

Lawyers file poorly researched briefs all the time. This judge is all indignant because he has smoked out a use of AI. He thinks he is making a virtuous stand on behalf of humans, as jobs get replaced by AI bots. Okay, but he accomplishes nothing. The next brief will just use software to check the citations.

[MaddogEngineer]

I didn't say poorly researched Here, I'll repeat the money line:

"Whether an AI generated false citations, or another human made one up, they are still false."

We are not talking about citing a case that exists but is not on point. I can't imagine any other outcome than termination if a junior associate made up a citation, because it takes a positive action to do so. I also imagine a partner who had an associate do this to him would lose his mind, because it's not just sloppy. This would be like a grad student fabricating data for a research professor's technical publication. Not an innocent mistake.

I don't particularly care whether the judge is grandstanding or not for all mankind, it doesn't make him wrong. An AI isn't even as good as the dumbest actual human. That places a greater burden on the person using it to scrutinize its output.

The current value of AI is not to replace human expertise entirely, because it's not capable of that. It's to remove repetitive drudgery, at a speed faster than any human could do it. If its users do not understand the limits of the technology, the consequences of that is on them. Others victimized by that are entitled to be as outraged as they want.

[Roger S]

If a human junior associate or grad student fabricated this, then you are right, that it would be a deliberate act that would get him fired. I agree. But this was AI, a soulless robot not capable of deliberate acts. Judges like to punish deliberate wrongdoing. But this judge is just ranting about a bot.

[David Nieporent]

No, it was not AI; it was a human lawyer who chose to submit it.

[KenveeB]

The human junior associate or grad student would be fired, absolutely. But you don't seem to understand that the senior attorney or researcher *would also be* fired or sanctioned in some way for filing it without having reviewed the important facts and cases. "I blindly accepted the word of a junior assistant I'm supposed to be supervising" is not a decent defense.

[David Nieporent]

This is a classic: come up with a stupid and fake argument — that the judge is somehow motivated by a desire to protect lawyer jobs from AI — and then argue that this plan won't work, and therefore is pointless.

Hint: that's not why the judge is indignant and not what his motivation is.

[Bubba Jones]

When did ChatGPT pass the bar?

[y81]

This is an interesting question. I have tried out ChatGPT on a number of law school-exam type questions. If you haven't done this yourself, you should. My conclusion, based on comparing ChatGPT to the multiple schlocky lawyers I encounter in practice, is that ChatGPT would pass the bar exam, but would not get High Honors at Berkeley Law. (Full disclosure: it's not like I got High Honors in every course at Berkeley. But often enough to know what it looks like.)

[mike petrik]

If citing fake cases is not sanctionable, it certainly should be. A lawyer has a professional duty of care. Relying on AI might be sufficient someday, but it absolutely is not today.

[Roger S]

Then you can suggest that the federal rules be amended to make it sanctionable to cite fake cases.

[David Nieporent]

As has been explained to you, that's what they already say.

[mike petrik]

No doubt Roger S thinks what they say is subjective, like all things in the universe. Using a pseudonym exhausted his wisdom.

[Noscitur a sociis]

I believe Roger S at least purports to be Roger Schlafly, disappointingly unimpressive son of Phyllis Schlafly.

[mike petrik]

If so, poor Phyllis.

[mike petrik]

Schlafly was trained as an electrical engineer, which would explain his disordered defense of AI.

[mike petrik]

I imagine he’s impressive in his proper environment, but that environment is not law. On this form he’s just another arrogant Internet ultracrepideriam talking through his hat.

[ThomasW]

How long until somebody using AI cite checks that they are all real cases but doesn't bother to read the case, which turns out to conclude the opposite of the argument it was cited for?

[Bubba Jones]

You just described much of the medical literature.

Most people, especially doctors, rely on the conclusion of the authors. They don’t review the actual results section and certainly not the methods section.

So they have no actual knowledge of whether the data support the conclusion.

And then when I cite other papers whose data you haven’t reviewed …

It’s nonsense on stilts.

[MaddogEngineer]

An AI is not an "author". It is a required presumption that the author of medical literature who will not make up stuff, because he knows that is bad and dangerous. An AI is not capable of that understanding. That's why it cannot be a author.

[KenveeB]

Attorneys do that now. I've had opposing attorneys cite a case for the opposite of what it says, or where the portion they quoted was the court quoting a party's argument before ultimately ruling against it. But, as they should, they get reamed by the judge for that.

[Ama Valde Intellectum]

I think you are trying to imply something by saying 'major' like 'What's the big deal?" the best firms are doing it !!

[Eugene Volokh]

That is a very odd inference to draw from my post.

[mike petrik]

My old law partner used to call that “taking ridiculous inferential liberties.”

[Noscitur a sociis]

I think you might need to amare the old intellectum a little valdius.

[Longtobefree]

A computer can tell a one from a zero.
A computer can add. Very fast.

The rest is programming.

[Stupid Government Tricks]

Let me introduce you to the IBM 1620 Mod I, aka "Cadet": Can't Add, Doesn't Even Try. It literally could not add except by table lookup. A decimal machine. Add tables were from 100-299, and multiply tables started at 300. Want to add 3+7? Read the digit at 137, unless there was a carry in, then read 237.

It could increment the instruction address, always in even numbers, but only for instruction addresses.

The Mod II could add, and even had floating point hardware, I think, but I never used that advanced machine.

[William Cheswick]

AI is already useful in many fields. I let it suggest code to me, and the suggestions have saved me a lot of time. But it is vital to remember:

AI provides rough drafts. It is your job to keep that in mind, especially if you are signing its result as your own.

[No One Of Consequence]

I'm shocked that we'd see something like this from a lawyer whose firm "brings a refreshing twist to old school law firm techniques and outdated methods" and who's "lovingly referred to as "The TikTok Queen.'"
https://www.womenintrialtravelsummit.com/speakers/taly-goody

[MaddogEngineer]

Ah, had no idea who the lawyer was here, had only seen mention in several places of being called out by the judge. I imagine charges of misogyny for making a big deal about this will be forthcoming.

[David Welker]

If you are going to cite a case, you should read it. Maybe even more than once.

[mike petrik]

Exactly.