They Just Can't Help It, AI Hallucination Edition

From Judge Judith Levy (E.D. Mich.) in Evans v. Robertson, the I've Seen Fake Cites on Both Sides Now case:

The litigants in this case … have been repeatedly warned that factitious citations will not be tolerated by the Court. Judge Stafford ordered Plaintiff and Defendant to "include a written representation that [they have] personally checked each legal citation and each citation is accurate and stands for its asserted proposition" and warned that a failure to do so may result in sanctions. Despite Judge Stafford's order, both Plaintiff and Defendant have violated the Court's express requirements and lied to the Court numerous times….

Defendant's objection, which was previously stricken by the Court, contained several factitious citations…. Plaintiff's objection contains at least two factitious citations…. Plaintiff references these factitious citations despite her declaration of verification, which states "I, the undersigned, verify that the legal authorities cited herein are accurately stated, relevant …, and drawn from controlling or persuasive precedent applicable to the issues presented in this case." …

Plaintiff's response to Defendant's objections also contains a factitious citation…. Plaintiff references this factitious citation despite her certification, which states "Plaintiff affirms that all legal authorities cited in this filing have been reviewed and verified for accuracy. All case citations were cross-checked using Westlaw and the University of Detroit Law Library, with the assistance of library staff, to ensure correct citation format, validity, and precedential status as of the date of filing." …

In addition, Defendant's response to Plaintiff's objections contains numerous factitious citations…. Defendant references these factitious citations despite her certification ….

Next, Defendant's objections to Judge Stafford's Report and Recommendation contains factitious citations. Despite Defendant's certification that "she has personally checked each legal citation in this filing and that each citation is accurate and stands for the proposition asserted," Defendant's citations are not accurate and do not stand for her propositions asserted…. It is obvious that Defendant's certification regarding accuracy is false….

Defendant's "motion to strike Plaintiff's limited rebuttal" contains factitious citations and lacks the required written representation that she personally checked each legal citation and that each citation is accurate and stands for its asserted proposition…. It is appalling that Defendant would use fabricated quotations in her argument that Plaintiff is misleading the Court….

Plaintiff's "consolidated response to Defendant's refiled objections" and Defendant's "motion to strike Plaintiff's limited rebuttal" also contains factitious citations, despite Plaintiff's verification that "the authorities cited herein are true and correct to the best of her knowledge and belief, and that all references accurately reflect the legal sources relied upon in this filing."  …

Plaintiff and Defendant's filings with factitious citations are stricken from the record and will not be considered. The Court will also strike filings that purport to be exhibits to filings with factitious citations….

Plaintiff and Defendant have repeatedly lied to the Court and wasted both the Magistrate Judge and the undersigned's time. It is apparent that further sanctions are necessary. Plaintiff and Defendant are ordered to show cause why they should not be sanctioned for repeated failure to follow Court orders and for dishonesty to the Court…. Sanctions may include:

• Monetary sanctions in the form of $100.00 per filing with a factitious citation, payable to the Court;
• A prohibition on future filings until the monetary sanction is paid to the Court;
• Requiring both Plaintiff and Defendant to appear in Court, in person, for a hearing on the record explaining their factitious citations and failure to follow Court orders, and
• Revoking both Plaintiff and Defendant's Pro Se Upload privileges, which would require them to file documents either by mail or by hand-delivery to the Clerk's office….

Comments

[Davy C]

How does a pro se litigant, whether using AI or not, verify that a particular precedent is still good without incurring significant expense?

[Stupid Government Tricks]

Indeed. One might almost think the legal system is designed to be as opaque as possible. Me, I'd get rid of all precedent and require every dispute be argued from first principles, ie, what does the Constitution say, what does the specific law say.

Even the way Amendments are listed with the Constitution is misleading. Start from the beginning with the Constitution, then come across the 12th Amendment and have to go back and see what it changed. Ditto for the 17th, 20th, 22nd, 25th. Why not revise the text directly? Read the 18th Amendment, then find the 21st changed it. Imagine reading a biology book which starts with Aristotle or some other ancient Greek, and piles up progress independently, so you read Lamarck as if true, then Darwin, when all you want to know is the current state.

Lawyers ruin everything.

[David Nieporent]

Indeed. One might almost think the legal system is designed to be as opaque as possible. Me, I'd get rid of all precedent and require every dispute be argued from first principles, ie, what does the Constitution say, what does the specific law say.

Chesterton called; he wants his fence back.

[Bunny495]

In addition to my wasting time commenting here I am also involved with the FSU Film School as a bit part actor. Recently I played a lawyer in a film about a young black female lawyer who had just graduated from Howard law school and was working for a firm dominated by older white male lawyers (most of them dissed her).

In the only scene I was in the head older white lawyer (who I knew from acting in a different Indie film) who was mentoring the young black female lawyer turned to me, nodded, and asked if I had anything to add to which I replied "I will be staying up late burning the midnight oil using Westlaw".

Problem with that was I had not read the entire script and the canonical scene was the young black lawyer pulling law books off the shelf, thumbing through them with almost cartoon like balloons of her thoughts flashing across the screen till she found the right site to win the case. Sad to say this did not fit well with my ad lib line which was cut from the final film.

So to answer your question (as I have posted many times before) Econ 101 teaches resources are rationed by money or time. The expense for a pro se litigant is in time.

[Kenneth Almquist]

The court order required the litigants to certify that they have verified that, “each citation is accurate and stands for its asserted proposition.” To do that, they need to read the cases they cite, and to check that the case wasn’t overturned on appeal.

[Davy C]

Ah. Seems like the plaintiff put more in her declaration than the court required.

[wvattorney13]

Google Scholar has most recent cases for free.

I think the issue with pro se litigants is that they believe that asking ChatGPT IS verifying the case. They have no experience otherwise.

[Kleppe]

Lawyers are, by and large, nothing more than rent-seeking assholes. They add no value to society as their raison d'être is to "protect" their clients from the consequences their brethren lawyers are seeking to impose on them.

e.g., "A lawyer in the district attorney's office is charging me with violating a law that lawyers in the state legislature wrote, and now I must hire a lawyer who is a defense attorney to protect me. If I am lucky, the lawyer who is a judge will only sentence me to a fine."

What a racket.

[Drewski]

Among other things, you are extremely unclear on what "rent-seeking" is.

[Stupid Government Tricks]

If you're a lawyer, it's pretty rich you complaining about not understanding economists' peculiar jargon.

[Stupid Government Tricks]

That it is, and a protection racket to boot. It's always amusing how immortal monopoly government uses its monopoly lawyers and judges to beat up on businesses whose innovations provide temporary natural monopolies which offend the monopoly guardians of morality.

[Drewski]

Why use a phrase you don't understand? You literally have no idea what you're saying when you do. (I know why Kleppe uses phrases he doesn't understand, though. It's because he's not very bright and is mentally unstable.)

[TwelveInchPianist]

Is it possible they misunderstood what factitious means?

[Absaroka]

It was a new word for me, and after looking it up it the first few definitions made fictitious seem like a better choice. But this definition seems apropos:

"factitious - not real but created deliberately and made to appear to be true"

[TwelveInchPianist]

Huh. Then it looks like maybe the citations are not factitious and didn't violate the judge's order.

It's usually better to be clear. I'm sure everybody already knows that the judge went to college.

[Drewski]

The judge found as fact that specific citations were factitious. For purposes of the case, they are.

[Bunny495]

One more time.

Westlaw has a $US1,600 a month single use license for it's AI addon. I have asked too many times to remember if anyone here uses it, knows about it, and if so how well it works.

I find it strange that personally I pay $US20 a month for the low tier Gemini AI LLM because it is worth a double sawbuck to me, yet no other poster here is willing to admit whether they use el cheapo AI or are willing to pay for better results. I am also aware of peeps who are paying way more than that for specialized AI engines.

Bottom line is even at it's current stage of development peeps (including me) finding it worthwhile to pay for upgraded AI. None of EV's posts on the topic of "bad AI" have detailed just which AI client was used and which tier of AI (paid or free) was used.

Point is that (just as with lawyers) not all AI engines are equal and as every internet knows YMMV.

[Soronel Haetir]

I sometimes use free local-only engines for working on fiction. Right now the only thing I'll use it for is suggesting places language should be changed (verb forms and the like). Even tracking conflicting descriptions is mostly still beyond it, particularly with wide separation.

[Bunny495]

I spend a lot of time hanging out with peeps at the FSU Film School. It is one of the better ones with good connections to Hollywood. I have spoken to well known directors and producers. The most common response they have to AI's involvement in cinema is they are "scared shitless". No question I can get a basic script and then format it to Hollywood standards from multiple AI engines that are better than 70% of what are submitted to the big studios.

I can still remember when WordPerfect wiped out secretary pools in big law offices. No question AI will only take over more jobs.

[JonFrum]

This would stop if it cost $10K per violation. And suspension from being able to practice for a year for a second offense. $100 is Starbucks money.

[ReaderY]

They are both proceeding pro se. it’s quite possible neither can afford a lawyer.

[David Nieporent]

Ah. Missed that. My thought also was that $100 was ridiculously light. But not for pro ses.

[TwelveInchPianist]

Additionally, Defendant cites “Smith v. Suburban Bars, Inc., 374 N.W.2d 480 (Mich. Ct. App. 1985).” (ECF No. 78, PageID.1012.) The Court is unable to identify or locate this case.

Interestingly (to me), there is an unrelated case called "Smith v. Suburban Restaurants, Inc. 374 Mass. 528 (1978)..."

[ReaderY]

Does the court have the power to order both sides to get lawyers?

It seems quite likely neither side has the ability to carry out the court’s orders. Indeed, neither may be able to even understand them. They may both just be feeding whatever the judge writes into the AI and then sending in whatever the AI writes back.

That said, if neither side can afford a lawyer, then effectively the defense wins. But because the plaintiff has the burden of making a prima facia case, perhaps, in a situation where neither side is capable of doing this, that’s not a completely unjust outcome.

[Zarniwoop]

I have to think that at this point the judge is asking herself “can both of them lose?”

[wvattorney13]

I think the judge needs to order "no more AI" for this case. Then it would be a clear violation that cannot be couched in terms of ignorance.

[Stupid Government Tricks]

How would the judge detect such a "clear violation" if it had been vetted properly? Couldn't. In other words, there is nothing to gain from such an order.

[neurodoc]

How about:
- court imposed sanctions for these failures to ensure non-hallucinated citations included required remedial legal research training and testing?
- a bounty system was instituted whereby fines to be paid by the responsible sides for phony citations would be used to pay those who discovered them and brought them to the courts attention.