The Volokh Conspiracy
Mostly law professors | Sometimes contrarian | Often libertarian | Always independent
"Counsel Relied upon Unvetted AI … to Defend His Use of Unvetted AI"
Plus, "I don't know how you can vehemently deny that when the evidence is staring us all in the face. That denial is still very troubling to me."
From Ader v. Ader, decided last week by N.Y. trial court judge Joel M. Cohen:
This case adds yet another unfortunate chapter to the story of artificial intelligence misuse in the legal profession. Here, Defendants' counsel not only included an AI-hallucinated citation and quotations in the summary judgment brief that led to the filing of this motion for sanctions, but also included multiple new AI-hallucinated citations and quotations in Defendants' brief opposing this motion. In other words, counsel relied upon unvetted AI—in his telling, via inadequately supervised colleagues—to defend his use of unvetted AI….
Plaintiff [had] identified inaccurate citations and quotations in Defendants' opposition brief that appeared to be "hallucinated" by an AI tool. After Plaintiff brought this issue to the Court's attention, Defendants submitted a surreply affirmation in which their counsel, without admitting or denying the use of AI, "acknowledge[d] that several passages were inadvertently enclosed in quotation" and "clarif[ied] that these passages were intended as paraphrases or summarized statements of the legal principles established in the cited authorities." Plaintiff then submitted a letter in rebuttal to the Surreply Affirmation, to which the Court invited Defendants to respond. Defendants did not respond.
As set forth in detail in the Rebuttal Letter and the papers in support of the instant motion, defense counsel's explanation of the citation and quotation errors as innocuous paraphrases of accurate legal principles does not hold water. Among other things, the purported "paraphrases" included bracketed terms to indicate departure from a quotation (not something one would expect to see in an intended paraphrase) and comments such as "citation omitted." Moreover, the cited cases often did not stand for the propositions quoted, were completely unrelated in subject matter, and in one instance did not exist at all.
Even as to the subset of fake quotations that happened to be arguably correct statements of law, "the court rejects the invitation to consider that actual authorities stand for the proposition that bogus authorities were offered to support." Use of fake citations and quotations that may generally support "real" statements of law is no less frivolous. Indeed, when a fake case is used to support an uncontroversial statement of law, opposing counsel and courts—which rely on the candor and veracity of counsel—in many instances would have no reason to doubt that the case exists. The proliferation of unvetted AI use thus creates the risk that a fake citation may make its way into a judicial decision, forcing courts to expend their limited time and resources to avoid such a result.
Unfortunately, the summary judgment briefing is not the end of the troubling conduct. Despite previously assuring the Court in defense counsel's Surreply Affirmation during the summary judgment briefing that "[e]very effort will be made to avoid the recurrence of such issues and to maintain the clarity and integrity of the submissions before this Court", Defendants' opposition to this sanctions motion contains another wave of fake citations and quotations. This time, Plaintiff has identified more than double the number of mis-cites, including four citations that do not exist, seven quotations that do not exist in the cited cases, and three that do not support the propositions for which they are offered. Separately, although not the subject of this motion, Plaintiff has also alerted the Court of still more fake citations in Defendants' opposition to Plaintiff's application seeking attorneys' fees in connection with the award of summary judgment.
In response to the overwhelming evidence skillfully marshalled by Plaintiff's counsel in the instant motion, Defendants "vehemently den[ied] the use of unvetted AI" and complained that "Plaintiff provides no affidavit, forensic analysis, or admission from Defendants confirming the use of generative AI" thus implicitly suggesting that AI was not used in preparing Defendants' summary judgment brief. At oral argument, the Court gave defense counsel the opportunity to set the record straight as to how exactly the many citation and quotation "errors" found their way into Defendants' briefs in this action. Defendants' counsel began with a prepared statement in which he acknowledged "some citation errors," but continued initially to maintain that "the cases are not fabricated at all."
Ultimately, however, upon questioning by the Court as to how a non-existent citation could possibly end up in the summary judgment briefing without the use of AI, counsel conceded that he "did use AI," contending that he "did verify and check the AI" but "must have missed" the false citation, which he conceded (for the first time) was an AI hallucination. With respect to incorrect quotations from actual cases, counsel acknowledged (also for the first time), under Court questioning, that they were not his own drafted "paraphrases" from the decisions but were instead AI-generated fake quotations that were not properly verified.
Turning to the inclusion of AI-generated false citations and quotations in Defendants' brief opposing the instant motion for sanctions, counsel began by indicating that he viewed the sanctions motion "as a minimal matter" relative to the other time-sensitive tasks on which he and his recently hired staff were working at the time. Ultimately, counsel conceded that a number of incorrect citations were AI-generated and not properly checked by lawyers he brought in to assist him. Indeed, he indicated that he identified at least one of the hallucinated citations during the drafting process and instructed his team to remove it from the brief, which they failed to do. In the end, it is undisputed that counsel signed a brief that contained the identified hallucination among several others.
Counsel expressed remorse for what occurred in this case, and at some points took responsibility (see e.g., id. at 15 ["[the false citation] was not removed. I take full responsibility for it being in this brief, and it had to be AI-generated. I believe it was"]; 17-18 ["I just want to explain my actions. And, obviously, some of this is sanctionable and extremely embarrassing and humiliating. I want to explain that I brought on additional staff because of the complexity of both of these matters. I was assured that all the quotations were 100 percent accurate. I went through the table of contents in each case to make sure they actually existed. I learned only after receiving the reply from opposing counsel that, in fact, there were inaccuracies"; 19 ["And, your Honor, when I say I told the staff, I take full responsibility. It's my staff, so I told myself to get rid of it and I did not get rid of it"]; 20-21 ["Your Honor, I am extremely upset that this could even happen. I don't really have an excuse. Here is what I could say. I literally checked to make sure all these cases existed. Then, you know, I brought in additional staff. And knowing it was for the sanctions, I said that this is the issue. We can't have this. Then they wrote the opposition with me. And like I said, I looked at the cases, looked at everything; so all the quotes as I'm looking at the brief—and I thought it was a well put together brief. So I looked at the quotes and was assured every single quote was in every single case, but I did not verify every single quote , , , . When I looked at—when I went back and asked them, because I looked at their [reply brief] last week preparing for this for the first time, and I asked them what happened? How is this even possible because, you know, when you read the opposition, I mean, it's demoralizing. It doesn't even seem like, you know, this is humanly possible"]).
On the other hand, counsel later muddled those statements of contrition by asserting: "I just want to clarify that I never said I didn't use AI. I said that I didn't use unvetted AI. I just want to be clear that I believe that I had checked everything. Number two, I did say that they were intended to paraphrase because in that rush of six days [to file the summary judgment brief] we made the decision that, hey, if this approximates this case, then let's remove the quotes. It's a paraphrase. Number three, retaliation was the reason for the immediate non-contrition. I thought, again, that the 800-lawyer firm [representing Plaintiff] was upset that I made a motion to have them relieved as counsel. I just wanted to point that out", to which the Court responded: "I'm not sure how that helps anything. The idea of unvetted AI, it sort [of] speaks for itself. If you are including citations that don't exist, there's only one explanation for that. It's that AI gave you cites and you didn't check them. That's the definition of unvetted AI. I don't know how you can vehemently deny that when the evidence is staring us all in the face. That denial is still very troubling to me."
{The troubling denial to which the Court referred is on page 12 of Defendants' brief in opposition to the instant sanctions motion and extends beyond the use of "unvetted" AI to suggest that AI was not used at all: "Plaintiff's speculation that Defendants' brief was generated using 'unvetted artificial intelligence ('AI') application' is purely speculative, unsupported by any evidence, and intended solely to cast Defendants' counsel in a negative light without factual basis. Defendants vehemently deny the use of unvetted AI. The errors, as explained, were human errors resulting from time constraints and the inadvertent misapplication of quotation marks to paraphrased content. Notably, Plaintiff provides no affidavit, forensic analysis, or admission from Defendants confirming the use of generative AI, nor do they show that any citation or quote was submitted with the knowledge that it was false. Absent such proof, accusations of bad faith or knowing deception fall flat under Rules of Professional Conduct Rule 3.3(a)(1)" (emphasis added).}
The court "award[s] Plaintiff her reasonable costs and attorney's fees incurred in connection with the sanctions motion, together with such fees attributable to addressing Defendants' unvetted AI citations and quotations in the summary judgment motion," and "directs Plaintiff's counsel to submit a copy of this decision and order to the Grievance Committee for the Appellate Division, First Department and the New Jersey Office of Attorney Ethics, copying Defendants' counsel and this Court on its transmittal letters. The Court will provide a copy of this decision and order to Judge Katz, who is presiding over a matrimonial matter in this Court in which Defendants' counsel is representing [the plaintiff]."
