The Volokh Conspiracy
Mostly law professors | Sometimes contrarian | Often libertarian | Always independent
"Houston Housing Authority Cited over a Dozen Cases in a Legal Brief. Almost None of the Quotes Exist."
Houston Chronicle (R.A. Schuetz) reported yesterday:
In a lawsuit over whether a woman should have lost her housing subsidy, the Houston Housing Authority's lawyer asked a judge not to force the agency to prevent the woman's eviction while the case was being decided. The brief, submitted by a law firm that frequently represents cities and agencies in the Houston area, cited over a dozen cases in support of its argument.
The only problem? Almost none of the quotes actually exist, a Chronicle analysis shows….
The firm's managing attorney "said in an email that because the court required the brief to be filed within a short timeframe, the quick turnaround 'prevented our usual multi-attorney review.'"
But he did not address why 11 of the 13 cases directly quoted did not actually contain those quotes or why many did not seem related to what his firm had quoted them as saying. He also did not respond when asked if artificial intelligence, which is known to "hallucinate," or say things that are not true, had been used to draft the brief.
Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Comments may only be edited within 5 minutes of posting. Report abuses.
Please
to post comments
It doesn't matter if he used AI or not.
You make up cases, you should be disbarred. Is is simple fraud.
LongToUnderstandSpecificIntent
Ironic. Volokh is posting about fictitious citations. The entire practice of law is based on fictitious, fake, supernatural doctrines. The legal word, constructive, means, made up lawyer shit.
https://www.legalbriefai.com/legal-terms/constructive
Volokh believes in mind reading, the forecasting of rare accidents, and that standards of behavior should be based on the feelings of a fictitious character. Why fictitious? So the standards can be objective, of course. This character is a thinly disguised avatar of Jesus, in violation of the Establishment Clause.
Decisions about the death penalty are based on these. Then, $trillions are transferred from productive entities to worthless parasites by the toxic lawyer profession. Not only does Volokh believe in these ridiculous ideas, but he indoctrinated thousands of intelligent young people into them on pain of flunking his courses and of ruining their futures.
Constitutions and bills of rights are really shitty things to have, aren’t they? People would be so much better off if we could just have a real leader, a Kim Jung Un, a Pol Pot, a Mebute Sese Seko, a Stalin, a Hitler, a real leader who could just line folks up against the wall and shoot them, without all this toxic lawyer crap getting in their way.
Eugene Volokh: Crypto-Theocrat. I KNEW it!
"Volokh believes in mind reading"
(Extended snickering)
Submitting false citations should be a strict liability offense.
"Is is simple fraud."
Is is??
Was it supposed to be "It is"?
Either way, it is not fraud (at least without more evidence). Laziness and incompetence are different from fraud.
I'll use that argument the next time I pass counterfeit money. I was just too lazy or incompetent to check.
Not sure why you think that is a witty riposte. That would be a defense to a charge of passing counterfeit money,
That's exactly why most people don't get charged for using counterfeit money. You, personally, would probably not benefit from that since you announced your plan in advance, in writing, on a public forum.
These days, who knows?
I yield to the superiority of the spell checkers.
Hear! Hear!
P.K. Dick: We remember it for you, wholesale
Modern AI: We make it up for you, wholesale.
IMO, any court that receives a filing that has hallucinated (non-existent) cases should require a full explanation from the attorney(s) that submitted the filing as to how it occurred, and serious court sanctions can, and should, be imposed as a deterrent.
I also believe that state bars should discipline offenders harshly.
At this point in time, yes. When it first became a public issue two (?) years ago, it was reasonable to grant leniency. But no longer can any attorney legitimately claim ignorance.
Yes, time to get back to lawyers doing there own lying and misrepresentation to the courts.
Your casual dismissal of the integrity and ethics of the majority of attorneys is sad, albeit expected.
Yes, lying and misrepresentation (aka, lack of candor) does exist, and it really makes most attorneys and most judges very angry- because it goes against the core ethics of the profession, and impedes the proper functioning of the court. Any officer of the court that thinks that lying is ever acceptable ... they are just doing it wrong.
Personally, I think that the following two things are true:
1. State bars should punish lack of candor much more aggressively and much more harshly.
2. Anecdotally, I have seen the same rot in this profession that I have seen in society- which is to say, a genuine disdain for the truth.
I'll let you decide how you view 2, generally.
Good cops who refuse to help expose and get rid of bad cops are not good cops.
Good lawyers who refuse to help expose and get rid of bad lawyers are not good lawyers.
All your excuses for good lawyers not helping get rid of bad lawyers put you squarely in the bad lawyer corner.
I like how people randomly post rants that are in no way responsive to the comment to which they're replying.
I like people who have no reading skills telling me what I didn't write.
"Your casual dismissal of the integrity and ethics of the majority of attorneys is sad, albeit expected."
You read "majority" into what I said.
As one bad apple can spoil the barrel, one bad lawyer can spoil the perception of many.
I agree with both of your takes and would add that I wish the courts would be less forgiving of transgressions; especially when they involve prominent attorneys and firms.
90% of lawyers make the other 10% look bad.
I subsequently looked this up; Mata v. Avianca, which was the case which brought this issue to public prominence, was almost exactly two years ago.
Can they deny the filing and proceed as if the party was nonresponsive?
That would be one available sanction, yes. Courts are reluctant to do that because they prefer cases to be decided on the merits, not on defaults. (It also punishes the client for the sins of the lawyer.) But it's a possibility.
My guess is the client knew whom they were hiring.
I agree, I think we've reached the point where attorneys have fair notice of these issues with hallucinated cases and they can't be excused away as just simple error. Having to fully explain yourself should be a minimum requirement if you're found citing a hallucinated case, and I think contempt or Bar sanctions should be used more often in such situations unless there's a *very* good explanation offered.
I'm waiting for a [ro se party to say "Your honor, his cases don't exist."
Former Libertarian vice presidential nominee Mike ter Maat and dozens of other members of the third party have joined a caucus within the Republican Party, hoping to usher in fiscally conservative policies.
Ter Maat, 2024 Libertarian presidential candidate Chase Oliver‘s running mate, left the Libertarian Party with over 40 members who have joined the Republican Liberty Caucus, a political organization working with the national party to promote limited government and free markets.
How f'ing difficult is it to verify that that cited authority is still good law or even exists?
As the search engines start rolling out AI 'assistance', I expect it's going to get harder, because the search engines will start 'finding' the same things the AI's hallucinated. And because the search results are bad enough you can't really assume that some cite not showing up in them is proof it isn't real.
It will really get bad when clicking the hallucinated link brings up a hallucinated page.
It doesn't matter, because a lawyer is not allowed to "assume" anything; when submitting a brief s/he has to actually read the case — not an AI summary of it — to determine not merely that it is real, but that it says what s/he thinks/hopes it says. And then has to Shepardize it to make sure that it's good law.
"And then has to Shepardize it to make sure that it's good law."
I once had a case that I won on summary judgment. After the MSJ hearing, we were allowed a supplemental brief of authority on an issue raised during the hearing when the other side brought up a case during the hearing. The supplemental brief was very short, as the case that was brought up was not good law and had been withdrawn (!!) for the exact reason it was cited at the hearing- because it was a misstatement of the law.
The coda? The trial court's order was appealed, and the other side brought in new appellate counsel. They regurgitate the arguments already used, and ... they also used the same case because they apparently didn't read the record until the end.
The fastest PCA I've ever received.
+1. And you have to make sure that the facts of the case you are citing are applicable to the current case.
I can pluck a nice sounding quote out of a case but when you put it in context it applies more to my opponent than me. A good lawyer on the other side is going to jump all over that. A copy/paste job out of a real case isn't good enough.
Representing the government in this kind of penny-ante dispute is the shittiest of shitlaw, whoever did this is likely entry level and overwhelmed. After the 14th hour of churning shitpaper, quality slips. Even in biglaw, stuff can slip through if the hours are long enough.
If it is so easy to check the cites, then the opposing party can do it, and there is nothing to complain about.
I don't understand the quick turn-around time excuse. IANAL but I thought WestLaw/LexisNexis had those look-ups a few keystrokes away. It shouldn't take a multi-attorney review to understand that any autogenerated citations need to be checked before submission. Amazed there are attorneys who do not understand that technology is not full-proof. Is it because they cynically hope no one will ever bother checking? Makes you wonder how much other fraud the profession has tolerated previously, that went unnoticed because it was less egregious.
The simple technical solution is for AI to flag any citation that it inserts. Then the humans know exactly what to double check.
full-proof??? I hope that's intentional...it definitely brought a smile to my face.
What's been really revealing about all this is that I would have assumed this was a problem for smaller firms rather than bigger ones. When one hires a biglaw firm with its $500/hour lawyers, one expects that one is getting quality work for its money. Apparently not.
Not that any lawyer has any excuse for it. But still, there's a reason major clients pay $1000/hour for a biglaw partner rather than 250/hour for the kid who just hung out his own shingle.
1) Major clients hire biglaw firms with $1,000/hr (ha! that's low for NYC) partners… and then demand that those partners delegate most of their work to the cheaper associates in the name of budget.
2) Most municipalities don't hire biglaw, and this is no exception. This wasn't biglaw:
I'm a bit puzzled about the "usual multi-attorney review," though, because Kevin Fulton, of Fulton Law Group, is the only lawyer at Fulton Law Group.
https://www.fultonlg.com/
And as long as I'm kicking a guy when he's down, his website says "Our principal office is located in Houston, Texas," but that appears to be their only office, not their principal one, and it appears to be a virtual office.
He probably has contract (non-employee) attorneys that he gives piece work to at like $50/hr.
Big Law economics have really changed have really changed since the 2007-08 financial crisis. I see partners billing hours that would have been unthinkable back when I was an associate. And staffing is much leaner -- usually a partner and a (very) junior associate, with each billing about the same number of hours. No more partner > senior associate > midlevel associate > first year associate teams, where the senior and mid-level associates account for 90% of the final bill.
That was a general comment not directed to this specific case. Last time the subject came up it was Morgan & Morgan, which is biglaw.
And yes, I'm well aware of the dynamic of partners giving the actual grunt work to associates, but still, the expectation is that somebody is reviewing their work. There's an old joke about two partners discussing whether sex with their wives fit under the heading of work or pleasure. They decided to ask an associate, who responded, "That's easy. If it were any work at all, you'd have me doing it for you."
"That was a general comment not directed to this specific case. Last time the subject came up it was Morgan & Morgan, which is biglaw."
Morgan & Morgan is a very large law firm.
I would not say that Morgan & Morgan is "BigLaw."
IMO, there is a very big distinction between those two concepts.
Agree. This stuff is pretty inside baseball, but "BigLaw" conventionally refers to large, full-service law firms primarily servicing corporate clients. In my time, they would have been called "White Shoe" firms, which I believe was originally coined to indicate "No Jews", but by the time I was a young associate in the '90s, all the firms were fully "integrated."
M&M, by contrast, is AFAIK just a large plaintiffs (i.e. personal injury) firm.
One benefit of this post is that the linked story provided this link to a database of court cases where AI hallucinations were included:
https://www.damiencharlotin.com/hallucinations/
It does not matter if the AI hallucinated this.
Lawyers are responsible for what they submit to courts in their name. It has been this way since before colonial times.
And yet there is no specific rule requiring the lawyer to check the cites.
There doesn't have to be, because there are rules requiring a lawyer not to bring or defend a proceeding unless there is a non-frivolous basis in law and fact for doing so. That means you have to check your law and fact to make sure it's correct. Asserting "there's a good faith basis in law because here's a citation I made up" is ludicrous and no different than fabricating a witness statement to support your factual claims.
That is not what is being asserted. It is more like: "there's a good faith basis in law because a research tool found what appears to be good support for it."
AI is not a "research tool," and also nothing it "found" can "appear to be" anything, because there's no way to determine whether it's "good support" without (stop me if you've heard this before) reading the case. Which the lawyer could not have done because it didn't exist.
(Or in this particular case, the case existed but didn't contain the specified content.)
As you have been told repeatedly, there is a specific rule requiring the lawyer to check the cites.