The Volokh Conspiracy
Mostly law professors | Sometimes contrarian | Often libertarian | Always independent
AI Prompts Used by Expert Are Subject to Compelled Discovery
From Judge Thomas O. Farrish (D. Conn.) last Monday in Conservation Law Foundation, Inc. v. Shell Oil Co.:
The defendants, Shell Oil Company and others ("Defendants"), have moved the Court for an order compelling the plaintiff, Conservation Law Foundation, Inc. ("CLF"), to produce materials on which its expert witness, Dr. Naomi Oreskes, relied upon in producing her expert witness report…. The parties … had resolved all issues identified in the letter briefs except for "their dispute concerning Defendants' request for the prompts Dr. Oreskes used in conducting her AI analysis and outputs." …
CLF first protests that artificial intelligence prompts used by an expert witness are not within the scope of discovery under Rule 26(b), but the Court disagrees. An expert witness's methodology is fair ground for discovery, and under the facts of this case, the process by which Dr. Oreskes culled down the defendants' document production into a subset to be worked with is an aspect of that methodology.
CLF then argues that the prompts are outside the scope of a Rule 29 discovery agreement between the parties, but the Court disagrees with this as well. It is true that Rule 29 permits parties to limit the scope of discovery by agreement, and that courts should enforce those agreements in appropriate cases. But before a court denies otherwise-relevant discovery based on a Rule 29 agreement, that agreement "must be quite clear." Here, CLF says that the parties had an agreement not to take discovery of each other's "expert notes, drafts, or communications needed by, and made during, the report drafting process," and at oral argument it contended that Dr. Oreskes' AI prompts qualified as "notes." But this is not so obvious as to be "quite clear," and accordingly it is not a sound basis for denying discovery that is otherwise within the scope of Rule 26(b).
CLF then argues that it has no additional responsive information to produce, and that it should therefore not be placed under any sort of order. CLF says that Dr. Oreskes did not use "prompts," but rather only applied "search terms," and it says that it has produced all such search terms.
When a requesting party seeks information within the scope of Rule 26(b), and the responding party claims to have no such information, the Court is presented with the question of whether it should nevertheless place the latter under an order to produce. "Under ordinary circumstances, a party's good faith averment that the items sought simply do not exist, or are not in his possession, custody, or control, should resolve the issue of failure of production since one cannot be required to produce the impossible[.]" "Yet this principle has been held not to apply when the requesting party has a strong reason for disbelieving the responding party's claim to have made a complete production—in other words, a reason backed up by solid evidence rather than mere suspicion." In this case, the defendants have an evidence-backed reason for doubting CLF's representation, because Dr. Oreskes' assistant, Dr. Alexander Kaurov, referenced "prompt[s]" in his declaration….
CLF is [therefore] ordered to revise its responses to any Rule 33 interrogatories or Rule 34 requests for production that call for disclosure of any artificial intelligence prompts and/or queries used by Dr. Oreskes or her team in the course of producing her expert witness report. If, after a diligent search, CLF determines that no additional responsive materials exist, it shall say so in its response, signed by the appropriate person under Rule 33 or 34. Rule 37(b) sanctions may then become available to the defendants if that representation is later revealed to be untrue.
Arnold & Porter (Melissa Weberman and L. Michel Marchand) wrote last week about the story; an excerpt from the post:
If you are retaining an expert who will use AI, treat the prompts as part of the expert's methodology from the beginning. Preserve them, understand how they are used, and assume they may need to be disclosed. Prompts that reflect a disciplined analytical approach may strengthen the credibility of the expert's work; poorly defined prompts may invite challenges to the methodology itself.
Address AI use explicitly in engagement letters, discovery protocols, and Rule 29 agreements. The parties here had an agreement that covered some of this ground, but its silence on AI tools specifically was enough to defeat CLF's position. An agreement that does not expressly address AI prompts and outputs may not protect them from discovery.
If you are opposing an AI-assisted expert, the prompts are a legitimate discovery target. They can reveal the assumptions underlying the analysis, the parameters the expert set for document selection, and any disconnect between what the AI was instructed to do and what the expert ultimately claims the documents show.
