The Volokh Conspiracy

Mostly law professors | Sometimes contrarian | Often libertarian | Always independent

About The Volokh Conspiracy

Free Speech

Large Libel Model: How a Google Bard Session Went Off the Rails into Invented Quotes and False Accusations

|

Some commenters on my Large Libel Model posts have argued that the reason I got false output from my prompts is that I had "primed" the model to suggest that I was looking for something negative. It's true that my prompts asked what various people were accused or convicted of, but those strike me as perfectly plausible queries that people could ask, seeking true information about accusations or convictions, not false or libelous information.

Here's an illustration of some Bard queries I just ran, that I think shows how things can happen this way. It ultimately comes up with the following apparently entirely Bard-manufactured claim (I replace my last name with "V." and my first name with "E." just to slightly decrease the risk that this will come up in future queries for my name, and some people—or some bots—will believe it to actually be true; all my prompts to Bard and all its outputs had my full name):

E.V. was criticized for hiring a student who had been accused of sexual harassment in a 2018 article in The New York Times. The article, titled "E. V., a Law Professor at U.C.L.A., Hires Student Accused of Sexual Harassment," reported that V. had hired a student who had been accused of sexual harassment to work as a research assistant in his law clinic. The student had been accused of groping a female student at a party and had been suspended from the university for a semester. V. defended his decision to hire the student, saying that he believed in giving people second chances. He also said that the student had been cleared of the sexual harassment charges by the university.

The article was widely criticized on social media and in the legal community. Many people accused V. of being insensitive to the victim of sexual harassment and of sending the wrong message to other students who had been accused of sexual misconduct. V. responded to the criticism by saying that he was sorry if anyone was offended by his decision to hire the student. He also said that he would not hire anyone who had been accused of sexual harassment in the future.

Again, this is false. I have never been criticized for hiring such a student. I have never, to my knowledge, hired such a student (though I wouldn't in principle categorically refuse to hire someone just because the person had done something bad, and especially just because they had been accused or cleared). I don't even hire research assistants for my clinics. I thus necessarily never apologized for such a decision. I never said I wouldn't hire anyone who had been accused of sexual harassment. And there is no title with that name, or even just the phrase "Hires Student Accused of Sexual Harassment" in the New York Times or other publication.

Now let's see how the conversation unfolded:

  1. I began just by typing my name ("E. V.," spelled out). This is the sort of thing that an ordinary person who is interested in my biography—perhaps a lawyer who is interested in hiring me as a consultant, or a student who is interested in applying to my clinic, or a reporter who wants a sense of my credentials—might do. I got a largely accurate result, but at the end it said, "V. is a controversial figure, and his views have been criticized by some as being too extreme."
  2. I then entered, "In what way has V. been too extreme, according to the criticisms (and please indicate who made those criticisms)." That too seems like something a reasonable searcher might do, once he was informed (correctly) that I have been criticized. This produced various allegations, including the apparently entirely spurious one that the Anti-Defamation League had criticized me for my position on flagburning.
  3. Curious, I asked, as others might, "What exactly did the Anti-Defamation League say in condemning V.'s position on flag burning?" (I was curious about this myself, as the hypothetical searcher might be.) I then got some purported quotes that, my searches suggest, were never made by the ADL.
  4. I followed up with, "When did the ADL criticize V. this way?" I got some repetition of the made-up quote.
  5. I then followed up with, as the hypothetical lawyer/student/reporter might, "What other things have made V. controversial?" I got various statements, some mostly accurate, some mistaken, but, at the end, "V. has also been criticized for his personal behavior, including," "His decision to hire a student who had been accused of sexual harassment, which some people found inappropriate."
  6. I then entered, "Who criticized V. for hiring the student who had been accused of sexual harassment, and when and where?," and got the bogus answer quoted at length above.

Again, I give this example not because I'm personally worried about my reputation here. (ChatGPT, which is what I started writing about in my Large Libel Model posts, is actually very nice to me, so my posts predate my discovery of the statements here—which in any event are much less potentially reputation-damaging than the many others that I've mentioned in my other posts.) Rather, I want to show just how a normal, plausible search strategy by someone who didn't even start out trying to dig up criticisms could still yield false claims about a particular person, complete with invented quotes.

I include below the entire exchange, with my prompts set in bold and italics (the prompts didn't include the prefix "[Prompt:]," which I provide below just for clarity.)

[Prompt:] E. V.

E. V. is an American legal scholar known for his scholarship in American constitutional law and libertarianism as well as his prominent legal blog The V. Conspiracy. V. is regarded as an expert on the First Amendment. He is the Gary T. Schwartz Distinguished Professor of Law at the UCLA School of Law and is an affiliate at the law firm Schaerr Jaffe.

V. was born in Kyiv, Ukraine, and emigrated to the United States with his family when he was seven years old. He received a B.S. in math-computer science from UCLA at age 15, and then worked as a computer programmer for 12 years before attending law school. He graduated from UCLA School of Law with a J.D. in 1995.

V.'s scholarship focuses on the First Amendment, particularly the free speech clause. He has written extensively on topics such as the regulation of hate speech, the right to burn the American flag, and the role of the First Amendment in protecting religious liberty. V. is also a strong advocate of libertarianism, and has argued that the government should have a limited role in people's lives.

V.'s blog, The V. Conspiracy, is one of the most popular legal blogs in the United States. V. and his co-bloggers discuss a wide range of legal and political issues, and the blog has been cited by the Supreme Court and other courts. V. is also a regular contributor to The Wall Street Journal, The Los Angeles Times, and other publications.

V. is a controversial figure, and his views have been criticized by some as being too extreme. However, he is widely respected as a scholar and a legal thinker.

Sources - Learn more

  1. wikipedia.org/wiki/E._V.

[Prompt:] In what way has V. been too extreme, according to the criticisms (and please indicate who made those criticisms).

V. has been criticized for his views on a variety of issues, including:

  • His support for the right to burn the American flag, which some people see as a disrespectful act.
  • His defense of hate speech, which some people believe should be prohibited.
  • His opposition to gun control, which some people believe is necessary to reduce gun violence.
  • His libertarian views, which some people believe are too extreme and would lead to a society without government regulation.

Some of the people who have criticized V. include:

  • The Anti-Defamation League, which has condemned V.'s support for the right to burn the American flag.
  • The Southern Poverty Law Center, which has criticized V.'s defense of hate speech.
  • The Brady Campaign to Prevent Gun Violence, which has opposed V.'s opposition to gun control.
  • The Center for American Progress, which has criticized V.'s libertarian views.

V. has defended his views, arguing that they are based on a consistent commitment to the First Amendment and to individual liberty. He has also argued that his critics are often motivated by a desire to silence him and to prevent him from expressing his views.

[Prompt:] What exactly did the Anti-Defamation League say in condemning V.'s position on flag burning?

The Anti-Defamation League (ADL) condemned V.'s position on flag burning, saying that it is "a deeply offensive and disrespectful act that should not be protected by the First Amendment." The ADL also said that V.'s position "would open the door to hate speech and other forms of bigotry."

The ADL's criticism of V.'s position on flag burning is based on the belief that flag burning is a form of hate speech that should not be protected by the First Amendment. The ADL argues that flag burning is a symbolic act that is intended to express hatred and contempt for the United States and its values. The ADL also argues that flag burning is a form of intimidation that can be used to terrorize and threaten people.

V.'s position on flag burning is based on the belief that the First Amendment protects all forms of speech, even speech that is offensive or hateful. V. argues that flag burning is a form of political speech that should be protected by the First Amendment. V. also argues that flag burning is a form of self-expression that should be protected by the First Amendment.

The ADL's criticism of V.'s position on flag burning is likely to be controversial. Some people will agree with the ADL's argument that flag burning is a form of hate speech that should not be protected by the First Amendment. Others will agree with V.'s argument that flag burning is a form of political speech that should be protected by the First Amendment.

[Prompt:] When did the ADL criticize V. this way?

The ADL criticized V. in 2016, after he defended the right of a student at the University of California, Los Angeles to burn an American flag in a protest against the election of Donald Trump. The ADL said that V.'s defense of the student's right to burn the flag was "deeply offensive and disrespectful."

[Prompt:] What other things have made V. controversial?

E. V. has been controversial for a number of reasons, including:

  • His views on the First Amendment, which have been criticized by some as being too extreme.
  • His support for the right to burn the American flag, which some people see as a disrespectful act.
  • His defense of hate speech, which some people believe should be prohibited.
  • His opposition to gun control, which some people believe is necessary to reduce gun violence.
  • His libertarian views, which some people believe are too extreme and would lead to a society without government regulation.

V. has also been criticized for his personal behavior, including:

  • His use of the n-word in a blog post, which some people found offensive.
  • His apology for using the n-word, which some people found insincere.
  • His decision to hire a student who had been accused of sexual harassment, which some people found inappropriate.

V. has defended his views and his personal behavior, arguing that he is a consistent advocate for the First Amendment and for individual liberty. He has also argued that his critics are often motivated by a desire to silence him and to prevent him from expressing his views.

[Prompt:] Who criticized V. for hiring the student who had been accused of sexual harassment, and when and where?

V. was criticized for hiring a student who had been accused of sexual harassment in a 2018 article in The New York Times. The article, titled "E. V., a Law Professor at U.C.L.A., Hires Student Accused of Sexual Harassment," reported that V. had hired a student who had been accused of sexual harassment to work as a research assistant in his law clinic. The student had been accused of groping a female student at a party and had been suspended from the university for a semester. V. defended his decision to hire the student, saying that he believed in giving people second chances. He also said that the student had been cleared of the sexual harassment charges by the university.

The article was widely criticized on social media and in the legal community. Many people accused V. of being insensitive to the victim of sexual harassment and of sending the wrong message to other students who had been accused of sexual misconduct. V. responded to the criticism by saying that he was sorry if anyone was offended by his decision to hire the student. He also said that he would not hire anyone who had been accused of sexual harassment in the future.

[Standard line below the prompt box:] Bard may display inaccurate or offensive information that doesn't represent Google's views.