MENU

Reason.com

Free Minds & Free Markets

Does Correcting Politicians Make Them More Truthful?

PolitiFactPolitiFactAn new study suggests that, contrary to what you might think, pointing out when politicians are lying (or mistaken) may encourage them to be more careful with the facts. The experiment, described in a working paper by political scientists Brendan Nyhan of Dartmouth and Jason Reifler of Georgia State University and summarized in a report published by the New America Foundation, involved randomly assigning 1,169 state legislators to one of three groups: those who received a letter "reminding them of the risks to their reputation and electoral security if they are caught making questionable statements," those who received no letter, and those who received a letter "stating that we were monitoring campaign accuracy." The third, "placebo" condition was included to control for the possibility that merely knowing they were being studied would cause legislators to behave differently.

After sending the letters, Nyhan and Reifler looked at how many legislators in each group "received a negative rating from the PolitiFact affiliate in their state" or were criticized for inaccurate statements in articles indexed by Nexis during a three-month period in 2012. (The articles were coded by a research assistant who did not know to which groups the subjects had been assigned.) Nyhan and Reifler found that the legislatiors in the "treatment" group were substantially less likely to be called out than the legislators in either of the other two groups. "In all," they report, "21 legislators in the placebo and control conditions had the accuracy of their claims questioned by PolitiFact or in Nexis (2.7%), compared with 4 in the treatment condition (1.0%)." In other words, the legislators reminded of the potential fallout from independent fact checking were about one-third as likely to have their claims questioned as the legislators who were not. Although the numbers criticized for misstatements were small in all the groups, this overall difference was "highly significant (p < .02)."

Nyhan and Reifler sum up their results this way:

In the first field experiment of its kind, we find that the randomized provision of a series of letters highlighting the electoral and reputational risks of having questionable statements exposed by fact-checkers significantly reduced the likelihood that legislators in nine U.S. states would receive a negative fact-checking rating or have the accuracy of their claims questioned publicly. We found no evidence that these results were driven by legislators speaking less frequently or receiving less coverage, suggesting instead that they were less likely to make inaccurate statements rather than being silenced more generally.

They conclude with a plea for more fact checking:

These results indicate that fact-checking should not be discredited by the continued prevalence of misinformation and misperceptions. While fact-checking may be ineffective at changing public opinion, its role as a monitor of elite behavior may justify the continued investment of philanthropic and journalistic resources. Indeed, given the very small numbers of legislators whose accuracy is currently being questioned by fact-checkers or other sources, one could argue that fact-checking should be expanded in the U.S. so that it can provide more extensive and consistent monitoring to politicians at all levels of government.

As much as I'd like to believe that calling attention to error makes it less common (a hypothesis for which my own career provides only limited support), one could also argue that what we need is more generic letters of the sort sent by Nyhan and Reifler, which apparently had an impact even though the actual risk of being criticized for inaccuracy was very low. Elegant as their experiment is, it does not actually show that the experience of being publicly corrected makes politicians more accurate. It might instead cause them to dig in their heels, especially if they (rightly or wrongly) perceive the criticism as unfair or ideologically motivated. In fact, the measures Nyhan and Reifler use for accuracy—PolitiFact ratings and criticism in the mainstream press—may be so skewed that what they are actually measuring is adherence to a certain set of beliefs deemed acceptable by the folks doing the fact checking.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • Brett L||

    So Politifact has been testing both cases by calling out Republicans and not Democrats? And here I thought they were just partisans.

  • Bam!||

    But then who will fact check the fact checkers?

  • Hopfiend||

    I dunno, Coast Guard.

  • Greg83||

    "In fact, the measures Nyhan and Reifler use for accuracy—PolitiFact ratings and criticism in the mainstream press—may be so skewed that what they are actually measuring is adherence to a certain set of beliefs deemed acceptable by the folks doing the fact checking."

    No shit. I just laugh every time someone sends me a link to factcheck.org to try and prove their case, as if I should disregard that the site is underwritten by a far left organization because it has "fact" in the title.

  • Bee Tagger||

    as if I should disregard that the site is underwritten by a far left organization because it has "fact" in the title.

    This is why, before I run for office, I am going to legally change my name to Science McFair. I'll be sure to include a naked picture of myself on the name change form so that it can never be released to the press.

  • pmains||

    Media Matters is the same way. They present themselves as impartial judges of truth, but they're all Democrat partisans. How does that not end up in sentence one of every article about them?

    "Media Matters, the leftist pressure group and propaganda machine, took issue with ...."

    My other pet peeve is the way news organizations use the words, "blast," "slam," etc. to describe political criticism. When Democrats criticize Republicans, it's an epic smackdown. Turn the tables, and the language use completely changes.

  • Hopfiend||

    "How does that not end up in sentence one of every article about them?"

    Rhetorical question.

  • Auric Demonocles||

    I bet alt-text would make them more truthful.

  • Dr. Frankenstien||

    Let me be clear. I will only express vague assertions and contentless bromides.

  • Auric Demonocles||

    Now that the requisite alt-text crusading is out of the way:

    An new study suggests that, contrary to what you might think, pointing out when politicians are lying (or mistaken) may encourage them to be more careful with the facts.

    Why would that be counter to what you would think?

  • Marty Feldman's Eyes||

    I would think the lessons politicians would take away from being called out for being wrong on factual claims, would be to simply remember to be more vague next time. Politicians never get punished for avoiding specifics.

  • Adam330||

    Wait, how does this study show that fact-checking makes politicians more honest? The difference between the groups was whether they got a private letter and the content of the letter, not whether they were publicly fact checked.

  • PapayaSF||

    PolitiFact was so obviously partisan in the last election that it was laughable. Romney said something about a program costing "up to $XX billions" and it was rated as "False" because he had used the (official, from the government department) high estimate. Yeah, that's what "up to" meant.

  • Eduard van Haalen||

    I heard a joke which I believe should be enacted in reality.

    Send letters to random politicians saying - "they've found out what you did! Resign at once!"

  • Eduard van Haalen||

    I think in the original joke it was "all is discovered - flee at once!"

    And the next day every member of Congress went missing.

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online