Expert Predictions Often Aren't

Just fresh off of my own stab at making predictions, I note that Louis Menand has a fascinating review in The New Yorker of University of California-Berkeley psychologist Philip Tetlock's new book, Expert Political Judgment: How Good Is It? How Can We Know?. Tetlock signed up 284 experts and asked them to make predictions about various events over a 20 year period. According to Menand, Tetlock found,

"that people who make prediction their business--people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtable--are no better than the rest of us…Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert's predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote. Our system of expertise is completely inside out: it rewards bad judgments over good ones."

Whole thing here.