Lynne Kiesling on Predicting Elections and Financial Markets
Human beings naturally look for patterns in the mess of events and data that surrounds us. Groping for hidden architecture is an evolutionary response to a complex world. In general it serves us well, but we risk detecting patterns where none actually exist. Sometimes we can learn after the fact that our pattern-based predictions were incorrect, and we update and move on, ideally with more humility and an updated mental model for the future. But biases often persist even after correction, especially when the subject of our attention is something with deep emotional roots, like the predicted outcome of an election.
Given the power of pattern recognition and our inherent biases, how do we get it right? In a dual review of Nate Silver's The Signal and the Noise and James Owen Weatherall's The Physics of Wall Street, Lynne Kiesling investigates.