Crowdsourced Amateurs Outperform CIA at Predicting World Events
Elaine Rich is a pharmacist in her 60s. She and a team of 3,000 other amateur forecasters in the Good Judgment Project (GJP) use Google to keep current on the news. The Central Intelligence Agency (CIA) employs over 20,000 professionals, operates with an annual budget north of $14 billion, and has access to oodles of classified information.
Which of these groups is better at predicting world affairs?
When it comes to "everything from Venezuelan gas subsidies to North Korean politics," reports National Public Radio (NPR), amateurs outperform the pros. Rich, in particular, has "been put on a special team with other superforecasters whose predictions are reportedly 30 percent better than intelligence officers." NPR explains how this is possible:
"Everyone has been surprised by these outcomes," said Philip Tetlock, one of the three psychologists who came up with the idea for the Good Judgment Project. The other two are Barbara Mellers and Don Moore….
But also, if you take a large crowd of different people with access to different information and pool their predictions, you will be in much better shape than if you rely on a single very smart person, or even a small group of very smart people….
"There's a lot of noise, a lot of statistical random variation," Tetlock said. "But it's random variation around a signal, a true signal, and when you add all of the random variation on each side of the true signal together, you get closer to the true signal."
The GJP has been operating for about three years. Tetlock's team provides people like Rich with some basic training in probability estimation, and then they're good to go.
This network of folk forecasters isn't likely to supplant the CIA, but it is looking to make changes in the way the intelligence community operates. The GJP blogged this week that "for many geopolitical forecasting questions, we see promise in a human-machine hybrid approach that combines the best strengths of human judgments and statistical models."
