Predicting the Future Is Hard
Building better models, from elections to financial markets
Facebook
Twitter
Tumblr
Blogger
StumbleUpon
Digg
Delicious
Reddit
Google(Page 2 of 2)
Human judgment enables a model to reflect human information, but it also introduces the potential for bias from using our perception to build models and interpret their results. This inclination can sometimes be a feature, as in weather prediction’s improvements over time by searching for and testing for patterns, or a bug, as in cases like finance and baseball, where bias can lead to less accurate prediction.
Some of Silver’s chapters cohere with the central Bayesian theme better than others do, and Silver does not consistently maintain the distinction between risk and uncertainty. Still, his skillful writing and storytelling make The Signal and the Noise an enjoyable read, even if you are not a prediction junkie. Overall it is a thoughtful, well-cited work with informative attention to detail.
Similarly, James Owen Weatherall’s The Physics of Wall Street is an engaging, well-written history of the work of physicists, mathematicians, and statisticians on modeling financial markets since the late 19th century. Weatherall, a physicist, mathematician, and philosopher, unearths research from some unjustifiably underappreciated mathematicians, and he narrates a lively story about their work while making challenging ideas easier to understand.
Some of Weatherall’s subjects, such as Benôit Mandelbrot, discovered entirely new fields of inquiry (in Mandelbrot’s case, fractal geometry and chaos theory) as they developed theories to solve concrete problems. Others, such as physicist Fischer Black, pioneered the application of physics models to complicated finance problems like options pricing. In all cases Weatherall shows that intellectual nonconformity and interdisciplinary collaboration were key to his subjects’ successes.
Weatherall’s main theme is that the methodology of physics involves developing appropriately simple models, being honest about their assumptions, testing those models, and then revising them based on their performance and/or when the assumptions are invalid. Based on this foundation, he argues that the physicists and other quants are not entirely to blame for failures to predict financial market downturns such as the recent 2008 crisis, nor even for having developed models and financial innovations that made financial markets more brittle and less resilient.
“Putting all of the blame for the 2007–2008 crisis on Li’s model, or even securitized consumer loans, is a mistake,” Weatherall writes. “The crisis was partly a failure of mathematical modeling. But even more, it was a failure of some very sophisticated financial institutions to think like physicists. The model worked well under some conditions, but like any mathematical model, it failed when its assumptions ceased to hold.”
Weatherall’s research and argument are broadly persuasive but incomplete. His account does not address the fact that physicists develop these models within a framework of human institutions, the sets of formal and informal rules that govern how individuals act and interact in financial markets and the broader economy. These institutions shape the incentives of all kinds of people—including quants and the people who employ them—in the complex network of markets.
So Weatherall’s conclusion is accurate, but the financial crisis was largely a failure of institutions and incentives that made financial markets more brittle, not solely a failure of mathematical modeling per se. While the Warren Zevon fan in me appreciates his epilogue’s invocation to “send physics, math and money!” to enable better outcomes in financial markets, it’s a prescription that overlooks the distorted incentives that existed, and persist, in financial markets.
These two books have several shared attributes that make them worth reading—lively writing that humanizes a difficult topic, attempts to understand modeling and prediction in the face of uncertainty, and application to well-examined case studies in finance, weather, earthquakes, and poker. A common theme is the danger of assuming that the risks of bad outcomes are independent of each other instead of related, especially in financial markets. Having faith that your model will work regardless of conditions leads to poor predictions and unexpected outcomes. Good modeling requires constant testing and humility, even (or especially) after a spectacularly successful election prediction.
Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time.
-
I think Silver lucked up in predicting the elections because of Romney's 47% comment, the storm (seeing government in action fixing things, Christie and Obama together, reminder of global warming), and a few other factors. Of course: the economy was showing signs improving slightly over a few years, in today's environment there is a severe distrust of rich people.
-
It is also worthy of noting that since the election, it has been revised to reveal that Obuma barely won the national election (due to the discovery of huge voter fraud in my loathsomely purple home state of Ohio). The popular vote has been revised to 51/49.
I think Nate Silver got lucky, I think Romney should have blown this bozo out of the water (in the absence of that "47%" comment in a room full of evul rich), and the screaming elephant in the room is: why the fuck does California have twice as many electoral votes as Texas? This question has to be put out on the national stage, because it allows a democrat to win the national election by taking a small handful of states.
-
California population: 38,041,430
Texas population: 26,059,203California EVs: 55
Texas EVs: 38Population ratio: 1.46
EV ratio: 1.44 -
But surely the fact that those numbers balance out quite fairly is just me having a good day. Tomorrow the nature of numbers will most certainly be different and my luck won't be so good.
-
Tomorrow California will collapse like every other collectivist "utopia", that might be just the kick in the ass America needs to start rejecting your backwards-ass "progress".
You'll also notice that anyone worth a fuck is fleeing the bankrupt sewer of California, and I will not be surprised when it turns out that the population numbers had been fudged all along.
Don't lecture me about "numbers", you fucking faggot--the ability to manipulate numbers does not change the basic nature of reality, as you and your ilk will soon discover.
-
You're making Tony look reasonable with your hyperbole and name-calling.
And people are fleeing at the margins, not wholesale -- California neither gained nor lost EVs in the last reapportionment, which means its population growth roughly mirrored the national average.
-
Huhn. States with large populations having more EVs than states with smaller populations?
What a shocker.
-
The population numbers are fudged. Nobody could possibly want to live in a coastal state with a yearlong temperate climate and perfect abs and breasts galore when they could go to fucking Texas and chaw on something.
-
California has good weather and horrendous governance, Texas has pretty good governance and crap weather.
Result? California treading water on population, Texas gaining four EVs.
I suspect that unless the governance in CA improves, the next census will cause them to lose EVs.
-
Some companies have moved from California to Texas, and including that, the main reason for California's small decline in population is its high cost of living. And the overwhelming factor in Texas's population growth is the growth of its Hispanic population.
I predict Texas will turn blue before California collapses into a socialist sinkhole. And it's pretty easy to be propped up as a success story of capitalism when you happen to be sitting on heaps of one of the world's most important natural resources.
-
I started to say that Romney was the worst possible choice and then I thought of all the other horrible candidates whom I would have never chosen. What a shitty field of candidates. With a bad economy, Obama should have lost but somehow that did not count this time.
-
The litmus test for Nate Silver will be how well his predictions do in retrospect when a Republican wins the Presidency. I think his predictions lean to the left. As long as the trend is left he'll do very well, if the trend is right will his predictions be as good? Only time will tell.
-
If Silver's predictions leaned to the left, he would have predicted Obama winning some states he lost.
Silver seems to have a pretty decent model. There is certainly room for improvement -- he didn't call the exact percentages of victory in each state -- but it is a proven rough predictor of what is likely to happen.
-
A good point.
-
The only surprise for me election night was Obama winning Florida, so I don't understand why people are acting so amazed at Nate Silver.
Also, unless you're planning and making decisions differently based on the outcome, how useful is this prediction to you? Unless you're heavily involved in high-level policy, law, and lobbying, what did you do with the knowledge that Obama was likely to win?
I'm more interested in predicting the stock market than which asshole is going to be holding a veto pen for the next four years.
-
so I don't understand why people are acting so amazed at Nate Silver
Because everyone else calling it was making predictions not in touch with objective reality, including blowhards here saying it would be a blowout for Romney or Obama, instead of the fairly close thing it was hinging on a few swing states won by a small percentage?
Perhaps you should be running a competing prediction model if you think Silver's model is not precise enough.
-
Because everyone else calling it was making predictions not in touch with objective reality, including blowhards here saying it would be a blowout for Romney or Obama, instead of the fairly close thing it was hinging on a few swing states won by a small percentage?
I won money on the election betting against my stupid friends who said Romney was going to win. If Nate Silver is a genius primarily because of how stupid other people are, I'm still not impressed.
Perhaps you should be running a competing prediction model if you think Silver's model is not precise enough.
To me, it seems a waste of time to go into complicated modeling, when my quick estimate of the result was already within 29 electoral votes of the outcome. However, I do have a background in statistics. Maybe I could start blogging like Nate Silver, and maybe, run my own webpage and become a politicial analysist. That way, I can make money telling people the outcomes of events that they will surely become aware of with absolute certainty, as long as they live past the election day. If I don't die of boredom first.
There's plenty of money to be made engaged in activities that I find much more preferrable. I'll pass.
-
Just leaving this here for shits and giggles.
-
Another review of Weatherall.
-
Thanks that was interesting and fun to read people letting him have it.
-
Sienna. I see what you mean... Edna`s c0mment is cool... on tuesday I got Smart ForTwo from earning $9836 this-last/5 weeks and a little over ten-k last-month. it's certainly the nicest-job I have ever had. I actually started 8-months ago and right away started making a nice at least $84, per-hour. I follow this website, http://WWW.FLY38.COM
-
this is really what I wanna know. Thanks for posting this clip!?abercrombie and fitch brussel
-
my best friend's step-sister makes $74/hour on the computer. She has been without work for eight months but last month her paycheck was $19194 just working on the computer for a few hours. Here's the site to read more http://WWW.FLY38.COM
-
til I saw the paycheck which was of $9582, I have faith ...that...my mom in-law woz like they say actualy making money in there spare time from there pretty old laptop.. there best friend has been doing this 4 less than fourteen months and resantly cleared the loans on there house and bourt a great Car. we looked here, http://www.wow92.com
-
my friend's step-mother makes $63/hr on the computer. She has been fired from work for six months but last month her payment was $15870 just working on the computer for a few hours. Read more on this web site
http://qr.net/ka6n