What Are Economic Models Good For?


Some people seem to have gotten the idea, based on my arguments about the use of models to measure the effects of the stimulus, that I think economic modeling is useless. I don't. Models can be useful for making informed guesses about the future. This is what the Congressional Buget Office does when it projects out over a decade. That's incredibly difficult, and, as the CBO will tell you, those projections are naturally subject to a lot of error. But from the limited-knowledge standpoint of the present, they can provide a loose guide to what we think might happen.

Where I'm skeptical, however, is that models can tell us what's happened in the past.

Eight weeks to a killer economy?

Let's say your friend Paul announced that they intended to lose 24 pounds. And the way this person was going to do it was to work out for 90 minutes a day, six days a week, for six weeks. First, he consulted with a personal trainer and an expert from the exercise department at the local college. Then he ran some tests and did some calculations about how many calories he could expect to burn at each workout, and he figured that, in combination with my average daily caloric intake, it ought to result in him losing four pounds each week. Paul is aware that some experts have begun to question whether exercise is the most effective way to lose weight. But he's also heard from friends and numerous local health and fitness experts that, executed properly, it can work. And his numbers—his model—show that it ought to add up. Six weeks, four pounds a week—it should work, right?

After six weeks, Paul calls you up on the phone and announces that the program has run its course. You ask if he's lost weight. And he says that, "Yes, as planned, I worked out for 90 minutes for six days a week. According to my model, I've lost 24 pounds!"

But that's not the question you asked. You asked if he's actually lost weight. Simply telling you that he performed the workouts as planned doesn't actually tell you if the assumptions he was using about how much weight loss he could achieve were correct.

Six pack jobs?

So Paul may have lost weight. He may not have. But the thing is, based on his answer, we don't know.

On the stimulus, the government is responding to the question of whether or not the stimulus worked in much the same way that Paul answered the question about his weight loss. Prior to the program's passage, the White House developed economic models to determine the stimulative effects of government spending: Put a dollar of government money into the right sector of economy, get a little more than dollar of productive activity back out. But when asked to measure the stimulus program's success, they're essentially saying that they spent the money mostly as planned, so, based on the model, it must have created jobs.

Obviously, the workout analogy isn't perfect. One big difference is that weight loss is pretty easy to measure directly. As Harvard economist Greg Mankiw points out, it's almost impossible to accurately measure the tangible results of the stimulus program—which is why we see folks rerunning models in order to attempt to provide answers. And although both the body and the economy are complex systems that produce tremendous uncertainties, the uncertainties in weight loss science don't quite match up to the uncertainties in macroeconomics.

But the point isn't to nitpick the differences (though I'm sure some people will). Instead, it's simply to say that the "proof" touted by stimulus advocates that the program worked isn't really proof of much at all. Models can provide a rough guide to what we assume will happen in the future, but they're not nearly as good at telling us about the past.


NEXT: Reason Morning Links: China's Economy Now Larger Than Japan's, the Border Crime Myth, Grim Month in Afghanistan

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. “What Are Economic Models Good For?”

    About as much as climate models.

    1. To be fair, there is no other way to assess the impact of the stimulus, except to use an economic model.

      Since you’re comparing what actually happened against a theoretical scenario (the “no stimulus” scenario which never happened), it’s not like you can just subtract the two sets of figures and report the difference (since one set of figures doesn’t actually exist).

      Instead, the best you can do is try to figure out what the path-not-taken MIGHT have looked like, and the only way to do that is with a model.

      Now, even if the model is a realistic and good-faith representation of the economy (which I don’t entirely believe), it’s outputs are still going to be a guess. So the only honest thing for the administration to say is “By our best estimates, the effects of the stimulus were likely in the range of X and Y”.

      Since I wouldn’t think this administration, or any other, would suddenly embrace honesty, I suspect, they’ll just plug whatever assumptions they want into their model until it spits out a result they’re happy with, and then report it as if it were a fact. (Having lots of personal experience with complex financial models at investing and consulting firms, I can assure you that this is exactly what happens.)

      1. Predicting what the economy would have done without stimulus is the same as predicting what the economy will do in the future.

        Curiously, Peter believes predicting the future is a useful guide, but predictions that can actually use input data from the time period of interest are not useful. But they are fundamentally the same: both cases are predicting outcomes for a system we have not observed (the future, and 2008-present without stimulus). Retrospective modeling has the advantage of using information about the system during 2008-present to enhance accuracy & precision over predictions of the future, about which we have observed nothing.

        If Peter thinks future modeling is useful, he should be hunky-dory with retrospective modeling. And your analogy is wrong too. The stimulus back-calculations are like retrospectively estimating how much of the weight I lost (the amount of which I observed) is due to my increased exercise, and how much of it is due to decreasing my caloric intake.

        Of course, you can monkey with a model to get it to say what you want; I would think folks are looking at those models used for predictions of stimulus effects to catch any shenanigans (maybe even some of that is on this here site…)

        1. “Predicting what the economy would have done without stimulus is the same as predicting what the economy will do in the future.”

          With one important difference. When modeling the future, you can simply wait and see how your predictions fared against reality to validate your model. When retrospectively modeling alternative histories, it’s impossible to apply a ‘real-world’ test.

          A model that can’t be tested is like a toilet that can’t be flushed. Neither one is good for shit.

  2. What’s with all of the gay shit, Peter?

    1. Fuck that shit – Fred Schneider in a dress!…

  3. Everyone knows the quickest, easiest way to lose weight is to ingest tapeworm eggs.

    1. South Bronx Paradise Diet, baby! All right!

    2. No, the quickest way to lose weight is to kill yourself – preferably in some tropical climate where your body will rot quickly.

    3. The quickest way to lose weight is to defect to North Korea.

      The easy part — well, they don’t really use that metric there.

    4. Nah, the quickest way to lose weight is to cut off your legs.

      1. No, you freeze your legs by packing with dry ice, then get the surgeons to remove your legs.

        1. I don’t think Obamacare will allow for that.

  4. “What Are Economic Models Good For?”

    Ratbagging teafucking, of course.

  5. We can all agree that Paul’s program allowed him to loose or prevented the gaining of 24 lbs.

    1. Actually, without his program, Paul would have GAINED 10 pounds a week. So he actually lost 14 pounds total. And you can’t prove otherwise.

      BTW love the Jillian Michaels ads on this post.

      1. I used that program and I have reached my goal of weighting negative 56 pounds (much, much less than 0!). Models make everything wonderful.

  6. The worst part of policy makers using models to “predict” the past is their policies cannot fail. Since the model used (or changes therein) is not public knowledge, no matter how far the original prediction and the results diverge, new “predictions” for the past can always claim victory.

  7. Paul was clearly “Too Big to Fail”

    So he couldn’t have failed.

  8. I’ve seen this basic point made by others, but I like the analogy. You should try to get this into a forum with a more mainstream audience.

  9. an expert from the exercise department at the local college

    I think it’s called Phys Ed, Peter. And the “expert” would be a gym teacher.

    1. Gym, is how feminists spell Jim.

    2. We don’t give Phys. Ed. degrees anymore; people get Kinesiolgy degrees now. You’ll see it all the time in College Football games. Although I never hear them called kinesiologists once they graduate.

      1. “Although I never hear them called kinesiologists once they graduate.”

        In Ontario, they’ve got their own professional association.

        1. Wow. In North Florida their professional title is usually “Coach” if they don’t get drafted.

  10. The Stimulus Diet: Keep ingesting cheeseburgers until the weight falls off. Also, ignore those Austrian weirdoes warning of imminent heart attack.

  11. Keep ingesting cheeseburgers until the weight falls off

    AKA the Atkins diet

    1. AKA “Stimulus”
      Get the digestive system cranking!

      1. Yeah sudden high levels of fat in the diet leading to huge amounts of floating fecal matter. Kinda sounds like the current administration and economy, so the stimulus must have worked 😉

    2. Can’t have the buns on Atkins, so they wouldn’t be cheeseburgers, they would be round steak with a cheese garnish. Uh, and no french fries or shake… there goes half the fun.

      1. Yeah, actually cheeseburgers was a bad example. A diet with lots of beef and saturated fat is perfectly healthy. Supersize fries and shakes would have been a better example.

  12. I cannot resist, as a rat-bagging nitpicker, the impulse to point out that, if you enter a vigorous weight training regimen, may are likely to “gain” weight, while reducing fat, strengthening the heart, and improving endurance. You will in fact, be healthier, but if all you look at is the scale, you will have FAILED.

    What does this have to do with the stimulus? If you allow people to strike out on their own, without government supervision, they might create the “wrong kind” of jobs.

    1. Not if you’re way overweight. The fat will burn at a faster rate than the muscle will build.

  13. If Paul would have used the Shake Weight, he could have lost more than 24 pounds. mMMahh, there it is.

  14. What’s with all of the gay shit, Peter?

    Seriously; the alt-text should be “BUT I’M NOT GAY!”

  15. Is there some reason these models can’t be evaluated in the same way as scientific theories, meaning predictions based on the model?

    You create your model, plug in your input values (carefully documented, of course),then document the models predicted measurable results for 12-, 18-, 24-months.

    If, at the 12-month point the predictions match reality you can claim it’s working. If they’re not right but close maybe you wait for the next point. If they are way off you could admit you have no clue what you’re doing and resign.

    Wasn’t there some prediction that without the stimulus unemployment might reach 9% based on the model? Where’s that resignation?

    1. Kilroy,
      These are the models that have performed best on the kind of tests you described. That is why they are the models that the private sector investors buy. You want a resignation–the predicted 9% unemployment came from the private-sector models of the economy, although in this case those models were being run by government economists who probably took an average of the best models.

      1. Part of the problem with modeling the downturn is that models use historical data to predict the future. This downturn was the biggest decline in GDP since WWII, so there was no historical data.

        1. Thanks for the reply, R2.

          So you’re saying there’s no way to tell if the stimulating didn’t work versus the model was flawed (for lack of relevant historical data or whatever)? That sounds a lot like “created or saved” to me.

          There was enough faith in the models and the selected input values to justify spending $800 billion but not enough faith to stand by its predictions 18 months later?

          I still think I’m owed a resignation.

          1. The models have built-in historical relationships and fundamental relationships between certain components, for example-the price of oil and inflation. In tail events (unlikely events), those relationships don’t always perform as they do in normal times. In this downturn, for example, the correlation between almost all asset classes increased massively as the price of everything dropped.

            Asking for a resignation is like asking the National Weather Service head to resign because they forecast a Cat. III huricane and it was actually a Cat. 5 hurricane. The problem with the economy is that the predictions actually change the result as the actors respond to the forecast. When you look at the GDP data broken down by month, Dec. ’08 was horrific. It actually dragged down both Q4 of ’08 and Q1 of ’09. It dragged down Q1 because GDP is an average of a quarter. Since Nov. and Oct. were higher than Dec. the average was above the level that economy finished the quarter at. So even if there had been no decline from the Dec. levels in Q1, the GDP data would have come in much lower.

            These models remain the best, however far, far from perfect way, we currently have to evaluate the stimulus.

            1. Yeah. I get that. Feedback, chaotic system, etc. (see Uncertainty Principle)

              Here’s the thing, using your analogy:

              If I asked the National Weather Service head to bet $800 billion on his hurricane prediction he’d (rightly) tell me to fuck off. He understands his model and the uncertainty of it. I see none of that in the stimulus BS.

              Also, when the hurricane landed and it was a Cat. 5, not a Cat. 3, he would very likely call it a Cat. 5, not try to tell everyone it matched his prediction and everyone’s storm gauges are wrong. That’s what I think is happening here.

              It doesn’t come down to whether I understand the models and the associated uncertainties. It comes down to whether they do.

              Scientific models bound the predictions they make with the inherent uncertainty and that goes for making predictions in systems that are every bit as complex as macroeconomics.

              What were the predictions (? uncertainties) for the initial values they used to say I bet $800 billion? Do they match what’s actually happened?

              1. Yes, economists do understand uncertainty bands for most models, it’s just that most consumers of the forecast don’t care or don’t understand such bands. CBO’s forecasts as they normally include a range of outcomes. I’m not sure what the uncertainties were with the initial forecasts for the stimulus. In general, the best evidence we have suggests that the stimulus worked to stabilize the economy. The big-bank bail-out in the TARP (not the stimulus) will turn a profit. Much like in the Asian financial crisis, fear is often stonger than greed, and when liquidity freezes up due to fear, government can provide a stabilizing investor of last resort that breaks the cycle of fear.
                What an economist might say about betting $800 billion on the stimulus–“the downside risk in the current forecast is huge. Injecting this money should stabilize the economy. Either way the deficit will be bad, because if the worst occurs, tax revenue will continue to collapse and automatic stabilizers will increase.”
                Although, I agree that scientists in many disciplines model such complex systems, there is one key difference. When you figure out a relationship in the economy, the system changes because you figured out that relationship. That is one reason that currency markets tend to be random walks with drift.

                1. In general, the best evidence we have suggests that the stimulus worked to stabilize the economy.

                  Is that based on predictions from the model that proved accurate? The one prediction that I’ve seen that was actually made was “a large economic-aid package would boost the economy and keep the unemployment rate below 8%.” This obviously is a failure.

                  If I constructed a model that predicted the acceleration due to gravity of an object near the earth’s surface would be 11.5 m/s? and the measured value was 9.8m/s? everyone would call my model a failure and so would I.

                  They definitely said it would be worse if they didn’t invest $800 billion. I’m looking for quantifiable predictions that can be empirically tested. At the very least you’d think they could publish the outputs of the model for a few common economic indicators with and without the stimulus. I haven’t even seen that.

                  What if the worst case prediction from the model was better than the current situation, like the unemployment numbers? It seems like the argument is “It had to work. It just had to!” to which I’m responding, prove it.

                  Personally, I don’t think the stimulus helped and I think it’s possible it was a net negative. I’d like someone to show that I’m just a government hating pessimist (and I appreciate your attempts here) but so far all I get are “created or saved” type answers.

  16. You forgot to mention that your friend actually gained weight.

  17. What’s with all of the gay shit, Peter?

    Seriously; the alt-text should be “BUT I’M NOT GAY!”

    Better alt-text: NO HOMO!

  18. Jillian Michaels is a man.
    In fact, she may be John Basedow. Their tits are about the same.

  19. You actually linked to a times article for words from the “experts”?!? Yeesh, it’s really falling off around here.

  20. You mean to tell me they threw a good chunk of a trillion with no more validation of the underlying logic than it was some unproven theory of some dead guy?

    [starts drumming lips with index finger while making ‘brbrbrbrbr’ sounds]

  21. These are the models that have performed best on the kind of tests you described. That is why they are the models that the private sector investors buy.

    I’m gonna need a citation on that, R2. I’m also gonna want to see whether what they actually did with the money matches what they told the model they were going to do with the money.

    Not that much, relatively speaking, got spent on new capital assets/infrastructure. The stimulus was was mostly a massive bailout of state budgets. Most of the stimulus money went to states to fill various budget holes, mostly in Medicare. Some of the rest went to various pet projects, like wrecking perfectly good cars and the like.

    What model do they have for that kind of stimulus spending, I wonder?

    1. Just trust me that there are only three major companies in the forecasting business and government economists use all of them.

      1. BTW–about 1/3 of the stimulus was tax cuts, including a payroll tax cut.

  22. Preventing larger cuts at the state level was a quick way to get the money into the economy and prevented further cuts at the state level which would have fed into the negative feedback loop.

  23. Peter, that was a lot of words to describe a really simple concept. Essentially, the administration is claiming that their original equation was correct by… working out the equation again on paper.

Please to post comments

Comments are closed.