Scatter Plot Punditry: Heritage Responds

I posted yesterday about a recent piece attacking the Heritage Foundation's economic freedom index. Heritage's Marc Miles replies below the fold—though for my money, what you folks came up with in the discussion of yesterday's post constituted just as effective a reply.

Every year for the last decade, the Index of Economic Freedom, which we at The Heritage Foundation publish with The Wall Street Journal, has shown a strong link between economic freedom and economic growth. For readers with any acquaintance at all with how markets work, this conclusion probably seems obvious. Yet Doug Henwood of the Left Business Observer recently demonstrated how hard it is for some people to grasp it.

In a March 26 "special report" that's been circulating on the Web, Henwood condescendingly notes the truism that "correlation doesn't prove causation." We agree, and we have never claimed otherwise. The data show only that changes in the Index score and growth rates rise and fall together.

But there are other ideas embedded in this test, such as "wealth is created by people not governments" and "reducing the barriers in the path of individuals allow them to use better their natural abilities to achieve their goals in life." Statistics ultimately can be used only to disprove. The data, when correctly tested, fail to disprove any of these statements.

Those are the facts. Unfortunately, from here Henwood's analysis deteriorates into the realm of ad hoc conjecture.

For example, in his attempt to conjure up an alternative theory to explain the data results, he asserts, "Corruption lowers a country's score, but when times are good, outstretched palms are often hard to notice." Some consistency, please: Corruption is a tax, for it raises the cost of doing business. The truism he chooses to ignore here is that "it is a tax in bad times, it is a tax in good times."

Henwood then descends into an argument about "how countries' scores in a base year [are] correlated with subsequent growth." In other words, how the level of economic freedom as measured by the Index in a particular year is related to changes in income over time. Who claimed that to be true? Certainly we did not. This must be Henwood's theory, which he proceeds to test as if it were ours.

In the process, he raises some red herrings about the use of purchasing-power parity (PPP) data and per capita growth data. Sorry, Doug, but no matter which data are used, our results are essentially the same.

But when Henwood tests his theory, the data do not support his assertion. Alas, his view of the world can be disproved by the data.  (For those who would like a sophisticated analysis of this point, see http://www.anderson.ucla.edu/documents/areas/adm/media/roll.pdf )

Most disturbing, however, is that this misrepresentation of our data is precisely the same as raised by Jeffrey Sachs in his new book "The End of Poverty." It is no more accurate when stated by Henwood as it is when written by Sachs. They proceed to test the level of the Index against the change in income, while our arguments have always been that the level of the Index is related to the level of income (see http://www.heritage.org/research/features/index/downloads/economicFreedomandPerCapita.gif), and changes in the Index are related to changes in income (See Figure 1 and explanation in http://www.heritage.org/research/features/index/chapters/Executive_Summary.pdf).

To paraphrase Henwood, anyone who lasted a week in basic statistics knows the difference between levels and changes in levels. Have those who have become ossified in 20th century economics forgotten the difference?

Sachs' and Henwood's screeds are not about good economics and good statistics. They are apparently about trying to hold on desperately to ideas of the past through misrepresentation and snide little potshots (e.g., "It's so much fun being on the right—you're liberated from the tyranny of having to make sense.") C'mon, boys, the public deserves better than that.

Comments

[Adam]

chthus noted yesterday the "card trick" used in this study - graphing ranks instead of graphing scores.

It happens to be the same card trick Common Sense for Drug Policy uses in an ad they regularly run in Reason, in a graph displayed at the bottom of their web page.

I emailed Nick Gillespie about it this summer, proceeded to get into a fight with Daddy Warbucks drug policy douchebag Robert Field, and the ad continues to run in Reason.

Yes, it's for a good cause, and no, the ends still do not justify the means. The graph is still bullshit, no matter how much I might agree with the policy views of those producing or publishing it, and yes, Gillespie, you should still pull that ad until CSDP gets off their lazy asses and makes a new graph. (And they have plenty of other ads that you still run that could be run in its place. Are you just afraid of pissing off Daddy Warbucks? I mean, yeah, Robert Field does whine like a little bitch when you tell him he's wrong.)

[Julian Sanchez]

Not to repeat the same fight here, but it's not obvious to me why the use of ranks instead of some other scale is wrong in that context. If the point is that pot isn't as bad as Alcohol on any of these dimensions, and not even as bad as caffeine on some of them, that counts as a sound argument that there's something amiss in our policy, regardless of the magnitude of the difference on any particular dimension.

[shyoung]

Rankings are a perfectly fine way to look at things. Certainly Adam isn't saying that ranking information should be suppressed?

[Chad Brick]

It is obvious to me that the problem is the switching between freedom score/rank and GDP growth RATE. You can compare the state of freedom to GDP, or changes in freedom to changes in GDP, but it makes little sense to compare the state of one to changes in the other.

Take China, for example. It has a low freedom score and a low GDP. It has a large positive change in freedom, and a large positive change in GDP. Both of these are consistent the trend of freedom causing higher GDP. However, what is the meaning of high GDP growth and low freedom? Not a whole lot. Likewise, Europe is the exact opposite - high freedom but little growth in it, high GDP and again little growth. Again, I would argue there is little relationship between their high freedom and low growth.

[jc]

Julian,

The problem is that the pious (on both sides of the argument) are so entrenched in their positions that the "trick" they use to persuade the other side winds up being used to the benefit of their adversary. In the case of the chart Adam mentions, a drug warrior could argue "We regulate the bejeeziz out of pot and look how low it ranks, therefore we should put even more regulation on alcohol and tobacco."

This is basically the argument I made yesterday, the chart arguing that there's no economic correlation to freedom is also an argument against regulation; the "honesty" of the chart has little to do with it. It's the stupidity or willful blindness (or lack of common sense?) of the person making the argument that he can't even see how his own chart can be used as an argument against his own position.

[shyoung]

Now that I have thought more about the rank vs rate comparison, I see now that yesterday's scatter plot proves nothing more than that where you are in no way correlates to what direction you are going or how fast.

[Adam]

It wouldn't be wrong if the chart didn't say at the top:

"Comparing Dangers of Popular Drugs (Lower score indicates less serious effect)"

If they stuck to the bottom chart of rankings - and disclosed that they are rankings instead of stating that they are "scores", it'd be a graphically dull ad, it'd be ambiguous for anything other than my-drug-is-better-than -your-drug analyses, but it'd be accurate.

By graphing the rankings and calling them scores, they're indicating that, for instance, the graph shows that cocaine is 1.5 times as "reinforcing" as alcohol, but that's not necessarily true, it's not likely true, and we certainly can't tell by looking at the rankings-masquerading-as-scores because the rankings are meaningless without the original scores. (If the original scores were, say, on a scale of 1 to 100, caffeine got a 5, marijuana got a 20, nicotine got a 25, alcohol got a 30, heroin got a 90 and cocaine got a 99, then cocaine would be more than 3 times as "reinforcing" than alcohol but the graph "shows" that it's only 1.5 times as "reinforcing".)

[Adam]

And no I'm not arguing against ranking - it's handy sometimes - but ranks have to be based on a measurement, and disclosing the measurement is generally more informative than disclosing just a ranking (unless, I suppose, the rankings happen to correspond directly to measurements).

[Jason Ligon]

The index is a Frankenstein monster of too many variables. I don't think much in the way of specific claims can be made based on this data. They might do better to isolate certain critical variables, compare those to growth, and not bother with the combined Index. It will always have the smell of being created to prove the point they want it to, and it contains enough dimensions that you could never prove otherwise.

[Adam]

Moral of my story: don't GRAPH rankings. Ranks can be reported in tables; scores can be shown on graphs; putting rankings on a graph is usually misleading.

[alkali]

Merits of the dispute aside, I would raise a yellow flag on calling Henwood's analysis a "misrepresentation." If someone shows you some data, graphed or otherwise, and tells you where they got the data from, and they are telling the truth, they are not dishonest. Wrong, maybe, sure, but not dishonest.

[The Real Bill]

My problem with the drug rankings is that they don't make sense with respect to my own experience. In the past, I've been addicted to both caffeine and marajuana. My dependence and withdrawal from marajuana were far worse than from caffeine. But, I did smoke a lot more pot than drink coffee, which might explain my situation. I suppose that it's all about the degree of use.

[The Real Bill]

Adam is correct, graphing rankings can be misleading.

[SixSigma]

Merits of the dispute aside, I would raise a yellow flag on calling Henwood's analysis a "misrepresentation." If someone shows you some data, graphed or otherwise, and tells you where they got the data from, and they are telling the truth, they are not dishonest. Wrong, maybe, sure, but not dishonest.

I would agree that this is probably the best way to look at it, but not the only way. If the author is aware of the trick used to make the statistics prove preconceived notions, then it is completely dishonest. The less industrious among us may never look into the actual data presented, regardless of whether it's easy to get. And after all, that's the market they're shooting for, the completely uninformed.

My main example, other than this one, would be Krugman and the many lies he likes to tell. His training as an economist gives him the ability to know he's lying, so it becomes not so much about ignorance as direct dishonesty.

[The Real Bill]

SixSigma,

Do you think Krugman is evil or just a hopeless ideologue that thinks the end justifies the means? Either the guy knows he's lying or he's undergone some serious memory loss. It's as if he's forgotten how to tell the truth.

[Rimfax]

Graphing the ranking instead of the value should distort the appearance of poorly distributed data, but the scores appear to be fairly well-distributed and the growth rates are likely to be moderately well-distributed. In other words, even though graphing rankings is a bad idea and it will cause varying degrees of distortion, the resulting graph in this case is very likely to be similar to one using the actual values. (I'd really like to see a graph of Henwood's data replacing the "growth rate rankings"/"score rankings" with "growth rate"/"score", if possible.)

Henwood is arguing that higher income correlates with a higher tolerance for economic freedom. In other words, he is arguing that economic freedom is a side-effect of a high income, possibly even that it is a negative side-effect.

His rankings graphs indirectly support this argument. Demonstrating that score correlates to income but that it fails to correlate to growth rate suggests that it wasn't economic freedom that made the rich rich.

Miles dismisses Henwood's arguments by saying that only changes in score, not static scores, are comparable to changes in per capita income. Leaving it at that would suggest that a country that was at the pinnacle of economic freedom would never again experience economic growth because its score can never improve.

It may be that economic freedom correlates to the sum changes rather than rate changes. Maybe lower scores lead to lower raises, that when viewed as percentage increases seem to be unrelated to score.

One way to reexamine the possibility of a correlation would be to compare the "growth rates"/"scores" of countries only to those with comparable income levels. That should help determine if "good" systems lead to higher incomes. Unfortunately, the low number of comparable countries in any economic group are likely to leave the results without statistical significance.

I think that dismissing Henwood's graphs and arguments as bad statistics (which they are to a degree) and nothing more is a missed opportunity to better understand the impact of economic policies, and perhaps the causes. If we can honestly evaluate the pragmatic impact of what we consider to be moral economic policies, we have an obligation to do so no matter what the findings may be.

Full disclosure: I am a card-carrying Libertarian and I've forgotten almost all of the statistics that I learned in college.

[SixSigma]

Do you think Krugman is evil or just a hopeless ideologue that thinks the end justifies the means? Either the guy knows he's lying or he's undergone some serious memory loss. It's as if he's forgotten how to tell the truth.

Hopeless is a term I think aptly aplies to Krugman. His insistance to lie about even mundane things, like whether SS needs help, leaves little doubt in my mind that he's lying versus memory loss. Of course this is an undefendable position as there's no proof without an admission or some real documentation some how proving he knew and lied anyway. Plausible deniability as it were.

Having said that, is it better he (or any other pundit, politician, know-it-all, etc) be lying or be ignorant? I've asked myself this question for a while now, focusing mainly on our elected officals as my reference, and I have no answer.

Of course being a smart truth teller would be the best choice here, but assuming the politician is wrong, which would you prefer?

[Brian Courts]

Hopeless ideologue, I would suppose. His defense of Fahrenheit 911 seemed to be that, even if it created false impressions, it performed an "essential service." So what can you expect? Too bad, he used to be a rather insightful and interesting writer before he became such an unabashed political hack. What happened?

[The Real Bill]

I prefer honest ignorance to purposeful lying. Not because the former is effectively better, but because I so dislike dishonesty. I'd have to say that it really depends on the situation, though. Smart and bad might be better than dumb and good when it comes to national security (as long as bad does not equal traitorous).

[Stretch]

The problem is the LBO report basically said two things that were in conflict with each other.

1. That Heritage was wrong, for various reasons.

2. That LBO plottings on dissimilar points are right.

The LBO report largely suggested that the EF index was completely wrong, not only because of what Heritage suggested (although snide comments made it a contributer), but because by using other startegies a different result could be represented.

Still, the point stands that if data can be manipulated and interpreted so easily and differently, that central planning of the economy is totally out of the question. Despite the faults of the free market and economic freedom in general, it is simply the best available (and consequently only) method.

Ultimately, Heritage provided information based on a certain set of criteria, and LBO distorted that for their own means. Ironically, the more the info is distorted, the greater the case against central planning.

[shyoung]

To use an analogy, Heritage says the if you have a bunch of balls rolling down a hill, there is a correlation between how fast each ball is spinning, and how fast it is moving. The LBO comes around and proves that how fast the ball is spinning says nothing about the rate it' velocity is increasing. Well no duh! Faster spin still means faster velocity, more freedom still suggests more wealth.

[Mark Bahner]

"Merits of the dispute aside, I would raise a yellow flag on calling Henwood's analysis a "misrepresentation." If someone shows you some data, graphed or otherwise, and tells you where they got the data from, and they are telling the truth, they are not dishonest. Wrong, maybe, sure, but not dishonest."

Henson's analysis *is* a "misrepresentation," because he uses it to claim that the Heritage Index of Economic Freedom is "meaningless."

Now, if he'd done a REAL analysis, like that analysis by Roll and Talbott, THEN Henson would have been doing some good stuff! But of course, you're not going to find anyone who claims the Index of Economic Freedom is "meaningless" who really cares about facts, or is capable of performing any serious analysis.

Back to that analysis by Roll and Talbott, that's got some fascinating stuff in it. Take a look at the low R squared for "government expenditures," for instance. THAT would be a legitimate point for LBO to make.

But of course, such analysis would take both honesty and expertise.

P.S. Seriously, take a look at that Roll and Talbott analysis. Some really, really interesting stuff. Look how economic growth after a "democracy event" is low initially...actually lower than after an "anti-democracy event". But the growth after an "anti-democracy event" stays flat, whereas the growth after a "democracy event" increases as time goes on. Cool!

P.P.S. And the R2 for a free press is pretty high, but the R2 for "political rights" is lower. Interesting.

[Rick Barton]

Julian made a crucial observation yesterday:

"It's actually the almost total absence of a correlation here that makes me suspicious. If they'd shown that the correlation wasn't as strong or tight as Heritage claimed, I'd be open to persuasion. But the idea that there's no link whatever? That there's no statistical link between heavy-handed economic regulation and stunted growth? That one doesn't pass the straight face test."

Giving qualitative evidence for a positive correlation between economic liberty and economic growth are some historical and current examples of nations that are in physical juxtaposition but widely disparate in economic freedom: East and West Germany, North and South Korea, Costa Rica and its neighbors, Hong Kong and China, Somalia and its neighbors. In some of these examples, the ethnic and cultural similarities of the two respective nations argues all the more for an economic freedom cause of the economic performance differences.

Cato has some interesting stuff that finds a positive correlation between economic liberty and economic growth/economic well-being:

Economic Growth and Freedom: A Causality Study (Examines the Heritage Foundation's economic freedom index.)

http://www.cato.org/pubs/journal/cj23n2/cj23n2-4.pdf

Economic Freedom, Political Freedom, and Economic Well-Being: A Causality Analysis:
(Uses Granger causality to establish a positive correlation between Economic Freedom (but not political Freedom) and Economic Well-Being. Granger causality is time series analysis.

http://www.cato.org/pubs/journal/cj18n2/cj18n2-5.pdf

Economic Freedom and human welfare: Some Empirical Findings:

http://www.cato.org/pubs/journal/cj18n2/cj18n2-8.pdf

[thoreau]

I know I'm late to the party, but I'd like to offer 1 thought:

A wealthy and free country might have only a modest growth rate, but prosperity is still a strong indicator of benefits from economic freedom.

Conversely, a poor and unfree country could probably experience significant percentage growth from modest changes in policy. Even a small gain in economic freedom might be enough to produce growth that is modest on an absolute scale but huge on a percentage scale. For instance, I could probably buy a large building in dowtown Pyongyang with the change in my pocket, install a call center, and even factoring in the low rates that I'd have to charge clients due to my employees' abysmal English skills I'd still double North Korea's GDP.

(OK, that was an exaggeration, but you get my point.)

[thoreau]

East and West Germany, North and South Korea, Costa Rica and its neighbors, Hong Kong and China, Somalia and its neighbors.

True, but these examples are of only limited applicability when evaluating economic policies in the developed world. Somebody on the left might argue that the relationship between economic freedom and prosperity is non-monotonic. A huge change in freedom, like the difference between the Koreas, clearly produces a huge change in GDP, but that comparison doesn't mean that modest increases in economic freedom will make a developed nation wealthier.

To refute that hypothesis, one would have to compare, say, France's 35 hour work week with Switzerland's 44 hour work week.

And even those comparisons of developed places with common languages and cultures have to carry caveats. For instance, a few months ago it was observed on this forum that heavily regulated New York attracts more business than lightly regulated Kansas. New York, despite its regulations, offers features that Kansas lacks (e.g. lots of suppliers and customers are already located there, not to mention an international port and cultural attractions).

As always, drawing conclusions from data is a tricky business. It can sometimes help if such efforts are informed by sound theory, but the line between rigorous theory and stubborn ideology can be a fine one. Analyses that are "informed" by ideology cannot be considered unreliable.

[fyodor]

thoreau,

Isn't your first post pretty much saying what Chad Brick said above? (Not criticizing if so, just checking to make sure I understand; I'm figuring you just missed his post.)

[thoreau]

As I said, I'm late to the party.

[Rick Barton]

thoreau:

True, but these examples are of only limited applicability when evaluating economic policies in the developed world. Somebody on the left might argue that the relationship between economic freedom and prosperity is non-monotonic. A huge change in freedom, like the difference between the Koreas, clearly produces a huge change in GDP, but that comparison doesn't mean that modest increases in economic freedom will make a developed nation wealthier.

I know the example comparisons that I gave are extreme cases, but I gave them as evidence against the idea that there is no positive correlation what so ever between economic freedom and economic growth. Also, in the case of Costa Rica and its neighbors, the freedom disparity is not so extreme. In the case of Somalia and its neighbors, we have a situation where, before the Somalian anarchy, (nice to be able to say "anarchy" in its positive, absence of government meaning), I'm thinking that it was on par with its neighbors as far as economic growth is concerned.

I'm not understanding your use of the term "non-monotonic" here. I'm thinking that to describe the relationship between economic freedom and economic growth as monotonic, ie an increase in the former *always* yields an increase in the latter is meaningless anyway since economic freedom is impossible to quantify with any exactness. However the general tendency of the relationship can still be observed.

...but that comparison doesn't mean that modest increases in economic freedom will make a developed nation wealthier.

But can't we say, based on the evidence, that modest increases in economic freedom tend to make developed nations wealthier? I know, I know, the exact meaning of "modest" is begged for here.

[thats right]

marajuana is bad for the mind but some people say that it makes them feel good about themselves but all they are really doing is killing themselves

[thats right]

I used to get hi thats when I was stupid the felling I got were not good they made me feel like hitting any body that got in my way including moms grandmas grandpas you got in my way i would kick your ass

[NikeFree Run Scarpe Uomo]

Accetto piano di sopra