Lott claims that I "used data up until 1997, but that is not possible since he limited the sample to only four years after adoption [of the laws]...." Clearly, he is mistaken, since my plots show data extending 10 years before the law's adoption.
My statement about the changes in slope in the various states was based on simple linear fits to the data two years on either side of t=0, without weighting the states by population. However, without doing any statistical analysis whatsoever, a mere glance at the graphs for the 10 states should allow readers to decide for themselves whether the data for the 10 states actually show anything particular happening at time t=0. (The data for robbery can be found plotted in my book or downloaded from the FBI Web site.)
Lott claims that his fitting procedure is not biased, because by using random data one is notvirtually guaranteed to find a drop or a rise at t=0, as I claimed. Instead, he points out that the random data might show an abrupt change in the slope, not the actual level, at t=0 (e.g., first rising then falling, or first falling then rising). But Lott's correction to my statement actually makes my basic point even stronger, since a decrease in slope is exactly what might be expected if Lott were right. Thus, if his fitting procedure would force random data to show a change in slope at t=0 -- equally often an increase or decrease -- we can't have too much confidence that any observed decrease in slope validates his theory.
It's difficult to find anything about mass murder amusing, but I find Lott's calculation for the greater deterrent effect of easing concealed-carry laws on multiple shootings very humorous. Essentially, he is saying that after concealed-carry laws are eased, mass murderers really are more deterred than ordinary murderers, because the chances are much greater that someone in a large group is actually armed. Now, I don't think mass murderers are totally irrational. But I find this type of probability calculation more revealing of Lott's thinking than that of mass murderers, some of whom I imagine would relish the idea of going out in a blaze of glory, in case someone in the group were armed. ("Suicide by police" seems to be a fairly common act by some psychos.)
In Lott's rebuttal on this same issue he fails to address the other inconsistency in his results: How could the laws act in reverse time, causing a big spurt of mass shootings the year before the laws were enacted? He also neglects to answer my question on how his analysis can show the murder rate dropping immediately after the laws are passed, but the aggravated assault rate not starting its drop until four years later.
Lott is right in pointing out that the omitted variables would need to change systematically in a way correlated with the dates of passing the laws. But given that the laws (according to him) account for such a tiny fraction of the change in crime rates, and given an extremely long list of possible variables, it seems likely that some of them could fit the bill. If Lott's claim that he really has accounted for all the key variables that affect violent crime rates were correct, then he really should be able to predict how the crime rates will change in the future in each state, based on all these variables. Moreover, if his predictions fail to be borne out in any state it would show that he has left out some factor. (We are all used to hearing about why the stock market did what it did on any given day, after the fact. But the failure to make such accurate predictions ahead of time tells us that maybe we really don't fully understand all the variables that make the market do what it does, any more than we understand the variation in crime rates.)
I am not alone in questioning Lott's statistical analysis -- see, for example, work by Daniel Webster, Jens Ludwig, Daniel Black, and Daniel Nagin. Lott notes that his F-test is the appropriate one to answer the question of whether there was a statistically significant change in the slope in crime rates at t=0. I don't dispute that the change in the slope of crime rates may be statistically significant at t=0. After all, there might have been a real change at that point in time for reasons unrelated to the laws.
However, I claim that the slope will probably also be found to change by statistically significant amounts at most other years as well, and that would show that there's nothing special happening at t=0, the year the laws were passed. The real test of whether it was the liberalized gun laws that made the difference is that a statistically significant change in slope was found at t=0 and only at t=0.
To see this basic flaw in Lott's statistical analysis, let's imagine that some lunatic has a theory that the Nasdaq drops every full moon. Presumably, according to Lott, the way to test this theory would be to do a linear regression involving as many extraneous variables as we can think of that might affect the Nasdaq -- and not to worry too much that we may not have gotten them all. Then using the regression, we need to see if the Nasdaq had a statistically significant drop on days when the moon was full. It very well might show a statistically significant drop on those days. Why not? However, I expect that the Nasdaq would also show drops (and rises) having comparable statistical significance for other lunar phases as well -- thereby proving exactly nothing.
Prof. Lott, wouldn't you agree that a finding of a statistically significant change in the crime rates at years before t=0 would invalidate your results? Will you tell us what your analysis shows for the statistical significance of changes in slope at years other than t=0?
John R. Lott Jr.
The Effect Is Clear
Disarming law-abiding citizens leads to more crime
To Prof. Ehrlich, the "basic flaw" in my statistical analysis is that concealed handgun laws are likely to be just accidentally related to changes in the crime rate. He takes a simple example of explaining how the stock market changes over time. Obvious variables to include would be the interest rate and the expected growth in the economy, but many other variables -- many of dubious importance -- could possibly also be included. The problem arises when such variables are correlated to changes in stock prices merely by chance.
An extreme case would be including the prices of various grocery store products. A store might sell thousands of items, and the price of one -- say, peanut butter -- might happen to be highly correlated with the stock prices over the particular period examined. We know that there is little theoretical reason for peanut butter to explain overall stock prices, but if you go through enough grocery store prices, it just might happen that one of them accidentally moves up and down with the movements in the stock market over a particular period of time. Similar problems can occur with other obviously unrelated variables, such as the incidence of full moons or sunspots.
There are ways to protect against this "dubious variable" problem. One is to expand the original sample period. If no true causal relationship exists between the two variables, this coincidence is unlikely to keep occurring in future years. And this is precisely what I did as more data became available: Originally, I looked at data through 1992, then extended it to 1994, then up until 1996, and then, in recent working papers, up through 1998. If Ehrlich understood this, he would realize that this is equivalent to his request that I should try to "predict how the crime rates will change in the future."
Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time.
nfl jerseys|11.14.10 @ 8:29PM|#
hxthgf