MENU

Reason.com

Free Minds & Free Markets

A Second Look at a Controversial Study About Defensive Gun Use

Criminologist Gary Kleck revises his paper on the incidence of the use of firearms for self-protection.

In April, criminologist Gary Kleck reported that he had uncovered evidence supporting his contention that Americans use firearms in self defense over 2 million a times a year. The survey he discovered had not been previously analyzed, but he reported that it matched what he found in the 1993 survey he conducted with Marc Gertz and published in 1995, known as the National Self-Defense Survey (NSDS).

His new report was based on surveys conducted by the Centers for Disease Control (CDC) in its Behavioral Risk Factor Surveillance System (BRFSS) survey in the years 1996-98. This finding was touted by many outlets—including Reason—as evidence in support of the utility of private gun ownership.

Shortly after that study was released, however, Robert VerBruggen of National Review (who has been of inestimable help in thinking through these issues) tweeted that he noticed, by studying the raw survey data himself, that Kleck had mistaken what were in fact surveys limited to small numbers of states per year for a national survey, analogous to Kleck/Gertz's own national surveys.

in direct response to queries from Reason, who first directly notified Kleck of his error, he worked through and has since issued a revised version of the paper, published as was the original as a working paper on the Social Science Research Network. In the new version, Kleck re-analyzes the BRFSS survey data accurately as limited to a small number of states, and ultimately concludes, when their surveys are analyzed in conjunction with his NSDS, that their surveys indicate likely over 1 million defensive uses of guns (DGUs) a year nationally, compared to the over 2 million of his own NSDS.

Here's how Kleck got to that new conclusion. The BRFSS, as Kleck describes it in his paper, "are high-quality telephone surveys of very large probability samples of U.S. adults...even just the subset of four to seven state surveys that asked about DGU in 1996-1998 interviewed 3,197-4,500 adults, depending on the year. This is more people than were asked about this topic in any other surveys, other than the National Self-Defense Survey conducted in 1993 by Kleck and Gertz (1995), who asked DGU questions of 4,977 people." The BRFSS asked about defensive uses of guns in seven states in 1996, seven in 1997, and four in 1998.

Kleck judged the "wording of the DGU question in the BRFSS surveys" as "also excellent, avoiding many problems with the wording that afflicted the DGU questions used in other surveys."

The BRFSS results were designed to exclude "uses by military, police and others with firearm-related jobs" and "uses against animals." The survey was designed to garner "yes" answers as long as a gun was used in presumed self-defense in any location (not just the home), whether or not the gun was actually fired (as, per Kleck's survey, around 3/4 of the time one needn't fire the gun to have found it useful in deterring an intruder or attacker).

Since Kleck's survey did not include Alaska and Hawaii and the BRFSS did (in 1996 and 1997 respectively), he kept them out of the comparison. The states for which a meaningful comparison could be made between his NSDS and this CDC survey, then, were, in 1996, Kentucky, Louisiana (also surveyed in 1998), Maryland, New Hampshire (also surveyed in 1997), New York, and West Virginia; in 1997, Colorado, Missouri, New Jersey (also surveyed in 1998), North Dakota, and Ohio; and in 1998, Montana and Pennsylvania.

Kleck notes that it's simply impossible to extrapolate meaningfully from the small set of states surveyed over the course of those three years to a solid national DGU figure from the BRFSS itself: "We cannot directly apply these estimates to the U.S. because the sets of states do not constitute a probability sample of the U.S. The prevalence of DGU could be far higher in some states than in the nation as a whole if the states have higher-than-average rates of gun ownership and/or crime, or could be far lower if the set of states had lower gun ownership or crime rates."

But he does think by comparing the national results from his NSDS to the results in the BRFSS-surveyed set of states in his NSDS you can make a tentative extrapolation (after adjusting for the fact that the BRFSS only asked the DGU question to households that had already said they had a gun, while his surveys "found that 21% of persons who reported a DGU had denied having a gun in their household at the time of the interview.")

In the group of states (minus Alaska and Hawaii) that BRFSS surveyed over those three years, the BRFSS found raw numbers of 55 (1996), 29 (1997), and 33 (1998) DGUs.

After a series of adjustments and weightings described at length in the paper, Kleck concludes the BRFSS survey indicates that the percentage of adults in gun-owning households who experienced a DGU in the states they surveyed were 1.33 percent for 1996, 0.89 percent for 1997, and 1.04 percent for 1998.

Again, while a straight national extrapolation for the BRFSS data alone can't be meaningfully done, Kleck tries, presuming that the ratio of the national DGU rate over the rate in the specific group of states that BRFSS happened to survey found in his NSDS should hold for the BRFSS as well, to make a national DGU rate guess from the BRFSS data. He ends up calculating national percentage rates for adults in gun-owning households nationally of 0.59 percent based on the 1996 states, 0.81 percent based on the 1997 states, and 1.82 percent based on the 1998 states.

After adjustments to get a guess for total adults, not just adults in gun-owning households, the range of total DGUs Kleck estimates for the nation with the above methods from the CDC's state-level surveys range from a low of 620,648 for 1996 to 1.9 million in 1998, for an average over the years of 1.1 million.

In Kleck and Gertz's NSDS, 10, 8, and 4 "weighted past year DGU cases" were found in the states BRFSS surveyed in the years 1996, '97, and '98 respectively. Given that very small number of actual pre-extrapolation DGUs Kleck found in the states that both he and the BRFSS covered in 1998, just four, making extrapolative adjustments based on them might seem to overstate his case.

As the adjustments work, for example, as spelled out for me initially by VerBruggen, had Kleck/Gertz found just two more DGUs in their surveys over the four 1998 states, the adjustments downward for the ratio of total U.S. DGUs over that group of states would be from 1.7 to 1.1, meaning that the national extrapolation for the BRFSS based on the NSDS for that year would be a whole number around 600,000 DGUs lower, and for that three-year average around 200,000 lower. That seems a lot of weight to place on such a tiny initial count. (There is also the wrinkle, as drawn out by economist Alex Tabarrok when discussing Kleck's first version of this paper, that with surveys regarding rare events, even a small percentage of liars, if the lies are distributed without any particular bias one way or the other, can very much overstate the phenomenon.)

A CDC representative, when asked about why no study using or publicizing the raw data on this DGU survey was ever issued, a matter Kleck speculates on quite a bit, wrote merely that "Data from the optional module data [asking the questions for those years' BRFSS were optional to the states, which is why only a few did] were made available to the public to analyze via the BRFSS public use dataset online" which is where Kleck eventually found them, though CDC never otherwise drew any conclusions from them in any publication nor drew anyone's attention to them.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • Dillinger||

    dude. the first sentence.

  • ||

    Do you ever wonder if maybe the writers don't have edit buttons either?

  • loveconstitution1789||

    They do. When the call out is bad enough, the article gets changed.

  • Dillinger||

    minimum pre-post standard should be "dude. read the first sentence."

  • Brett Bellmore||

    Every writer, if they don't have a formal proof reader, should at least have a critical friend they run stuff by. It's very difficult to spot your own typos, your brain autocorrects as you read them.

  • scJazz||

    and then the acronym spam... wtf... omg... lol... huh... googling... blah... blah... blah
    some more blah blah blah with acronym spam...
    wtf does this article (strike that) random shit word spam actually mean?

  • loveconstitution1789||

    I have pulled my concealed pistol 4 times in my life.

    3 times that nobody but the criminal and I know about and 1 time that the criminal will never hurt anyone again.

    The majority of people that carry weapons dont always tell authorities when they prevent a crime with their weapons. These instances are never counted.

    These studies are ridiculously under-guesstimated numbers.

    Even the guy mentioned in this article guy swung by 1 million events.

  • bvandyke||

    Wow, 3 times. Never had too, so far not even come close to it.

    Do you live in a not so good place or are you an outlier?

  • loveconstitution1789||

    2 times in Atlanta and 1 time in Las Vegas.

    Its always dum-dums who are really careless and desperate. They try shit at night. One guy had a NY accent so probably wasnt even from Atlanta.

    They always run away.

    Well, except for the one guy. He learned that some people can draw fast and know how to protect themselves with a gun.

    It will happen when you least expect it. I always keep a vigilant eye out. I hope you never have to deal with it.

  • BambiB||

    I've reached for my concealed weapon twice. In neither case did I allow the opposing person to know I was armed. Both times I was able to talk them down from a suicide attack.

    If you call my house and ask if I own any guns, you may get a "none of your business" answer. Or I may just lie and say, "no". I'm not keen on setting myself up for a robbery and I don't know who the fuck you REALLY are when you call.

    Under-reports like myself are likely more often the rule than the exception. For example, the number of firearms in private hands is widely assumed to be around 300 million. But there is good reason to believe that the actual number is in the 400 to 600 million range.

  • Exsqueezeyou||

    "But there is good reason to believe that the actual number is in the 400 to 600 million range."

    Agreed. FFS I would argue that by virtue of the proliferation of guns in America that many unarmed people are simply protected by a savvy criminal's risk assessment. High gun ownership rates in a given neighborhood or state will raise the risk of being shot and criminals (except meth and bath salt users) will be more risk averse. So, anti-gun persons benefit directly from actual gun owners exercising their rights. Even the anti-gunners without "Protected by Smith & Wesson" signs, get protected without touching a gun if they are perceived to be in sufficient proximity. This would be more akin to actual "herd immunity." Never see any anti-gunner progs with signs proudly declaring themselves and/or property to be gun free, do we. They are proud to declare and demand attention to their other choices.

    One need only look at the hot and cold burglary rates for Canada and the US to support this argument. And, no, it has nothing to do with the respective average temperatures of the 2 countries.

  • DiegoF||

    Whoa, four DGUs? All four on the street, concealed carry? I don't know anyone with nearly that many--although I know people with civilian kills. My condolences; no one should have to go though that. I hope you are at peace; if you are not, I know plenty who even now are in your shoes, even though they fully understand that they did what they had to and should bear no guilt.

    Anyway, where the fuck do you live, New Orleans?

  • Quo Usque Tandem||

    I used to live in New Orleans, but heeded the first rule of self protection [to avoid circumstances, as reasonably possible, that might potentiate such a conflict] and got the hell out.

  • loveconstitution1789||

    Outside Atlanta. Once was in Vegas.

    I dont lose a wink of sleep. My lady is more freaked out because of it. Telling her that violent crimes has gone down is not reassuring for her.

    I refuse to be a dead victim without fighting back.

  • Live Free Or Diet||

    I have pulled my concealed pistol 4 times in my life

    I carry CCW, and though I've never actually had to pull it, I have been very close a few times. One of those, a cop intervened for us.

    The defensive shooting I was in was a home invasion. I begged the guy to please just go away, but he tried to get me to hesitate by claiming my rifle was on safe as he started a rush. Two rounds of 5.56, and his mother and sister (each and together) sued me 5 different ways inside 10 years. Winning cost me over $200,000, never mind his break-in, weapon-waving and threats to my family were on video. I was awarded legal expenses, but "blood from a stone" and all that.

    What a nightmare.

  • BambiB||

    So the lesson from that is if you shoot... kill the entire family?

    Florida at least has the enlightened bit of law that if a criminal attempts to sue for being shot, their time in prison is compounded by a loss of privileges.

  • Earth Skeptic||

    Probably. It would help clean out the gene/culture pool.

  • Live Free Or Diet||

    Seems like it sometimes, huh?
    No, I don't mean that. I'm just glad they finally left me alone.

  • loveconstitution1789||

    It sounds like a nightmare.

    I get not telling us what state you live in but it sounds like a state where property rights and self defense are not respected.

    The lawsuit against you should have been laughed out of court in a summary judgment costing hundreds of dollars rather than hundreds of thousands of dollars.

    At least youre alive and the piece of shit criminals cant hurt anyone else.

  • Live Free Or Diet||

    I get not telling us what state you live

    I live in Virginia, and won each case rather easily, but it was still expensive. The plaintiff simply have nothing like the money to compensate me for my expenses. I've collected a total of about $12,000.

    The money part isn't really the bad part. I can afford the money. It was having to relive the whole thing so many times. It encourages you to second-guess yourself as they argue the "cruelty" of using "a military caliber" and "hunting ammunition." They tried to make a big deal of the Ruger Mini I used, but it was a very traditional-looking rifle without any optics or lights or anything else.

    The whole thing was recorded on 911's system, and on our home security system. Virginia and our county both quickly stated it was a clean, clear defensive shoot. Local police detectives volunteered their time testifying for me. I really cannot fault the government here in any way.

    My wife and children are alive and well, and, yes, that's what matters. I would do it all again in a heartbeat. It eats at me a lot less than it used to. I just suck it up and keep going. Better him than my kid.

    Sorry for this big emotional puke, but it has been helpful.

  • Exsqueezeyou||

    "...he tried to get me to hesitate by claiming my rifle was on safe ..."

    It was on safe...for you and not him. Hope he last heard, "Are you sure? click, bang!
    I can sincerely say I'm glad you made (it) home safe.

  • Live Free Or Diet||

    That you, sir.

  • FlameCCT||

    Thankfully you and your family are safe. I almost pity the people that sued you because they were just being used by dirtbags looking for a payday on their and your misery.

  • vek||

    I'm sorry to hear you had such a tough time just for defending yourself. In a sane world that kind of thing should have been laughed out of court in 30 seconds. What nonsense.

  • DrZ||

    Why would you ever want to tell LE about a thwarted mugging when you pulled the weapon and put it to a stop. There are horror stories about LE taking away guns (when they are used) and people never getting them back or getting them back after many months. If I made a bad guy run by using a weapon, I would not tell the police either.

  • TxJack 112||

    True because if you pull the gun and do not use it, it could be argued that the threat did not warrant the use of deadly force so your actions were "brandishing a firearm" and could result in your arrest. However, I think there are many instances when simply pulling a gun is enough to stop an incident. I know of at least one personally where a gun was used to stop a road rage attack.

  • Henry||

    Because if the perp reports the "ADW" to the cops before you do, he automatically becomes the "victim," and gets to leverage the justice system against you as his second bite at winning the ghetto lottery.

  • Oli||

    That's what I mean when I say the US has a violence problem. Not people pulling guns in self defense, but people having to pull guns in order to save their life.

  • vek||

    Thinking about it, I can recall 4 times in my life where I may have been inclined to bust out if I had a CCP. 1 was technically when I was too young to have been able to carry anyway, but 3 were as an adult. 2 of the 3 as an adult I managed to get out of the situation unscathed physically or monetarily, but a little rattled. The 3rd I basically had to toss some crack head a chunk of the money in my wallet, but not all of it, to avoid it turning into something sketchier. I actually started carrying pepper spray awhile back, but didn't have it on me when one of those incidents above happened! Argh! It was one I got out of though, so I guess all I loss was the chance to blind and kick the shit out of a mugger...

    I should really get a CCP though... That's just too many times for a person to have shit like that happen, and I'm only in my early 30s! Seattle isn't even an especially dangerous city either. I can only imagine being in a proper dangerous city.

  • Jerry B.||

    "Criminologist Gary Kleck revises his paper on the incidence of the use of firearms for self-protection"

    Or, as the Washington Post will headline it...

    "Pro-gun criminologist admits he lied about self-defense numbers"

  • bvandyke||

    Very good!

  • DiegoF||

    By the way, in case you haven't read Kleck's books, the only political organization he has ever belonged to is the ACLU, and he is a prog with absolutely no ideological affinity for gun rights. He is just that rare creature--the public policy academic who cares more about telling the truth than in being a soldier for some political cause and putting his academic status in service of that end.

  • Quo Usque Tandem||

    As rare as...an honest politician?

  • Scarecrow Repair & Chippering||

    Not quite. This one actually does exist.

  • AD-RtR/OS!||

    A million here, a million there, pretty soon that adds up to some serious life saving.

  • JesseAz||

    So even with the lower number... 30 defensive uses per death. If you remove suicides like an intelligent person it is near 50 more defensive uses. Take away justified uses, defensive, it is 60-70 times per death... Yes. Run on this liberals.

  • Otto Didact||

    No sense trying to use this on liberals. They know (believe? feel?) what they know (see earlier) and have no desire to be confused bu the facts. Remember, Liberals have their "truth" even if it bears no resemblance to what you or I might consider truth.

    Michael Savage says that liberalism is a mental disorder. I disagree. Liberals aren't crazy; they're "otherly sane". By their lights WE'RE the deluded ones.

  • scJazz||

    Hey Brian... make sure you and your editor are actually not totally hung over before submitting!

  • Michael Cook||

    I have been forced to display a weapon for defensive purposes twice and neither time did it result in a police report. Why would it? The guy who ran away isn't telling and I know enough about our problematic legal system not to open a can of worms.

    My first wife and my sister each had one such experience, with my sister firing one shot through a very expensive picture window. She didn't even report that to police, although it meant she had to replace the glass out of pocket in order to not involve the insurance company.

  • Earth Skeptic||

    These types of DGU seem to fit an "active" category, i.e. a weapon in hand or otherwise deliberately brought to attention. Has anyone tried to quantify the inhibitive effect of more "passive" weapon roles? Like a potential criminal act discouraged by someone with open carry, a driver with a long gun in a window rack, etc. And the even more abstract level, where a bad guy figures that some target just might be armed and goes away?

    Kinda like how we "use" our army and navy every day, since just knowing they exist affects how other countries make decisions.

  • FlameCCT||

    IMHO the deterrence affect is better than our nuclear arsenal MAD (Mutual Assured Destruction) policy because defensive usage usually ends with DAG (Dirtbag Assured Destruction).

  • TxJack 112||

    The one issue that gun owners never address that should become our point of rebutting gun control claims is one that gun grabbers use which is a total fabrication. Police do not prevent crime, the investigate crime. It is impossible to prevent crime and at best you can deter a very minimal amount of it in very specific locations. Police always arrive after a crime has occurred, never before. Since it is impossible for police to actually protect you in a life or death struggle, the only choice is protecting yourself.

  • Michael Cox||

    Well said.

  • FlameCCT||

    Well said.
    When you only have seconds, police are minutes away.

  • AZ Gunowner||

    All of this of course is a moot discussion.

    The right to arms, the right to self-defense, are not subject to cost/benefit calculations.

    Even if it were true that the ratio was the reverse - more criminal use of guns than DGU - that wouldn't morally justify removing any individual's right to defend him/herself.

    To do so makes the individual a sacrificial offering to "the greater good".

    A reprehensible idea.

  • Desert Raton||

    My guess is that reporting of DGUs in states like Maryland and NJ (where it is almost impossible to get a CCW permit) is significantly lower than in states which are less hostile to the 2nd Amendment. Did the extrapolation to national numbers of DGUs include adjustments for this?

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online