A Survey Not Designed to Measure Defensive Gun Use Finds Little of It
Do less than 1 percent of crime victims defend themselves with firearms?
A study in the latest issue of Preventive Medicine estimates that less than 1 percent of crime victims use guns in self-defense. The authors, Harvard health policy professor David Hemenway and University of Vermont economist Sara Solnick, find that using a gun seems to be effective at reducing property loss but "is not associated with a reduced risk of victim injury." It will surprise no one familiar with the long-running debate about defensive gun use (DGU) that the source of the data for this study is the National Crime Victimization Survey (NCVS), which consistently generates much lower DGU estimates than other surveys do. At least some of that gap can be plausibly explained by weaknesses in the NCVS that Hemenway and Solnick do not seriously address or, for the most part, even mention.
The biggest strength of the NCVS, which is conducted annually by the U.S. Census Bureau on behalf of the Justice Department's Bureau of Justice Statistics, is its large, nationally representative sample, which includes 90,000 households and about 160,000 individuals. The survey's biggest weakness in this context is that it is not designed to measure DGUs, and there are good reasons to think it misses a lot of them.
Looking at data from 2007 through 2011, Hemenway and Solnick found "14,145 crime incidents in which the victim was present at the incident." Just 127 of those victims, about 0.9 percent, used a gun defensively, and their injury rate (4.1 percent) was about the same as the overall injury rate (4.2 percent) for victims who took any sort of protective action. Judging by the injury rates, a gun looks more effective than other weapons, unarmed resistance, or screaming, but less effective than running away or calling the police.
As Hemenway and Solnick concede, such comparisons may be misleading. "Instances of self-defense gun use [SDGU] may differ in many ways—including ways we could not control for—from instances where the victim used other forms of self-defense or took no self-protective action," they write. Fleeing and calling the police, which on the face of it seem like the safest options, are not always feasible, and the situations where victims use guns in self-defense may be more dangerous to begin with. Notably, victims who reported any kind of protective action were almost twice as likely as the other victims to have been injured before reacting, which suggests, as you might expect, that victims adjust their responses according to the threat they perceive from their attackers.
When they considered property loss, Hemenway and Solnick found stronger evidence that guns can help. While 85 percent of victims who took no protective action lost property, the rate for victims who used guns was 39 percent. Victims who used other weapons did a little a better, losing property 35 percent of the time. By comparison, most victims had something taken when they resisted without weapons, argued, cooperated, screamed, or called police. "Compared to other protective actions," Hemenway and Solnick conclude, "the National Crime Victimization Surveys provide little evidence that SDGU is uniquely beneficial in reducing the likelihood of injury or property loss."
That interpretation seems too strong, since the survey does provide evidence that guns (and other weapons) are pretty effective at preventing theft. But the real problem is that the 127 DGUs identified by this survey may represent just a fraction of respondents' DGUs, for reasons that Florida State University criminologist Gary Kleck has long emphasized. The NCVS, unlike surveys by Kleck and others that have generated higher DGU estimates, is not anonymous. Respondents have to supply their names and contact information, they are initially interviewed in person by a representative of the federal government, and they know the survey is commissioned by the Justice Department, a law enforcement agency. Hence it is plausible that some respondents remain silent about their DGUs because they worry that their actions could be legally questionable, given all the restrictions on where and when people may use firearms.
Furthermore, as Hemenway and Solnick acknowledge, respondents are never directly asked about defensive gun use, and they are given an opportunity to describe it (along with any other actions they may have taken in response to an attack) only after identifying themselves as crime victims. Kleck notes concerns that the NCVS seriously undercounts crimes such as spousal assault and rape, which implies that it undercounts DGUs by victims of those crimes. People who use their guns to prevent crimes—say, scaring away a would-be burglar or rapist simply by announcing that they are armed—may not see themselves as crime victims and therefore may not get a chance to report their DGUs in this survey, even assuming they are inclined to do so. A combination of these factors may explain why Hemenway and Solnick find not a single sexual assault prevented by a gun in five years of NCVS data.
It's possible, of course, that some respondents asked about DGUs in surveys like Kleck's make up stories (despite his best efforts to distinguish between real events and tall tales), misconstrue or misremember ambiguous situations, or report DGUs as more recent than they really were (which would affect the estimate of annual DGUs). There is an enormous gap between the 100,000 or so DGUs per year indicated by the NVCS and the estimates generated by other surveys, which range as high as 2.5 million. It is tempting to conclude that the truth lies somewhere in between, although Kleck argues that even 2.5 million may be an underestimate given people's reluctance to report, even in an anonymous survey, actions that might get them into legal trouble.
Hemenway and Solnick predictably argue that Kleck's numbers are improbably high, and it is tempting to surmise that the truth lies somewhere in between. But while Kleck has responded at length to criticism of his methods and conclusions, Hemenway and Solnick's article barely alludes to the NCVS weaknesses he highlights. They do mention that the survey includes "no specific questions about self-defense gun use." That's a pretty big flaw for a study aimed at measuring self-defense gun use, which the NCVS isn't.
Brian Doherty has more on the controversy over counting DGUs.