A Lott More Lies: Debating More Guns, Less Crime
By: Devin Hughes and Evan DeFilippis
John Lott and I had a debate on the David Pakman Show over the question of whether or not more guns decrease the crime rate and whether “gun-free zones” result in more mass shootings. Below are a few of the more salient points in the debate I would like to highlight. For a more thorough analysis of Lott’s work, please check out our article “Shooting Down the Gun Lobby’s Favorite ‘Academic’: A Lott of Lies.”
“The Vast Majority of Academic Studies Support More Guns = Less Crime.”
For the reasons I highlight in the debate, Lott’s assertion that the vast majority of studies agree with him is deeply disingenuous. In the last minutes of the debate, Lott stated that “Cherry picking would be if I ignored them.” Below is the list of studies Lott ignores in his list of studies on concealed carry, and each has a note showing whether he mentions the study in “More Guns, Less Crime.” In the debate I mention that he excludes 7 studies in his cherry picking. The number is actually at least 9:
Regression to the Mean, Murder Rates, and Shall-Issue Laws
http://www.tandfonline.com/doi/abs/10.1198/000313008X362446#.VBi20fldXVY (Briefly mentioned in footnote 91)
The Effect of Nondiscretionary Concealed Carry Laws on Homicide
Hepburn, Miller, Azrael, and Hemenway
http://www.ncbi.nlm.nih.gov/pubmed/15128143 (Not Included)
Myths of Murder and Multiple Regression
http://crab.rutgers.edu/~goertzel/mythsofmurder.htm (Not included)
Flawed gun policy research could endanger public safety
Webster, Vernick, Ludwig, and Lester
http://ajph.aphapublications.org/doi/abs/10.2105/AJPH.87.6.918 (Not included)
Two Guns, Four Guns, Six Guns, More Guns: Does Arming the Public Reduce Crime?
Concealed Handguns: The Counterfeit Deterrent
Zimring and Hawkins
The Final Bullet in the Body of the More Guns, Less Crime Hypothesis
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=431220 (Included and mentioned on the list, but not given a number)
An evaluation of state firearm regulations and homicide and suicide death rates
Rosengart, Cummings, Nathens, Heagerty, Maier, Rivara
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1730198/pdf/v011p00077.pdf (Not included)
One of these two studies (they both occur in the same year in the same journal, so it isn’t possible to determine which one Lott is excluding, as he only lists one):
Yet Another Refutation of the More Guns, Less Crime Hypothesis — With Some Help From Moody and Marvell
Ayres and Donohue
http://econjwatch.org/articles/yet-another-refutation-of-the-more-guns-less-crime-hypothesis-with-some-help-from-moody-and-marvell (Not included)
More Guns, Less Crime Fails Again: The Latest Evidence From 1977-2006
Ayres and Donohue
http://econjwatch.org/articles/more-guns-less-crime-fails-again-the-latest-evidence-from-1977-2006 (Not included)
One is forced to wonder why Lott excludes entire fields of well qualified academics, especially those who work in public health. Also, unlike what Lott insinuated in the debate, all these studies deal with crime on a national level. For a more detailed discussion of why many of the studies Lott includes on his side are fatally flawed, see our previous article on Lott’s work.
Lott also claims that a majority of academics agree with him. Let’s do a simple counting exercise of the academics who have published on each side of this including his own list of studies:
|Reduces Crime||No effect or Increases Crime|
*Indicates a panel member of the National Research Council.
** Not included in Lott’s list
Author of study showing a crime increase.
(1) Moody and Marvell have studies showing RTC Laws reduce crime and others that show no effect. (2) Both Maltz and Plassmann have backed away from their own studies showing a reduction in crime.
Even according to Lott’s own list of the studies, the number of academics that disagree with him heavily outnumber those on his side. When we include the academics that Lott ignores, it’s not even close. It is readily apparent that the academic community is not on Lott’s side.
“No Study Finds Negative Effects from Right-to-Carry Laws”
Lott opens the debate with the bizarre and obviously false claim that ‘No study finds negative effects from RTC laws.’ He then retreats to the safer assertion that no study has found evidence of an increase in murder after concealed carry laws are passed. This too is also false. Let’s examine both the studies that find evidence of crime increases as well as those that find increases in the murder rate (emphasis is added).
From the conclusion: “A ‘‘shall issue’’ law that eliminates most restrictions on carrying a concealed weapon may be associated with increased firearm homicide rates.”
As Ayers and Donohue explain:
“Moody and Marvell have given us a single new regression using the Lott county-level data that we corrected back in 2003 and analyzed in Ayres and Donohue (2003b) and that the National Research Council analyzed in its 2004 report. Moody and Marvell use this single regression – a state-specific hybrid model estimated on county data for the years 1977-2000 – to estimate the overall change in the cost of crime attributed to RTC laws for the period that these laws have been in effect for the 24 states they evaluate. Their own table indicates that RTC laws on net increase the costs of crime (albeit statistically insignificantly) in aggregate for 23 of the 24 jurisdictions they examine, but cause massive benefits in the single state of Florida.
In our view, Moody and Marvell’s state-specific estimates support the view that RTC laws generally do not lower and may increase overall crime costs – at least if one endorses Marvell’s own paper (with Kovandzic) arguing that the issuance of RTC permits did not alter crime in Florida. Since the Kovandzic and Marvell paper examining the impact of RTC law on Florida crime is listed in Moody and Marvell’s Table 2 as standing for the proposition “Shall Issue Has No Significant Effect on Crime,” one might assume that they conclude that RTC laws have not lowered crime in Florida.”
From the conclusion:
“What is to be made of the aggregated evidence concerning the impact of RTC laws when one extends the state panel data through 2006? The one consistent finding that is statistically significant for the hybrid model in Tables 2, 3, and 4 is that RTC laws increase aggravated assault. The point estimates across all three tables are generally consistent with higher rates of murder and robbery, although these estimates are not statistically significant. In general, one might assume that the biases from inadequate controls for crack and general measurement error would tend to bias those results to zero, so it may be the case that better information and models would reveal that RTC laws increase murder and robbery as well as aggravated assault. The mixed evidence on rape and auto theft leaves little basis for conclusion with respect to these crimes.”
From the abstract:
“Across the basic seven Index I crime categories, the strongest evidence of a statistically significant effect would be for aggravated assault, with 11 of 28 estimates suggesting that RTC laws increase this crime at the .10 confidence level. An omitted variable bias test on our preferred Table 8a results suggests that our estimated 8 percent increase in aggravated assaults from RTC laws may understate the true harmful impact of RTC laws on aggravated assault, which may explain why this finding is only significant at the .10 level in many of our models. Our analysis of the year-by-year impact of RTC laws also suggests that RTC laws increase aggravated assaults. Our analysis of admittedly imperfect gun aggravated assaults provides suggestive evidence that RTC laws may be associated with large increases in this crime, perhaps increasing such gun assaults by almost 33 percent.
In addition to aggravated assault, the most plausible state models conducted over the entire 1979-2010 period provide evidence that RTC laws increase rape and robbery (but usually only at the .10 level). In contrast, for the period from 1999-2010 (which seeks to remove the confounding influence of the crack cocaine epidemic), the preferred state model (for those who accept the Wolfers proposition that one should not control for state trends) yields statistically significant evidence for only one crime — suggesting that RTC laws increase the rate of murder at the .05 significance level. It will be worth exploring whether other methodological approaches and/or additional years of data will confirm the results of this panel-data analysis and clarify some of the highly sensitive results and anomalies (such as the occasional estimates that RTC laws lead to higher rates of property crime) that have plagued this inquiry for over a decade.”
Dr. Moody and his colleagues report that RTC laws decrease murders and there is no significant evidence on both a state and county level for any other crime category. However, they misread their own results, which actually find that RTC laws significantly increase aggravated assaults, providing further evidence that Donohue and his colleagues are correct about the crime inducing nature of these laws.
Here is the data table I referred to in the debate, with the error surrounded by a red box (the t-stats both clearly show a statistically significant effect):
Nowhere in the Moody paper does it explain why a significant T-stat is un-bolded or undiscussed in the conclusion, despite the fact that it directly undermines the thrust of their entire paper. One wonders if the thank you note at the beginning of the paper has something to do with it: “The authors thank The Crime Prevention Research Center for its support.” (John Lott’s organization).
“You Misinterpret the Results from the National Research Council”
Not true. They found, closely following Lott’s model and data, that there was insufficient evidence to support the more guns, less crime hypothesis. To refute a hypothesis, you don’t have to prove the opposite. You merely have to show that the evidence does not support the original claim. This is quite literally Statistics 101—evidence is either sufficient or insufficient to reject the null hypothesis.
The fact is that Lott has managed to construct a career out of the singular and extraordinary claim that More Guns = Less Crime, and has provided little substantive evidence, let alone extraordinary evidence to support that claim (as concluded by 15 of the most prominent academics in the United States). Indeed, writing about the weaknesses of Lott’s work, David Hemenway from the Harvard School of Public Health wrote, “Lott’s approach virtually ensures that, whatever the true relationship between guns and death, his analysis will not find them.”
The conclusion of the National Research Council can be read here. A curious note on page 126 reads, “Through communication with Lott, the committee learned that the data used to construct Table 4.1 of Lott (2000) were lost and that the data supplied to the committee are a reconstruction and not necessarily identical to the original data.” How convenient.
To be clear, the reason that the National Research Council’s conclusion contained any ambiguity at all is simply because they followed Lott’s methodology too closely, despite it being riddled with obvious errors. First, Lott’s dataset that was used by the panel had several errors, which Donohue corrects. Both Philadelphia and Idaho, for example, had their year of adoption for concealed carry laws coded incorrectly.
Second, the panel failed to incorporate proper criminal justice control variables by paralleling Lott’s model. Specifically, Lott fails to include controls for police or incarceration, which are included by many studies on crime. In addition, Lott’s use of arrest rate variables (which is echoed in the NRC review) is extremely problematic. Donohue explains in a footnote, “The Lott and Mustard arrest rates instead are a ratio of arrests to crimes, which means that when one person kills many, for example, the arrest rate falls, but when many people kill one person, the arrest rate rises since only one can be arrested in the first instance and many can in the second. The bottom line is that this “arrest rate” is not a probability.”
Third, and perhaps the most important omission corrected was the lack of clustered standard errors. As Donohue explains, not clustering standard errors (which is now standard practice among econometricians) drastically increases the odds that a spurious result will incorrectly be deemed as statistically significant.
When correcting these flaws, which by any objective standard are errors, the conflicting results the NRC model’s show vanish. Instead, there is clear evidence that RTC laws increase crime, particularly aggravated assaults. Dr. Donohue then tweaks the model, using a far more reasonable set of control variables than Lott’s 36 demographic ones along with adding other critical controls that were left out. The results consistently show an increase in aggravated assault.
Also during the debate, Lott makes some rather bizarre claims about the most recent study by Dr. Donohue, which shows significant increases in aggravated assaults after concealed carry laws are passed. He mostly relies on the claim that there are massive errors in the Donohue study, and after correcting those errors the results go away. This is blatantly false. Not only does the study Lott cites (and coauthored) not show that the results vanish (it merely points out the errors), but the most recent version of the Donohue study has corrected all of these errors, which had no significant impact on the results.
“All the mass public shootings in the United States, with the exception of 2 shootings, occurred in gun-free zones”
- This is simply incorrect. The Everytown Institute on Gun Safety identified 18 instances of mass shootings in the last 6 years that did not take place in “gun-free zones.”
- John Lott definition of a “gun-free zone” is intentionally imprecise. He goes to extraordinary efforts to insist that a place is a gun-free zone when there is clear evidence to the contrary.
To give some glaring examples of Lott’s deliberate deceptiveness:
- Hialeah, Florida: In the debate, Lott asserts that he called the restaurant, and that’s how he learned that the restaurant was a gun free zone. He doesn’t specifically mention that he called the restaurant in his report, though given how deceptively written his section on this shooting is, it is certainly possible. Either way, the evidence clearly shows that the restaurant legally is not a gun free zone. Florida law separates the bar area from the rest of the restaurant for concealed carrying. This partition is clearly evident in pictures of the place, and concealed carriers by law would be allowed to carry everywhere but the bar area. Further, we can clearly see from the menu that this is a restaurant dedicated to food, not alcohol.A letter from the concealed weapons division of the Florida Department of Agriculture clearly explicates this issue: the law is written in such a way as to “allow the carrying of firearms in restaurants or similar businesses that primarily serve food but that also happen to serve alcohol as well. In other words, a restaurant may in fact have a bar where alcohol is dispensed for consumption on the premises. However, the serving area in the restaurant where patrons are dining would not constitute the part of the establishment primarily devoted to the sale and consumption of alcohol.”
- Boston: John Lott sloppily classifies ENTIRE CITIES as gun-free zones because, it is difficult to get a concealed carry permit in those cities. There’s no objectivity or precision. He conveniently ignores facts such as Boston being a commuter city that doesn’t place restrictions on people traveling into Boston with concealed carry. We also know that there are an estimated 250,000 people with concealed carry licenses in Massachusetts, so one wonders why Lott considers Boston a “gun-free zone.”
- Geneva County, AL: Lott excludes this as a mass shooting because not all of the victims were killed in public, due to the fact that some were killed in their homes. Lott ignores the fact that homes are not “gun-free zones”, either.
- Lakewood, WA: In Lakewood, four police officers were murdered in a coffee shop by a mass shooter. Lott defines Lakewood as a “gun free zone” even though all the persons killed in the shooting were armed. Lott claims that the presence of a gun has a tremendous deterrence effect on mass shootings, and then simply ignores shootings where every single victim has a gun.
The list goes on. It is clear Lott is simply massaging the data to support a particular conclusion.
“Gun Free Zones Result in More Mass Shootings”
To support his claim that gun free zones attract mass shooters, Lott cites one of his own studies that purportedly shows that concealed carry laws decrease mass shootings by substantial margins. He further provides the examples of the Moncton shooter in Canada and the Isla Vista shooter in California as support for his contention. However, all of this evidence quickly falls apart on closer inspection.
A study by Duwe, Kovandzic, Moody in 2002 evaluated 25 RTC laws from 1977 to 1999 and found, “virtually no support for the hypothesis that the laws increase or reduce the number of mass public shootings.” Lott’s mass shooting results simply aren’t supported by other academics, even those like Moody who typically side with him on concealed carry laws.
The Moncton shooter Lott references certainly did mock gun free zones. But that’s all he did, mock. When he initiated his attack, he didn’t target gun free zones. Quite the opposite. He targeted the Canadian Mounted Police, and all of his victims were armed. It is bizarre that Dr. Lott would see a mass shooter targeting men with guns as proof of his theory that mass shooters try to avoid men with guns in order to maximize casualties.
In the case of the Isla Vista shooter, his motivation was clearly revenge against women for rejecting him. While he did try to plan his attack to minimize resistance, choosing a sorority house as a target clearly reflected his revenge motivation, and his plan to avoid resistance clearly wasn’t well thought out as he was forced to exchange gunfire with police officers twice.
The one case that would bolster Lott’s case the most is the Aurora Theater shooting. Here we have case of somebody seeking to inflict the maximum amount of casualties in a well-planned attack. However, this is just one isolated case. And even the gun free zone motivation isn’t as clear cut as Lott would have us believe.
If the motivation behind public shootings is the important variable to study, we should examine those shootings in public that fail to reach high casualty counts as well. The FBI has compiled such a study, examining 160 active shooting incidents between 2000 and 2013. In 71 of the 112 (63%) shootings that occurred in commercial and educational areas, the shooters had some relationship with the school or business. If the shooter’s driving motivation is to inflict as many casualties as possible it is really hard to explain this pattern. Further, there are many of these shootings that occur in places typically not seen as gun free zones. The evidence clearly shows that public shooters are not casualty-maximizing machines meticulously planning so as to avoid armed resistance. Instead, these tragedies are most often the result of the shooter snapping due to psychological trauma, and seeking to harm the people they know.
A Lott More Lies
Lott’s reliance on falsehoods to carry him through the debate was unfortunate, though not surprising given his extensive history of ethical transgressions which we have detailed previously. Almost all of Lott’s points were either outright false or highly misleading. You can examine the evidence detailed in “Shooting Down the Gun Lobby’s Favorite ‘Academic’: A Lott of Lies” and determine whether Lott is worthy of your trust.