Did the Education Department Badly Err in Its Count of Shootings at Schools?

An illustration, it seems, of how badly statistics can go wrong (and of the principle "Garbage In, Garbage Out").

NPR (Anya Kamenetz) reports:

This spring the U.S. Education Department reported that in the 2015-2016 school year, "nearly 240 schools … reported at least 1 incident involving a school-related shooting." The number is far higher than most other estimates.

But NPR reached out to every one of those schools repeatedly over the course of three months and found that more than two-thirds of these reported incidents never happened….

We were able to confirm just 11 reported incidents, either directly with schools or through media reports.

In 161 cases, schools or districts attested that no incident took place or couldn't confirm one. In at least four cases, we found, something did happen, but it didn't meet the government's parameters for a shooting. About a quarter of schools didn't respond to our inquiries.

The Education Department report is based on surveys returned by schools, apparently without any real checking, and some of the source data seems to have been badly wrong:

[The survey] reports 26 shootings within the Ventura Unified School District in Southern California.

"I think someone pushed the wrong button," said Jeff Davis, an assistant superintendent there. The outgoing superintendent, Joe Richards, "has been here for almost 30 years and he doesn't remember any shooting," Davis added. "We are in this weird vortex of what's on this screen and what reality is."

Likewise, 37 incidents were listed for the Cleveland school district, but apparently "37 schools reported 'possession of a knife or a firearm,' which is the previous question on the form." "The number 37, then, was apparently entered on the wrong line." A reminder of how cautious we should be in evaluating data, even data reported by seemingly authoritative federal agencies.