Did the Education Department Badly Err in Its Count of Shootings at Schools?

An illustration, it seems, of how badly statistics can go wrong (and of the principle "Garbage In, Garbage Out").

NPR (Anya Kamenetz) reports:

This spring the U.S. Education Department reported that in the 2015-2016 school year, "nearly 240 schools … reported at least 1 incident involving a school-related shooting." The number is far higher than most other estimates.

But NPR reached out to every one of those schools repeatedly over the course of three months and found that more than two-thirds of these reported incidents never happened….

We were able to confirm just 11 reported incidents, either directly with schools or through media reports.

In 161 cases, schools or districts attested that no incident took place or couldn't confirm one. In at least four cases, we found, something did happen, but it didn't meet the government's parameters for a shooting. About a quarter of schools didn't respond to our inquiries.

The Education Department report is based on surveys returned by schools, apparently without any real checking, and some of the source data seems to have been badly wrong:

[The survey] reports 26 shootings within the Ventura Unified School District in Southern California.

"I think someone pushed the wrong button," said Jeff Davis, an assistant superintendent there. The outgoing superintendent, Joe Richards, "has been here for almost 30 years and he doesn't remember any shooting," Davis added. "We are in this weird vortex of what's on this screen and what reality is."

Likewise, 37 incidents were listed for the Cleveland school district, but apparently "37 schools reported 'possession of a knife or a firearm,' which is the previous question on the form." "The number 37, then, was apparently entered on the wrong line." A reminder of how cautious we should be in evaluating data, even data reported by seemingly authoritative federal agencies.

Comments

[Chem_Geek]

There was that "school shooting" the other year in Lansing, where someone shot himself for suicide, in a parking lot of a school building that had been closed for years.

But yeah, OMG, ban all teh gunz now for the children!

[Rock Lobster]

"A reminder of how cautious we should be in evaluating data, even data reported by seemingly authoritative federal agencies."

As reported elsewhere, the statute of limitations on corrections has apparently passed, so we'll just have to deal with these gross inaccuracies until next year, when they'll make it all better.

This is a sterling example of the dispassionate "experts" the progressives who would have overseeing every aspect of our lives if they were given the power to do so.

[Stephen Lathrop]

Wasn't it the Trump administration which published the goofy survey? And the progressives at NPR which busted the inaccuracies? Must every single right wing comment on this forum be a pre-scripted attack?

[Toranth]

The survey has been going on for decades, but these questions were added in 2015. The data was gathered in late 2016.

Good for NPR for busting these numbers, though. Anyone with a touch of common sense should have seen they were false - just like the "400% increase" claim that was in a different article - but it served the agenda of some groups. It's good to have a major media source publicly exposing these problems.

[Brett Bellmore]

Yes, nice to see that NPR isn't all the way down the gun control rabbit hole, they've still got some standards.

[Bob from Ohio]

"Trump administration "

Bureaucrats published it. Most non military, non law enforcement federal civil "servants" are leftists [especially in make work departments like Education].

[Sarcastr0]

I'd be surprised if Ed. allows any publication that isn't vetted by a political first.

It's just ordinary stupidity; no need to gin up it being some liberal plot.

[Bob from Ohio]

Its not a plot, its just reality. Liberals dominate the "apolitical" civil "service".

[Sarcastr0]

The scenario you posit is that the liberals in the civil service are throwing aside any sort of professionalism and publishing (easily exposed) lies on purpose, I'd call that a plot, though a dumb one.

[Bob from Ohio]

"professionalism"

LOl. From bureaucrats?

Lazy, stupid, lazy and stupid. The 3 types of bureaucrats.

[Sarcastr0]

Well, alright then.

[DjDiverDan]

Sarcastro, you are right, this is just ordinary stupidity. And the biggest difference between government and the private sector is that in the private sector, ordinary stupidity gets punished, while in government, there is no cost to pay for ordinary stupidity. And that is a very good reason not to trust government to do anything but its essential functions, and even then keep a very close eye on it.

[NToJ]

"...while in government, there is no cost to pay for ordinary stupidity."

This is false.

[gormadoc]

Yeah, those NYC project workers having orgies on the job were reassigned to other positions. That was a big cost to them.

[Naaman Brown]

The survey period was 2015-2016.
The results were published during in the Trump Administration.
The President did not publish them.
The White House staff did not publish them.

Which reminds me that Barak Obama, the White House staff, that administration and Democrat politicians did take the 1994 NSPOF stat that 60% of a sample of uninstititutionalized adults living in homes with phones who owned guns bought them from licensed dealers and they did claim that meant 40% of criminals acquired guns from unlicensed dealers at gun shows. The Obama administration misrepresented survey results to push an agenda.

When did Trump use this DoE survey?

[apedad]

"This is a sterling example of the dispassionate "experts" the progressives who would have overseeing every aspect of our lives if they were given the power to do so."

Well how should we work on problems and address issues?

Follow God's word or the take the advice of ignorant Tennessippians?

[Absaroka]

Tennessee and Mississippi are close, but what we should do is look to Missouri - the 'Show Me' state, and not blindly trust people who assert they know what they are doing.

People in government or think tanks don't live lives of monastic academic excellence - they can be sloppy, wrong, and have agendas just like anyone else. They ought to be viewed like people on Craigslist selling their aunt's only-driven-to-church-on-sunday car: trust, but verify.

I'm going to hazard the guess that you thought the invasion of Iraq was a mistake (I sure thought it was). The ranks of the neocons who were pushing that were chock full of impressive credentials; that decision wasn't made by a bunch of high school dropout Walmart clerks. If the other side's impressively credentialed team can get things very wrong, you ought to be open to the possibility that your own side's impressively credentialed team can be equally wrong.

[Bob from Ohio]

"ignorant Tennessippians?"

A bigoted statement. Unsurprising from the "tolerant" left.

[Rev. Arthur L. Kirkland]

Mississippi and (less severely) Tennessee have been political, educational, economic, cultural, and moral drags and stains on the United States for as many centuries as the United States has.

Mississippi and Tennessee should be unincorporated territories, not states.

Recognizing or mentioning these points =/= bigotry.

[bernard11]

I lived in TN for many years, have friends there, and am generally fond of the place.

While I don't like the way political trends in TN have gone lately - even before Trump - it is simply wrong to call it a "political, educational, economic, cultural, and moral drag and stain on the United States."

The state's Republican party used to be dominated by political descendants of Howard Baker. That it has been overrun by imbeciles like Marsha Blackburn is unfortunate, but I think it likely that Bredesen will win that Senate race.

Regardless, it's a far cry from your description.

[Naaman Brown]

In ALK's fan-fic of The Hunger Games, he is President Snow of the Capitol and Tennessippia is District 13.

[Procyon Mustelid]

Well, padt of it depends on whether you treat East, Middle, or West as the just-plain Tennessee you're talking about. I never had much to do with West Tennessee and Middle Tennesse's alright, but fuck East Tennessee.

[TwelveInchPianist]

Don't ever change, Arthur.

[Very definition of bigotry] =/= bigotry

[FlameCCT]

Start with a root cause analysis and address the real issue(s) instead of addressing the symptoms.

[James Pollock]

Depending on who's doing the counting, "gun-related" incidents can vary widely. School shootings with actual bodies are hard to ignore, unless you're high on Alex Jones, but things like having firecrackers go off in the schoolyard and some neighbor reports hearing shots, to things like having a cop draw on a potentially-threatening situation, are "gun-related"... but not really gun-related.

Literally everybody should go find a copy of a book called "How to Lie With Statistics", originally published back in the 1950's or so.

[Rossami]

Agreed.
- 1954 edition
- reprinted in 1993

That said, this is still my favorite statistics text by a long shot.

[tkamenick]

My two favorite statistics:

The average adult has about one testicle and one breast.
The vast majority of humans have an above-average number of legs and arms.

[MatthewSlyfield]

"The vast majority of humans have an above-average number of legs and arms."

Amputees drag the "average" down. 🙂

[Procyon Mustelid]

Umm... men have breasts. How do you think we have nipples?

So it should be two breasts, but then there's mastectomies dragging the average down. Plus, being nitpicky, I don't think there's actually a 50/50 male/female split among adults, and don't testicular cancer patients have to get testectomies?

[MatthewSlyfield]

"Plus, being nitpicky, I don't think there's actually a 50/50 male/female split among adults"

It depends on age, but overall, it's pretty darn close to 50/50, the global gender ratio is 102 males to 100 females (that's 50.495 to 49.505).

The reason it depends on age is that woman on average live longer than men, so the % of women increases with age.

US Male to Female Ratio by age

for the US, the male / female ratio is

1.05 at birth

0.99 at age 35

0.90 at age 65

0.51 for 85 and older

[Naaman Brown]

We had an incident in Johnson City during which armed robbers fled through a school zone pursued by police prompting a school lockdown. I have wondered if that had ever been reported in one of those lists as a school shooting. I did find out that a non-student body with a gunshot wound had been dumped by non-students on a school ground and listed as a school shooting, even though the shooting actually occurred elsewhere.

[Naaman Brown]

Anya Kamanetz, " The School Shootings That Weren't", NPR Morning Edition, 27 Aug 2018.

I'll repeat myself. Such quality of data in that US Dept of Edu report criticized by NPR!

Omro school district in Wisconsin asked the US Dept of Edu:
should a consensual paintball-gun fight involving students be reported as an "attack with a weapon" or as a "possession of a firearm."

NPR contacted Santa Monica-Malibu Unified district about their four reported shootings. The district told NPR no shootings happened. When the district tried to get the government to correct the report, the district was told they were a month late submitting the correction and the report would remain as published, would not be updated and reissued.

Am I a total cynic to suggest that if the next years stats are prepared with better vetting, the reported school shootings will be down and some unrelated change in between will be created with reducing school shootings?

[MatthewSlyfield]

Omro school district in Wisconsin asked the US Dept of Edu:
should a consensual paintball-gun fight involving students be reported as an "attack with a weapon" or as a "possession of a firearm."

How about a a consensual paintball-gun fight involving students should not be reported at all?

[ToddR]

Possibly some of the people analyzing the data were searching for a specific outcome and their enthusiasm revealed itself.

[Chem_Geek]

DING! DING! DING!
Thank you folks, no more calls please, WE HAVE A WINNER!

[PoxOnBothYourHouses]

The thing is that the DOE's number will become a zombie statistic. No matter how often it is killed it will rise again.

And do damage.

[BillyG]

I'll through in this statistic too: "Every year since 1950, the number of American children gunned down has doubled." Yes, its an error. That didn't prevent it from being repeatedly used.

More background on this here.

[RobinGoodfellow]

Such a terrible testament to our state of innumeracy that anyone could believe this statistic.

Even if only 1 child were killed in 1950, the number of children gunned down would exceed the entire US population sometime in 1978.

And surpass the entire population of the earth in 1983.

[WJack]

"[S]eemingly authoritative federal agencies" are mostly staffed with agenda driven, progressive members of the deep state. So is NPR, but this one was apparently too obvious for spin.

[Sarcastr0]

Spoken like someone who has never met a real live career bureaucrats. Trust me - they are not the liberal zealots you think they are!

I also like how NPR's counterexample is so easily tossed off.
The left? They call NPR 'Nice Polite Republicans.'

[VinniUSMC]

You're right, they're not liberal zealots, they're progressive zealots.

[WJack]

"Spoken like someone " who is a victim of progressive educators."

[Smooth Like a Rhapsody]

NPR is just the leftish radio network of what we have started calling The Swamp. The Swap knows no party.
The Swamp is the Church of D.C. and their clients and lobbyists at all levels of government. David Brooks and David Gergen--two putative Republicans--are two of the most revered saints of the Church.

[Sarcastr0]

More like NPR is status quo radio, and no one likes a centrist. (Plus they are a bit too pleased with their deal IMO.)

Your ranting about the swamp kinda weird from you though. You're smarter than to go whole hog into just the same boring blanket anti-government populism that's been fleecing the rubes since the country was founded.

[Smooth Like a Rhapsody]

My terminology is just being de rigeur.
If you are insinuating however that you dispute the fact that we are governed to an alarming extent (or an extent that many non-ranting people would be alarmed by) by that group (whether you call it The Swamp, or The Deep State, or K Street, or just The Bureaucracy) of unelected people which has developed it's own agenda and inertia independent of either party or the political process--then I would disagree with you.

[Sarcastr0]

An agenda implies coordination. There is no evidence of coordination like that going on.

If it's just some sort of emergent shared incentives, well, that's every government ever.

If you just don't like unelected people in government, I would posit there's a reason an unelected civil service is part of every modern government.

[Bob from Ohio]

"The left? They call NPR 'Nice Polite Republicans.'"

The left is stupid.

National Pinko Radio is more accurate.

[Rev. Arthur L. Kirkland]

Still angered by NPR's failure to hire Alex Jones and treat Pizzagate and birtherism with respect, Bob?

[WJack]

NPR, (one of many progressive propaganda pumps) never mentions the taxpayers as being among its donors . . . perhaps some creditiability might be gained by doing so.

[Absaroka]

"never mentions the taxpayers as being among its donors"

The first hit searching for 'NPR tax funding' is a page from npr.org that has a breakdown. Governments supply 14% of their funding (and universities another 12%, which might or might not be public money).

Individuals are 37%, and evil corporations 20%.

[Sarcastr0]

The right doesn't like NPR, which means they are biased liberals.
The left doesn't like NPR, which means the left is stupif.

That's some mighty unbiased reasoning there.

[ActualRightWingPatriot]

Liberals are dishonest hacks who have no qualms about lying and cheating in their quest to impose global socialism and cultural Marxism.

[Rev. Arthur L. Kirkland]

Should women be permitted to vote?

What about renters?

[Hey Skipper]

Women, renters, absolutely.

You, on the other hand ...

[Smooth Like a Rhapsody]

What about Ranters?
And Diggers, what do we do with Diggers?

[MatthewSlyfield]

Ranters can vote as long as they are current on their rant payments.

Diggers can vote, but only if they have a ladder to climb out of their hole so they can reach a polling place. I'm not letting them borrow my ladder.

🙂

[ActualRightWingPatriot]

No, and yes, if they are net payers of federal income taxes.

[Goju]

Statistician's blues:

https://www.youtube.com/watch?v=IUK6zjtUj00

[Michael Cook]

It is passing ironic that the Education Department in the USA can't count. . .

[WJack]

If progressives were capable of understanding irony they would be conservatives.

[Curly4]

These shooting statistics as reported here are very similar to other statistics reported. Even educational statistics are made up. I know this professor who was the director of developmental reading and writing at a community college and the only person to report to the college the stats for those who were taking reading or writing courses because they did not score high enough on the entry test. This professor reported the required stats to the school then at a later time saw these stats (for the time this professor reported them) and there were not the stats that the professor reported. This was reported to Institutional Research and was told that the stats would NOT be changed because it would to disruptive.
Using this as an example of the way the education systems work I would not trust any stat that is given in response to a request.

[Naaman Brown]

Anya Kamanetz, " The School Shootings That Weren't", NPR Morning Edition, 27 Aug 2018.

The report said 240 schools reported shootings. NPR checked with three-fourths of schools who would answer (about 180): 161 denied a shooting occurred, 11 confirmed, 4 cases something not a shooting had happened.

When NPR contacted Santa Monica-Malibu Unified district about their four reported shootings, the district told NPR no shootings happened. When the district tried to get the DoE to correct the report, DoE replied the district was a month late submitting the correction. The report would remain as published, would not be updated or reissued.

A report of 240 school shootings when there were about 15 or 16 actual incidents should be corrected.

[Longtobefree]

No. The Department of Education did not err in its count of school shootings.
It deliberately lied in compliance with the liberal agenda.
#deepstatesucks

[Sarcastr0]

Always chalk up to evil what could just be incompetence!

Guys, paranoia isn't a virtue.

[dantheserene]

If the mistakes go in both directions, it's probably incompetence. If they always go in the same direction, it's probably evil. See the difference?

[Sarcastr0]

Sure, except 1) this is one data point, and 2) confirmation bias is a helluva drug.

[dantheserene]

Every bit of false data collection here bulks up the number.

While we're at it, Why are you so keen to give incompetence a pass?

[Sarcastr0]

Did I say this was cool and good? I did not - should be dealt with on both a personnel and policy basis.

So you're saying this shouldn't count as one act, but many, because it released many incorrect numbers? I hope not - that's some legalistic BS right there, proves nothing about the motives behind this act, and a lot about your outcome-oriented reasoning.

But I think I'm misunderstanding you.

[Longtobefree]

It's only paranoia when they are not actually after you - - - - - - - - -

[Sarcastr0]

But that's a joke about how a paranoiac rationalizes their own...never mind.

[MatthewSlyfield]

Just because you are paranoid, that doesn't prove that no one is out to get you. 🙂

[Sigivald]

They failed before they started by using "school shooting" to include incidents where nobody was even shot at, let alone hit.

When "school shooting" includes "school cop negligent discharge", you're setting up to inflate the numbers, or you're too incompetent to be doing research in the topic.

(I mean, that's not something to ignore, but it's not a "school shooting"; it should be recorded separately, same as "firearm discharges unrelated to the school itself", such as the hoary old trope of "drug deal gone wrong in the school parking lot at 2AM" - it happened "at a school", but it simply is not a school shooting in any meaningful sense for policy analysis.)

[Stephen Lathrop]

Sigivald, that supplies a comment which begs an implied question. That implied question is, "Is this about a statistical attempt to measure hazards related to people having guns in schools?" If it were, then negligent discharges would be relevant.

There are other implied questions, too. One of them is, "Is this about the Trump administration trying to make a case for arming school personnel, by inflating school shooting statistics?"

I suggest extreme caution about both those implied questions, and most of the others commenters here seem to be relying on. If forced to choose an implied question of my own, my choice would be, "Yikes, what if this shows us what a mess the gun statistics industry has been right along?"

[Charles Epperson]

As to the first question I do not know what the definition is from the survey but most peoples interpretation of "school shooting" is guns brought to school and used in a manner intended to kill people. If this is the case then the negligent discharges are not part but I do not know.

As the statistics are from before Trump was president, 2016, you second questions is already answered.

As to your third question, it does. The statistics are sporadic and so open to skewing they are basically meaningless.

[Harvey Mosley]

"..."Is this about a statistical attempt to measure hazards related to people having guns in schools?" If it were, then negligent discharges would be relevant."

No, negligent discharges by cops would not be relevant because every anti gun law that I have seen exempts cops. So any information gathered using negligent discharges by cops (or ANY shootings by cops at schools) would only be used to pass laws against non cops and would therefore have no effect on any shootings at schools involving cops.

[Naaman Brown]

The Bureau of Justice Statistics, Firearms Use by Offenders (periodic surveys of representative samples of prison inmates whose last crime involved possession and use of firearms) in 2004 showed less than one percent acquired guns from gun shows.

The National Institute of Justice. National Survey on Private Ownership and use of Firearms NSPOF (a survey of non institutionalized adults living in homes with phones) in 1994 showed that of adults who acquired firearms, 60% bought them from retail sources (licensed dealers), 19% received them as birthday etc, gifts, 13% bought used guns from private owners, 5% inherited them, 3% swapped guns with another gunowner.

The Obama Administration, Democrat polticians, gun control advocates, hammered and hammered the claim that 40% of crime guns were bought from unlicensed dealers at gun shows and we needed to close that gun show loophole with universal background checks. When that 40% claim was tracked back to the NSPOF source, it was not even close to crime gun acquisition from gun show loopholes.

Bureau of Justice Statistics, the National Institute of Justice and other sources in the "gun statistics industry" do make reasonable efforts to acquire reliable statistics. The users of those statistics -- gun prohibition advocates -- are the ones making the mess.

[Naaman Brown]

The stats were for the school year 2015-2016 ending when school closed for summer 2016. Very clever of Trump to orhestrate that survey for the purpose of proposing arming teachers.

"The 2015?16 Civil Rights Data Collection (CRDC) is a survey of all public schools and school districts in the United States . . . a valuable resource for other federal agencies, policymakers, researchers, educators, school officials, parents, students, and other members of the public . . . . for the first time, the CRDC required schools to report on school-related shootings . . . ."

CRDC gives 240 as the number of schools that reported shootings but says that is not the total number of shootings for the 2015-2016. Hit shure aint. NPR: 37 schools in Cleveland district said the new "school shooting" box was checked for a different item. 26 schools in Ventura County CA also misreported nonexistent shootings. Two districts had 63 of the 161 NPR-proven false reports: an anomaly that would raise flags to most data collectors, expecially if a new question has been added.

[Michael Ejercito]

Why do we need a Department of Education?

[Bob from Ohio]

We don't . The worst president of the 20th century thought it a good idea.

[Milhouse]

Woodrow Wilson?

[Buddy Bizarre]

2nd Worst? He did sign a law allowing beer to be home brewed as well as de-regulating the airlines.

[Charles Epperson]

We do not. it would be much better to spend the money elsewhere or best delete it from the budget andf not redistribute it.

[ActualRightWingPatriot]

Modern education is great for learning how to perform oral sex using a banana. Not so good for anything else.

[Smooth Like a Rhapsody]

How do you perform oral sex using a banana?

[jsfreason]

Josiah Stamp: "The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the chowky dar (village watchman in India), who just puts down what he damn pleases."