Scottish Smoking Ban Study Leads to Huge Drop in Journalistic Credibility

Two months ago, a study reporting that the imposition of Scotland's smoking ban had been followed by a 17 percent drop in hospital admissions for heart attacks prompted credulous headlines like these:

Smoking ban brings big cut in heart attacks in Scotland, study finds (The Guardian)

Smoking ban 'reduces heart risk' (BBC News)

Scottish smoking ban cuts heart attacks (The Telegraph)

Scottish Smoking Ban Leads to Huge Drop in Heart Attacks (Der Spiegel)

Now it turns out that the complete admissions data for the year following the ban show a drop less than half as big as the one claimed in the study and in all those news stories. And as Michael Blastland shows in a recent BBC News article, the 8 percent drop may simply be part of a long-term trend:

Heart attacks have been falling steadily for some years now. The percentage falls in the three years before the ban were 5.1%, 4.7% and 5.7%. So the fall since is still bigger than the trend would lead us to expect, but bigger only by about three or four percentage points—an improvement, but retreating fast from the magnitude of 17.

The latest release also makes clear that even an 8% fall in heart attacks is not unprecedented. There was another, larger drop between 1999 and 2000 of about 11%.

This seems to demonstrate significant variability around the trend, suggesting that last year's 8% drop might even be the result of chance. It is conceivable, although perhaps unlikely, that the smoking ban had no effect at all. The figures could be a result of no more than the ordinary ups and downs of statistical variation from one year to the next.

Reporters can't be expected to look at data that haven't been released yet. But the idea that a smoking ban could cause an immediate, dramatic drop in heart attacks, whether by reducing secondhand smoke exposure or by encouraging smokers to quit, is so scientifically implausible that journalists should be automatically skeptical. Random variation and/or continuation of pre-existing trends are much more likely explanations for a decrease in heart attack admissions that happens to follow the imposition of a smoking ban. In the absence of more evidence, including additional years of followup and data from other jurisdictions with smoking bans (most of which apparently have not seen the big changes found in the few places that anti-smoking activists like to cite), the breathless reports prompted by studies like this one are absurdly premature.

[via The Rest of the Story]