Why Do Smoking Bans Prevent Heart Attacks Only in Small Towns?

Jacob Sullum | May 25, 2007

A study in Preventive Medicine finds that a smoking ban in Bowling Green, Ohio, was followed by a 47 percent drop in hospital admissions for coronary heart disease. According to the researchers, "The findings of this study suggest that clean indoor air ordinances lead to a reduction in hospital admissions for coronary heart disease, thus reducing health care costs."

Isn't it funny that jaw-dropping results like these seem to happen only in small towns with small, highly variable numbers of heart disease admissions? Why is it that smoking bans are so much more effective in places like Helena, Montana; Pueblo, Colorado; and Bowling Green, Ohio, than they are in places like New York, Boston, Los Angeles, San Francisco, and Florida, where much larger samples should make dramatic reductions in hospital admissions easy to see?

A look at the raw hospital-admission numbers for Bowling Green, as reported by Michael Siegel, may help resolve this mystery:

1999: 35
2000: 24
2001: 24
2002: 36
2003: 22
2004: 26

Although the smoking ban took effect in March 2002, Siegel notes, the researchers treat that year's admissions as if they all occurred before the ban, which conveniently helps magnify the apparent post-ban drop, since 2002 had an unusually high number of admissions. They also take the number for the first half of 2005 and simply double it to estimate the number for the full year—a maneuver that ignores seasonal variations in admissions but also helps make the post-ban drop seem bigger. Leaving both of those problems aside, how can we reasonably conclude anything about the cause of a post-ban drop in admissions when changes of comparable magnitude occurred in earlier years, for no apparent reason? As I've said before, it's inevitable that heart attacks, purely by chance, will fall substantially in some of the hundreds of jurisdictions subject to smoking bans. That does not mean the smoking bans prevented heart attacks.

In this case, the idea that the smoking ban cut heart attacks in half within three years is especially implausible, since the ban did not apply to all businesses, exempting stand-alone bars and separate bar areas of restaurants. It seems unlikely this law had a dramatic impact on smoking rates or secondhand smoke exposure, and the researchers present no evidence that it did. Given commonly cited estimates of smoking's contribution to heart disease, the ban could not have produced a drop in hospital admissions of this magnitude even if everyone in the whole town stopped smoking.

Comments

|5.25.07 @ 1:06PM|#

Jacob,

Are there any studies that show heart-attack rates rising in a few towns, as we'd expect if these are ordinary statistical variations?

Chucklehead|5.25.07 @ 1:09PM|#

I remember reading a report that said if you were alive, you had a 100% chance of dying.

Scary stuff.

|5.25.07 @ 1:11PM|#

Why Do Smoking Bans Prevent Heart Attacks Only in Small Towns?

It's called the Mellencamp Effect.

|5.25.07 @ 1:13PM|#

Shelby-
You really think they'd report those kind of data to the public? Then people might not stop doing whatever the media wants them to do; why do you think most new sites don't site their exact data?

This reminds me of an old reason artile mentioning that due to the data shown in recent studies, the less smoke you're exposed to, the more damaging it is--good stuff. I love stats class; you learn how easy it is to BS, and 80/90% of the population can't call you on it.

thoreau|5.25.07 @ 1:15PM|#

My gut suspicion is that if these dramatic reports only come from small towns then it can probably be explained, as Shelby suggests, by ordinary statistical variations. When you have a small pool, the fluctuations will tend to be larger percentage-wise.

(For those who want the technical details, keep in mind that from the Central Limit Theorem the fluctuations scale as sigma = sqrt(N) where N = sample size, so the percentage fluctuations are of order sigma/N = sqrt(N)/N = 1/sqrt(N).

If that's the case, then most of the dramatic examples will come from these small towns, and anybody who wanted to find a dramatic counter-example would also be most successful at finding them in small towns.

Now, somebody might say to me "Wait, you haven't read the study, how can you comment on their statistical methodology?" That's a fair point, but I'm going to observe that no matter how careful your methodology is, a small sample is still more likely to pose problems. If the most surprising claims are always coming from studies of small samples, I'm going to say that I'm unimpressed. Especially since the large number of small towns means that you have ample room to cherry-pick.

Now, if a surprising claim came from a large city, or from an aggregate of small towns that collectively offered a large sample size, then I'd say that we have to look beyond statistical variation.

Bottom line: I'm suspicious of any line of hypothesis for which the most dramatic examples involve small samples.

ed|5.25.07 @ 1:17PM|#

I'm suspicious of any line of hypothesis for which the most dramatic examples involve small samples

Or researchers with an agenda.

Brian|5.25.07 @ 1:22PM|#

Yeah, the 2006 Ohio employee protection smoking ban in all public buildings - I'm sure in the next few years heart disease rates amongst restaurant servers and bartenders will drop dramatically in the next few years. What a joke.

Guy Montag|5.25.07 @ 1:24PM|#

It's called the Mellencamp Effect.

Didn't he give away a PCB contaminated house, that was painted pink, as an MTV stunt?

Perhaps there are other agents that can damage health besides tobacco.

Guy Montag|5.25.07 @ 1:25PM|#

oops, I did not close the tag

|5.25.07 @ 1:37PM|#

thoreau nailed it.

Someone - an ex-Mayor running for President, say - could similarly claim that New York City had an 80% decline in homicides between 2001 and 2002.*

*stats here: http://samoa.istat.it/Eventi/sicurezza/relazioni/Langan_rel.pdf
[I've added in the numbers for 9/11, which are not actually included in the survey.]

|5.25.07 @ 1:49PM|#

I see another trend. Look at the difference between 1999-2000. That's right, it's clear proof that Prince's "Party like its 1999" was prescient about the level of partying (and drug induced heart attacks) in 1999.

What does this say about the effects of doves crying? It could be disasterous.

thoreau|5.25.07 @ 1:50PM|#

Chris S. wins the thread.

Thomas Paine\'s Goiter|5.25.07 @ 2:15PM|#

Khuder SA, Milz S, Jordan T, Price J, Silvestri K, Butler P

I want to kick them in the balls.

Chucklehead|5.25.07 @ 2:23PM|#

I sense a theme in TPG's posts today.

MattW|5.25.07 @ 3:06PM|#

Thoreau said:

"Now, if a surprising claim came from ... an aggregate of small towns that collectively offered a large sample size, then I'd say that we have to look beyond statistical variation."

I am always skeptical of these "meta-analyses", because this only works if you have a random sample of "small town studies". However, as mentioned by Barakku, samples of studies tend to be biased toward 'positive results'. This is called 'publication bias' in the biomed lit I think. This has caused a movement to document negative results (i.e. journals should accept papers with negative results, to promote reliable meta-analyses)

thoreau|5.25.07 @ 3:15PM|#

MattW-

I'm not thinking of a traditional meta-analysis of selected published studies, for the reasons that you point out.

Rather, I'm thinking of some researchers who decide that they will do a study of all the small towns that enact smoking bans in such-and-such time period, or that they will randomly select a dozen such towns, or something like that.

So it's done as a single study by the same researchers applying the same methods (or applying methods that are closely related, if data sets from different towns come in different forms).

I'm not sure how feasible such a study would be, but my point was that it would at least have a large sample size.

Bottom line: I want a large sample size. Then I'll be more impressed.

|5.25.07 @ 4:25PM|#

I think this study is UNCONTROVERTABLE PROOF that smoking bans which exempt bars and allow for seperate smoking areas in restaurants cut heart attacks in half. It is therfore only a legislator with no respect for public health who demands a total smoking ban.

|5.25.07 @ 7:27PM|#

Although the smoking ban took effect in March 2002, Siegel notes, the researchers treat that year's admissions as if they all occurred before the ban, which conveniently helps magnify the apparent post-ban drop, since 2002 had an unusually high number of admissions.

Contrarily, if one wants to play statistical games, one could argue that the stress brought on by the ban caused the spike in 2002.

;P

|5.25.07 @ 8:16PM|#

On top of that, they seem to imply that exposure to second hand smoke is completely reversible once removed from the source, which contradicts the scare story that SHS is almost as bad as smoking.

How the hell did such a shoddy piece of research get by peer review?

|5.25.07 @ 8:49PM|#

How the hell did such a shoddy piece of research get by peer review?

If all the "peers" are anti-smoking activists...

Guy Montag|5.26.07 @ 8:29AM|#

On top of that, they seem to imply that exposure to second hand smoke is completely reversible once removed from the source, which contradicts the scare story that SHS is almost as bad as smoking.

Actually, from the study, a law makes the "effects" of second-hand smoke completly reversable.

|5.26.07 @ 10:05AM|#

So who gose around to small towns and conducts these studies i live in a small town and have,nt seen anyone doing any such thing

|5.26.07 @ 6:45PM|#

Because the study is not a true study but something paid for by nonsmoking advocates who don't care about the law, truth or the free enterprise system, they just want their way.

Harley|5.27.07 @ 7:13AM|#

Wally, these studies are done by examining data, not by scientists visiting these places, taking samples, and examining people.

|5.28.07 @ 12:05PM|#

The results are a statistical artifact. In the last issue of American Scientist (whose website seems to be down today, Memorial Day) discussed the fact that disease rates in small towns are lower than those in large towns, and ascribed it to a fact about degree of variance and extremes (a law of statistics whose name I have forgotten, and I just threw the original article out, so I can't find the reference).
Consequently, the article should never have been published, although the American Scientist article gives a number of other examples of non-results that were published to great fanfare because folks don't know the variance facts.

Michael J. McFadden|5.29.07 @ 12:51PM|#

"Why Do Smoking Bans Prevent Heart Attacks Only in Small Towns?"

There are two things I'd recommend for people who want answers to this question.

First, go to

http://bmj.bmjjournals.com/cgi/eletters/bmj.38055.715683.55v1

and read the Rapid Responses to the Helena study. The Helena study was the prototype for the Pueblo, Bowling Green, and other copycat studies to follow. You'll be amazed at the amount of twisting that went on with this study and the almost complete abdication of responsibility by its authors to defend their work or their claims. You'll also be amazed at some of the statements about the study that were simply, and clearly, false.

Second, go to

http://www.smokersclubinc.com/modules.php?name=News&file=article&sid=2385

and read about a much larger study that was done over a year and a half ago that completely discredits the findings of these small studies. The larger study, carried out by David W. Kuneman and myself, is based upon completely available and easily accessible public records for verification and has never been publicly challenged on its content.

That study has since been formalized and put into proper form for submission to medical journals. It was turned down by the British Medical Journal (publishers of the Helena Study) primarily on these grounds:

"Our main problem with the paper was that we did not think it added
enough, for general readers, to what is already known about smoking and health."

Evidently, despite the fact that the search string < Helena smoking ban > garners 189,000 hits on Google, and despite the fact that our study reached diametrically different conclusions than Helena, and despite the fact that it was 1,000 times as large and based on publicly verifiable data... it was judged not of interest or value to "general readers".

Follow up submissions to Circulation (which published the Helena copycat Pueblo study) and to Tobacco Control resulted in similar rejections without options for rewrites and resubmission.

So why do "Studies" show bans reducing heart attacks only in small towns?

Maybe because medical journals refuse to print any studies showing otherwise.

Michael J. McFadden
Author of "Dissecting Antismokers' Brains"
http://pasan.TheTruthIsALie.com