Panic to the People

Nitrites, cyclamates, 2,4,5-T…government creates needless fear when it jumps to conclusions.

It's becoming a habit. A government agency performs or commissions a scientific study designed to answer questions about the possible health effects of a substance. The results seem to indicate a hazard. An immediate announcement is made. The news media publicize it extensively, greatly alarming the affected individuals. The agency establishes policies or takes regulatory action. Later, independent experts examine the report and find that the original conclusion was in error. The study was insufficient, flawed, or misinterpreted.

The agency may reverse its decisions, but it cannot help those who suffered economic losses, psychological distress, and so on as a result of its earlier actions. Furthermore, public confidence in the agency—and in scientific findings—diminishes each time its scientific work is discredited. Retractions of earlier government warnings don't reassure consumers. Instead, many people fear that the government, under pressure from this or that industry, is trying to deceive the public so that something that is "known" to be dangerous can continue to be sold and used.

Incidents of this type have occurred all too frequently in the last few years. It would be reassuring if only trivial decisions were affected, or if a single regulatory agency were involved. But, as the following case histories will show, this is not true. Government reliance on studies later discredited by the scientific community has become commonplace.

NITRITE: THE NEWBERNE STUDY

In August 1978 the Food and Drug Administration (FDA) and the US Department of Agriculture (USDA) announced the banning of sodium nitrite as a food additive. Nitrite has long been used as an ingredient in cured meats such as bacon and lunchmeats for two reasons: (1) it inhibits the growth of bacteria, including those that cause botulism, a deadly type of food poisoning; and (2) it contributes to the distinctive color and flavor of these products. The agencies realized that if they banned nitrite abruptly, the risk of botulism would increase. Therefore, they planned a gradual phaseout of this additive, combined with research to develop alternate methods of preventing bacterial growth in cured meats.

What prompted this action? The agencies had received results from a study, performed under contract to the FDA, indicating that nitrite had caused lymphoma, a type of cancer, in rats. Soon after the study's results arrived, a joint FDA/USDA press release stated: "Nitrite has been found to induce cancer.…The FDA has received from Dr. Paul Newberne of the Massachusetts Institute of Technology (MIT) a report showing that nitrite…causes cancer when ingested by laboratory rats." Dr. Donald Kennedy, then commissioner of the FDA, said, "We know more than enough about the Newberne study to be convinced that it is well done and strongly supports the hypothesis that nitrites are carcinogenic."

The agencies seemed very sure that nitrite causes cancer—sure enough to publicize this alleged hazard widely and to plan a phaseout that would have required substantial changes in the way consumers handle and store cured meat products. Yet others were not so sure, including Dr. Newberne himself. Various parts of his report to the FDA stated: "There was not…a convincing dose response.…[There was a] somewhat less than convincing case that nitrite is lymphomagenic [capable of causing lymphoma] in rats.…[The] data are only suggestive."

The agencies' confidence in the quality of the study was strange, since an FDA inspection of Dr. Newberne's laboratory, conducted while this study was in progress, had shown significant deviations from acceptable laboratory procedures. The researchers didn't identify the animals in a foolproof way; mixups were possible. They used improper procedures for recording data. Necropsies (examinations of dead animals) were performed in the rooms where the live animals were housed, despite the fact that formalin, a chemical used in the necropsies, might have been harmful to the rats. Furthermore, the FDA inspectors observed a laboratory worker attempting to feed the wrong food mixture to a group of rats.

Questions about the Newberne study arose almost immediately after the FDA/USDA announcement. The FDA sent the report to several scientists for comment; half of them stated that it lacked sufficient data for a proper review. Several pointed out flaws, including improper statistical methods and the failure to determine possible exposure to nitrosamines—known cancer-causing agents that can be formed when nitrite reacts with other compounds that may be present in food.

Scientists identify lymphomas by examining microscopic samples of body tissue. The identification of this cancer is not straightforward; it requires expertise in pathology and the exercise of professional judgment in questionable cases. Several expert pathologists questioned Newberne's identifications of lymphomas. These were the most serious challenges to the study, since all its conclusions depended upon the assumption that the cancers were identified correctly.

Realizing that their decisions might have been too hasty, the FDA and USDA then asked a group of independent experts to review the pathological aspects of the Newberne study. After examining 50,000 slides of tissues from 2,000 rats, the independent pathologists found a "much lower incidence of lymphoma" than Newberne had reported—so much lower, in fact, that there was no longer any evidence that nitrite had caused cancer in the rats. The researchers concluded that the incidence of lymphoma in rats fed nitrite was 1.15 percent rather than the 10.5 percent Newberne had reported. In rats not fed nitrite, they found that the incidence was 1.5 percent (slightly higher, in fact, than in the treated group) instead of the 5.75 percent reported earlier.

For a variety of legal and political reasons, the agencies had never officially initiated the nitrite phaseout. In August 1980, after the reevaluation of the Newberne study and two years after the first press release, came another joint FDA/USDA communiqué: they had dropped all plans to phase out nitrite. But can consumer fears of this substance be eliminated as easily?

OCCUPATIONAL CANCER: THE "ESTIMATES" PAPER

In September 1978 Joseph A. Califano, Jr., then secretary of Health, Education and Welfare, told an AFL-CIO conference on occupational safety and health: "At least 20 percent of all cancer in the United States—and perhaps more—may be work-related.…It is estimated that 10 to 15 percent of all cancer deaths in the United States will be associated with previous exposure to asbestos."

Secretary Califano's statements shattered the general consensus that the incidence of occupationally induced cancer is relatively low—from one to five percent of all cancers, according to a widely accepted World Health Organization (WHO) estimate. Where did Califano get his numbers? His figures, which have shown up in almost every subsequent federal document dealing with cancer and the environment, came from an unpublished, unreviewed paper by 10 scientists entitled "Estimates of the Fraction of Cancer in the United States Related to Occupational Factors."

This paper has become the focus of widespread professional criticism and scorn. It was heavily criticized at a conference on cancer prevention shortly after Secretary Califano's announcement. There, Dr. John Cairns, director of the Mill Hill Laboratory of Cancer Research in London, said, "There are several parts of it which seem to be manifestly silly, and anyone who could sit down and do calculations could see how stupid it is." A distinguished epidemiologist, Sir Richard Doll, flatly stated: "I regard it as nonsense."

In an editorial in Nature, Dr. Richard Peto of Oxford University noted: "The document was based on the flimsiest of arguments, and reached conclusions which are grossly at variance with easily accessible data.…A group of reasonable men can unfortunately collectively generate an unreasonable report." An editorial in the prestigious medical journal Lancet said that "its framework is insubstantial, remarkably so," and concluded, with admirable British courtesy, that "it is sad to see such a fragile report under such distinguished names."

The distinguished names are those of ten respected American scientists who "contributed" to the "Estimates" paper. (No one has publicly admitted authorship.) An investigator from the New York-based American Council on Science and Health (ACSH) found that 8 of the 10 scientists responsible for the statistics Califano quoted—the statistics to which they "contributed"—believe they may be incorrect.

When ACSH researcher Ruth B. Schwartz spoke with the "Estimates" paper contributors, she found that many of them disapproved of all or part of the paper. At the time of Califano's speech, they said, they didn't regard the paper as a finished product; they considered it a draft that they intended to revise. Califano should have known this.

Some contributors stated that they had expected the paper to be revised and submitted for publication in a scientific journal. Unfortunately, this step, which would have led to careful prepublication review of the report by cancer experts, never took place.

Several "Estimates" contributors told ACSH that, insofar as it is serving as a basis for policy planning, the report is being misused. (Unfortunately, this is exactly how it is being used.) They also said that the paper clearly does not warrant the degree of confidence it is receiving from federal regulatory agencies.

Most cancer epidemiologists believe that the lower WHO estimate of the frequency of occupational cancer is far more realistic than the higher figure quoted by Califano. But the secretary's "20 percent…and perhaps more" received extensive publicity and implied that Americans face a major occupational cancer threat. That's what the public and the regulators remember.

With regard to the "Estimates" paper, Dr. Philip Handler, then president of the National Academy of Sciences, stated: "The necessity for scientific rigor is even greater when scientific evidence is being offered as the basis for the formulation of public policy than when it is simply expected to find its way into the market place of accepted scientific understanding.…Surely public policy should not rest on observations so preliminary that they could not find acceptance for publication in an edited scientific journal."

2,4,5-T: THE ALSEA II STUDY

On March 1, 1979, the Environmental Protection Agency (EPA) issued an emergency order suspending most uses of the herbicide 2,4,5-T and declaring it an "imminent threat to human health." Why? The EPA had received, one day earlier, a report of an epidemiologic study claiming to show a correlation between the use of 2,4,5-T and the incidence of miscarriages in the Alsea basin area of Oregon.

The Alsea basin is a forested area where 2,4,5-T has been used for a number of years. In July 1978 the EPA stepped up its continuing investigations of the health effects of 2,4,5-T after receiving a letter from eight Alsea women who had experienced miscarriages and suspected the herbicide as the cause.

The first phase of the investigation, Alsea I, examined the circumstances surrounding the reported miscarriages. The EPA investigators concluded that there was no cause-and-effect relationship between the use of the herbicide and these miscarriages and that "there was no real evidence of an epidemic" of spontaneous abortions.

The second phase, Alsea II, began in October 1978. It consisted of a retrospective study of spontaneous abortions in a six-year period (1972–77) in the Alsea area and in two control areas (one urban and one rural) where 2,4,5-T was not used.

The Alsea II report, which prompted the emergency suspension order, presented the following conclusions: (1) there was a significantly higher spontaneous abortion rate in Alsea than in either control area; (2) there was a seasonal peak of spontaneous abortions in Alsea in June, two to three months after the area was sprayed with 2,4,5-T; and (3) the spontaneous abortion rate in Alsea correlated with the quantity of 2,4,5-T sprayed.

An EPA press release called the correlation between the miscarriage rate and 2,4,5-T spraying "alarming" and the June peak "dramatic." One agency official told the press, "Now we have a lot of dead bodies, a lot of dead fetuses, in Alsea."

Since March 1979, 18 different individuals or groups, including epidemiologists, statisticians, and foreign government agencies and commissions, have reviewed all or part of the Alsea II study. Each and every one of the 18 reviewers found that the EPA's data did not support the EPA's conclusions.

One reviewer wrote that "the Alsea II study…is not a sound epidemiologic investigation. The Report contains severe deficiencies in data collection, in analysis, and in evaluation which render its conclusions baseless." Another critic stated that the "EPA erred seriously in each of the three conclusions in the Alsea II report." A statistician concluded, "The Report has fundamental statistical and logical flaws.…[It] is self-contradictory.…the data do not support the suggested correlation.…It would not be reasonable to accord any weight to the Report's conclusions."

What were the statistical flaws? All three sets of data collected by the EPA (records of hospitalized cases of spontaneous abortions, live-birth statistics, and 2,4,5-T spray data) were inaccurate and incomplete. The "June peak" in Alsea miscarriages actually occurred in one year only, and the two control areas showed similar single-year peaks in other months.

A meaningful comparison of June miscarriage rates could only be made if the populations of the three areas were constant throughout the year. But populations change in the Alsea II study areas. Alsea is a coastal area that attracts a large summer population. The urban control area, the city of Corvallis, is a college community, the home of Oregon State University. In contrast to Alsea, this area loses residents in the summer.

The EPA collected data only on hospitalized spontaneous abortions, even though not all women who miscarry go to a hospital. In Alsea, an estimated 70 percent of all miscarriages resulted in hospitalization; in Corvallis, where the pattern of medical care was different because of the presence of clinics, the estimate was 30 percent; there was no estimate for the rural control area. Yet the EPA compared the rural data with the other two sets without correcting for these crucial factors.

When the rates for Alsea and the urban area were compared, no correction was made for the 70/30 difference. If the total miscarriage rates in Alsea and Corvallis were exactly the same, this error alone would have made the Alsea rate appear to be more than twice the Corvallis rate. One reviewer reanalyzed the data, making the 70/30 correction. He found that the ratio of miscarriages to live births in Alsea was actually lower than that in Corvallis. In fact, the ratio in all three areas was lower than the national average.

One reviewer of the Alsea II study, commenting on its larger implications, wrote: "The critical issue is how the EPA makes a risk assessment and the quality of the research they use.…The EPA needs to be able to evaluate research quality, even when it is developed by its own personnel.…No one was well served by the incorporation of the Alsea II study into the regulatory process…least of all the EPA itself, whose very credibility has been tarnished."

THE LOVE CANAL CYTOGENETICS STUDY

On May 17, 1980, the EPA announced that a study commissioned by the agency had found chromosome damage in 11 of 36 residents of the Love Canal area in New York State. Although there had been several other studies of the health effects of hazardous wastes buried at Love Canal, the press and the public regarded this one as the most significant. For the Love Canal residents, who had been frightened and confused by two years of conflicting reports about long-term effects on their health, this announcement was the proverbial last straw. They demanded to be evacuated from the area, and President Carter approved an emergency relocation several days later.

The examination of white blood cells for chromosomal abnormalities (cytogenetics) is a sophisticated but now well-recognized procedure. Even so, the health significance of such abnormalities is uncertain. A group of people exposed to a toxic substance may have more damage than a similar unexposed group, but these effects may be transient and may not represent a true disease condition. Chromosome abnormalities may also be caused by other factors, including virus infections, sunlight, X-rays, cigarette smoking, and aging. Thus, it is difficult to attribute these changes to one single factor.

Despite these uncertainties inherent in the test, the EPA contracted with Biogenics Corporation of Houston to conduct a chromosome study on 36 Love Canal residents. The agency needed support for its lawsuits against Hooker Chemical Company, the firm that had dumped the wastes at Love Canal many years earlier. It was never intended to be scientifically rigorous. Unfortunately, it was so utterly lacking in rigor that its results are scientifically (and probably legally) meaningless.

As several cytogenetics experts noted after the study had been completed and announced, the validity of this test depends on a comparison of results from exposed subjects with unexposed control subjects. Samples from both groups should be processed and examined together. To avoid experimenter bias, laboratory workers should conduct their tests without knowing which samples belong to which group.

The EPA study, however, included no control group, making meaningful analysis of the study group impossible. The only significant results of the cytogenetics study were increased psychological suffering by the Love Canal residents—and an enormous expenditure of the taxpayers' money for the relocation of these people.

Unfortunately, this study has not been the only problem in the Love Canal investigations. Despite an immense amount of work by researchers from federal and state agencies, the actual health effects caused by the Love Canal wastes are still unknown. According to Lewis Thomas, the chairman of a review panel commissioned in 1980 by New York's Gov. Hugh Carey to evaluate studies of the area, "Many of the events of the past two years have fueled rather than resolved public anxiety and questions regarding the possible health effects of hazardous wastes at Love Canal." The ill-conceived cytogenetics study was certainly one such event.

GOVERNMENT MISUSE OF SCIENCE

These incidents share a common feature—a government agency announced and acted upon a scientific finding first and permitted independent experts to review the report later. As these incidents show, the reverse order of events would clearly be preferable. It would also be normal scientific procedure—to evaluate results first, and make use of them later.

Evaluation of a study by other specialists in the same field, a step known as peer review, is an essential part of the scientific process. A report cannot even be published in a reputable scientific journal until it has undergone and passed this type of review. Furthermore, if the results of an experiment are unusual or particularly important, the scientific community reserves judgment until other researchers have produced the same results.

In all of the cases detailed above, peer review and replication were side-stepped. Major government decisions were made on the basis of single, unreviewed, unvalidated studies.

After the release of the EPA's Love Canal cytogenetics study, a representative of the New York State Health Department said, "I for one will never believe anything the EPA says or releases again unless it has been peer reviewed." Incidents such as these invariably damage the government's scientific credibility, but that is not their only consequence.

A government announcement of an adverse health effect can never be fully retracted; the public gives more credence to bad news than to good. With regard to the cytogenetics study, Governor Carey's panel stated: "The damage done…is perhaps beyond mending; many of the Love Canal residents have by now become so distrustful of governmental agencies and their scientific reliability that they are unwilling to believe anything except the worst of news about themselves."

Unfortunately, the four incidents described above are not the only examples of premature government actions based on preliminary scientific information. The American public will have to live with the consequences of other mistakes for a long time. For instance, the findings of the original animal study that led to the banning of cyclamate have not been confirmed by later investigators, yet cyclamate remains banned. If the ban were lifted, many consumers would regard this sweetener with understandable suspicion.

The Canadian epidemiology study that related the use of saccharin to an increased risk of bladder cancer in men has been decisively contradicted by later work. Yet saccharin remains controversial.

Another unfortunate side-effect of the publicity surrounding ill-founded preliminary conclusions was reported recently in Science. In the summer of 1979, residents of a Memphis, Tennessee, neighborhood were complaining of various health problems. A former local health official made mention of a chemical waste dump in the area, and soon there followed picketing, highly charged meetings, and even calls for a Love Canal–style evacuation. A congressional hearing was convened. The EPA and the Centers for Disease Control moved in to study the problem.

But it turns out that there was no problem—nothing more than background levels of pesticides, no historical or aerial evidence of a source of chemicals, and no data that would support toxic illness or severe health effects. There can be little doubt that this case of a "phantom dump," as Science called it, is at least partly a result of the inadequate but well-publicized studies of the Love Canal situation.

The increasing incidence of scientific shortcuts on the part of government agencies is appalling. Peer review, statistical integrity, and replication of important findings are essential to science. When government agencies take shortcuts and needlessly spread panic among the public, science itself loses credibility. It is likely that people will be less and less willing to accept any scientific results. The disturbing prospect is a citizenry functioning in an increasingly complex world on the basis of ever more ill-informed choices.

Elizabeth Whelan is executive director of the American Council on Science and Health (ACSH) and the author of Preventing Cancer. Kathleen Meister is an associate editor of ACSH News & Views, from which this article is adapted, by permission of the authors.