The Intergovernmental Panel on Climate Change's Loose Relationship with Peer-Reviewed Science

The U.N.'s Intergovernmental Panel on Climate Change (IPCC) issues a report summarizing climate science every five years or so. Those reports represent "the consensus science position on climate change, directly engaging more than 2,500 scientists from more than 130 nations."

Is that so? What percentage of literature citations in the IPCC's Fourth Assessment Report are derived from the peer-reviewed scientific literature? Peer-review is not all that high a standard, so one would hope that most, if not all, would be. After all, as the editor-in-chief emeritus of the American Physical Society's nine journals, Martin Blume once said, "Peer review doesn't necessarily say that a paper is right. It says it's worth publishing." It turns out that a good bit of the literature cited by the IPCC in its latest report doesn't meet even that standard.

In an op/ed in The Australian, journalist Matt Ridley points out that a provocative new book,The Delinquent Teenager Who Was Mistaken For The World's Top Climate Expert, by Canadian journalist Donna LaFramboise finds that a lot of the "scientific" references in the IPCC report are from the "grey" literature. Ridley explains: 

The impression the UN gave was that they were composed by thousands of senior scientists.

In the words of Rajendra Pachauri, the head of the IPCC: "These are people who have been chosen on the basis of their track record, on their record of publications, on the research that they have done . . . They are people who are at the top of their profession."

In fact, as Laframboise meticulously documents, world experts on malaria, hurricanes and other topics are excluded because of their sceptical views; while a relatively small clique does the actual writing, many of whom are young and have such a short "track record" that they barely have higher degrees….

Laframboise set out to test Pachauri's claim that "we carry out an assessment of climate change based on peer-reviewed literature, so everything that we look at and take into account in our assessments has to carry (the) credibility of peer-reviewed publications -- we don't settle for anything less than that."

In March last year, Laframboise recruited 43 private citizens in 12 countries online to audit the entire IPCC 2007 report and count the number of non-peer-reviewed references. Each section was audited by three people and the lowest (most conservative) estimate used.

Even so, the audit showed that 5587 of 18,531 -- fully one-third -- were non-peer-reviewed sources: including newspaper articles, activist reports, even press releases. The IPCC had a rule that such sources must be flagged as such. It had been ignored. When criticised for this last year by a panel of the world science academies, it simply changed the rule.

So about one-third of the IPCC references are not from peer-reviewed sources. In fact, many are from activist studies and pamphlets as well as the popular press, e.g., the infamous IPCC assertion that Himalayan glaciers would melt away by 2035. Earlier this year, environmentalist Mark Lynas called out the IPCC for its conflict of interest in appointing a Greepeace-affiliated scientist as the lead author of its report on renewable fuels. Not too surprisingly, that report highlighted Greenpeace's no-doubt-objective "research" on renewables. 

However, instead of tightening its procedures, the relevant IPCC technical committee has decided that it would be "impractical" to flag or highlight the fact that information from the "grey" literature cited in future reports actually comes from non-peer-reviewed sources.