U.N. Climate Change Panel Processes Critiqued

In the wake of last year's Climategate scandal, the InterAcademy Council (IAC), an Amsterdam-based organization of the world's science academies, is issuing its critique of the U.N. Intervovernmental Panel on Climate Change's (IPCC) processes and procedures. In the measured language of science, the critique appears to be fairly damning. Tidbits from the report and accompanying press release below:

…amid an increasingly intense public debate about the science of climate change and costs of curbing it, IPCC has come under closer scrutiny, and controversies have erupted over its perceived impartiality toward climate policy and the accuracy of its reports….

The report recommends that "to enhance its credibility and independence" the IPCC should be overseen by an executive committee that 'should include individuals from outside the IPCC or even outside the climate science community." Depending on how the members of the executive committee are chosen, this might help avoid the kind of group think that has arguably afflicted IPCC assessments.

The IAC also recommends that the IPCC chair, and the heads of the various working groups be limited to participating in just one five year assessment "in order to maintain a variety of perspectives and fresh approach to each assessment." Is this a hint that the current controversial head of the IPCC assessment process, Rajendra Pachauri, should step down now? In addition, the IPCC should develop "a rigorous conflict-of-interest policy to be applied to senior IPCC leadership and all authors, review editors, and staff responsible for report content."

Given the controversies over the meaning and significance of climate data, the IAC report further recommends

…IPCC should encourage review editors to fully exercise their authority to ensure that all review comments are adequately considered.  Review editors should also ensure that genuine controversies are reflected in the report and be satisfied that due consideration was given to properly documented alternative views.  Lead authors should explicitly document that the full range of thoughtful scientific views has been considered.

As an example, the IAC highlighted the Himalayan glaciers fiasco in which the IPCC, on the basis of the slenderest of evidence (basically a popular magazine science story), reported that all the glaciers in the Himalayas could melt largely away by 2035. The IAC report notes that the six experts who reviewed the first draft of this section said nothing relevant to this claim and that only two of the 12 reviewers of the second draft flagged it by citing peer-reviewed studies that pointed out that actually some Himalayan glaciers were expanding. The lead authors failed to address these reviewers' critical comments and the alarming assertion was published in the IPCC Fourth Assessment Report.

Later, when outside researchers suggested that the claim was an error, the IPCC assessment leader Rajendra Pachauri magisterially dismissed their criticisms as "school boy science." Transparency and openness indeed.

The IAC report finds that the IPCC characterization of the amount of scientific uncertainty concerning data and projections about future warming and its effects is confusing. The IPCC currently provides a mish-mash of confidence, likelihood and level-of-understanding scales.Confidence and likelihood scales tend to be based on the subjective views of participants. The IAC makes the sensible recommendation that the IPCC chiefly apply a level-of-understanding scale as a way to characterize scientific uncertainties. A qualitative level-of-understanding scale…

…describes the level of scientific understanding on a particular point in terms of the amount of evidence available and the degree of agreement among experts. There can be limited, medium, or much evidence, and agreement can be low, medium, or high. According to the guidance, when the level of confidence in the scientific findings is "high agreement, much evidence," authors may use one of the quantitative scales to calibrate the level of confidence in their conclusions or the likelihood of an outcome.

The IAC report further recommends:

The confidence scale should not be used to assign subjective probabilities to ill-defined
outcomes.

Quantitative probabilities (as in the likelihood scale) should be used to describe the
probability of well-defined outcomes only when there is sufficient evidence.

Authors shouldindicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).

The IAC concludes:

…that because intense scrutiny from policymakers and the public is likely to continue, IPCC needs to be as transparent as possible in detailing its processes, particularly its criteria for selecting participants and the type of scientific and technical information to be assessed.

Trying to reform the highly politicized and dysfunctional IPCC may be a fool's errand, but implementing the IAC recommendations would be a good first step. 

Go here to get a copy of the IAC report.