100120_Clarifier

The UN’s Intergovernmental Panel on Climate Change (the IPCC) assesses the scientific, technical and socio-economic information relevant for the understanding of the risk of human-induced climate change.

This week, the IPCC was thrown into controversy when it was revealed that its long-held claim that climate change will melt most of the Himalayan glaciers by 2035 has was sourced from pure speculation published in a non-academic pop-science magazine.

So just how does the IPCC arrive at its conclusions? What sort of material does it trust and what are the checks and balances that weed out the fact from fiction? Andrew Macintosh, associate director at the ANU Centre for Climate Law and Policy, explains:

Given that there are so many levels of review within the IPCC, how could they possibly miss this kind of error?

The IPCC process basically works like this. Experts in relevant fields are nominated by governments and others to be lead authors and contributors to specific parts of the assessment reports. After the lead authors are appointed, relevant material is submitted by people in the field for consideration in the report. A substantial proportion of the submitted material consists of, or is drawn from, peer-reviewed articles that have been published in academic journals. However, other material can be accepted in certain circumstances. On the basis of the material submitted, and other peer-reviewed and internationally available literature, first drafts are prepared. The drafts are then reviewed by experts in the field and revised to take into account their feedback. The revised drafts are then reviewed by governments and a wider group of experts and authors. After this, the final report is prepared.

This process is supposed to be objective, transparent and representative. It is also meant to be exhaustive and robust. These goals are supposed to be guaranteed by the fact that material contained in the final report usually undergoes four layers of review — first when papers are peer reviewed for publication in academic journals, second when the materials are reviewed by the lead authors before the preparation of the first draft; third when the first draft is reviewed; and finally when the revised draft is reviewed by governments and experts. So how could an error like that concerning the Himalayan glaciers get through this process? (I assume for these purposes that the statement about the glaciers is erroneous)

The answer is that the process is human and all human processes are fallible. It would be astounding if the IPCC reports did not contain errors. The four reports that constitute the IPCC’s Fourth Assessment Report contain thousands of facts, covering a multitude of areas. Authors make errors, as do reviewers. In this case, a statement made by a scientist to a reporter from New Scientist appears to have been taken as scientifically based when it was conjecture. The scientist may even have been misquoted. Either way, what appeared in the New Scientist report was not backed by rigorous science. This should have been identified in the IPCC review process and the material should have been omitted from the final report. It wasn’t and that is deeply regrettable.

I have no doubt that the lead authors of the relevant chapter and other reviewers are embarrassed by the mistake. As they would know, this type of error undermines public confidence in the IPCC and its processes. The only positive that can be taken from this incident is that it has demonstrated how science tends to be self-correcting. People questioning facts identified the error and the error should now be removed from the report, or corrected in the IPCC’s Fifth Assessment Report.

How common is it for the IPCC to cite lobby groups such as WWF or magazine interviews as sources?

Material that is not from an academic or governmental source can be included in IPCC reports. However, it is supposed to meet certain quality standards. In this case, it appears the process has broken down and the gatekeepers of quality have failed to perform their task.

How does this kind of error compare to the leaked Climategate emails in terms of denting the IPCC’s credibility?

We are still waiting for the full story behind Climategate and this incident. However, from what has been reported, this error appears to be of greater import. The expected rate of melting of the Himalayan glaciers is not a trivial matter. Their rate of loss will have important implications for several significant human and natural systems. Claims of rapid glacier loss should have been thoroughly checked and it appears they weren’t. If the story as reported is correct, this is a serious error and the IPCC should take steps to ensure that all similar statements are subject to more rigorous evaluation.

Sceptics’ hysteria aside, what does this mean for the credibility of the IPCC within the scientific community?

This incident should not permanently stain the credibility of the IPCC. Most people recognise that the peer review process is not infallible and that errors are made. In the case of IPCC assessment reports, the error rate is likely to be very low, and well below that in most other scientific publications. The challenge is to ensure that errors such as this are not repeated.

How can the IPCC best address this problem?

The existing processes should have weeded out this sort of material. In the future, the IPCC should ensure that greater attention is paid to its existing processes, reviewers are given greater time and resources to verify facts, and that claims about key issues are thoroughly evaluated.

Given that this report is from 2007, how significantly has the science around glaciers changed since then? What does this mean for IPCC reports in the future?

Beyond my area of expertise.

How much of this and the Climategate affair is a lesson in PR for the IPCC, something scientists don’t usually have to contend with to such an extent?

If the IPCC needed these incidents to alert them to the political context in which they operate, I think we are in trouble. Climate change is a highly politicised issue that attracts strong ideological responses. This has been the case since climate change became a significant issue in the late 1980s and, no doubt, it will continue to be the case. Most people working in the field are aware of the context in which they operate and the potential for their work to be misconstrued and for errors such as this to have serious consequences. The glacier incident has provided a vivid illustration of the need for the IPCC to be exacting in its standards and for scientists to be cautious when engaging with the media.