Quality, Transparency, and Rigour

The Intergovernmental Panel on Climate Change (IPCC) reports are likely the most cited documents on the subject of global warming. The organization, established by the United Nations, doesn’t do any original research – it simply summarizes the massive amount of scientific literature on the topic. Their reports, written and reviewed by volunteer scientists, and published approximately every six years, are a “one-stop shop” for credible information about climate change. When you have a question about climate science, it’s far easier to find the relevant section of the IPCC than it is to wade through thousands of results on Google Scholar.

The main problem with the IPCC, in my opinion, is that their reports are out of date as soon as they’re published, and then everyone has to wait another six years or so for the next version, which is subsequently out of date, and so on. Additionally, because there are so many authors, reviewers, and stakeholders involved in the IPCC, the reports come to reflect the lowest-common-denominator scientific understanding, rather than the median opinion of experts. In particular, government officials oversee the writing and reviewing of the Summary for Policymakers, to make sure that it’s relevant and clear. However, some governments are beginning to abuse their power in this process. The late Stephen Schneider, in his 2009 book Science as a Contact Sport, recounts his experiences with government representatives who absolutely refuse to allow certain conclusions to be published in the IPCC, regardless of their scientific backing.

The result is that the IPCC reports frequently underestimate the severity of climate change. For example, in the most recent report, the worst-case estimate of sea level rise by the end of this century was 0.59 m. Since then, scientists have revised this estimate to 1.9 m, but it won’t show up in the report until the next edition comes out around 2014.

Another example concerns Arctic sea ice: the worst-case scenario from the IPCC was an ice-free Arctic in the summer beginning around 2100. These estimates have come down so rapidly that there’s an outside chance the summer sea ice could be gone before the next IPCC report has a chance to correct it (presentation by Dr. David Barber, media coverage available here). It will more likely disappear around 2035, but that’s still a drastic change from what the IPCC said.

Despite this conservative stance, there are still some who think the IPCC is alarmist (this is usually paired with something about a New World Order and/or socialists using a carbon tax to take over the world). Naturally, the IPCC has become a favourite target of climate change deniers, who wish to obscure the reality of human-caused global warming. Last year, they claimed to have found all kinds of errors in the latest report, somehow proving that global warming wasn’t happening. In fact, most of these so-called “errors” were nothing of the sort, and the worst of the two real mistakes in the report involved a typo regarding which year certain glaicers were expected to disappear. Not bad, for a three-thousand-page document, but it created quite the media firestorm. Apparently scientists are expected to have 100% accuracy at all times, or else they are frauds.

Just a few weeks ago, the IPCC made some changes to their policies in response to these events. Their press release about the new policies featured the phrase “Boost Quality, Transparency and Rigour” in the title.

No, no, no. That’s not what the IPCC needs. These are very admirable goals, but they’re doing just fine as it is. Actions to “further minimize any possibility of errors in future reports” should not be their top priority. Further extending the review process will only further delay the publication of each report (making them even more out of date) and further enhance their lowest-common-denominator position. When you have an error rate on the order of 0.67 errors/1000 pages, should you spend your energy getting that all the way down to zero (a virtually impossible task) or on the real issues that need to be addressed?

I think the IPCC should adopt a continually-updating online version of their report. This would solve their chronic problem of being out of date, as well as help the organization adapt to the increasing role of the Internet in our world. Any future errors the deniers liked to yell about would be fixed immediately. Governments would be forming policies based on the best available evidence from today, not a decade ago. Everything would still be in one place, and version control would allow transparency to remain high.

The IPCC should also make it more clear when their estimates are too conservative. When a single sentence that didn’t even make it into the summary is shown to overestimate the problem, the climate science community ties itself up in knots trying to correct its tattered image. But prominent conclusions that underestimate the problem go unacknowledged for decades. If it were the other way around, can you imagine the field day deniers would have?

Luckily, the changes made to IPCC policy are not all aimed at appeasing the bullies. A long-overdue communications plan is in development: a rapid response team and Senior Communications Manager will develop formal strategies for public education and outreach. Hopefully, this will counteract the false claims and defamation the IPCC has been subject to since its creation.

Another new plan is to create an Executive Committee, composed of the Chair, Vice Chairs, Working Group Co-Chairs, and advisory members. This will “strengthen coordination and management of the IPCC” and allow for actions to be taken between reports, such as communication and responding to possible errors. A more structured administration will probably be helpful, given that the only people in the organization currently getting paid for their work are the office staff (even the Chair doesn’t make a cent). Coordinating overworked scientists who volunteer for a scientific undertaking that demands 100% accuracy can’t be an easy task.

Will the IPCC continue to be the best available source of credible information on climate change? Will its structure of endless review remain feasible in a world dominated by instant news? Should we continue to grant our governments control over the contents of scientific reports concerning an issue that they desperately want to avoid? Should we continue to play to the wants and needs of bullies? Or should we let scientists speak for themselves?


5 thoughts on “Quality, Transparency, and Rigour

  1. A great and important posting… Thanks.

    The IPCC is the Al Gore of publications. Truthful, but none-the-less a political animal not quite up to date.

    I recall seeing some controversy around the founding of the IPCC… there was plenty of political maneuvering just in putting it together.

    Even today, there are many climate scientists who refuse to consider any data that is not either IPCC or inside a peer reviewed publication.

    This kind of scientific professional posturing is totally inappropriate for this issue. There is so much raw data that flows strongly through government agencies and even pre-publication academic studies.

    It depends on the sub-topic, in my opinion. For WG2 and WG3 topics, grey literature is necessary. For WG1, the peer-reviewed literature is extensive enough, and the smears elsewhere are frequent enough, that I wouldn’t take anything else too seriously (with the likely exception of government agencies). -Kate

    The IPCC is now cited widely precisely because it is 5 years behind the current research. This is an atrocious situation for science.

  2. > These estimates have come down so rapidly that there’s an outside chance the summer sea ice could be gone before the next IPCC report has a chance to correct it (presentation by Dr. David Barber, media coverage available here)

    There is no chance at all of this occuring. OTOH, if you believe it, are you interested in betting on it?


    What odds would you require on an “outside chance”, and over what timeframe?

    No, I wouldn’t bet on it. I really doubt it’s going to happen that soon. But I wouldn’t rule it out. The multi-year ice is nearly gone, so all we need is another year with meteorological conditions like 2007. -Kate

  3. I wish I knew why William Connolley was so very sure of the validity of ice extent modeling over ice volume modeling. True, he knows much more than I do about the subject. But it is also true that his outlook gets more pessimistic with the size of the bet.

    While a thin wide expanse of ice might be a valid winter result it is not a valid summer result. William wants a monthly average, but the difference between the September one day minimum and the averaged monthly minimum is around 200,000 to 300,000 sq Km. The forcasts diverge by a greater amount.

    Back to the topic, agreed the IPCC process needs to continually updating, the lag between data, publication of papers, cosideration of the reports by the IPCC and then the delay to IPCC reports being issued means that the science is very out of date
    . It is a field that is changing so very fast.

  4. I have a problem with the, by now infamous, “mistake” made by the IPCC regarding ice in the Himalayas. It is clear that not all ice will be “gone” by 2035 for I am fairly sure that new ice will form every winter for a long time to come well past 2035.
    The real issue is whether or not the Himalayan glaciers will provide the same environmental service to the many people living downstream in India, Pakistan, China etc.
    As the snow and ice melts going uphill to higher and higher latitudes there will come a time that the remaining snow and ice will be insufficient for the people living downstream. This may come much sooner than what the corrected version of the IPCC report claims.
    A single mountain is usually conical in shape and it’s surface area decreases as one moves up the mountain.So does the area to store snow and ice. This is not so obvious for mountains that melt into each other to form a mountain chain nevertheless even the Himalayas decrease in surface area as one moves up the mountain chain.
    It could very well be that snow and ice decrease as global warming erodes the existing glaciers that the remaing ice is just inadequate for the people living downstream.
    As the population increases, more and more people depend on an ever decreasing resource.
    The real disaster that really matters, a shortage of melted water during the dry season, might indeed happen by 2035 regardless of whether there is any ice left in the Himalayas at that particular time. Indeed the capricious nature of “weather” may cause this at any time from now on.
    If and when this happens the IPCC “mistake” will no longer look like a mistake.

  5. “I think the IPCC should adopt a continually-updating online version of their report.”



    “…government representatives who absolutely refuse to allow certain conclusions to be published in the IPCC, regardless of their scientific backing”

    will never allow such a thing :(

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.