Climategate Update: About That Peer-Reviewed Stuff

|

Back when the Climategate affair first hit the news, head of the Intergovernmental Panel on Climate Change Rajendra Pachauri angrily defended the IPCC's Fourth Assessment Report (4AR) by asserting that it had been rigorously peer-reviewed. As the Guardian quoted Pachauri:

"The processes in the IPCC are so robust, so inclusive, that even if an author or two has a particular bias it is completely unlikely that bias will find its way into the IPCC report," he said.

"Every single comment that an expert reviewer provides has to be answered either by acceptance of the comment, or if it is not accepted, the reasons have to be clearly specified. So I think it is a very transparent, a very comprehensive process which insures that even if someone wants to leave out a piece of peer reviewed literature there is virtually no possibility of that happening."

The Daily Telegraph is reporting that claims of rigorous peer-review may have been exaggerated:

… a new study put this claim to the test. A team of 40 researchers from 12 countries, led by a Canadian analyst Donna Laframboise, checked out every one of the 18,531 scientific sources cited in the mammoth 2007 report. Astonishingly, they found that nearly a third of them – 5,587 – were not peer-reviewed at all, but came from newspaper articles, student theses, even propaganda leaflets and press releases put out by green activists and lobby groups.

The group auditing the 4AR references was organized by Canadian global warming skeptic Donna Laframboise. You can find the report here and judge its accuracy for yourself.

NEXT: Reason.tv: We The Living Trailer for 1942 Italian Film Version of Ayn Rand's Novel

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. “The processes in the IPCC are so robust, so inclusive, that even if an author or two has a particular bias it is completely unlikely that bias will find its way into the IPCC report, especially if that bias does not agree with our pre-determined conclusions,” he said. “Also, Piltdown Man.

    1. “Also, Piltdown Man.”

      What does Piltdown Man have to do with anything? Is that a really ignorant way of saying you think that it’s some kind of indictment of the peer review process? Piltdown Man was kept away from scientific scrutiny for years, and it was scientists who figured out it was a fake when they finally got their hands on it.

      And what proportion of the shoddy citations were from the Working Group II report? As I understand it that’s the sociology professor wing of the IPCC, so I wouldn’t be surprised to see lots of crap coming out of it.

      1. Really? Piltdown Man was hidden from scientists and it is them who found the hoax? What about the similarity between “lost” data by Phil Jones? And the refusal of others to supply their data? Looks a lot like a Piltdown skull to me.

    2. What a bunch of cunts.

      I won’t take any one of these fuckers seriously until they off themselves and thus reduce their carbon footprint.

  2. The science is settled.

    1. Fine, even if I can concede to you that the science is settled, the resulting politics is that the left wants to spend trillions (10^12) of dollars correcting global warming. What is definitely unsettled is whether spending that amount of resources will sufficiently mitigate the effects of global warming to our satisfaction. Spending such a massive sum of money requires more certitude about the results we will achieve.

      1. We are on the eve of destruction. We must avoid AGW at all costs.

        1. Alright, can you come up with a process of avoiding AGW that actually has certifiable results and doesn’t cost trillions? Let’s start with the former, and then we can negotiate costs.

          The greatest minds in world politics, attempting to develope a solution to this problem, only came up with transfer payments from developed to emerging nations. Who knew the world was warming because Sri Lanka wasn’t getting enough American tax dollars?

          1. Look over there? See the camera?

            You’ve just been Tony’d.

            1. We are all Tony now.

          2. Alright, can you come up with a process of avoiding AGW that actually has certifiable results and doesn’t cost trillions? Let’s start with the former, and then we can negotiate costs.

            Look up the TTAPS study.

        2. These are not particularly good impersonations of Tony.

          He is usually much more wordy when spewing his nonsense.

          1. Yea I had my doubts, nonetheless, I expect there to a sharp and verbal response to that eventually from the maestro. I kept writing to save myself future effort.

          2. Sorry. It’s difficult to type and suck cock at the same time.

            1. Come on Tony! No excuse. Wylie’s cock is the size of a toothpick.

    2. The denialism is settled!

  3. The Daily Telegraph is reporting that claims of rigorous peer-review may have been exaggerated:

    *Jaw crashes to floor*

    1. No, you’re lying! You’re all lying! [Runs to room and sobs.]

  4. “The group auditing the 4AR references was organized by Canadian global warming skeptic Donna Laframboise.”

    Well if she’s a skeptic, then obviously nothing she says can be trusted.

    …unless, of course, her work is peer reviewed through newspaper articles, student theses, propaganda leaflets and press releases put out by green activists and lobby groups.

  5. Buh-huh-husted!

  6. It just keeps getting better, doesn’t it? I have to admit that I am getting many lulz. I can’t wait for the next bullshit scheme, and the people who want it so badly to be true, and to have them tell me not to be skeptical. Hilarious.

    1. “I can’t wait for the next bullshit scheme, and the people who want it so badly to be true, and to have them tell me not to be skeptical.”

      I thought that was called “ObamaCare”?

      Oh, wait, no… The “next” scheme is when we remake Wall Street in Obama’s image?

      I get lost–it seems like there’s a new scheme every day!

      1. Aw, you’re too kind. Well, maybe not every day. I have to fit a round of golf in every few days, you know.

  7. “Studies have shown that accurate numbers are not any more useful than numbers you make up.”
    Dilbert May 8, 2008

    How many studies say that?
    87

  8. How thorough is the usual peer review process, anyway? Does a reviewer read the article and look for gaping logic holes and obvious misstatements? Or does the reviewer go back and check all the footnotes, verify the math, and do everything short of rerunning the experiment to make sure the article is sound? I really don’t have a feel for the level of effort going on here.

    1. Here’s a good article that I read in the SIAM newsletter some months ago that explains some of the common abuses in the peer review process.

      http://www.ima.umn.edu/~arnold…..attack.pdf

    2. It depends on the field and the nature of the work but basically your first choice; the second warrants a second paper.

    3. Totally agree.

      Peer review is a standard of research, but it is no substitute for experimental proof.

    4. Depends on the reviewer and the article. They should be (at least) working in the same or a close related field. The generally are expected to have published a few papers of their own. They are sometimes experts in the field.

      With that basis, they should (at least) read the paper closely and *think* about anything therein that is not obvious. Maybe that’s a little stronger than your first alternative.

      The purpose of review is to insure that stuff getting published:

      1) is not *totally* bonkers [*]
      2) describes the work or calculation well enough for someone else to attempt to repeat it
      3) and are on-topic (and for prestigious journals, important enough) for the rag in question.

      That’s it, really. It’s not an enormously high hurtle.

      In reality there are some subjective criteria. Papers from un-credentialed authors may get closer scrutiny. A lot of fields seem to have a preferred style, and deviating from it will draw criticism. And, of course, when controversies rage there may be some boosterism.

      [*] Meaning it uses the language and terminology of the field and either build on the agreed foundations or—if purporting to rewrite the basics—approaches them circumspectly and with careful attention to things already known (new theories must retain the successes of the old one as well as beating new problems…)

      1. Thanks for the replies. It was out of my range of experience.

        I ask because I work in engineering and I supervise junior engineers. When I sign off on their work, I’m saying it’s correct to the best of my (and their) ability. When shit breaks, they don’t came after junior, they come find me. So checking the work is almost as involved as doing the work yourself sometimes.

      2. To complement EscapedWestOfTheBigMuddy’s good explanation, the reviewers are humans and have prejudice like anybody else: for instance, it has been shown for example that in some fields, the same paper gets more rejection when authored with a woman’s name than with a man’s name (but I don’t know if that study itself met scientific standards!!!)

  9. Muy lulz. Muy muy.

  10. Muy lulz. Muy muy.

    (It’s not spam, dammit!)

  11. According to the good doctor T, peer review just ensures that an article is suitable for publication. The article itself must provide enough information for independent researchers to reproduce your work. That is where verification happens, not in the peer review.

    1. I’m looking for the difference in this case, and I’m not seeing it.

      The question is whether the the conclusions of the IPCC’s Fourth Assessment Report might be contaminated by bias.

      The answer is that it cites propaganda leaflets from environmental activists.

      1. I think it’s important to remember too that this stopped being about the science a long time ago…

        They were using this study to support public policy questions that have nothing to do with science. Whether human activity is leading to climate change is a scientific question…

        How much of a sacrifice I’m willing to make personally in order to assuage the effects of global warming on future generations?

        …That isn’t a scientific question.

        There are scientists who tried to make it one–and they got busted.

        So the rednecks got one right. It was bound to happen eventually.

    2. She was very delighted with this new idea, and went to her
      professor. And his reply was, no, you cannot do that, because the
      experiment has already been done and you would be wasting time.

      Scientific verification in practice often leaves much verification to be desired. So I wonder how much verification there really is in climatology.

      1. Careful here, students (at least doctoral students) need “new” results to get Ph.Ds, so they shouldn’t replicate other work unless they’re going to overthrow it or that’s a step on the road to something new.

        That said, in my end of the business, important results do get regularly verified. Often there are competing experiments in the first place, and in any case subsequent generations of machine are likely to repeat your measurement as a side-effect of what they’re really doing.

  12. Oops Doctor T would be Thoreau who used to post frequently in the past. I didn’t mean to refer to the current person posting as “T”.

    1. Whatever happened to that thoreau guy?

      I really respected him.

      1. He gave up on us, and now only hangs out on Unqualified Offerings.

    2. Dammit, kinnath, don’t retract the statement that I’m a doctor. Send me a handsomely framed certificate, instead. It’ll look better than my lowly MBA.

      1. Thoreau is my hero. PhD Physics. He was a great poster.

        You need provide much evidence to take his place.

    3. I thought maybe you were referring to a scene from Dr. T and the Women.

      1. movie ever

  13. The entire report was peer-reviewed, unlike lame Daily Telegraph articles. Nothing out of the ordinary has been found to have been done in this report. It is truly impressive the lengths deniers go to sound like they know what they’re talking about when they’re parroting cheap anti-science propaganda, though.

    1. Note to puppeteer: Work on your ventriloquism; the voice isn’t quite right.

      1. Exactly. Less angry but more cluelessly snide.

  14. but came from newspaper articles, student theses, even propaganda leaflets and press releases put out by green activists and lobby groups.

    Newspapers do this same thing when they write articles on gun statistics. They include “facts” handed them through press-releases from activist groups. 30,000 people a year killed by gun violence…sheesh.

    1. Do we know where the statistic of 45,000 die from lack on insurance comes from?

      1. I believe that comes from the University of East Anglia. Something about a computer model running adjusted data .

  15. The global warming hoax deniers are dying a death of a thousand cuts.

    1. Just as we would have it!

  16. “Nothing out of the ordinary has been found to have been done in this report.”

    That’s the problem. You see, lying for person gain, corruption and laziness are all quite ordinary. It turns out that “scientists” are quite ordinary and U.N. scientists are especially ordinary.

  17. Donna the Raspberry. How Sweet.

  18. How thorough is the usual peer review process, anyway?

    Sometimes, people read the papers. If a reader thinks it’s his job to find and correct material mistakes in what he reads, he’s overruled and doesn’t get asked to review things anymore. It’s very thorough.

  19. Joshua Corning called this the day the climate e-mails broke. The most damaging part of the e-mails was the West Anglia crew’s efforts to subvert the peer review process. Basically, they got all of the academic reviews to agree that skepticism was not acceptable and then discounted any skeptical claims by saying they were not “peer reviewed”. The fact that the peers refused to review them wasn’t mentioned. The flip side of that was the shoddy review that went on for work that affirmed global warming.

  20. You people don’t even have a clue how preconceptual science works.

  21. It all comes down to what you mean by “peer” review. Apparently the global warmenist climatologizers defined their peer group as other global warmenist climatologizers.

    See, its all good.

    1. LOL! Good point; in fact, you are quite right: peer-review in itself is a sociological construct. In fact, it has often been used to block progress when it came from outside the peer-group, instead of fostering progress for its own sake, as science should do.

  22. My favorite part was the phrase “completely unlikely”.

  23. Let’s see: over 12,000 peer reviewed citations, and the remaining 5,000 probably consist of primarily government reports (from such agencies as the EPA or the UN). Funny how the denalists who conducted this “audit” failed to mention this.

    In the meantime, the denalists have what, a couple dozen papers that back their view…maybe?

    1. Like I wrote earlier, it’s not so much about whether the earth is warming or not, nobody has a stake in arguing one way or another. The stake lies in requiring us to spend Trillions to fix a problem we are not even sure Trillions in spending can even affect. Furthermore, a swath of that money is proposed to be transfer payments to developing countries which does absolutely nothing but enrich [insert corrupt official’s name here].

      1. So when our industrial machine becomes responsible for poor countries being thrust into misery, starvation, and even sinking into the ocean, we should just say “sorry, I really had to have this Hummer.”

        1. Or … sorry, I have to have refridgeration, more like it.

          Hummers are litle a fraction of a percent of the problem.

          The real drivers of are manufacturing and air conditioning/refridgeration.

          You want to get rid of those things?

        2. These payments are supposed to be an “I’m sorry” payment to the third world then? Mitigation of the effects or perhaps reversing the effects was never the point of these payments? It was simply to relieve our guilt by enriching their dictators. Suggesting this idiocy should prevent the person who proposed it from ever managing public money as they’ve immediately shown themselves unworthy of competently managing resources.

    2. And of course, the Governmental agencies issuing the reports had absolutely NO interest in the outcome.

      Great crippled Christ on a broken crutch! Why do Choads always maintain that GO’s and “non-profit” NGO’s have no ulterior motives for their actions/assertions?

      Give me the profit motive any day. Even dishonest traders are at least still traders. I may not trust someone who’s primary motive is a $ more than I trust someone who’s primary motive is political power or acclaim. but I at least UNDERSTAND them.

    3. Let’s see: over 12,000 peer reviewed citations, and the remaining 5,000 probably consist of primarily government reports (from such agencies as the EPA or the UN). Funny how the denalists who conducted this “audit” failed to mention this.

      In the meantime, the denalists have what, a couple dozen papers that back their view…maybe?

      And that is relevant how ?

      1. It means you’re wrong.

  24. even propaganda leaflets and press releases put out by green activists and lobby groups.

    What kind of scientist uses information from those sources in reports?

    It would be like psychologists doing a paper on homosexuality and using God Hates Fags Dot Com as an authoritative source.

Please to post comments

Comments are closed.