Climategate and FOIA in Virginia

|

Mathematical mirage or climate history?

Last week, the University of Virginia turned over nearly 4,000 pages of data and emails relating to the research of climatologist Michael Mann (who was then at UVA), the lead researcher on the notorious "Hockey Stick" temperature graph. The data underlaying that graph purportedly showed that the 20th century as the hottest century in the last 1,000 years. A couple of skeptical Canadian researchers Stephen McIntyre and Ross McKitrick later attacked the statistical method used to process the raw data. In 2005, Technology Review reported

McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but it handles data normalization in a way that can only be described as mistaken.

Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called Monte Carlo analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!

The Hockey Stick remains controversial, but some more recent research has tended to confirm its general conclusion about the trends in global warming.

Mann was also one of the researchers whose sometimes less-than-objective emails were released in the Climategate scandal. The Climategate emails and data have spurred skeptics to seek further information about how climate science has been conducted and negotiated among researchers. In Virginia, the state Attorney-General Ken Cuccinelli (R) and the market-oriented think tank, the American Tradition Institute (ATI) have been seeking access to Mann's emails and other research under the Virginia Freedom of Information Act (VFOIA). In May, a state judge ordered UVA to release the Mann data and emails under the VFOIA. Last week, ScienceInsider reported

Months after a request from a Virginia politician and a conservative think tank, the University of Virginia (UVA) has turned over documents related to embattled scientist Michael Mann's research into the history of Earth's climate. But what the petitioners really want to see may still be at large…

In a press release this afternoon, ATI said it had received "a 4.3 megabyte disk that contains 3,827 pages" of data. Paul Chesser, ATI's executive director, said ATI staff members had not had a chance to review the contents. "We think we got a third" of the documents requested, Chesser said. Mann, who says the university is keeping him abreast of the documents it releases, says they consist of routine e-mail messages and similar "boilerplate." …

"U.Va has not turned over emails related to discussions of research, unpublished manuscripts, private discussions between scientists about science, etc.,–i.e., any of the materials that are exempt from release by state law," Mann wrote in an e-mail message. "U.Va has simply turned over the non-exempt emails, and many of these were turned over to ATI months ago."

Interestingly, the National Science Foundation which funded some of Mann's research issued a research misconduct investigation closeout memorandum concerning Mann. That memo concluded [PDF]: 

To recommend a finding of research misconduct, the preponderance of the evidence must show that with culpable intent the Subject committed an act that meets the definition of research misconduct (in this case, data fabrication or data falsification).

The research in question was originally completed over 10 years ago. Although the Subject's data is still available and still the focus of significant critical examination, no direct evidence has been presented that indicates the Subject fabricated the raw data he used for his research or falsified his results. Much of the current debate focuses on the viability of the statistical procedures he eniployed, the statistics used to confirm the accuracy of the results, and the degree to which one specific set of data impacts the statistical results. These concerns are all appropriate for scientific debate and to assist the research community in directing future research efforts to improve understanding in this field of research. Such scientific debate is ongoing but does not, in itself, constitute evidence of research misconduct.

Lacking any direct evidence of research misconduct, as defined under the NSF Research Misconduct Regulation, we are closing this investigation with no further action. …

Finding no research misconduct or other matter raised by the various regulations and laws discussed above, this case is closed.

It will be interesting to see if ATI actually uncovers anything that suggests witting misconduct, but my suspicion is that the most the institute will find is research confirmation bias, if that. 

See also my earlier post on Climate Change FOIA Hypocrisy where I urge: 

Instead of engaging in FOIA battles which fuel conspiracy theories, the real solution is for publicly funded researchers to embrace research transparency, putting all their data and methods online for everyone to see.

Advertisement

NEXT: Washington Post Ombudsman: Paper's Coverage of Ron Paul Has Been "sparse"

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. The Hockey Stick remains controversial, but some more recent research has tended to confirm its general conclusion about the trends in global warming.

    Science, Ron, doesn’t this very article demonstrate that the “Hockey Stick” is complete and total BS, rather than “controversial”?!

    “It has been shown to be entirely made up, and yet some people still refuse to admit its truth”?!

    1. The infamous “Hockey Stick” graph was featured prominently in the IPCC TAR Summary for Policymakers. It was important in that it cast serious doubt on the notion both of a global Medieval Warm Period warmer than the 20th century and of a global Little Ice Age, both long-time (cautiously) accepted features of the last 1,000 years of climate history. It seems these periods were regional, not globally synchronized — though the LIA seems to have been more widely experienced.

      This caused quite an uproar in the skeptic community, not least because of its visual efficacy. Two Canadians, an economist and a petroleum geologist, took it upon themselves to verify this proxy reconstruction by getting the data and examining the methodology for themselves. They found errors in the description published in Nature of the data used — errors that prevented them from duplicating the study. Mann et al., the hockey stick’s creators, published a correction in Nature, noting where the description did not match what had actually been done. The Canadians, McIntyre and McKitrick, then published a paper purporting to uncover serious methodological flaws and problems with data sets used.

      ‘The hockey stick is broken’?Well, no … but who’s playing hockey anyway?
      http://www.grist.org/article/the-hockey-stick-is-broken

    2. Did you see the link to the article titled “The ‘Hockey Stick’ Lives”? It was right there in the bit of text that you quoted and italicized.

      1. I did. I simply find it difficult to believe.

        How many different ways does Mann have to be shown to be full of shit before his lies cease to be “controversial”. He “massaged” the data but coincidentally got it exactly right?

        1. So much politics and rent seeking depends on it being true.
          Admit to it being false and now there is no excuse to control what fuel people use for their cars, what light bulbs they may purchase, or get grants for “green technology”.

          Even if AGW is proven without a doubt to be false, it will still be considered fact.

          1. Even if when AGW is proven without a doubt to be false

          2. So much politics and rent seeking depends on it being true.

            ^This. The argument has never been about warming.

    3. AFAIK random numbers fed into Mann’s “model” result in the “hockey stick.”

      There is nothing controversial.

      Nice attempt at soft-pedaling bad science, there. Even if someone doesn’t look at the e-mail evidence that strongly supports intent, this is BAD SCIENCE.

      1. It ain’t science.
        No scientific method, no science.

      2. Were they over ANY group of random numbers? Because a random group of numbers could experience chains of data that result in what looks like a hockey stick. The question is, how did Mann’s model analyze the data in relation to how other (“right”) models analyzed the data? This was not demonstrated here. Comparing the output of a model to a priori information about the statistical distribution of the dataset proves nothing. Hell, you could disprove probability theory with that mindset over short enough data sets.

        1. Were they over ANY group of random numbers?

          The fact that you asked that question, and you were serious, disqualifies you from having your opinion considered.

          Hint: read up just a TINY BIT on what you’re running your discksucker about before wasting page space.

          1. I’m talking about, did they take a small subset of the data? I’m not talking about a selective subset, but, say, samples 11,000 through 20,000 in a data pool of, say, 1,000,000.

            Anyway, my real point was that I wanted to see how the two models analyzed the data, not how one model analyzed the data based on information we know about the data before we even run the test. It’s the difference between results that credit one method over another, not that one sucks, so you’re left with another.

      3. What it does demonstrate, at the least, is that Mann did not make any meaningful effort to develop a way to see if his hypothesis was falsifiable. A basic part of scientific research (as opposed to humanities, sadly) would be to verify that your research methodology can actually detect what it purports to detect and can correctly distinguish between a real effect and a spurious effect. At the very least Mann’s methods have been shown to fail at that level.

        More critically, however, these results may indicate (but do not confirm) intentional tweaking of the model. (I say may, because, however unlikely, it is possible that the mistake was unintentional, that he had a flawed methodology that he honestly thought was correct.

        But either way (intentional or not), it does mean that, as sarcasmic notes, his science was not science since his research appears to be incapable of falsifying his hypothesis.

        If others can demonstrate the same result from his model (random data consistently generates the hockey stick) then his results have to be discarded entirely and any other studies that confirm the results should be subject to the same testing with random data. In fact, the authors of any such studies should welcome that extra layer of scrutiny since it would allow them to demonstrate that their results are accurate. If they resist this scrutiny it should be a red flag about their research. (I imagine a number of climate researchers are now quietly doing this to their own previous research: whether they will release those results is another question, but if I were a researcher who had relied on Mann’s conclusions in any way, I’d be very uneasy right now.)

        1. Re: White Laplander,

          If they resist this scrutiny it should be a red flag about their research.

          The infitely gullible (i.e. leftists) do not seem to consider this, instead blaming the skeptics for not being just as gullible.

          1. I’m not sure anyone is gullible. I think that the “findings” serve the purposes (financial, psychological, etc.) of the people who choose to accept them without critical thinking — and to attack those who don’t.

            1. Re: Barry D,

              I think that the “findings” serve the purposes (financial, psychological, etc.) of the people who choose to accept them[…]

              Oh, posh! Why, that would mean they’re nothing more than self-interested rent-seekers that…. Oh! I see what you mean!

      1. Comment From Here

        While I agree that the hockey stick has decayed, I am somewhat stunned by Von Storch and Zorita’s new found pollyanish view of paleoclimate science culture.

        As they well know, the hockey stick debate advanced in spite of intolerance and obstructionism by their peers. It advanced because of two investigators outside of the walls of climate science (McIntyre and McKitrick 2003 & 2005) relentlessly investigated, and published pointed critiques that outraged the science’s gatekeepers. And then it continued to advance because a few vetted “non-skeptic” climate researchers (starting with Von Storch and Zorita) were willing to risk the firestorm to challenge the methods behind the stick’s orthodoxy.

        And finally it advanced because of unwelcome politics – a Congressional investigation and request that made scientists confront the issues.

        More disappointingly, the authors also seemed to have forgotten that the hockey stick was jsut the immediate symptom, that both they and M&M had raised more fundamental issues regarding paleo-climate science core culture – the lack of full disclosure, the acceptance of journal unenforced policy, the lack of informed and robust peer reviews, etc.

        What then has really changed? Von Storch, et. al. have recently gotten fairer hearings, and they are back within the walls of acceptable opinion. None the less, the cultural practices that led to this crisis have NOT changed, and that Von Storch and Zorita found it necessary to studiously ignore mentioning the two researchers that are most responsible for the stick’s decay suggest they know the walls remain, and the gatekeepers are vigilent.

        Von Storch and Zorita should not confuse their own serenity with peers as a change in culture – nor should they give up fighting for their (former?) ideals.

    4. There have been half a dozen independent papers confirming the hockey stick using boreholes, ice cores, tree rings, lake sediments and stalagmites as proxies. You can read some of them here:

      http://www.skepticalscience.co…..-stick.htm

      This is an issue settled years ago: the hockey stick is real and science has moved on.

  2. the real solution is for publicly funded researchers to embrace research transparency, putting all their data and methods online for everyone to see.

    I will make the same comment for privately funded researchers.

    Science is science.

    1. Wrong.

      Are real world results of things like GMO crops and such not enough to make you see the correctness of the science behind it?

      Old men get hard dicks without publicly displaying the science behind viagra et al.

      Can these rent seekers researchers say the same about their science?

      When corporations are trying to extort hundreds of billions of dollars via the use of government force, I’ll buy the “we need to see their science” line. In that vein, when government funded universities can show the real world correctness of their science via real world results, and not via graphs and charts which are based on manipulated and cherry picked data, I’ll stop demanding that they come clean.

      1. ???

        I was saying that SCIENCE is open source.

        If you are doing R&D, keep it secret, but if you want it to be called science, you need to be open.

        1. People often get science and technology confused.

      2. Re: Mad libertarian guy,

        Can these rent seekers researchers say the same about their science?

        Well, they can tell you that if they attach the words “[…]under the effects of Climate Change” on the title of their investigation for their government grant application, they will succeed more often than not. Now, that’s science!!!

  3. Making science is like making sausage.

    1. Only Religion (Mine) is Pure

    2. Herman Cain agrees that no muslim should be involved in it.

  4. How to Talk to a Climate Skeptic – Grist
    http://www.grist.org/article/series/skeptics

  5. I remain an AGW agnostic. I don’t know, don’t pretend to know, and don’t really care.

    1. Lots of reasons to care. The Canadian economy won’t have to rely on maple syrup exports w/ all that new farmable land, and my outdoor swimming pool season can be extended. Plus more caribou to hunt with all the polar bears drowning.

    2. You’re a smart man; one of the few who know the truth.

      1. ^observe the wingnutz know-nothingism in which not-knowing or caring passes for wisedom

        1. Re: O2,

          observe the wingnutz know-nothingism in which not-knowing or caring passes for wisedom

          Which would make him a wise-ass. But not doing something after inconclusive evidence is showing wisdom.

          If you were really serious about your do-something-ism, you would be the first one to string up a person at the mere suggestion he rapes children. Otherwise you’re just another hypocrite.

          1. fact is the “evidence” will never satisfy deniers since their denial is political not evidentary.

  6. McIntyre and McKitrick obtained part of the program that Mann used,

    Is it SOP in the science world to share part of your model, but not all of it, when your conclusions are being reviewed and tested?

    1. It is SOP in the science world to provide all of the information necessary for someone else to replicate your results. Whether or not this includes computer code depends on the situation. Generally, re-running someone’s code and getting the same results tells you nothing. It would be like going over to the someone else’s lab and running the experiment there. If there is some systematic error in the code or the experimental setup, it will still be there.

      That being said, if you run the experiment or attempt to reproduce their model based on the provided information and fail to do so, it is common to attempt to either 1) work with the other scientist to figure out where your error or their error is, or (2) write a paper saying that your results were different and plainly laying out the details of your experiment or model. Harassing the scientist in an attempt to get the information is not particularly common.

      1. Harassing the scientist in an attempt to get the information is not particularly common.

        I’m sure its not, because most scientists operating in good faith will share their data and models without being harassed.

        1. most scientists operating in good faith will share their data and models without being harassed.

          The need for the VFOIA request tells everyone all they need to know about the ‘good faith’ of the AGW crowd.

          1. But the VFOIA requests are not for data, or anything remotely related to the actual work. It’s for corespondence between a grab bag of people, many of whom have nothing to do with Mann’s paper.

  7. Scientific Method: Hypothesis->Experiment->Data->Conclusion

    No where do I see “come to a consensus” or “ridicule, mock and ostracize anyone who goes against the consensus” in the Scientific Method.

    Politics? Yes.
    Science? No.

    1. Exactly. This has come to reek, abjectly, of politics, and no longer has anything to do with science. Which means it’s now just rank partisanship.

    2. Scientific Method: Hypothesis->Experiment->Data->Conclusion

      you forgot replication.

    3. You forgot one…

      Independent Replication

      1. That was implied, but yes.
        And that is also something non-existent in climate “science”.

        1. And that is also something non-existent in climate “science”.

          That is a false statement.

          1. What is there to replicate?
            Computer models that spit out the same result regardless of what data is put into them?
            Please. Climate “science” is a joke.

          2. NM has a point here.

            People have replicated results in climate science. Running the same cooked data through the same kludged-up model gives the same results, every time!

            1. People have replicated results in climate science. Running the same cooked data through the same kludged-up model gives the same results, every time!

              This is an ignorant statement.

          3. That is a false statement.

            There has been no independent replication. The models have zero predictive success – even over much shorter time periods than they claim.

            1. There has been no independent replication.

              That is a false statement.

              The models have zero predictive success – even over much shorter time periods than they claim.

              That is an exaggeration.

  8. ridicule, mock and ostracize anyone

    I thought that was our job. Damn scientists, takin’ our jerbs.

  9. The science is settled meddled.

  10. There’s a huge difference between the “general conclusion” that there is an upward trend and an upward trend that shows exponential growth. It’s like saying my living standard increased over the last 30 years, and so did Bill Gates. There’s sure to be a general conclusion that both of our living standards increased…

    1. Exactly.

      A model isn’t valid when it shows that a number changes in a positive direction, and it does. A model is valid when it predicts HOW much the number will change.

    2. Plus its the sharp upward trend that is used to claim that human activity is the cause. A gradual, consistent rise opens the door to consider natural climate shifts as the cause.

  11. Here is an interesting article. About how there is a theory of a massive earth fart that was responsible for the triassic extinction…anyway, I saw this Killer Lakes episode thing on Discovery. Awesome, scary but awesome. Dealt with CO2 and Methane saturating some lakes then being released all at once by a landslide (Killed thousands of people). If I were a crazy conspiracy guy I would think that might be one more cause for CO2 increases and not humans…weird. There is no way the earth is as strong and powerful as teh mankind. We are clearly the quizarts haddarach(sp).

    1. This was televised on a coast-to-coast flight on which I was a captive viewer. It was really freaking scary. Total fear mongering. I bet it convinced half of the people on that 747.

    2. Yes. You. Are. Irrelevant. Arrogant. Little. Man.

  12. One among many reasons academics and scientists should be free from politically-motivated witch hunts: every out-of-context email mined makes stupid people think they know something, and are therefore free to blather on about a complicated subject they clearly know nothing about except what they read on a right-wing blog.

    1. I have 5 years of post-grad education in statistics. This is called “fitting the data.” I don’t have to know anything topical about the experient to know that Mann is a fraud.

      1. Yet Mann could never have been born and there’d still be a huge amount of data suggesting that increasing concentrations of heat-trapping gases in the atmosphere traps heat. One guy’s alleged shenanigans don’t undermine the laws of physics.

        1. The scientific genius has thus spoken. Farts trap heat, let’s ban frijoles.

          1. actually the global cattle herd is a prob causing methane plus ammonia-contaminated groundwater.

          2. Even the Afghanis knows farts trap heat and they are offended when Marines ask them to pull their finger.

    2. That’s fine. We should also free them from political/public funding as well. Then we don’t have to be concerned with making them publicly & politically accountable.

  13. One among many reasons academics and scientists should be free from politically-motivated witch hunts

    Would those include the witch-hunts that defend the consensus, as well?

  14. Ron, given that they’ve spent tons and tons of money to prevent these e-mails from getting out, it wouldn’t surprise me if they contained evidence of misbehavior.

  15. Why does the “Hockey Stick” even matter?

    It’s not an actual temperature record.

    It’s a reconstruction using “proxy data”, which begs the question of whether or not the proxies track temperature reliably.

    Unfortunately, for the last 60 years or so, the tree ring proxy data and actual temperature measurements haven’t correlated very well. Measured temperatures have risen, while tree ring proxies have shown a decline. You can Google the “Divergence Problem” for details.

    This divergence is well known to dendroclimatologists, and remains inexplicable. Which raises a simple, but critical question… If proxies and temperatures are known to diverge today, how can we know they didn’t do the same thing in the past?

    An honest scientist would acknowledge the problem, describe it as completely as possible, and exercise appropriate caution when drawing conclusions from the data.

    The scientists who created and promoted the “hockey stick” reconstruction instead chose to hide the declining part of the proxy record, and quietly replace it by grafting on the actual rising temperature trends to make it look like there wasn’t any uncertainty.

    Berkeley professor Richard Muller explains in detail: http://www.youtube.com/watch?f…..BQpciw8suk

    1. The word “proxy” is meant to confuse stupid people. Kind of like the phrase “quantitative easing.”

    2. How do we know that proxies don’t diverge from the actual numbers? When the proxies support our foregone conclusions?

  16. The Hockey Stick remains controversial, but some more recent research has tended to confirm its general conclusion about the trends in global warming.

    Which can only tell one that these researchers are committing the same statistical error than Mann. You cannot simply sweep under the rug the procedural error that MkIntyre and McKitrick discovered:

    “This method of generating random data is called Monte Carlo analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!

    1. Actually, when their work was reviewed it was found that M & M’s analysis did not use “random” data, that downmard hockey sticks were as common as upward hockey sticks, that the magnitude of the hockey sticks produced by M&M were much smaller, and that for their paper, they mined the 1% of hockey sticks that looked the most dramatic while failing to show any of the non hockey sticks or downward pointing hockey stick.

      Clearly, this “statistical error” cannot account for the hockey sticks seen in all subsequent papers. (nor even Mann’s)

      1. “Actually, when their work was reviewed it was found…”

        Link?

      2. Actually…

        By the way, tonylurker, there is nothing I love more than someone who starts their sentence with actually, while being wrong.

        McIntyre and McKitrick said that using the same steps as Mann et al., they were able to obtain a hockey stick shape as the first principal component in 99 percent of cases (counting both upwards and downwards-pointing “blades”) even if simulated red noise without any inherent trends was used as input.

        http://www.uoguelph.ca/~rmckitri/research/trc.html

        They did not “mine the 1% of hockey sticks that looked the most dramatic while failing to show any of the non hockey sticks or downward pointing hockey stick.”

        I do not understand your motivation for openly lying, but it’s sad. Knock it off.

  17. Instead of engaging in FOIA battles which fuel conspiracy theories, the real solution is for publicly funded researchers to embrace research transparency, putting all their data and methods online for everyone to see.

    Let me up one better: How about getting rid of public funded anything?

    The idea that pure science requires stolen loot to fund it is LUDICROUS. Most of the great discoveries of the 20th Century were done by young scientists working on their doctoral papers.

    1. Or out of their garage/basement laboratories.

  18. Of course it would be Canadians who would investigate the hockey stick.

  19. Instead of engaging in FOIA battles which fuel conspiracy theories, the real solution is for publicly funded researchers to embrace research transparency, putting all their data and methods online for everyone to see.

    But Mann’s data and methods are available for everyone to see (at least all the data and information needed to independently asses his work and conclusions), yet the FOIA battles continue.

    This isn’t about Data and methods, this is about conversations between scientists; The gripes, the back and forth of the editing process, and any other communication that might have something useful for propaganda purposes.

    1. This isn’t about Data and methods, this is about conversations between scientists; The gripes, the back and forth of the editing process, and any other communication that might have something useful for propaganda purposes.

      Funny enough, that’s exactly what the NYT thought when they released the Palin emails. How did that propaganda campaign go?

  20. I think Bailey has had one cup too much of the AWG koolaid.

    A better discussion can be found at Watts Up With That:

    http://wattsupwiththat.com/201…..more-45982

  21. Yet while the attacks continue, the “hockey stick” graph’s basic premise ? that the planet’s recent warming is unprecedented over at least the last millennium ? continues to draw support from a growing number of independent studies.

    Oh really?

    http://www.climategate.com/wp-…..-10000.jpg

Please to post comments

Comments are closed.