Science

Most Preclinical Life Science Research Is Irreproducible Bunk

$28 billion in research funding wasted every year.

|

BrokenTestTube
twocities

Biomedical science is broken, according to a new study. In their article, "The Economics of Reproducibility in Preclinical Research," published in the journal PLoS Biology, a team of researhers led by Leonard Freedman of the Global Biological Standards Institute reports that more than half of preclinical research cannot be replicated by other researchers. From the abstract:

Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible—in the United States alone.

Back in 2012 I reported on other studies that also found that the results in about 9 out of 10 landmark biomedical papers could not be reproduced

The scientific journal Nature published a disturbing commentary claiming that in the area of preclinical research—which involves experiments done on rodents or cells in petri dishes with the goal of identifying possible targets for new treatments in people—independent researchers doing the same experiment cannot get the same result as reported in the scientific literature. 

The commentary was written by former vice president for oncology research at the pharmaceutical company Amgen Glenn Begley and M.D. Anderson Cancer Center researcher Lee Ellis. They explain that researchers at Amgen tried to confirm academic research findings from published scientific studies in search of new targets for cancer therapeutics. Over 10 years, Amgen researchers could reproduce the results from only six out of 53 landmark papers. Begley and Ellis call this a "shocking result." It is.

And ten years ago in his groundbreaking article, "Why Most Published Research Findings Are False," Stanford Uninversity statisitician John Ioannidis found:

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

Science is supposed to build and organize knowledge in the form of testable explanations and predictions. If reported research results cannot be reliably replicated, they are not science.

NEXT: Reason Weekly Contest: Begin a 'Viral' Moral Panic Facebook Post

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. If we ran this experiment again, would we still only get 6/53?

    1. oooh, a very meta statement. Nicely done.

    2. Dunno. Why don’t we run it up the old wood chipper and see if anyone salutes?

    3. One would hope that the quality of published science would improve such that a 6/53 result would never again occur.

      Much of what we’re observing is p-hacking. If scientists do lots of experiments, many will confirm a hypothesis with statistical significance of p less than 0.05.

      I’m not hopeful though. NOAA has just published a paper that refutes that their has been any pause in global warming. It’s based exclusively on a single dataset out of eight major temperature datasets. They couldn’t get p less than 0.05, so they settled for p less than 0.10.

      This is what now passes for science … consensus science at that!

      1. When you can’t find enough outliers to get yourself to 0.05, your data are shit.

        1. Actually, it’s your hypothesis that is shit. Your data may or may not be shit.

  2. “when there is greater financial and other interest and prejudice,…”

    DING! DING! DING!

    1. Exactly follow the money. If we made it illegal for tax dollars to fund any scientific research, I dare say that scientific research would quite suddenly have a lot more integrity.

  3. So if most of the work done in something as straight forward as preclinical life science is bunk, I am sure the work done in something as complex and vague as climate science is totally on the up and up.

    1. How is life science more straight forward than climate science? I would have thought the opposite to be true. Repeatability is necessary in science. Think of an experiment performed on a group of people 20 years ago. Can that be repeated today? Those original people are 20 years older, and besides they are already aware of their previous participation. You have to use different people, and repeatability goes out the window, unless you want to maintain the fiction that the first group subjects is not different from the second.

      With climate science, at least you are working with rocks and gases rather than people, which smooths out some of these wrinkles.

      1. I would look at it the other way. Biology is biology. I can’t get those people but I can get people just like them. I can control for outside factors with people a hell of a lot easier than I can with the weather of the entire planet.

        1. “but I can get people just like them”

          Are you so sure? I assume that any group of people 20 years ago is going to be different in any number of ways from a similar group today. We are an inseparable part of a dynamic environment and we change as our surroundings change. Same problem for climate science, the earth is not a laboratory, and experimental conditions can never be truly replicated. Just with medical science, the issue of human consciousness enters the picture and that adds a whole dimension of complexity absent in physical sciences.

          1. I can get them pretty close. And I can get them a lot closer and eliminate outside factors a hell of a lot more than you can eliminate outside factors studying the climate.

            1. “And I can get them a lot closer and eliminate outside factors…”

              It seems the article, which is about pre-clinical life sciences, should make you question that confidence.

              1. The difference is that with biological studies one can put in lots of controls and test thousands or millions of subjects. We only have one climate and can’t control it at all. Statistics!

                1. “millions of subjects.”

                  Besides mounting costs, I think Ioannidis points out that enormously scaled up experiments introduce statistical problems of their own. I’m no statistician, so you might want to follow the links to the source.

          2. “Think of an experiment performed on a group of people 20 years ago. Can that be repeated today? Those original people are 20 years older, and besides they are already aware of their previous participation. You have to use different people, and repeatability goes out the window,”

            That’s idiocy. Your hypothesis and what you’re are testing for has to be broad enough to encompass such changes. If you can’t produce consistently repeatable results then it’s not fracking science.

            1. “be broad enough to encompass such changes”

              Assuming you know what all the relevant changes are. It may not be as easy as you think it should be. That seems consistent with the article.

              ” If you can’t produce consistently repeatable results then it’s not fracking science.”

              Again you are assuming that the initial conditions in any two experiments are identical. This is simply not tenable. You don’t have access to a time machine, for a start, not to mention identical test subjects.

      2. If you studies did use the same people, they would be of less applicability to the populace at large, so picking a different set by the same criteria would improve the value of the data when the study was repeated.

        1. Heraclitus, everything is in flux. You can’t put your foot twice in the same river. Modern science pretends you can.

          1. Unless there’s been a change in the composition, the fact that it’s not the same water molocules makes no difference.

            1. “the fact that it’s not the same water molocules makes no difference.”

              You’re assuming that every water molecule is identical to every other water molecule. Or every H atom is identical to every other H atom. Such gross simplification is bound to catch up with you sooner or later and bite you on the backside.

              1. “You’re assuming that every water molecule is identical to every other water molecule. Or every H atom is identical to every other H atom. Such gross simplification …”

                You truly are an imbecile.

                1. “You truly are an imbecile.”

                  Are you trying to say that smart people all know that every H atom is identical to every other H atom? Or is your point something even more absurd? Maybe just don’t like those who question your idealism.

      3. A study done on 20 subjects isn’t likely to be worth much anyway.

    2. John, I don’t know if you are aware of the Mann v Steyn suit; there’s a development that I think you will admire.

      Mann sued Steyn for libel claiming Steyn called him to a child molester in a passage where Steyn showed the commonalities between Penn State’s exoneration of Mann for scientific misconduct, and Penn State’s investigation of that assistant coach that was sodomizing boys.

      Steyn has announced his intention to go to trial; has repeatedly claimed that Mann is, in his opinion, and based on the facts surrounding Mann’s actual work, a complete fraud. “Dr Fraudpants” he calls him.

      Here is the beautiful bit. In a move reminiscent of the Air Pirates challenging disney, Steyn is publishing a book that Mann might find of interest.

      This is what the cover looks like.

      It’s at moments like this when I am reminded of that episode of the Dilbert TV series:

      Todd does seem to have those other intangible qualities as well– the kind that make women love him and men wish they were women.

      1. I still can’t believe Mann brought the suit. What a arrogant prick.

      2. Mark Steyn is a beast. Who knows Mann is working the “process is the punishment” routine, so Steyn is pushing for the trial to happen as soon as possible. Even to the point of gleefully and immediately complying with all of Mann’s fishing expeditions… only for Mann to deny his.

        The whole process so far shows how much of a piece of shit Michael Mann is.

        1. I read an interview with Steyn (forget where now) in which he said his main purpose was to stop Mann’s “process is the punishment.” Apparently there was another academic scientists (in BC perhaps) who made critical statements about Mann and, by the time the lawsuit was done, the guy was basically broke, including losing all his retirement savings.

          1. Michael Mann is, in my opinion, a giant, putrid, hamster-fellating, douchebag-goateed shit-weasel.

            I write, of course hyperbolically about a horrible public figure, relating to his conduct on a matter of public concern under my First Amendment rights to free speech and to petition the Government for redress of grievances. This comment has no mention of wood-chippers.

            1. “Michael Mann is, in my opinion”

              You’ve met him I take it. Otherwise you are just repeating the opinion of others. Passing idiotic platitudes off as your own doesn’t say much for you.

              1. What idiotic tripe. I bet you’ve never met Obama, Charles Manson, and a ton of others, that you nevertheless have opinions on.

                You likely see all your BS as somehow poignant, like biology is impossible to study due to differences in humans (which if true moron, would mean medical science is impossible and should be nonexistent).

                In the end, you’re just stupid, and you’re points are meaningless and irrelevant.

                But I am curious, what did you ask mommy to get you for your upcoming 12th birthday? Almost a teenager!

                1. “What idiotic tripe.”

                  I thought so too. Still, wouldn’t it be nice to hear about these celebrities from people who actually knew them. Relying on 3rd hand accounts from the likes of tarran doesn’t satisfy my celebrity gossip needs.

  4. Is the scientific method unraveling? If so, why? Is it all the government funding?

    1. Yes. You don’t get grants to disprove theories. You get them to prove them. The best example of this is of course Ron Bailey’s beloved climate science. No one gets a government grant for work that disproves AGW.

      1. I should’ve phrased that slightly differently, in any case. Is adherence to the scientific method unraveling?

        1. Eh. Not really. The thing is, if it works consistently, you don’t have to hack the process. So all the good science still gets done, there are just more false positives than there should be. OTOH, a rigid adherence to the method would disqualify faint but real signals. Epigenetics, for example. If there weren’t people stubbornly clinging to the idea that there must be more to heritability than having certain genes, we might get stuck in a cul de sac for a long time.

          1. I suppose part of the problem isn’t just government interference and biases, but also that much more complex systems are the subject of much research. Reality can be a real bitch to understand when you get into the weeds.

            1. What is being hurt is the public perception of science. It is harder to discredit the people who claim autism results from a substance in vaccines or that a sample of water that statistically contains no active molecules of a substance has healing powers.

              1. I think it also requires established scientists and people promoting science to be tougher in public about the frauds and errors. The autism/vaccine thing is a good example. I’ve heard it talked about on a number of science and/or skeptical podcasts. And it’s nearly always presented along the lines of, “the autism/vaccine link was given new life by a single paper which has since been withdrawn.” True as far as it goes but it was published in a peer-reviewed journal. That needs to be addressed, explanations as to why it happened, and steps taken to prevent it happening again. The way it is being presented now makes it sound like a lone kook published it in some obscure journal.

                1. We’re seeing a lot more looseness around professional ethics in general. No surprise that extends to science.

      2. Yes. You don’t get grants to disprove theories. You get them to prove them.

        You get grants to come EVER CLOSER to proving theories, and thereby proposing a further theory that needs additional grant money to prove.

        1. The trouble is, if you are doing anything interesting, there is at least one other group chasing the same theory as you. So attempts to deliberately slow the pace of research are about as common, and about as successful, as price fixing schemes in low-end retail.

      3. In biosciences and medicine it’s really more about the pressure to produce “promising” results. Your PhD advisor or PI wants results that will pop so they can get more grant proposals funded, so the incentives and pressure is there for grad students and postdocs to, well, produce the desired results. It’s like that in all the other sciences to a lesser degree. You see it all the time. Great results published in prestigious journals that create some buzz for a while, but then you never hear about it again. Why? It was irreproducible bullshit. But it doesn’t matter. The PI and grad students got their names in a high impact journal. So they proudly put that on their CV and if nothing more comes of it at least they have that feather in their cap. Doesn’t seem to matter that it was fudged. At least not in the long run.

    2. “Is the scientific method unraveling? If so, why? Is it all the government funding?”

      I think there is a danger of creeping giganticism. Consider Newton’s experiments with optics. A room darkened by curtains, and a couple of prisms. Budget, less than a guinea. Compared this to the billions necessary for those enormous circular particle accelerators in use today. Growing costs for results of diminishing utility could conceivably kill science.

      1. An explosion of wealth brought on by weakening of political institutions would mean there’s a whole lot more wealth in existence to spend on scientific research that carries little return.

        1. I believe that great expenditures which bring little in return depend precisely on political institutions which are at their strongest. Think of the Egyptian pyramids, the west coast Indian’s potlatch, Europe’s WWI, or America’s moon exploration.

          1. Think of the Egyptian pyramids

            Massive amounts of slavery, theft and government predation.

            the west coast Indian’s potlatch

            A voluntary sharing economy=strong central state? Ridiculous analogy.

            Europe’s WWI,

            A blood bath and an economic and social catastrophe that is still being felt.

            America’s moon exploration.

            A proxy conflict for a country sitting on the edge mutually assured statist annihilation. But a moot point all the same.

            Your choices of analogy paint a pretty hellscape that you would subject everyone to.

            1. “Your choices of analogy paint a pretty hellscape that you would subject everyone to.”

              My choices are not analogies and I don’t know where you get the idea that I would like to subject anyone to them. They are all examples of man’s extravagance, and how man has wasted resources over the years. A strong state is not necessary, but there has to be social agreement among the populace for this kind of expenditure.

              The pyramids were not built by slaves, no matter how badly Charlton Heston was treated in the movie version. The potlatch was not carried out in a statist society, but still there is surplus that has to be destroyed, consumed or wasted – it happens in any organized society. That brings us to WWI, where young men fell over themselves trying to volunteer.

  5. No, YOUR biomedical science study is broken! We’re rubber, your glue. Everything you say about our research standards bounces off our reproducibility rates and sticks to your critical metastudy. Na na na na CAN’T HEAR YOU over the sound of grant money slapping our desks.

    1. Exactly that. I have been at many many basic science presentations where they show the one result that supports their hypothesis, not the 20 that didn’t. And they even admit to it. But there are grants that need to be justified and renewed and tenure to be ‘earned’. There are good ethical scientists out there who are successful, but many who wash out because they don’t play ‘the game’. The system is very broken.

      1. Same as it ever was. See the rat study referenced in Feynman’s famous Cargo Cult Speech.

      2. Why isn’t a large chunk of government grant money earmarked for reproducibility experiments? Their whole argument for government science is funding science that the private sector won’t, and I would think repeating obscure experiment done by graduate students who just want out of college already would be a big field of scientific study the private sector isn’t likely to fund.

        1. My proposal is that just as wveryone who gets a PhD in the sciences has to perform original research, anyone who takes a masters should have to do a reproduction of some likely scientific study.

  6. Regarding the ten-year old study (not just restricted to biomed), I wonder how many of these studies were on environmental science?

    1. “Environmental Science” has no experiments that can be reproduced. It has ‘corrected’ data and the ‘models’.

      1. AND consensus. Never forget “consensus”.

      2. And if old corrected data don’t confirm the models, climate scientists correct it some more.

        It’s laughably absurd to this old engineer who’s been following this since the GHWB Administration and Rio.

  7. Please God, let this happen. Pleas let us be worthy to live to see this.

    Peter Sterne at Capital New York reports that the popular website Gawker could be undone by none other than former wrestler Hulk Hogan, who is suing the company for $100 million for publishing portions of a 2006 sex tape featuring Hogan having sex with Heather Clem, wife of a shock jock named Bubba the Love Sponge Clem. The case goes to trial in a Florida court on July 6, and things could get existential for Gawker, according to Sterne:

    A loss, and an award of even a fraction of the $100 million Hogan’s attorneys are seeking, could empty the company’s coffers, forcing [Gawker Media CEO Nick] Denton to either sell the company outright or to hand much of its equity over to deep-pocketed investors. [Capital New York]

    http://theweek.com/speedreads/…..roy-gawker

    1. Nooooo, I had rather hoped unionizing would torpedo their operation in the end.

      1. Be still your tongue. If you can’t understand the pure joy associated with watching that site go bankrupt and Denton end up broke, you are bigger monster than Episiarch. No amount of “we are losing money because of our union” could ever match that.

        1. I agree with Commodious. I wanted to see the effects of a union-run tween info-site.

        2. I suppose putting the thing out of our misery quickly rather than slowly benefits humanity in the long run. To speak metaphorically, of course. Nobody at gawker should be physically harmed in the process of stripping their operation down to the plumbing to pay for their legal damages, and I have nothing but utmost concern for Denton’s wellbeing.

          1. I bet you are the kind of person who licks their ice cream cone rather than just eating it. I like my please and schadenfreude in big chunks.

      2. Yeah, I was hoping the union would fuck them over as well, not some pedestrian libel claim.

        But then again, Hogan + union might be twice as delicious.

        1. This would be an excellent moment for a Gawker strike.

        2. It is not really a pedestrian libel claim. It is an invasion of privacy claim. It is really a suit against Gawker for being assholes. And if the lose, it will be because the jury conclude they are assholes and deserve to lose. There is a lot of karma in that.

        3. As long as they’re crying, I don’t care what caused them to cry.

    2. Where is sarcasmic when you need him? Principals, not principles. Fuck Gawker.

      Does the Hulk have a case?

      1. Legally, he probably shouldn’t win. But it is going to a jury in Hogan’s home town. The fear is that the jury will rule in his favor and slam Gawker with a $100 million judgement because Denton is such a fucking douche bag and the site so dislikable. If that happens, Gawker can’t appeal without putting up the entire judgement as bond. They don’t have anything close to a hundred million dollars and would be screwed. Denton would have to sell or declare bankruptcy and shut the site down.

        1. Excellent. This calls for a celebration.

        2. NAL. Is a $100M judgement even in the realm of possibilities? Also, won’t the Denton lawyer ask for a change in venue based on probable jury bias?

          1. The demand is for $100 million and a jury can award up to the demand. So, yes it is possible. And they have tried to change the venue and the judge denied the motion. The trial is happening there and the case will go to a jury. If the jury rules against Gawker, they will likely have a good case on appeal, but will have to come up with the total amount of the judgement as bond before than can make that appeal.

            1. Has any similar sextape case ever garnered such a large judgement? I know American juries can get excessive in their civil judgements, but I have my doubts that 100 million will be the amount. Hopefully though, 10% of that figure will be enough to kill Gawker.

              1. No. But again, that doesn’t mean this one won’t. There has never been a sex tape case involving a jury from the plaintiff’s hometown sueing someone who was as big of a douche bag as Denton. Juries do get pissed off occasionally and do give outrageous awards.

      2. That Bubba guy is a giant douche, no play on his name intended. One can hope that he was being cuckolded.

        1. Since he was taking and directing the tape, that is not a hope it is more of a fact. He is apparently into watching other men nail his wife. Not the weirdest or most uncommon kink but certainly a pretty far out one none the less. If that is his thing, what business is it of Gawker’s? How is it not a terrible invasion of privcy to publish the tape over his objection?

        2. Yeah, he’s bad people.

    3. Huh. Perhaps the Kochs could buy them.

      Oh, yes, I went there.

      1. Reason should crowd source a fund to buy it in bankruptcy.

        1. Torture aggrieved Progs.

        2. Keep the same blogs, but slant the fuck out of them to libertarianism. Just for a while, then make them focused on their purported subject areas. You know, like making io9 a science fiction blog, rather than a Dr. Who/transgender blog.

    4. “Peter Sterne at Capital New York reports that the popular website Gawker could be undone by none other than former wrestler Hulk Hogan, who is suing the company for $100 million for publishing portions of a 2006 sex tape featuring Hogan having sex with Heather Clem, wife of a shock jock named Bubba the Love Sponge Clem.”

      This is hilarious given that Gawker threw itself into fits of righteous outrage over the stolen celebrity nude pictures during ‘the fappening.’

      So naked video of Hulk Hogan you stole = okay, naked pics of female celebrities someone else stole = morally objectionable. I think a 100 million dollar suit over this is completely absurd and think it would be wrong for Gawker to lose something like that, but the hypocrisy on display is staggering.

      1. It wouldn’t be wrong at all. It would be proof that sometimes justice is served albeit not directly.

        1. It would be wrong because it’s a ludicrously large and unjustified reward. I hate Nick Denton and Gawker as much as anybody, but if a similar suit were being brought against, say, Reason (though I don’t know why they would have published a sex tape) I imagine you’d oppose the idea that they should lose that kind of money over publishing a sex tape.

          I don’t think anybody should lose that kind of money for publishing a sex tape, even people I hate.

          1. I do. The people at Gawker would love it if something like this were to happen at reason and have no principles at all. They therefore do not deserve to benefit from mine. Moreover, if they lose this, it will be because they are complete assholes and the jury hates them. That is justice.

          2. Hogan will never recover anywhere near that much, but Gawker could get nailed pretty hard if they acted willfully.

  8. Why Most Published Research Findings Are False,”

    I KNEW it! Global Cooling was false!

    1. Global cooling and warming are dueling. They’re at a standstill now, which explains the sixteen-year warming pause. But believe you me, the earth will cool and/or warm to some unknowable degree at some point sooner or later.

      1. “[…]But believe you me, the earth will cool and/or warm to some unknowable degree at some point sooner or later.”

        I’m certain you can get a grant to ‘prove’ that, so long as you couch it in dooms-day verbiage.

        1. Our planet is suffering a prolonged period of catastrophic climate inertia. If this torpor is allowed to persist for much longer we may enter an era characterized by total meteorological inactivity: no climate and no weather at all.

          Where can I pick up my grant check?

  9. And to think all this time I was eating oat bran to stay healthy!

    (Wups…brain got locked into 1990…)

    P.S: Wood chipper. I’d say more, but I’m afraid that those busted papers would take legal action against me.

  10. See what? Gawker implode or the sex tape?

    1. Yup. And Heather Clem is quite the MILF. Damn does it pay to be a retarded radio host.

      1. A retarded radio host with a penchant for watching his milfy wife get railed by celebrities, apparently.

        1. I guess she likes the benefits that come with the job of being wife of retarded radio host.

      2. *quickly googles Heather Clem sex tape.*

        Huh. Well, here is a, um, blow-by-blow narrative of it, but I bet you’ve already read it.

        http://gawker.com/5948770/even…..-it-anyway

  11. So pre-clinical life science has something in common with climate science?

    1. NO! Pre-clinical life science would never settle for p less than 0.10.

      Well, unless it was doing a meta-analysis on the effects of second-hand smoke.

  12. researchers at Amgen tried to confirm academic research findings from published scientific studies in search of new targets for cancer therapeutics.

    Giant evil profiteering kkkorporation tries to steal ideas from noble academicians!

  13. This is what comes from not towing the lion. And neglecting the wood chipper.

  14. A loss, and an award of even a fraction of the $100 million Hogan’s attorneys are seeking, could empty the company’s coffers, forcing [Gawker Media CEO Nick] Denton to either sell the company outright or to hand much of its equity over to deep-pocketed investors

    WTF?
    “Of course I want to buy your blind three-legged asthmatic race horse. NAME YOUR PRICE!”

    1. UNIONIZED asthmatic racehorse.

    2. Are we talking about Gawker or Greece here?

      1. At least Greece has assets! Also, dude, a Greek horae is a trap.

        1. Horae? Greek Goddesses of the seasons are a trap?

  15. I read a decent amount of science history (David Bodanis is a favorite), but I still wonder – what’s the moment when research became so politicized? If you look at the greats like Niels Bohr or Ernest Rutherford or Marie Curie, they were doing research for the sake of figuring shit out in general (like, WTF an atom is made of). And many of the great breakthroughs in electronics/technology came about because of a desire to make bank, as in Edison.

    So what’s going on nowadays? Is it the source of money for fields like biomed and chemsitry? Is it the scientists’ own agendas (like that fucker Neil de Grasse Tyson)? Is it too much government money, or too much corporate money?

    I dunno. It just seems like something as shifted in the last generation or two that has politicized the heck out of scientific research.

    It would make a cool doctoral thesis if one were majoring in history.

    1. Two things happened. First was World War II and the Manhattan project. Before that governments pretty much stayed out of science. With the Manhattan project, governments realized the value of science in producing weapons and suddenly got very interested in it. Once the government started funding so much of it, it was bound to become politicized if for no other reason that the confirmation bias that results from wanting to please your government masters.

      Second, Marxism claimed to be science. Lysyenkoism was real. And the effects of Marxists commitment to having science prove their ideas lingers to this day. A good number of scientists are leftists and view their work as a vehicle for their politics.

      So you combine government funding with a slow influx of Marxists and Marist thought and you end up with politicized science.

      1. It was bad enough when some brought religion into science. Then it got even worse; others brought government and their politics into science. Now what often passes for “science” is political religion.

        1. I think the problem is the opposite. It wasn’t that religion got involved in science. It was that science got involved with religion and scientists started thinking they could make claims on truth that they can’t and that their work had necessary political and ethical implications rather than being the value neutral knowledge that it was.

        2. To be fair, Faraday did all his work because of his religion. And he is still one of the greats, despite that built-in bias. But I get the impression the Sandemanians were particularly open-minded and chill.

          1. It wasn’t just Faraday. Most of those involved in setting the foundations of modern science were highly religious. Newton seems the most important example.

            1. Leibnitz too.

        3. Please. An enormous proportion of early physicists considered themselves to be theologists more than scientists. Religion in science is not a new phenomenon. Without religion, the modern scientific method probably never gets off the ground.

      2. There’s something to that, actually. I was about to suggest that popularizing science and the ego-driven personality cults cropping up has something to do with it, but I think even that is explicable as a facet of creeping cultural Marxism.

    2. It just seems like something as shifted in the last generation or two that has politicized the heck out of scientific research.

      I blame Bush.

    3. Actually something interesting. A while back when the story about poor repeatability in bio research first broke, the reason anyone found out was because all the pharmaceutical companies were funding repeatability studies. When they kept finding study after study that couldn’t be repeated, they started to complain, and only then, did the rest of the scientific community become aware of the problem. So big pharma way better at science than government funded scientist.

      1. “You don’t know what it’s like out there! I’ve worked in the private sector. They expect results!”

    4. The level of scientific rigor has certainly decreased, and scientists are essentially trained cowards (e.g. need more information to make a decision), so there is little challenge to suspect work from those with a technical understanding.

      Hypothesis testing has become the current “approved” scientific method, which also leads to extreme bias, and a priori assumptions. Add to this a reticence to publish null data, lack of leadership by PI’s (just leave it to the grad students!), and you have an extremely murky picture.

      While I was a grad student, I literally watched a post-doc in our lab publish dozens of papers/abstracts over 2-years, and ultimately land a faculty position at Penn State. Interesting that the equipment used, NEVER functioned during the entire period…

      1. Careerism has a lot to do with it. You make a living as a scientist in one of three ways. You get a company to hire you and work at their lab, you get a tenured position at a university, or you freelance from university to university living on grant money.

        Working for a corporation is good since corporations, unlike universities have to make money and thus can’t subsidize bullshit. But they also rarely subsidize the kind of new ground breaking research that you want to have happen. The universities do that. But to get a position at a university, you basically have to publish in volume, which encourages the kind of behavior you describe. Grants are just as bad since you never get a grant to disprove a hypothesis.

        1. You forgot the govt labs: Sandia, LBL, Los Alamos, Argonne, PNNL, NREL, NIST, Oak Ridge, NASA, Brookhaven, etc. They are more politicized in some ways than the Universities. But also some excellent work.

    5. Given that the Soviet Union literally believed in ‘socialist science,’ I think this has been going on awhile.

      1. And Aryan science. The Nazis screwed themselves by going down that path.

        1. That’s not even getting into the various pseudo-sciences regarding phrenology, eugenics and race which were definitely based on the racial biases of the people who believed in them.

    6. Bad science pushing poor results is not just a present phenomenon. Take Milliken’s measurement of the electric charge. He got a value that was substantially off from the presently-accepted value with a tighter uncertainty estimate than was warranted. Future studies aiming to reproduce his results got data which were at odds with his figure and were reinterpreted to get a “more correct” answer (that is, one in agreement with Milliken’s figure). Over time, we saw estimates of the electric charge gradually depart from agreement with Milliken’s initial study and move more toward today’s presently-accepted value. People didn’t want to disagree with the authority even though their data necessitated that. And this is an example of a seminal result from that period. Imagine how bad other, less remembered studies were. I don’t think that there’s anything worse going on in the science of today than there was in the science of yesteryear. It’s just that we’re living through the science of today and get to experience the bad parts of it. The only thing we retain from the science of yesteryear are (what we believe to be) the correct results.

      1. I think fraud was far less institutionalized in the past, only because everything was far less institutionalized. Fudging by individual scientists is bad and has been going on for a long time, but it’s also easy enough to catch, once someone tries to reproduce the results.

        1. It’s also really difficult to tell what is fraud and what isn’t when you’re talking about work at the cutting edge of human knowledge, where no one really knows what the real answer is. Milliken’s result had an enormous impact and established new physics (quantization of electric charge). He wasn’t completely clear on his analytical methods but gave tight error bars. What position is the future researcher in when he gets a similar result with numbers that are substantially different? He doesn’t have Milliken’s apparatus or know his analytical methods. Maybe his instrumental parameters are off or he’s doing a calculation wrong. Maybe he’s not thinking of some crucial thing. He’s epistemically limited. Does that make his re-analysis of his data fraudulent? I don’t know. It’s the problem with working at the edge.

          1. I guess that there’s a good way to settle the question of which era produced better science. Have Ioannidis go back and do the same analyses of the papers published in various fields decades ago. I would guess that reproducibility back then was just as bad as (if not worse than) it is today. The only difference is that we don’t have immediate knowledge of the poor results from the past.

    7. Read Eisenhower’s farewell address. Everyone remembers the MIC part and no one recalls yhe SGC part.

    8. Science has always been politicized. The eugenics movement was “scientific”. Communism was “scientific “. The wandering womb was “scientific”. Ever since science overtook religion as the dominant means of describing how the world works, various assholes have been using it as a justification for their assholery.

      It’s the same mindset as the divine right of kings, or any other use of religion to support someone’s political ascendency.

      Science is powerful, and power will always attract politicians.

    9. Planck’s Law (E = hv) came from trying to figure out how incandescent light bulb filaments worked.

  16. Who needs reproducibility when you’ve got CONSENSUS??!?!?

    1. Most “sound science” today is based on popularity contests and Facebook likes.

      1. John Galt, why don’t you fucking love science?

        1. It’s impossible to remain objective when working with something you love.

  17. Garbage in, garbage out.

  18. Scientists who knowingly present shoddy research like this should be taken out and…

    1. Spanked?

    2. …if you don’t want to face a grand jury, or worse, dare not speak figuratively nor even jest.

    3. Given a stern yet respectful and understanding talking-to, followed by an ice cream cone to assure them there are no hard feelings and everyone gets another chance?

      1. Ice cream cones are ok, but remember Sprinkles are for winners. Even Progressives know that.

        https://pbs.twimg.com/media/B7Mz6nSCcAEuFaw.png

  19. Most Preclinical Life Science Research Is Irreproducible Bunk
    $28 billion in research funding wasted every year.

    I wouldn’t say the money is really wasted. The money is spent to expand knowledge, and most often fails to accomplish this objective in the sense of understanding what is and what works. But, like Pirsig notes in Zen and the Art of Motorcycle Maintenance, there is no such thing as a failed experiment. From so-called failed experiments, we learn what is not and what does not work. Even if the failed experiment obtains a false result, the inability to replicate the experiment’s result in subsequent work advances knowledge about what does not work. And the result must be reproducible, and it must actually be reproduced, before a hypothesis confirmed by an initial experiment can be called a scientific theory. Otherwise, it remains a hypothesis.

    The problem is not wasted money on scientific research that fails to yield reproducible findings.

    The problem is money wasted on fraudulent scientific research that obtains findings that reward researchers with future grants, tenure, and accolades, their political benefactors with additional power, and assorted cronies with rents coerced from taxpayers.

    1. there is no such thing as a failed experiment. From so-called failed experiments, we learn what is not and what does not work.

      This reminds me of the quote attributed to Edison: “I have not failed. I’ve just found 10000 ways that won’t work.”

      1. Did he steal that quote from Nicola Tesla too?

        1. ZING!

        2. “Did he steal that quote from Nicola Tesla too?”

          Bah, any true Tesla acolyte knows that Tesla never failed at anything. Any failures were purely Edison’s fault.

          Or maybe Bush’s.

  20. Science is supposed to build and organize knowledge in the form of testable explanations and predictions. If reported research results cannot be reliably replicated, they are not science.

    Hogwash, science is based on CONSENSUS!!!11!! /tard

  21. I get a 404 when I click on the link.

    1. I was able to reproduce those results.

      1. CONSENSUS!

  22. Speaking of scientists (oh yes, I’m going there!) – what are the odds that Tim Hunt has a few hush-hush sexual harassment settlements in his past?

  23. “And what else floats?”

    *pause”

    “A WOOD CHIPPAH!”

    “ExACTly! Who are YOU, who is so wise in the ways of Science??”

  24. If reported research results cannot be reliably replicated, they are not science.

    Yeah, um, that’s pithy and all, but it’s not actually true.

    1. Ugh, did I just get myself involved in a “what is science?” debate? I hate these.

      1. Dude – THE SCIENCE IS SETTLED!

        You lose!

        PWND

      2. Baby, don’t test me. Don’t test me. No more.

    2. PS Also – I agree with you about this 🙂 Mister Stuckey, Chem – “You can never be 100% sure. You can be nine point nine nine nine nine (then he said “nine” a bunch more times) percent sure….but never 100%. If you did an experiment to boil pure water, but it did not boil at 100C….”

      It’s all aboout the method – not the results.

      Doesn’t mean it’s “not science” (stipulating one is following good experiment design and actually following the Scientific Method?, etc. etc. etc.)

  25. Do you want to improve your SAT scores? Then you should have a look here.

Please to post comments

Comments are closed.