Some Korean Stem Cells Apparently Real

Early leaked reports from the academic committee that is investigating the claims made by Korean stem cell researcher Hwang Woo Suk suggest that Hwang did in fact clone cells from five different patients. There is some question about whether the stem cells derived from those cloned embryos have developed sufficiently to qualify as self-perpetuating cell lines. However, it appears that these cloned embryonic cells were produced after he published work in Science claiming to have made 11 patient specific human embryonic stem cell lines.

The mystery deepens: if Hwang can legitimately produce patient specific stem cell lines, why not wait until the work is done, instead of lying about it? It can't be because he feared that he would be scooped by other research teams. All of the competing stem cell labs are way behind the Korean lab.

The investigating committee has ordered further tests of the cells from three other labs. The final report should be issued next week.

Comments

[Warren]

Yeah, tis a puzzlement. Whatever the truth about Hwang, science will continue to uncover the truth about stem cells. Bailey's post from last week is still the key point. The setback to science will come overwhelmingly from the political fall out.

[Ed]

What political fallout? How will it "set back" science?

[thoreau]

The setback to science will also come from the fact that an apparently talented person with at least some real results has discredited all of his future work. If the man had stayed honest he could have made some real contributions to science. But now he has rendered himself unreliable, and he will no longer be able to contribute significantly to the advancement of science.

[Warren]

Ed,
I refer you to Ron's post.

[Ed]

What, this?
"opponents of stem cell research will point to Hwang's fraud arguing that stem cell researchers are so unethical that they cannot be trusted to develop this technology. Third, because this deception undermines the public's trust in the scientific enterprise which has consequences for future support and regulation."

That's rather hyperbolic. You know, one bad apple and all that. Scientific research is not dead. The wringing of hands is premature.

[Answers Dep't]

The answer is: because he thought his particular balance of real reseach and sham research would make him the most money. It is impossible for us to know whether he chose correctly under this criterion.

[joshua corning]

What is even more interesting is Science magazine's reaction to this.

here from a pbs transcript:

http://www.pbs.org/newshour/bb/science/july-dec05/scandal_12-27.html

"the journal has to trust its reviewers; it has to trust the source. It can?t go in and demand the data books." -DONALD KENNEDY Editor chief Science

WTF so science (small s) is now about trust? What about verifiable facts? What about peer review?

So Science will publish a researchers results thus certifing the work as being "published" and "peer reviewed" yet makes no effort to ensure that work is of any scientific merit? or to ensure that the work can actually be peer reviewed?

How can a work be peer reviewed if the data is not even available?

there is more discussion on this subject at climateaudit:

http://www.climateaudit.org/?p=477

[thoreau]

How can a work be peer reviewed if the data is not even available?

Checking every single facet of the experiment before publication is too tedious. Replication is usually done after publication. What the reviewers look for is a report that adds up.

Do the authors describe taking adequate measures to control for other variables and check their results?

Do their interpretations fit with the data reported?

Do they discuss and quantify sources of error?

If they report something that is unexpected, do they also report taking steps to rule out more mundane explanations?

Do they control for known sources of error?

If the report adds up, and doesn't raise any red flags, it's approved for publication. That doesn't mean that the result is accepted (that will only happen when other authors replicate the results), but it means that it's deemed worthy of further scrutiny by the broader scientific community.

Most frauds make mistakes. They report data that's too good to be true. They ignore subtle effects that would creep up in real experiments. The committee that investigated the Bell Labs fraud found lots of that sort of thing.

[thoreau]

I should also note that some journals now impose strict requirements on how images are enhanced and manipulated. Authors must report exactly how they enhanced their images, and enhancements that seem unnecessary (to experts) may be rejected. And if they report that an image is unenhanced, but a reviewer (who's familiar with the technique) thinks that it's "too perfect" he might demand some explanations before approving it.

Peer review is about honest reporting and the plausibility of the report. Replication comes after peer review.

[joshua corning]

Thoreau,

yup you completely missed the point.
Science does not in any way require researchers to make thier data available to the "broader scientific community" for "further scrutiny".
So lets see I publish then I withhold my data and surprise suprise no one disputes my results. What does science say about it? We have to "to trust the source". Give me a f$%*g brake

[thoreau]

So lets see I publish then I withhold my data and surprise suprise no one disputes my results. What does science say about it?

Then your results were ignored. Which is very different from being accepted.

[thoreau]

BTW, journal reviewers could always insist that more data be included in the paper before it's accepted. If somebody only included enhanced or filtered data (enhancement and filtering can be legitimate if properly reported), a suspicious reviewer could demand that the original data be reported as well. (e.g. the author has to produce a graph with both the original and the filtered data on it.)

But I've never heard of reviewers actually showing up and going through notebooks.

So reviewers can demand more data, even raw data. But total audits and inspections of labs are unusual.

[joshua corning]

"So reviewers can demand more data, even raw data. But total audits and inspections of labs are unusual."

you just said reviewers do not do replications and I said broader sceintific comunity. which was a quote from you before. Reviews as you put it only check the plausability of the results not the actual data used to generate the results.

You are playing word games and it is getting anoying. And you have not adressed my point...if the broader scientific community is not given the data used to generate the results then they will be unable to replicate those results.
who cares if the reviewer "can" get that data when they aren't even trying to replicate it and it isn't even thier responsiblity to replicate it.

"I should also note that some journals now impose strict requirements"

I think there is a differance between "impose" and having it on the books somewhere. The failers of Science point out that the latter was the case.

[joshua corning]

Note:

there is also another point in the DONALD KENNEDY quote and that is the idea of trust in science. Science is not and should not have anything to do with trust. It is a system and a method with implicit checks and balances to insure quility not a cliq reliant on trust and presige.

And I think Science has harmed itself and science (small s) by reliing on star power over the method.

What was that I said about science as faith and science as method with the ID descussion...I think my point on that is germain here as well.

[thoreau]

You are playing word games and it is getting anoying. And you have not adressed my point...if the broader scientific community is not given the data used to generate the results then they will be unable to replicate those results.
who cares if the reviewer "can" get that data when they aren't even trying to replicate it and it isn't even thier responsiblity to replicate it.

First, "data" goes through many different cycles. A lab notebook may contain some numbers inscribed by hand during the execution of the experiment, or a computer may contain a data file generated during the initial run of the experiment.

Then that data is either loaded or entered manually into a data processing program. It may be smoothed to remove noise (a step that a good author will report), it may be plotted, etc. If the data is plotted without enhancement, and the plot is put into the journal article, then the scientific community has been given the data, in one sense. They can try to replicate it.

However, if they want to detect fraud they'll need original lab notebooks and data files, to try to piece together whether the data was actually obtained as reported. That's a forensic effort: Who actually did this? When did he do it? Was he really using the tools that he claimed to be using? Is this really his handwriting? Why is the ink on this page so fresh, while everything else in the notebook shows signs of age? Does the allegedly original data file contain any of the annotations that the software for that instrument normally adds to data files? Do the data files contain time stamps consistent with the order in which experiments were done?

And paper reviewers are simply not equipped to do that as a matter of routine.

Also, other scientists don't usually need original data files to replicate results. Rather, they need a description of the methods followed. They then follow the same method and see what results they get. They then compare their results with the results reported in the papers.

You could get picky and say that they should get all of the original graphs in electronic form so the graphs can be compared in a data processing program. But that's not the way it works in practice. Graphs are usually noisy, so what you look for are key features, and you report those features in the text of the paper. If somebody is trying to replicate the work he subjects his data to the same analysis and looks for the same key features, and compares those key features with what the other author reported.

Now, for some fields the scientific community does need the original data. If somebody is analyzing data from a space telescope, or the human genome, or a particle accelerator, or some other project that would be really, really difficult to replicate, the raw data is usually made available. The interpretation of that data, however, can be tricky. Somebody may claim that he poured through the very large data set and analyzed it statistically and found a certain trend. Those who try to replicate his work won't request his notebooks to play detective and see if he did what he claimed to do. They'll just take the same data and perform the same tests that he claimed to perform, and see if they get the same result.

[thoreau]

Out of curiosity, joshua, are you a scientist?

You don't have to be one of us to criticize us, of course, but experience produces more informed criticism.

[joshua corning]

Out of curiosity, joshua, are you a scientist?

Nope. I majored in molecular and cellular biology at the University of Washington but I do not use it directly at my current or any job I have had.

You don't have to be one of us to criticize us, of course, but experience produces more informed criticism.

You will get no arguement from me on the above statement.

[thoreau]

Thanks for the info.

[joshua corning]

one thing thoreau...being a scientist i think you should expect more of this. The internet and the blogverse is only going to encurage more semi informed people like me to question real scientists.

I doubt in thoery that you would find this disheartning...although in practice i am sure you will and do find it anoying

[thoreau]

joshua-

I'm fine with curious questions. I'm less fine with hostile questions. I admit that we don't always communicate well, so you'll get statements from journal editors who claim that data isn't checked. And then people who aren't practicing scientists will be like "WTF?"

They don't mean that there's no inspection whatsoever. What they mean is that the reviewers don't go to the lab and do a forensic examination of data sources to see if everything was obtained honestly, nor do they try to replicate every experiment themselves before accepting the paper.

That still leaves lots of room for inspection of data. They can examine the data as presented, and see if it's consistent with the statements in the text. They can compare it with their experience and standard practices to see if it seems real. They can request to see what the data looked like before it was enhanced, or corrected for instrument calibrations, or whatever. But they won't actually roll up their sleeves and start checking notebooks like a detective would.

Anyway, please don't assume the worst of us. But keep asking questions.