Do Law Journals and Law Firms Check for Retractions of Cited Scientific Articles?

Law review editors, lawyers, and paralegals are trained to check whether cited legal authorities have been overruled or otherwise abrogated. This is the right thing to do, but it's also the smart thing to do, since it helps avoid embarrassing and credibility-destroying errors. Fortunately, law has long had tools for doing this, from the old process of "Shepardizing" to electronic tools available in Westlaw, Lexis, and elsewhere.

But I'm not sure whether there are similar protocols for checking whether scientific articles or other academic articles have been retracted. My sense is that there are tools for doing this in various fields, but they aren't well-integrated with each other. Does your law journal or law firm have guidance for doing this? It would be good to share this information, especially if (as I suspect) many journals and firms don't actually have systems in place to do this.

Comments

[Jerry B.]

The University of Massachusetts library has some guidelines and suggestions on finding retracted articles, but it appears that you have to look in many places if you don’t know who originally published them.

https://guides.library.umass.edu/c.php?g=672689&p=4736833

[NellieBly]

A little bit of legwork but a good resource to have on hand. Thanks!

[Archibald Tuttle]

The bigger problem is excluding papers published by dubious journals. Regrettably, for-profit scam journals outnumber the legitimate ones. They charge the authors fees. Their claims about peer review are false.

Another complication is that papers are often followed up by letters that correct errors but without withdrawing the whole paper. Without searching those too, you risk the same embarrassment.

For a list of acceptable journals, but limited to physics, not other sciences. Search "physics forums acceptable journals"

My best advice is to retain a scientific consultant to double check any scientific facts you plan to cite as evidence. And of course, offer the consultant the same hourly rate your attorneys charge.

[Tom for equal rights]

Archibald Tuttle 51 mins ago
Flag Comment Mute User
"The bigger problem is excluding papers published by dubious journals. Regrettably, for-profit scam journals outnumber the legitimate ones. They charge the authors fees. Their claims about peer review are false."

The peer review problems run a lot deeper through out the science community. Lots of borderline academic fraud that gets passed/blessed in the peer review process. Recent examples are most all the pro covid masking studies, the gas stove/asthma study with the PAF methodology, Jacobsons 100% renewable study with the claim that the reliability was tested in his model every 30 seconds for the model period 2050-2052.

[SimonP]

"Do law review editors, who typically lack any kind of training or graduate education apart from a year-long hazing ritual, but who are for some reason entrusted to be the gatekeepers of legal academia, endeavor to double-check scientific sources cited by lazy law professors who can't be arsed to evaluate them, themselves?"

Another howler, Eugene. By the way, did you ever get a fool to take you up on your offer for three weeks of unpaid bluebooking experience for your pet journal?

[NellieBly]

This is an important question, which has me wondering what the coded language(s) of scientific journals may be. In law there is incrementalism and there are many ways of eroding controlling precedent without casting it our entirely.

But is this done in science and how is it done? Ham fisted, as in "we were unable to replicate Newton's apple experiment"? or more subtly, as in "after several assays, we were unable to cause lead to become gold," but we did not control for x, y, or z." (Since this would be an impossibility in any circumstance, the 'control' language is a face saving gesture for a previous researcher.

I know of no government repository for discredited or retracted work. (I would not trust one at this time if one were found to exist.)

Recently I found a bit of a d.i.y. effort which is nonetheless interesting: https://retractionwatch.com/

[Artifex]

But is this done in science and how is it done?

To a large extent it is not done in the real sciences. The fundamental difference between law and science is that science that is first and foremost a model of our shared reality. It is repeatable and testable. You can push narratives and redefine words all you like, but in the end, reality does not bend. Law has no such sanity check and its correctness is simply the current most popular shared delusion.

You will find that specialists in any scientific field have a pretty good idea of which papers are good and which are crap based on how well they fit the parameters of previous experience. This makes a database of little use. If you have read the prior work and understood it, you already have a good idea of where the bullshit lies. You tend to have more major retractions in areas that are pure pseudoscience or grant grift where the signal to noise is so low that understanding counts for little.

[Tom for equal rights]

Artifex - quality peer review is likely done in the hard sciences, though, as I noted above, the quality of peer review drops dramatically when the science becomes political. (covid & sub sets of climate science - such as the paleo reconstructions, renewables)

[Artifex]

Peer review is certainly done and this is a good thing. That being said, databases of retracted papers are not necessary because if you are current on the subject, they are redundant. In general, you have a good idea of what work is being done and who is credible.

[Tom for equal rights]

Artifex - I concur that peer review is a good thing, and is effective in most areas of science. That being said, I see a lot of studies that get through peer review that are quite dubious ( the most polite word I can use). Jacobson's 100% renewable studies is good example. Jacobson 's claims - A) overstating land use for fossil fuel electric generation by a factor of 2-3x, understanding land use for renewables electric generation by a factor of 5-6x. B) using 20% of nameplate capacity for wind and solar during winter months, when the average name plate is 7-9% during the winter months north of 46 pararell

[Tom for equal rights]

typo - should be understating land use for renewables by a factor 5x-6x. (not understanding)

[Toranth]

https://retractionwatch.com/

I was going to mention this one. It's probably the biggest single resource for listing retracted papers, with more than 30,000 retracted papers listed (along with the reason, where known).

[ducksalad]

Have to distinguish retracted or corrected from discredited or merely superseded.

If EV is worried about citing a formally retracted or corrected paper, that’s relatively easy in the case of any halfway legitimate journal that is still publishing. Open the latest posted electronic version and if will be flagged with a publishers notice somewhere obvious near the top. Surely if one is citing it, one has opened it...

On the other hand, finding whether a paper is discredited or superseded is much harder, and it’s not an objective yes-no question. Early theories that turn out to be incomplete or even in error are a legitimate part of the history of the field and might retain quite a bit of utility. No one’s going to put a red flag on a Maxwell or Rayleigh paper just because of quantum theory.

[John R. Ellis]

"a bit of a d.i.y. effort which is nonetheless interesting: https://retractionwatch.com/"

It's much more than a DIY effort:
https://en.wikipedia.org/wiki/Retraction_Watch

[Rev. Arthur L. Kirkland]

Is the Volokh Conspiracy ever going to get around to refraining from publishing humiliating, repeated, credibility-destroying mistakes with respect to Today In Supreme Court History?

Does the Volokh Conspiracy have any quality controls in place? Does it care about research and publication quality? Can the Volokh Conspiracy provide any guidance concerning the frequency with which it embarrasses itself with Today In Supreme Court History?

[James_Herms]

Can the Volokh Conspiracy provide any guidance concerning the frequency [of mistakes in] Today In Supreme Court History?

You can help get this underway by citing a few!

[James_Herms]

[lhfry]

I recently had occasion to do some research on “Social and Emotional Learning,” the latest public school fad. I used the website of Second Step which claims that its program is supported by research. The irreproducibility problem is writ large there. I selected a few studies at random and the conclusions generally state that the studies need to be replicated, show that the data suffers from confirmation bias and that more research is necessary. Meaning these studies are largely invalid and yet Second Step and other programs are being adopted everywhere in the US.

This kind of thing can’t be easily checked and most school board members either don’t or don’t know how to identify valid research.

[Tom for equal rights]

Lhfry – as noted above, the peer review process in agenda/politically driven studies are exceptionally bad.

the second point is there is a serious defeniency in the ability to identify valid research or even identify blatant math errors or obvious red flags.

[weibullguy]

“the peer review process in agenda/politically driven studies are exceptionally bad.”

Actually it’s really good. Never has a peer reviewer allowed a dissenting opinion to slip through.