Medical vs. Legal Experimentation


Many clinic trials are exploring the safety and efficacy of potential treatments for COVID-19. Such clinical trials generally employ the "gold standard" of randomizing patients either to a treatment or to a control (or to multiple treatments and controls). Meanwhile, across countries, we see many different public health and legal responses to COVID-19. But these never seem to be randomized.

We are left to casual empiricism to evaluate important public health questions. Are Asian countries doing relatively well in fighting COVID-19 because of masks? Or is it some other factor? Meanwhile, California and Washington, which have succeeded to a great deal in flattening the curve, have taken many different measures in fighting COVID-19. How much has each measure contributed? Once countries and states start relaxing measures, it will be important to learn about the effect of different policy aspects of lockdown, so that we can measure at least the benefits of restrictions in the cost-benefits analysis. Perhaps we will get lucky, and some governments may pick arbitrary geographic or numerical cutoffs in implementing certain policies, allowing for regression continuity designs. Eventually, patterns will become sufficiently clear from uncoordinated policy actions around the world that we will learn more about policy effects. But our understanding would improve much faster if we used randomization.

A simple example: Suppose a state is considering whether to ban work on construction projects (or, equivalently, is considering lifting such a ban). Just as a clinical trial is appropriate when we have genuine uncertainty about the effects of a treatment, so too might a randomized governmental trial be appropriate here. Rather than apply a change to all worksites at once, apply it initially to half, chosen at random. Keep in contact with construction workers to find out how many test positive (and if testing becomes cheap, test them all at the beginning and end of the experiment). This may help us learn, for example, whether outdoor contact at work is considerably lower risk than indoor contact.

Many who reflexively support the idea of medical experiments oppose legal experiments. One reason might be informed consent, which may not be possible in some legal experiments. But this is really just a question of policy baselines. Ian Ayres, Yair Listokin, and I explained in a paper about the case for randomized legal experiments a decade ago:

[M]edical experiments can generally be viewed as equivalents to policy experiments.
Subjects in medical experiments who give informed consent presumably would prefer a guarantee of receiving the treatment rather than a chance of receiving a placebo. The status quo is a legal regime that constrains liberty by forbidding distribution of the treatment. Let us assume that the legal prohibition on what Eugene Volokh has called "medical self-defense" is permissible. When the government authorizes a medical experiment, it is effectively authorizing a new legal regime that permits patients to have access to a treatment. The government, however, does not authorize this new legal regime in a universally applicable way; instead, it insists on randomization. Only some patients will legally have access to the treatment. It is thus sometimes permissible for new legal policies, including potentially pernicious ones, to be introduced randomly.

Just as legal coercion will stop some people from receiving their treatment of choice in a medical experiment, so too will legal coercion stop some people from going to work in a legal experiment on construction projects during the pandemic. If we are very confident that we know the correct answer to whether such projects may proceed (and we may well be confident for now that a wide range of activities should be categorically banned), then we should not have an experiment. But at some point as we move into an emergency situation and at some point as we move out of an emergency situation, we will be quite uncertain about the correct policy response. In those cases, legal experimentation may be justified. If we are engaged in such experimentation, we ought to try to do it in a rigorous way so that we can make better decisions.

We are likely, alas, to have too few legal experiments in this emergency. Any one jurisdiction obtains only a portion of the social benefits of information arising from the experiment. If a single jurisdiction employs such experiments, the entire world can benefit from the resulting information, but that also means that a jurisdiction might not undertake an experiment in which social benefits exceed social costs, if the private costs exceed the private benefit. Meanwhile, the private costs include discomfort with legal experimentation, with doing something weird that is just not done. One can easily imagine a parallel universe in which people accepted legal experimentation but recoiled at the idea of medical experimentation or another parallel universe in which both kinds of experimentation were accepted, with appropriate concern for ethics regardless of the experiment type. My hope is that we move eventually toward the latter state of affairs.

NEXT: Today in Supreme Court History: April 2, 1980

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. Just a quick reaction. A lot is being made of CA’s much lower infection rate than that of NY, and esp NYC. But part of that may be the result of the state’s apparent inability to ramp up testing. NY is far ahead of CA in testing. I am not saying that CA isn’t faring better, but rather that we probably don’t know definitively that that is the case, due to the testing gap.

    1. Semi-related, here is a quick comparison of countries, their infection / death rates, and how many mitigation steps they have taken. The only pattern he finds suggests that face masks are the only mitigation which makes any difference.

      Rigorous, peer-reviewed, it's not. Interesting it is.

    2. There is also a population density issue -- how many *residential* buildings in California are more than 5 stories tall? 20 stories tall? How many are physically touching the adjacent buildings?

      That's the norm for NYC and (to a lesser extent) Boston, and much like in Wuhan, population density helps transmit an airborne virus.

    3. Now this skepticism I can get behind. I don't think the numbers and models are robust enough to form any kind of conclusion with that kind of specific resolution.

      We'll have a wealth of good data...after the fact.

    4. I am not saying that CA isn’t faring better, but rather that we probably don’t know definitively that that is the case, due to the testing gap.

      I dunno -- I think the death gap is remarkably telling. NY reported its first death a few days after CA, and now its total number of reported deaths is over 10x CA's. I find it implausible that CA isn't using some of its several thousand tests per day to check the status of a few hundred candidate deaths per day, as all other states appear to be.

  2. The US have about 75% population of the EU. The EU have, as of today's report, 33,100 deaths from COVID-19. The US have 5,140 deaths. If we had the same ratio of deaths per population we should have 24,500 dead at this point. The US death rate is about one-fifth.
    California has just shy of 40 million people. The ratio for them compared to the EU is .09, so they should have about 3,000 dead. They don't.
    Probably the less dense housing for most of the state plus the lack of public transportation accounts for most of the difference with New York and the EU.
    Construction workers are not among the elderly or the health compromised. They will get sick, but not die, just like during the flu season last year and years before.

    1. You're assuming an identical timeline for when day zero of the epidemic in the US versus the EU is. That's definitely not true.

      Also different testing rates and standards for death reporting.

      No good conclusions about comparative burden between cohorts, nations, or states can be drawn in any definitive manner until this is all over.

      I also disagree with your construction worker point. Plenty of young healthy people will die as well. Lower rate plus higher absolute number equals far from no deaths.

  3. The ethical issue I have with medical tests is that half the population is denied treatment. I don't think that should be done in a situation like this....

  4. Good lord.

    First, this is the same utilitarian mindset that leads us to voluntary coronavirus infections. It's just not a moral paradigm most of America follows.

    Second, the cost-benefit isn't there. For risking some deaths, you get some work done, and some data that's going to be hinky as hell - this is no way to do field work.
    As soon as those deaths manifest (which will happen, just due to numbers), everything is going to shut down again, only harder, and dumber.

  5. Look, the critical difference here, is that the legal system has access to coercion. It's not like you perform medical experiments on people seized off the street, you know.

    Legal experiments? Yeah, you might be doing just exactly that.

    1. Correct.

      If I agree to participate in a clinical trial I am informed not only that I might get a placebo, but also that there are certain risks if I get the experimental medicine.

      How does the construction worker get to decide whether to accept the risk of working during the pandemic? And what of other people the worker may come into contact with? And extend this further. Should we give some sites, and not others, protective gear to see how much good it does?

      Not such a hot idea.

      I don't think so.

  6. A construction company's functioning (vel non) has nothing to do with peer-reviewed scientific analysis of microorganisms. That would be like saying that someone who gets a speeding ticket could challenge the laws by proving that the stopping power of modern cars exceeds the models used by the engineers that decide on speed limits.

    Imagine an old wooden church in New England where the village meets to decide on collective questions. The scientists show up, and the village decides how far apart everyone has to be, what sort of sanitation has to be in place, etc. (Side note -- the "cordon sanitaire" was actually invented by Marcel Proust's father, which might explain the Proustian sensibility starting to seep into Instagram and Twitter from the navel-gazing quarantined.) After they're finished, the burly folks who do construction get up and argue with the leadership about how many rules they have to follow, and what exceptions can be made. Due to the nature of their trade, there's a lot of shouting and physical intimidation.

    For obvious reasons, you don't let the multibillionaire heavies upon whom we rely to build things start arguing with the mousy and fearful scientists, pale and wan from working in the lab all day -- either in the New England town hall or in the payola-fed corridors of power. The lawmaker gets as much information as possible, and then issues the laws. The Weberian disenchantment that comes from realizing that one process is somehow connected to the other is merely an adventitious side-effect.

    Mr. D.

Please to post comments