Medical vs. Legal Experimentation

Many clinic trials are exploring the safety and efficacy of potential treatments for COVID-19. Such clinical trials generally employ the "gold standard" of randomizing patients either to a treatment or to a control (or to multiple treatments and controls). Meanwhile, across countries, we see many different public health and legal responses to COVID-19. But these never seem to be randomized.

We are left to casual empiricism to evaluate important public health questions. Are Asian countries doing relatively well in fighting COVID-19 because of masks? Or is it some other factor? Meanwhile, California and Washington, which have succeeded to a great deal in flattening the curve, have taken many different measures in fighting COVID-19. How much has each measure contributed? Once countries and states start relaxing measures, it will be important to learn about the effect of different policy aspects of lockdown, so that we can measure at least the benefits of restrictions in the cost-benefits analysis. Perhaps we will get lucky, and some governments may pick arbitrary geographic or numerical cutoffs in implementing certain policies, allowing for regression continuity designs. Eventually, patterns will become sufficiently clear from uncoordinated policy actions around the world that we will learn more about policy effects. But our understanding would improve much faster if we used randomization.

A simple example: Suppose a state is considering whether to ban work on construction projects (or, equivalently, is considering lifting such a ban). Just as a clinical trial is appropriate when we have genuine uncertainty about the effects of a treatment, so too might a randomized governmental trial be appropriate here. Rather than apply a change to all worksites at once, apply it initially to half, chosen at random. Keep in contact with construction workers to find out how many test positive (and if testing becomes cheap, test them all at the beginning and end of the experiment). This may help us learn, for example, whether outdoor contact at work is considerably lower risk than indoor contact.

Many who reflexively support the idea of medical experiments oppose legal experiments. One reason might be informed consent, which may not be possible in some legal experiments. But this is really just a question of policy baselines. Ian Ayres, Yair Listokin, and I explained in a paper about the case for randomized legal experiments a decade ago:

[M]edical experiments can generally be viewed as equivalents to policy experiments.
Subjects in medical experiments who give informed consent presumably would prefer a guarantee of receiving the treatment rather than a chance of receiving a placebo. The status quo is a legal regime that constrains liberty by forbidding distribution of the treatment. Let us assume that the legal prohibition on what Eugene Volokh has called "medical self-defense" is permissible. When the government authorizes a medical experiment, it is effectively authorizing a new legal regime that permits patients to have access to a treatment. The government, however, does not authorize this new legal regime in a universally applicable way; instead, it insists on randomization. Only some patients will legally have access to the treatment. It is thus sometimes permissible for new legal policies, including potentially pernicious ones, to be introduced randomly.

Just as legal coercion will stop some people from receiving their treatment of choice in a medical experiment, so too will legal coercion stop some people from going to work in a legal experiment on construction projects during the pandemic. If we are very confident that we know the correct answer to whether such projects may proceed (and we may well be confident for now that a wide range of activities should be categorically banned), then we should not have an experiment. But at some point as we move into an emergency situation and at some point as we move out of an emergency situation, we will be quite uncertain about the correct policy response. In those cases, legal experimentation may be justified. If we are engaged in such experimentation, we ought to try to do it in a rigorous way so that we can make better decisions.

We are likely, alas, to have too few legal experiments in this emergency. Any one jurisdiction obtains only a portion of the social benefits of information arising from the experiment. If a single jurisdiction employs such experiments, the entire world can benefit from the resulting information, but that also means that a jurisdiction might not undertake an experiment in which social benefits exceed social costs, if the private costs exceed the private benefit. Meanwhile, the private costs include discomfort with legal experimentation, with doing something weird that is just not done. One can easily imagine a parallel universe in which people accepted legal experimentation but recoiled at the idea of medical experimentation or another parallel universe in which both kinds of experimentation were accepted, with appropriate concern for ethics regardless of the experiment type. My hope is that we move eventually toward the latter state of affairs.