A word of caution before offering the bar exam online

Remember the Iowa Caucus app and HealthCare.gov?

California and Massachusetts have now committed to offering the bar exam online. Let me offer a few words of caution: Technology is hard, especially for lawyers. Transitioning a multi-day exam to an online platform is a significant undertaking, and it cannot be done in a few months.

Does anyone remember the Iowa Caucus App? (Did we ever actually receive the final results, or is it a moot point now?). Does anyone remember HealthCare.gov? The ACA almost failed on arrival due to the botched website.

Recent graduates are stuck in a very, very stressful situation. Rushing through an online testing system, without adequate planning, will only increase the stress. Even worse, it is conceivable that errors in the testing process could alter results. Imagine if a student sits for the bar, and some of the questions are not uploaded; or data is corrupted; or a computer crashes and there is no way to retake the exam till six months later. One of the reasons my faculty opted for a credit/no-credit system is that we lacked confidence in the integrity of online examinations.

My colleague Derek Muller highlights additional problems.

Let's put aside the security issue for a moment and simply focus on reliability of software. Six years ago, ExamSoft had an issue during the July 2014 bar exam where thousands test-takers were unable to upload their answers for hours. Some (I think, wrongly) even blamed this debacle on a decline in bar passage rates that cycle. Exam software is not sufficiently reliable even in the best of times. Add to that the remote (and secure) delivery of materials that have previously been printed, and the collection of those materials after the exam.

In-room security is a huge problem, too. Bar exams are notorious for picayune requirements, like a small clear plastic bag containing limited personal effects, sign-in sheets to use the restroom during the exam, and so on. Remote proctoring software purports to watch the eye movement of exam test-takers during the exam, to scan the room before and after to make sure no one else is present, and other rather theatrical promises. Let's face it—those probably work in much lower stakes tests.

Given enough time, these sorts of contingencies can be planned around. But is impossible to create a quality product under these circumstances.