A legitimate reason for disputes over mismatch

Justice Antonin Scalia's inartful effort to raise the issue of mismatch in the Fisher oral argument last Wednesday generated two waves of responses. In the first wave, dozens of web commentators and several public figures (including Sen. Harry Reid) denounced the comments as condescending and racist. In the second wave, many leading news organizations (commendably) published stories pointing out that Scalia was actually referring to actual research and ideas developed in briefs filed in the case (including one by me). These stories generally gave some explanation of the mismatch hypothesis, but then, very often, quoted "experts" who dismissed the idea as either wholly discredited or at least undercut by a vast array of evidence.

The Chronicle of Higher Education offered this quote from Matt Chingos of the Urban Institute: "The serious question [about mismatch], Mr. Chingos said, is whether colleges' affirmative-action policies admit students who are not likely to succeed there. 'There is no high-quality empirical evidence in support of that hypothesis,' he said."

The New York Times quoted Oren Sellstrom, of the Lawyers Committee for Civil Rights, as saying "there is a vast body of social science evidence that shows exactly the opposite of what the mismatch theory purports to show, that actually minority students who benefit from affirmative action get higher grades at the institutions they attend, leave school at lower rates than others, and are generally more satisfied in higher education…" The Times did not cite any specific evidence in support of mismatch.

(The Times also published a letter coauthored by William Bowen—a former president of Princeton and a leading scholar on affirmative action—that "there is an abundance of empirical evidence refuting the mismatch hypothesis and no credible evidence supporting it.")

The Guardian reported, "[E]xperts said Scalia's comments are couched in the so-called 'mismatch theory', a controversial concept that has been 'thoroughly discredited,' said Richard Lempert, law professor at the University of Michigan. (Lempert was also recently prominently quoted in the Times dismissing mismatch.)

Statements of this kind about mismatch research are, of course, widely at variance with the facts. The dominant finding by a long series of impressive studies published over the past several years is that mismatch is real and occurs in many different contexts. Efforts by investigative teams that reach across different perspectives, like the Journal of Economic Literature study that I discussed last Thursday, conclude that, while much remains unknown, there is quite substantial evidence for some types of mismatch and the phenomenon should be viewed seriously.

Why, then, do reporters both fail to discuss the evidence on mismatch and let broad claims from the anti-mismatch camp stand unrefuted? There's an obvious answer here: The "mainstream media" see affirmative action (including racial preferences) as closely intertwined with racial inclusivity and racial healing, and thus something difficult, and even risky, to seriously question in one's reporting. There's also the widespread journalistic habit of quoting strong, bold statements and giving short shrift to nuanced discussion.

But I think there are two other phenomena that complement media skittishness: the complexity of mismatch research, and the conscious strategy of academics who oppose mismatch on ideological grounds. I'll address the second point—the strategy of pro-affirmative action academics—in my next post. Below are some thoughts about the complexity issue.

A useful way to think about mismatch is in terms of "first-order" and "second-order" effects. First-order effects are specific phenomena that could happen directly to a student place in an environment with mostly academically stronger students; second-order are more conditional, indirect consequences.

Nearly all mismatch literature implicitly or explicitly concerns one of three "first-order" effects. (1) Learning mismatch occurs if a student actually learns less in a classroom because, for example, the professor is pitching her teaching to students with greater academic preparation. (2) Competition mismatch occurs if a student is learning well but nonetheless gets relatively low grades simply because of the strength of the competition, and consequently becomes discouraged or concerned about her academic future. (3) Social mismatch can occur because of the tendency of students to form friendships with other students who have similar academic backgrounds or performance; if a school uses large preferences that correlate strongly with race, then social mismatch can produce racial self-segregation.

The best example of a "second-order" mismatch effect is college graduation. A student might experience any or all of the "first-order" effects described above, but still graduate from college if, for example, the college has a very high graduation rate (i.e., it's hard to flunk out), or strong academic support services (a supplemental class for students having academic difficulty will, almost by definition, avoid mismatch), or if students switch from hard majors to soft ones.

This context helps to at least partly explain why the two sides of the mismatch debate can talk past one another. The evidence on first-order effects is pretty overwhelming. Learning mismatch has been demonstrated in randomized experiments and is at the heart of the finding that large law school preferences seriously damage one's chances of passing a bar exam on one's first attempt (bar exams are one of the rare examples in higher education where even an imperfect measure of learning exists). The much lower grades that result from large preferences probably mostly reflect competition mismatch; this association has been shown repeatedly and is conceded by most scholars.

Very high rates of minority attrition in the sciences probably reflect both "competition" and "learning" mismatch, although the half-dozen peer-reviewed studies that demonstrate science mismatch have not tried to parse out the two distinct effects. Social mismatch has also been convincingly demonstrated in both a randomized experiment and in longitudinal research.

I can't claim to have encyclopedic knowledge of all these literatures, but I think it is safe to say that 90 percent of the peer-reviewed studies testing these first-order effects that has found evidence—usually very strong evidence—of mismatch.

The same can't be said of research on second-order effects. The evidence on graduation, in particular, is very mixed. Even in the case of law school mismatch, evidence on graduation effects is weak in the 1990s data and would almost certainly be even weaker today, as graduation rates at most elite schools has edged closer and closer to 100 percent. There are undoubtedly many students -especially low-SES students—attending community colleges who would improve their chances of graduating if they could go to a four-year-college instead, even if with an admissions preference.

Thus, often when an academic argues that mismatch is "not a problem," he or she is thinking of the research on graduation or some other second-order effect, while scholars who believe mismatch needs to be addressed will usually cite some first-order effect in evidence. In this sense, the disagreement is bridgeable (as reflected in the Journal of Economic Literature piece).

It remains the case, however, that scholars who believe mismatch is real tend to far more measured and nuanced in their claims than the often fervent, and absolutist, anti-mismatch scholars. Even the most eminent members of the anti-mismatch camp often go wildly beyond the evidence—something explored in detail in my next post.