Do the Divergent Results of COVID-19 Antibody Studies Reflect Real Differences?

The infection fatality rate probably varies from one place to another.


How deadly is COVID-19? It has become increasingly clear that the answer varies from place to place. Local conditions affect not only the percentage of the population that is infected but also the percentage of those people who die as a result.

That point has been largely overlooked in the often heated debate about U.S. antibody studies, which has focused on methodological issues such as sampling bias and the accuracy of the tests used to determine who has been infected by the COVID-19 virus. While those issues are important, it's misleading to assume that we are trying to identify the one true infection fatality rate (IFR) for the entire country, let alone the world.

Two Australian public health researchers, Gideon Meyerowitz-Katz and Lea Merone, recently reviewed 13 studies that used various methods, including virus tests, antibody tests, "excess deaths," and epidemiological modeling, to estimate the IFR for COVID-19. The estimates covered a wide range, from 0.05 percent in Iceland to 1.3 percent in Northern Italy and among the passengers and crew of the Diamond Princess cruise ship.

The weighted average was 0.75 percent. But "due to very high heterogeneity in the meta-analysis," Meyerowitz-Katz and Merone warn, "it is difficult to know if this represents the 'true' point estimate. It is likely that different places will experience different IFRs. More research looking at age-stratified IFR is urgently needed to inform policy-making on this front."

It is clear that the COVID-19 death rate varies with age. A study of excess deaths in several Italian towns, based on a comparison of mortality during the pandemic to mortality during the same period in earlier years, found that the IFR ranged from 0.02 percent among people in their 40s to more than 15 percent among people older than 90. Another Italian study found an overall IFR of 1.3 percent, rising to 4.3 percent among people older than 60.

Age demographics help explain why death rates in Italy (median age: 47.3) and among people infected on the cruise ship (mean age: 58) were relatively high. "One reason for the very high heterogeneity is likely that different countries will experience different death rates due to the disease," Meyerowitz-Katz and Merone note. "It is very likely, given the evidence around age-related fatality, that a country with a significantly younger population would see fewer deaths on average than one with a far older population, given similar levels of healthcare provision between the two. For example, Israel, with a median age of 30 years, would expect a lower IFR than Italy, with a much higher median age."

Alan Reynolds, a senior fellow at the Cato Institute, argues that age is a proxy for serious preexisting medical conditions, which are also associated with a much higher death rate and become more common as people get older. He notes that underlying conditions such as diabetes, heart disease, liver disease, kidney disease, respiratory illnesses, and immune system suppression have been involved in at least 99 percent of COVID-19 deaths in New York City.

"The absolutely critical and widely misunderstood point here is that 'underlying conditions' are THE only risk that virtually all fatal cases of COVID-19 had in common—not age," Reynolds writes. "That misunderstanding arose because old people are far more likely to have one or more of these conditions (and because more old people die of this and almost every other fatal risk). But it's about time to stop echoing the fallacy that this virus kills old people, rather than sick people."

The prevalence of preexisting health problems is obviously relevant in understanding why COVID-19 seems to be especially deadly in some places. Another widely cited explanation is the quality and capacity of the local health care system. Other things being equal, it makes sense that a jurisdiction where hospitals are stressed by a large number of COVID-19 cases would see not just more deaths but a higher IFR.

Antibody studies in New York and Indiana have yielded estimated IFRs of about 0.6 percent, three times the estimates from antibody studies in Miami-Dade County, Los Angeles County, and Santa Clara County. Some of that gap may be due to methodological problems with the latter studies, which could have exaggerated the prevalence of infection and therefore underestimated the IFR.

The Santa Clara County study, which was conducted by researchers at Stanford University, has been widely criticized. The authors respond to that criticism in the latest version of their preprint. One of the study's harshest critics, Columbia University statistician Andrew Gelman, offers a mixed review of the revised version.

One outstanding issue regarding the Santa Clara County study is its solicitation of subjects through Facebook ads, which may have biased the sample toward people who were especially eager to be tested because they were especially likely to have been infected. The researchers in Los Angeles County and Miami-Dade, by contrast, used random samples designed to be representative of the local population, as did the researchers in Indiana. The authors of the Los Angeles County study, which was published today as a letter to The Journal of the American Medical Association, nevertheless note that "the estimated prevalence may be biased" because "symptomatic persons may have been more likely to participate." The New York subjects were randomly selected from shoppers, which could have created a bias in either direction.

Another major issue is the accuracy of the antibody tests—in particular, their specificity, which indicates how often they correctly identified negative samples as negative in validation tests. Even a seemingly high specificity (say, 90 percent) can generate more false positives than true positives when researchers test subjects from a population in which the prevalence of infection is relatively low.

The New York study used a test developed by the state health department that has been validated by the Food and Drug Administration (FDA). According to the FDA, that kit has a specificity of 98.8 percent, meaning it incorrectly identified negative samples as positive about 1 percent of the time. Assuming that 5 percent of the population has been infected, the FDA says, the New York test has a positive predictive value of 79.4 percent, meaning roughly one in five positive results will be wrong. The Indiana study used Abbott's IgG test, which according to the FDA has a specificity of 99 percent and a positive predictive value of 84 percent when prevalence is 5 percent.

The Los Angeles County and Santa Clara County studies both used tests manufactured by the Chinese company Hangzhou Biotest Biotech and distributed in the United States by Premier Biotech, which is based in Minneapolis. The manufacturer reported a specificity of 99.5 percent, but that rate has not been validated by the FDA, and the Stanford researchers who conducted the Santa Clara study independently tested just a small number of validation samples. The actual specificity of the Premier Biotech kits, which is important in the prevalence calculation and therefore the IFR calculation, is a major point of contention between the Stanford researchers and critics of the study.

The Miami-Dade study used tests produced by the North Carolina company BioMedomics. The company reports that its test, which has not been validated by the FDA, generated 12 false positives out of 128 samples from uninfected people, meaning the results were erroneous more than 9 percent of the time. If the true prevalence of infection were 5 percent, the BioMedomics test would generate more false positives than true positives. It's not clear whether and to what extent the University of Miami researchers who conducted the study took false positives into account. That study, like the New York and Indiana studies, has not been published, even as a preprint.

In short, methodological weaknesses—in particular, low test specificity—could help explain why the California and Florida studies generated much lower IFR estimates than the New York and Indiana studies. But that is not the whole story.

The crude case fatality rate (CFR)—known deaths as share of reported cases—is substantially higher in New York (nearly 8 percent) and Indiana (6.2 percent) than in California (4 percent) and Florida (4.3 percent). For reasons that may include overburdened hospitals, COVID-19 patients seem to have fared worse in New York and Indiana than they have in California and Florida. In other words, the divergent results may to some extent reflect actual differences in fatality rates.

The authors of the Santa Clara County study note that the IFR may "be substantially higher in places where the hospitals are overwhelmed (e.g. New York City or Bergamo), or where infections are concentrated among vulnerable individuals (e.g. populations without access to healthcare or nursing home residents). For example, in many European countries, 42–57% of deaths occurred in nursing homes and
the same appears to be true for 25% of deaths in New York. Infection fatality rate estimates may be substantially higher in such settings." While this interpretation is obviously appealing as a response to the study's critics, that does not mean it is wrong.

There are similar variations in IFR estimates from Europe. Antibody tests in Gangelt, Germany, covering some 80 percent of the local population, found that 15 percent of residents had been infected, suggesting an IFR of 0.4 percent—half the estimate for France and less than one-third the estimate for Northern Italy in studies considered by Meyerowitz-Katz and Merone. The difference is not surprising, given that Germany's crude CFR (4.6 percent) is much lower than the crude CFRs for France (15.7 percent) and Italy (14.2 percent).

In Sweden, meanwhile, the Public Health Authority recently raised its IFR estimate to 0.6 percent, which is similar to the estimates from New York and Indiana but still substantially lower than the estimates from Italy and France. Not everyone agrees with the new estimate for Sweden. "State epidemiologist Anders Tegnell and Anders Wallensten have both said they believe the mortality rate is below 0.4 percent," Sveriges Radio reports.

The plausibility of relatively low IFR estimates ultimately depends on the plausibility of the prevalence estimates on which that calculation relies. The latest version of the Santa Clara County study, for example, estimates that nearly 3 percent of the local population had been infected by early April, about the same as the prevalence estimate for Indiana in the last week of that month. Since Indiana reported its first COVID-19 death on March 16, more than a month after the first confirmed death in Santa Clara County, the prevalence estimate for the latter jurisdiction seems, if anything, surprisingly low.

It is still possible, of course, that the Stanford researchers substantially overestimated the prevalence of infection in Santa Clara County, depending on the impact of sampling bias and the actual specificity of the antibody test they used. Yet even if their IFR estimate is too low, it seems likely that the true IFR in Santa Clara County (and in California generally) is lower than the estimated IFRs in New York and Indiana, given the relatively high death rates among known cases in both of those places.

In other words, the contrasting results of the antibody studies may reflect both methodological issues and differences in the underlying reality. That possibility tends to get lost in the argument between people who uncritically embrace low IFR estimates and people who automatically dismiss them.

[This post has been updated to note today's publication of the Los Angeles County antibody study.]

NEXT: State Supreme Courts Stand Up for Economic Liberty While SCOTUS Falls Down on the Job

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. All these results show is that the testing is not yet accurate enough to develop a policy that can rationally justify suspending the US Constitution.
    Just repeal all the bullshit edicts, rules, regulations, etc, and remind people every once in a while that washing you hands, covering coughs and sneezes, and staying away from sick people is good idea for everyone, and a very good idea for the elderly and immune system compromised.

    Just like every other flu season ever.

    1. What is this “rationally justify” of which you speak?

      1. Six months ago I lost my job and after that I was fortunate enough to stumble upon a great website which literally saved me• I started working for them online and in a short time after I’ve started averaging 15k a month••• The best thing was that cause I am not that computer savvy all I needed was some basic typing skills and internet access to start••• This is where to start… Read Details Articles

  2. “Do the Divergent Results of COVID-19 Antibody Studies Reflect Real Differences?”

    Better question, why do you think anyone cares what you think about it

    1. Yeah I’m pretty tired of all of the hair splitting and hand wringing. Both the economy and the constitution have been destroyed. None of this crap matters at this point.

      1. Sarah Paul Walker, Six months ago I lost my job and after that I was fortunate enough to stumble upon a great website which literally saved me• DCf I started working for them online and in a short time after I’ve started averaging 15k a month••• The best thing was that cause I am not that computer savvy all I needed was some basic typing skills and internet access to start•••

        This is where to start… For More Detail

  3. until the covids are politically inexpedient nobody will know.

  4. //A study of excess deaths in several Italian towns, based on a comparison of mortality during the pandemic to mortality during the same period in earlier years, found that the IFR ranged from 0.02 percent among people in their 40s to more than 15 percent among people older than 90.//

    Nobody in my entire family has ever lived to 90. I don’t plan to either. An 85% survival rate IN FUCKING NONAGENARIANS?

    GTFO. “Deadly” virus that can’t even kill 90-year olds. What a fucking joke.

    1. This made me laugh out loud, and my grandmother is in her 90s.

    2. yeah lol. my 94 year-old aunt “feh”s at this nonsense.

    3. Right, I mean getting out of the bathtub at that age has to be what, a 2-3% fatality rate?

      Seriously, if you are 90 years old the chance of you dying in the next 365 days just under normal circumstances is >15% last I checked. And that percentage goes up pretty rapidly as you get into your 90s.

    4. My dad turned 90 this last Saturday. He told me he’s shooting for 100. And I wouldn’t be surprised if he gets there. You pick up a lot of immunities in 90 years.

  5. The Los Angeles County and Santa Clara County studies both used tests manufactured by the Chinese company Hangzhou Biotest Biotech and distributed in the United States by Premier Biotech, which is based in Minneapolis. The manufacturer reported a specificity of 99.5 percent, but that rate has not been validated by the FDA, and the Stanford researchers who conducted the Santa Clara study independently tested just a small number of validation samples.

    I eagerly look forward to all the reasons why Chinese lies about their data are crap when that fits what we want to see and good when that fits what we want to see.

    1. China lies.
      Also, the virus isn’t especially deadly.
      There you go

  6. Shorter Sullum: “We don’t know.”

    Good survey, all kidding aside.

  7. It turns out that when you force covid patients into nursing homes, you do badly. (NY/NJ/PA/MI/UK at least)

    In fact, more and more it’s looking like the bar on this thing was not killing your own people. Sadly, many of our “leaders” managed to wriggle underneath it.

    1. Isn’t the pompous governor of NY a great guy or what. He was thoughtful enough to bundle every covid patient returned to a nursing home with several extra body bags. NY officials knew exactly what they were doing.

  8. Do you ever get bored writing about how much we don’t know?

  9. Another issue is in the probable heterogeneity of the antibody targets in the viral structure – How do we know know which antibody assay is looking at an antibody that actually has function against the viral particle at a critical function or structure?

  10. Wrong byline? Email says Bailey.

  11. Already going south of 0.5, and likely to hit 0.3 by the end.

    So not really much cause for concern to anyone, except the terminally unhealthy.

  12. Stop trying to rehabilitate the Santa Clara study. I know you’re embarrassed because you declared it a breakthrough result when it was published, but that’s in the past and everybody’s already forgotten.

  13. This is the biggest thteat for humans covid 19 has already destroyed America and china is now going to take advantage no matter what we do there will be world war 3 i m pretty sure abput that btw visit my newly created website

  14. This was a remarkably balanced take on how to interpret the anti-body studies.

    Of course, whether the true IFR is 0.2% or 0.6%, the panic and the lockdowns were never justified. (Nor is a simple population-average IFR very informative. With the significant age-skew in mortality, even if age is just a proxy for comorbidities, it doesn’t make much sense to talk about IFR outside of age-categories, or potentially comorbidity categories).

  15. This is a statement of the obvious. I suppose this falls into the category of when there is nothing useful to state, state the obvious.
    More important is that antibody tests using different platforms are giving dramatically different results.
    Before reporting any more conclusions/interpretations:
    First, standardized the antibody test.
    Then test a random population (based on standard polling technique) for both the presence of the virus and the presence of the antibody.
    Add the antibody plus the virus to get the infection rate.
    The infection rate doesn’t matter. It is the infection fatality rate (death due to virus/(all infections, past and previous).
    The second important metric is case fatality rate
    (death due to virus/all symptomatic infections) as this is a proxy for utilization of health services.
    Outside of getting more publications under one’s name, I don’t understand how any reputable researcher/academic can publish this information.

  16. 0. The main point is that the initial proclamation by the WHO that the IFR was 3.4% was complete bogus and the author should start with that. The estimate that pushed the world into a panic and recession was up to 10 times (Or more) to high. Why don’t start with that?

    1. Mr. Sullum, did you read the second version of the Santa Clara study? I guess not, because the specificity estimates have been significantly sharpened. Andrew German acknowledges this point now.

    2. For those of us who know a bit of math beyond high school, it is easy to calculate the ratios of IFRs given the age pyramids in different countries. The fatality rate vs age grows exponentially (literally exponentially, not in the sense used by media idiots And other mathematically illiterate types). This is the so called Gompertz distribution and using it shows that the IFR can vary by a factor of 10 between old and young countries.

  17. Given the flaws in some experiments, all the results are consistent with a IFR of 0.5% ± 0.3% averaged over the whole general population. We know fatality is highly dependent on age, and men have twice the fatality rate of women in all age groups. The biggest unknowns are exposure dose effects and human genetic variations in response to the virus. It is entirely possible that exposure doses in some environments, like riding in a packed NY subway car without wearing masks every day, are on average higher or more repeated than exposures chatting in a bar or getting a massage from an infected masseuse. Infected people singing (as in a choir) seems to produce high levels of virus in the air around them. There is some evidence that high dose exposures are more likely to be fatal, as a lot of young health care workers have been killed, particularly in the early days of the pandemic in China. The best way to reduce exposure doses is for everyone to wear masks in indoor businesses and confined spaces as much as possible.

  18. Lost in the weeds, obsessed with the impossibility of perfect information, haunted by the politics of ‘data we don’t like’ (and therefore reject), we fail to see the larger picture.

    So what is that larger picture?

    1) The truth is, 8K of us die every day. Every normal, average, unremarkable day is filled with 8000 family tragedies….none of them headlined by Lester Holt. Kim Kardashian’s cleavage is MUCH bigger news than the death of 3M Americans this year, last year, every year. This is a given.

    What else?

    2) 99% of WuhanV deaths are associated with comorbidities. That means we still don’t know (and won’t know for quite some time) how many of these ‘virus deaths’ are truly ‘incremental’ (above and beyond the normal 8K/day) and how many are simply ‘pushed’ by WuhanV.

    3) The vast majority of WuhanV infections are asymptomatic or so mildly symptomatic that the ‘victim’ does not even go to the doctor. We can quibble the limitations of the Santa Clara testing…but the fact remains that although the County had 956 recognized WuhanV cases (people sick enough to seek treatment & testing), the serology tests conducted by Stanford revealed that between 50,000 and 80,000 individuals had already been infected. And of those thousands? The truth is….the estimated hospitalization rate (for those who are truly sick) is about .05% (or 50/100,000).

    Put all that together and you begin to construct an understanding of the Virus-side of the social/economic equation we now wrestle to balance..

    The virus is highly infectious….most people who get it don’t know they have it….those who become sick are very likely to NOT require hospitalizations….and those who are hospitalized will most probably not die. The risk of death is highest for those who are already comorbid, but the mortality rate for the infected population would seem to be well under 1%…and perhaps as low as .1%.

    What else do we know of that larger picture?

    4) The Lockdown — triggered by a horribly flawed pandemic forecast model — has effectively killed the Global Economy. Hundreds of millions of people unemployed…. without an income….unable to provide for their families. The world as we knew it is withering.

    5) The Media Virus Hype (initially keyed to that same horribly flawed pandemic model) has frightened millions and opened the door to petty tyrants, eager to use the Crisis to tell us all exactly what we can and cannot do (pretty much as dictated by whim and their personal perspective on what is and is not ‘essential’).

    When we tie all that to the irrationalities prompted by the Lockdown (too many to count) we discover insanities: hospitals laying-off staff, on the verge of bankruptcy because 1) people are too scared to seek treatment for critical health problems…and 2) because the State mandated an end to non-essential surgeries and procedures (fearing the predicted tidal wave of Virus Victims).

    In sum … more damage has been done — in 3 short months — to the United States than was ever done by any of our enemies in all the wars we’ve ever fought, combined.

    On one side of the equation we have the virus and the very minimal threat of death (primarily for the comorbid). On the other — a destroyed America. Tens of millions unemployed. Thousands of businesses crashed & burned (and probably unrecoverable). Life savings trashed. Homes lost. Domestic violence up. Suicides up. Depression up. More people dying of ‘normal’ and manytimes treatable medical issues because they’re too scared to seek treatment in hospitals financially crippled by the mandated ban on the non-essential.

    Initially we were told the damage to our nation was a sacrifice required to ‘flatten the curve’…to avoid overwhelming an ill-prepared Medical System with a tidal wave of WuhanV victims.
    Now we hear we must wait more testing….we must wait for a vaccine…we must wait until….”It’s Safe!”.

    But — the truth is — it will never be safe. It can never be safe. Even if we develop a vaccine (a far from certain outcome, given that we’ve never succeeded in developing a vaccine for a Corona Virus), there will be — inevitably — something else. We cannot afford to wait. We especially cannot afford to wait for some Godot-like Panacea which may never actually arrive.

    So what do we really know?
    The truth is…life is risk. And WuhanV just another kind of risk. The question before us: what is all this worth? What is the value we assign to a ‘normal’ (albeit, risky) existence…one which includes baseball games, grandkids, grocery stores, restaurants, 1st dates, proms, funerals, and graduations….all of the ordinary hustle-bustle which used to surround us? Texas’ Dan Patrick put it this way: “There are more important things than living, and that’s saving this country for my children and grandchildren and saving this country for all of us,” He’s absolutely right.

    And it doesn’t matter that we don’t yet know with 99.99% certainty what the actual IFR may be….the Big Equation remains the same: huddle in fearful isolation and thereby destroy exactly what we’ve spent the last 244 years trying to defend….OR….take arms against this sea of troubles, and by opposing, end them?

    Are we so afraid of dying that we become afraid to live? The choice is ours.

Please to post comments

Comments are closed.