Want Useful Data on Obamacare? The Census Bureau Won’t Help You.

Whitehouse.govWhitehouse.gov

One of the big unknowns about Obamacare is what effect it has had on reducing the number of uninsured in America. Over the last few months, we've seen a number of attempts to measure this, but they've all been relatively narrow surveys that for a variety of reasons don't provide a complete picture. As a result, many analysts on both sides of the debate generally agreed that the best way to answer this question would be to use information gathered by the Census Bureau, which provides some of the most consistent, comprehensive data on the nation's uninsured.

Well, now it looks like we won't be able to use the Census to determine the health law's effects on insurance coverage over time. That's because the Bureau is in the process of radically altering the way it asks questions about insurance status. The update will render useless comparisons between data reported last year and data reported this year, according to The New York Times

The Census Bureau, the authoritative source of health insurance data for more than three decades, is changing its annual survey so thoroughly that it will be difficult to measure the effects of President Obama's health care law in the next report, due this fall, census officials said.

The changes are intended to improve the accuracy of the survey, being conducted this month in interviews with tens of thousands of households around the country. But the new questions are so different that the findings will not be comparable, the officials said.

An internal Census Bureau document said that the new questionnaire included a "total revision to health insurance questions" and, in a test last year, produced lower estimates of the uninsured. Thus, officials said, it will be difficult to say how much of any change is attributable to the Affordable Care Act and how much to the use of a new survey instrument.

The Times reports that the update was not politically motivated, and that it was made in the interests of gathering more accurate data. The old survey methodology is said to have been prone to measurement errors that overestimated the number of uninsured; the new data supposedly corrects that problem.

Now, to be clear, the change will give us one year of pre-expansion data: The new measurement methodology will start with 2013, which means we’ll be able to compare last year's insurance pre-coverage-expansion stats with this year's. That will give us some sense of the changes under the Affordable Care Act. But there are still big problems with the timing of the transition. As Aaron Carroll notes, it still destroys the long-term trendline. The trend info would have had a lot of value generally, and also would have set the 2013 results in a helpful context. Now it's going to be difficult to discern the impact of the upheavals that we saw in the health insurance market in 2013, which featured, among other things, large unexplained spikes in coverage, and the cancellation of millions of individual insurance policies. 

This is infuriating and unfortunate. Ignore, for a moment, how politically convenient it is. It's just poor social measurement.

If the interest is in getting the best and most useful data possible, then we should want a time series without breaks or alterations, one that can be used to reliably compare the years before and after the single largest and most controversial policy change intended to affect the provision of health insurance in the last four decades. One might think that creating a consistent measure for this effect would be a priority for people whose job is to create socially useful guides to the makeup of the nation's population. Apparently not. The data may be more accurate going forward, but it will be far less useful for measuring this crucial, once-in-a-generation transition.

As to whether this is politically motivated, there's no evidence that it is. And I would be surprised if this move were intentionally cooked up simply to obscure the figures. But back in 2009, the White House moved to exert more direct control over the Census Bureau and its operations. What that means is that, at the very least, the White House should have been involved. If the Obama administration was truly concerned about transparency in measuring the impact of the health law, then its officials should have known about the change, should have noted it publicly before the end of the open enrollment period, and should have pushed to stop or delay it.

Because as it stands now, we simply won't have the robust information set we need to confidently assess the impact of the Affordable Care Act on the uninsured over time. Nor will we have a good picture of exactly how Obamacare is affecting various types of coverage—the particular ways in which people are, or aren't, accessing health insurance under the law.

These aren't the sort of questions you can answer based on first principles, or what feels right. You need robust measurements. Solid Census Bureau data, taken consistently over time, would have helped answer these questions. But it appears that's not something we’re going to have. Instead, we'll be left with ongoing arguments and unanswered questions, especially if the administration continues to keep mum on other critical unanswered questions about the law. A decade from now, we'll be able to definitively determine that Obamacare was a thing, that happened, and did something, and cost some money. But that may be as much as we can know.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • Adam330||

    "in a test last year, produced lower estimates of the uninsured."

    Don't worry. Once the new figures come out, they will be trotted out to claim Ocare reduced the ranks of the uninsured, conveniently leaving out the fact that the methodology is different.

  • ||

    Yes... yes they will. And all the usual suspects will be repeating it ad nauseum. Just like the 'rise' in autism cases caused by the double medical threats of... changing the definition so more people fell under it, and looking more closely for autism cases. The press was ALL OVER that for years. Did they ever ONCE point out that the 'rise' was illusionary? No, well, OK, Stossel probably did. and that's why he got along so well with his colleagues...

    It just makes me want to vomit.

  • TCop19||

    Not to mention the definition of insured and previously uninsured.

  • Major Johnson||

    Don't forget, they had to change the definition of insurance first.

  • JohnZeus||

    Oh puh-LEEZE! How naive are you if you don't think this is politically motivated. C'mon. This Administration has proven over and over that they NEVER do anything that is NOT politically calculated. This is all about muddying the waters in order to render all analysis meaningless and demagogue-able.

  • PapayaSF||

    No kidding. When someone has been caught driving drunk dozens of times, and you see their car weaving on the road, it's silly to say: "Oh, maybe they're sneezing or swatting at a bumblebee in the car."

  • craiginmass||

    Is there something wrong with political motivation? Whenever more than 1 person is involved in something, politics is the game.

    What do you expect? Hey-Zeus?

  • Pro Libertate||

    TRANSPARENCY.

  • PRX||

    let Obama be clear

  • Dances-with-Trolls||

    Isn't this an implicit admission that the numbers used to justify this chucklecuk were BS?

  • R C Dean||

    The old survey methodology is said to have been prone to measurement errors that overestimated the number of uninsured;

    Funny how nobody mentioned that when OCare was being debated and passed.

    the new data supposedly corrects that problem.

    Funny how the change to reduce the reported number of uninsured only goes into effect when Obama needs data showing that there are fewer uninsured.

  • JohnZeus||

    The infuriating part of this is that the media will ignore this, so very few will know about it, and then in a year or two they'll announce the "miraculous" success of Obamacare in reducing the number of uninsured. Without skeptical investigative journalism that questions everything and makes it very public, we're doubly screwed as a nation. Collusion between govt and the press is the absolute kiss of death.

  • paranoid android||

    It has to be a complete coincidence. This is the most transparent administration ever, after all, I'm sure they wouldn't blatantly lie about their motivations for doing something. This idea that politicians are slimy and dishonest is just a right-wing lie funded by the Koch brothers, probably.

  • TCop19||

    Dishonesty and corruption are long-standing trademarks of the Obama Clown Show.

  • BigT||

    Obama Clown Show.

    Racist! (most clowns wear whiteface)

  • CampingInYourPark||

    Change in methodology? Is answering the question: "Do you have health insurance?" solving a fucking riddle?

  • ||

    Change in methodology? Is answering the question: "Do you have health insurance?" solving a fucking riddle?

    After healthcare.gov? Yes.

  • creech||

    Not to worry. Sometime after the election, there will be a bunch of pundits and pols who claim statistics show not enough uninsured signed up - due entirely to the subsidies being inadequate - so its high time the Teathuglicans and Kochsuckers get out of the way so poor people can receive free health care.

  • Stevie OneLeg||

    I would be surprised if this move were intentionally cooked up simply to obscure the figures. But back in 2009, the White House moved to exert more direct control over the Census Bureau and its operations. What that means is that, at the very least, the White House should have been involved.

    Oh, Peter, Peter, Peter. I find your naivete utterly adorable.

  • JWatts||

    "The old survey methodology is said to have been prone to measurement errors that overestimated the number of uninsured; the new data supposedly corrects that problem.
    ...
    But back in 2009, the White House moved to exert more direct control over the Census Bureau and its operations.
    ...
    The Times reports that the update was not politically motivated...
    "

    Bullshit.

  • datcv||

    The fact that the times felt the need to mention that is a guarantee that the opposite is true.

  • James Ard||

    Is there any question now why they moved the Census Bureau into the White House? Not politically motivated my foot.

  • Paul.||

    Keep your friends close, and the people who can shape your legacy closer.

  • Gilbert Martin||

    "The Times reports that the update was not politically motivated, and that it was made in the interests of gathering more accurate data"

    Ha! Ha! Ha! Ha!.

    That's a knee slapper.

  • Jackand Ace||

    "Now, to be clear, the change will give us one year of pre-expansion data: The new measurement methodology will start with 2013, which means we’ll be able to compare last year's insurance pre-coverage-expansion stats with this year's."

    Yes, to be clear...that's the whole point, is it not? We will have data for the year before the ACA, which then can be compared to data after. Its a bit of a tempest in a teapot, if you ask me.

    Regardless, you still have other polls, such as Gallup, who continue with the same methodology from year to year. Of course, they just said the uninsured numbers are dropping.

    http://www.gallup.com/poll/168.....-2008.aspx

  • politicsbyothermeans||

    I'm certain no one here would ask you.

  • Ornithorhynchus||

    I'll assume that the new survey is more accurate than the old one. Still, that's no reason they couldn't do both for five or six years. That way they could see how well the trends match up on the two surveys, which might even allow them to extrapolate the new method back a few years.

  • JWatts||

    "Yes, to be clear...that's the whole point, is it not? We will have data for the year before the ACA"

    Obamacare started in October of 2013! Were you born yesterday? How did you fail to realize that?

    Oh, you were repeating the talking points that were emailed to you. Nevermind.

  • astronomical object||

    Proposal for a new, news feature:

    When publishing articles that include government-agency-x-did-this or government-agency-y-did-that *also* include a mouseover/dropdown menu/etc. which shows the names and titles of the individual(s) who made the decision(s) to implement the policy, etc. being discussed.

    No more hiding behind borg sectors.

  • Duelles||

    So the WH moved the census bureau into its domain of influence, the NYT says changes in the reporting of data are not politically motivated? Two sources that I do not trust pretty much says it all. Any competent agency would provide transitional data.

  • craiginmass||

    The very idea that folks here - or other Americans - want or care about useful data on health care...is fantastic! If that were true, why have they put up with corporate greed at 2 to 3X the real price - for decades.

    Here's a quick study into what we pay compared to others - nothing to do with the ACA. You folks have been brainwashed to just say "bad bad Obamacare" and take your eyes off how you've been shafted for decades. It's corporate greed which has hurt us and, if anything, the ACA puts some caps on that by many means (please don't make me list them out).
    http://www.scribd.com/doc/2186.....ice-Report

    You should be cheering, but like so many others in this country you have been somehow convinced to think and vote against your own interests. As long as their is a web site or newletter that allows you to "ditto", everything else will work out, eh?

  • Chris Conover||

    This change was 10 years in the making, so it’s hard to picture a conspiracy afoot. http://www.census.gov/hhes/www.....014-16.pdf

    The new method assuredly is superior in that we’ll now get both a point-in-time and over-time estimate of uninsured status, whereas previously we got a number that was SUPPOSED to be coverage in the prior calendar year, but turned out to be a blend of point-in-time and last-year-coverage figures. It would have clearly been better to have all this in place well before 2010 so that we could have better measured impact of the ACA. But it made more sense to switch to a superior measurement method in time to capture 2013 and 2014 as opposed to putting this off for yet another half-decade etc.

    Given the timing of the change, admittedly, having side-by-side comparisons of old and new numbers might have been nice hypothetically so we could better deduce changes attributable to ACA vs. other factors.

  • Chris Conover||

    As well, the MUCH larger American Community Survey (also conducted by Census) does provide us with annual snapshots that pre-date President Obama’s inauguration, so this will provide us with the capability you describe. The only problem is we’ll need to wait until September 2015 before we have definitive ACS figures that compare CY2014 vs. CY2013. But realistically, that’s how long we would have had to wait for CPS figures if we interpret them as being estimates of coverage in prior calendar year. That is, CPS figures that will have come out this September would have told us about CY2013, not 2014.

    Moreover, the smaller NHIS reports quarterly estimates of the uninsured going back to 1998: by December we will have NHIS estimates for Q1 2014 to compare against 2013 figures. http://www.shadac.org/publicat.....urveys2013

    Thus, one could wish for more timely data, but we won't be entirely devoid of data to assess the impact of this misguided law.

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online

  • Video Game Nation: How gaming is making America freer – and more fun.
  • Matt Welch: How the left turned against free speech.
  • Nothing Left to Cut? Congress can’t live within their means.
  • And much more.

SUBSCRIBE

advertisement