The Gatekeeper

How a little bureaucratic office became the biggest impediment to Barack Obama's health care plans

It was January 2009, and Democrats were triumphant. Their party had won major victories in both the House and the Senate, and Barack Obama, arguably the most economically left-wing president in decades, had just won the White House on a promise to finally achieve what had eluded liberals for so long: universal health care.

As the new era unfolded in Washington, plans for overhauling one-sixth of the economy began to take shape. Health care reforms, Democrats vowed, would extend insurance to every American and be fully paid for without requiring middle-class tax hikes, all while cutting costs significantly enough to save the country from financial catastrophe. To sell these claims the party trotted out one of the most respected number-crunchers in town, Office of Management and Budget Director Peter Orszag, a former Brookings Institution health care expert obsessed with cost cutting. With 60 votes in the Senate, nothing seemed to stand in the Democrats' way.

Nothing, that is, except the Congressional Budget Office (CBO), a nonpartisan federal agency that until this year was run by none other than Peter Orszag. As drafts of various health care bills began to emerge on Capitol Hill, the CBO, responsible for devising Congress' official legislative cost estimates (known as "scores"), released a series of reports that demolished key Democratic claims. According to the CBO, both the "tri-committee" bill proposed in the House and the bill proposed in the Senate Finance Committee would cost in excess of $1 trillion over 10 years, might leave tens of millions uninsured, and would not curb rising health care costs. Indeed, both would add substantially to the budget deficit in the long term. As the year progressed, the CBO proved a more effective check against key elements of the Democrats' domestic agenda than anything concocted by Republican strategists or libertarian wonks. In an October article, The Washington Post concluded that the CBO had "essentially condemned two legislative proposals by slapping them with trillion-dollar price tags."

Created as an afterthought and initially intended as a low-profile congressional calculation service, the CBO has quietly risen to a place of unique prominence and power in Washington policy debates. Widely cited and almost universally respected, it is treated as judge and referee, resolving disputes about what policies will cost and how they will work.

But the agency's authority is belied by the highly speculative nature of its work, which requires an endless succession of unverifiable assumptions. These assumptions are frequently treated as definitive, as if on faith. In practice, this means the CBO is not merely an impartial legislative scorekeeper but a keeper of the nation's budgetary myths, a clan of spreadsheet-wielding priests whose declarations become Washington's holy writ.

Birth of a Budget Office

The CBO's history goes back to the 1974 Congressional Budget and Impoundment Control Act, one of many mid-'70s attempts to wrest back power from an executive branch that had expanded to new levels under President Richard Nixon. The act, which passed over Nixon's veto, was part of an ongoing effort to get Congress more involved in the budgeting process, which many legislators felt was too dominated by the White House. It launched the House and Senate Budget Committees, changed the way the annual budget was prepared, and created the Congressional Budget Office.

Few in Congress knew what the new agency would look like. As CBO Special Assistant David Mundel recalled in a 1988 case study prepared for Harvard's Kennedy School of Government, "I don't think anybody had the faintest idea what this place would be—if it would be two people, if it would be a group that advised the budget committees only, if it would ever come to exist."

Before birthing their new bureaucracy, the House and Senate had to reconcile sharply different ideas about how the agency would be run. The House, according to then–Special Assistant and later CBO Director Robert Reischauer in the Harvard study, wanted a quiet, low-profile organization, "basically a manhole in which Congress would have a bill or something and it would lift up the manhole cover and put the bill down it, and you would hear grinding noises, and twenty minutes later a piece of paper would be handed up with the cost estimate." It was to be "noncontroversial, the way the sewer system is."

The Senate, by contrast, envisioned a more prominent office, one that would provide not only calculations but extensive analysis—a full-service congressional think tank. After a protracted power struggle, the Senate conception won out. Senate Budget Committee Chairman Ed Muskie (D-Maine) helped pick the economist Alice Rivlin as the first CBO chief, with a mandate to remain objective and nonpartisan, providing detailed analysis but avoiding policy recommendations.

Rivlin had been (and remains) a senior fellow at the center-left Brookings Institution, where she has worked, on and off, in various capacities since 1957. Now 78, Rivlin is a petite woman with short, dark hair, and the wry, knowing confidence of a longtime Washington hand. She holds a visiting professorship at Georgetown University's Public Policy Institute, serves on the board of the New York Stock Exchange, and was a 1983 recipient of the MacArthur Foundation's "genius" grant. Rivlin was not merely the first head of the CBO; she was one of the agency's architects as well, quickly expanding its mission beyond what even the Senate had in mind.

Early on, Rivlin pushed for, and received, independence in hiring. Staffing on Capitol Hill back then was rife with nepotism—Rivlin has described it as a "schmoozy, good ol' boy Hill culture"—and those making hiring decisions were expected to play along when powerful legislators sent their former staffers out with recommendations. Muskie largely shielded Rivlin from such expectations.

Thanks both to Muskie's protection and the agency's vague conception, Rivlin had a great deal of leeway to staff and organize the CBO as she saw fit. Disregarding the Senate's estimate that 158 staffers would be sufficient, Rivlin and her initial team hired 259 instead, every one of whom she made a point to interview herself.

Rivlin's goal was for the CBO to be respected by the budgeting community. Analysts kept their judgments conservative by relying on pre-existing economic models. Even when pressed by legislators, Rivlin says, she "religiously" avoided policy endorsements. The result was an agency that quickly attained a high degree of respect and authority on Capitol Hill.

From the beginning, CBO figures provided meaningful reality checks for big-ticket items on both parties' legislative wish lists. In 1975, for example, the agency estimated that a health care bill proposed by Ted Kennedy would cost $185 billion over five years—three times higher than the number put forth by Kennedy's staff. In the 1980s, Rivlin turned the CBO into a staunch opponent of Ronald Reagan's supply-side economics, insisting that tax cuts could not bring in enough revenue to pay for themselves.

Though there has been some periodic pushback against the agency, over the years the CBO has accumulated enough authority to be seen as one of the last honest arbiters in Washington. As Brookings budget analyst Henry Aaron recently told The American Prospect, "It's not infrequent to hear people say it doesn't make any difference what [a bill] really costs. It only matters what CBO says it costs." When the president declared during a September 2009 prime time address that he would not sign a health care bill unless it was deficit neutral, the universal assumption was that the CBO would make that particular call.

But looking into the future is an inherently uncertain enterprise, as much magic as science. Just how accurate is the CBO's crystal ball?

Making Assumptions

During the last major push for health care reform, under President Bill Clinton in 1994, the CBO drew up a model of the existing system, extended that model into the future, and calculated how the future would change under the proposed legislation. In doing so, the agency made scores of estimates and assumptions, both about the state of the existing system and how millions of individuals might react to the proposed changes.

In order to determine how much the bill would cost, it had to account for dozens upon dozens of variables, including how many people had health insurance and how many didn't, how many people with insurance got it through an employer, what types of plans those people had, the total amounts of their premiums, the percentages of the premiums the employers paid, the coverage and cost-sharing requirements of those plans, whether or not uninsured individuals had options to obtain insurance and what those options might be, the health of the insured, the frequency and type of care used by the insured, the nature and size of the companies providing insurance for their workers and the business reasons that led them to do so, regional breakdowns of health care use patterns, how new treatments and new medical technologies would change medical practices and costs, the impact of hospital consolidations and other developments in the business of medicine, how enforceable and effective new cost controls would be, and what choices patients would make with regard to doctors, medicines, and treatments under the proposed legislation.

To build their working model, CBO analysts worked with data pieced "together from several inadequate or dated surveys and sources," according to a 1995 article in the journal Health Affairs by Robert Reischauer and Linda T. Billheimer, the CBO's director and deputy assistant director, respectively, during the 1994 debate. Surveys and census data were used to the greatest extent possible, but by and large the necessary information simply did not exist. So the analysts did what they could, and what they still do today: They guessed. Specifically, they estimated that the Clinton plan would raise the deficit by about $70 billion over the first six years, contradicting the White House's estimate that the proposal would lower the deficit by $60 billion.

To be sure, their guesswork is as good as it comes. On the left and the right, most economists will testify to the CBO's general excellence. Even those who criticize the office tend to do so delicately. In an August New York Times op-ed arguing that the agency's health care numbers were consistently unreliable, Jon Gabel, a senior fellow at the University of Chicago's National Opinion Research Center, began by praising the office for its competence, integrity, and well-earned respect.

Nevertheless, even good guesswork is still guesswork. As the George Mason economist Arnold Kling says, "it is literally an impossible task" to accurately make the sort of projections the CBO specializes in. "We don't do controlled experiments in economics," Kling says. "So when you're talking about figuring out the effects of health care policy, it's very difficult."

Part of the difficulty is that the CBO is trying to replicate systems it can't really see. To understand the problems with building an economic model, consider what it takes to make a working scale-model train. To build that train, you'd first need accurate information about how the full-size train works: how big its parts are, at what speed those parts move, its power consumption and control system. Imagine trying to build a model train without ever being allowed to look inside the engine compartment. A smart engineer would be able to make reasonably educated guesses about the internal workings by measuring the outside and by looking at various external controls, but those guesses would almost certainly come with a high margin of error.

That's no small part of the problem for the CBO. When scoring legislation, they're essentially trying to build small-scale working models of systems using fairly limited data sets. For example, according to Reischauer and Billheimer, the National Health Interview Survey provided CBO analysts with data on "health insurance coverage, health states, use of health services and socioeconomic variables." But these sources provided "no data on premiums or cost-sharing requirements, and no indication of the exact share of premiums paid by employers."

These days, CBO analysts are scoring bills using intricate computer simulations based in large part on survey data. The raw information is interpreted through academic research on how human beings respond to various economic assumptions. In an interview with The Washington Post, the CBO's chief health care analyst, Phil Ellis, compared the process to playing Sim City, a computer game that simulates urban development. But even the best model is still only as good as its input data. And for policies that have no real-world antecedent, it's extremely difficult to come up with accurate input data.

In theory, it would at least be possible to know with reasonable precision, say, what the spread of employer contributions to the cost of health insurance actually looks like. But all the data in the world still wouldn't solve the main problem: human beings. According to Jim Manzi, a senior fellow at the Manhattan Institute and the founder of an applied artificial intelligence company, "The fundamental problem is that no one has a good model of the human mind." We can know what people have done, but not why they did it, and certainly not how that reasoning will translate into future decisions. Moreover, Manzi notes, "It's not just one mind. It's many human minds interacting."

A capable modeler can introduce variables to adjust for any number of factors. But the more detailed a model, the more moving parts it needs. Inside every CBO formula is a patchwork of sketchy assumptions about both human behavior and the state of the current system, many of which, as in the case of Clinton's health care plan, are backed by remarkably little data. Each of these assumptions increases uncertainty.

The CBO, to its credit, is entirely open about both the significant uncertainties in its reports and the general problem of modeling. Testifying to Congress in 2002 about the agency's budget projections, former CBO chief Rudolph Penner was blunt: "No one forecasts anything very well. That is true whether one looks at pundits forecasting the course of the war in Afghanistan, demographers forecasting worldwide birth rates, or pollsters forecasting the French presidential election." He was equally candid in his assessment of the CBO's work. "I recently studied the history of errors," he said, "and I would like to submit my results for the record. They are pretty discouraging."

Penner's testimony focused on the CBO's woeful record in both budget forecasting and estimating "revenue and expenditure feedbacks"—basically, in correctly identifying behavioral responses to policy changes. On the budget, Penner noted that "the average error made in the forecast of the budget balance used to formulate the budget resolution is over $100 billion for the first year covered by the resolution and over $400 billion five years out" and that "the projection for the budget balance in 2007 changed over $800 billion between early 1997 and the summer of 2000."

Less wonky are the criticisms made by partisans. CBO scores frequently provoke annoyance from both Democrats and Republicans, although the complaints tend to center on different issues.

For Democrats, the CBO's biggest sin is its failure to score savings from changes to government-run health care programs. In July Bruce Vladeck, formerly a top staffer at the Health Care Financing Administration, argued in Roll Call that "the CBO has routinely overestimated the costs of expanded government health care benefits and underestimated the savings from program changes designed to reduce expenditures," pointing specifically to its overestimation of the five-year cost of the Medicare prescription drug benefit. Sen. Max Baucus (D-Mont.), chairman of the Senate Finance Committee, huffed at a February hearing that "we're not in the old situation where whatever the CBO says is God."

Republicans, on the other hand, tend to worry more about the office's record on taxes. According to a report that the Joint Economic Committee, a panel of economic advisers to Congress, prepared for then-Rep. Dick Armey (R-Tex.), CBO analyses in 1989 and 1990 "failed to take into account the effects of higher capital gains taxes after 1986, producing huge forecasting errors." In a 2003 essay for National Review Online, Rep. Mark Kirk (R-Ill.) noted the office's repeated failures to accurately forecast the budget and complained that the agency did not do enough to provide "real-world estimates that account for the interaction between federal taxes, federal programs, and individual behavior."

But the agency's inaccuracies must be judged relatively. "They're the second worst source of forecasts on the effects of health care policy," says Arnold Kling. "As far as the worst, all the rest are tied. I'm sure that their forecasts are flawed and subject to huge error. But the next question to ask is: Compared to what?"

Manzi concurs. "No matter how bad the situation is," he says, "would no scorekeeper at all be even worse?"

Other approaches are unlikely to produce better results. Releasing a range of scores, for example, would underline the uncertainties, but it would also provide greater ammunition to partisans, who would simply pick the numbers that favored their position. Rivlin contends that the question is moot. "Decisions have to be made in light of the best information available," she says. "There isn't really an alternative."

Not Just a Scorekeeper

With the estimated costs for the Senate and House health care bills both coming in at $1 trillion or more, the CBO proved one of the most powerful roadblocks for the president's agenda over the summer. But anyone depending on the CBO to blockade reform reliably will be disappointed.

When the Senate Finance Committee released the preliminary draft of its health care bill in September 2009, the accompanying CBO analysis reported that the legislation, as written, would reduce the budget deficit over the next two decades. This projection was based on the highly dubious assumption that Congress would follow through on the bill's proposed Medicare payment cuts—cuts Congress has been loath to make in the past. The report acknowledged these past failures, but the CBO is prohibited from making judgments about the likelihood that Congress will keep its own promises.

After a year of relying on the CBO's scores to prove the high cost and disastrous deficit effects of health care reform, congressional Republicans and other ObamaCare opponents were suddenly faced with an analysis that confirmed the Democrats' most contentious fiscal point: that over time the changes would more than pay for themselves. The CBO projection of deficit reduction was arguably the best break the Democrats had gotten in months. The bill's prospects improved overnight.

But based on what? The CBO admitted in its report that its "estimates are all subject to substantial uncertainty," particularly those dealing with deficit effects in the second decade. Yet neither the inherent uncertainty nor Congress' history of reneging on planned Medicare cuts had much effect on the ensuing debate. All that mattered was the score.

The CBO had effectively given the bill a go-ahead—and revealed, once again, how influential the agency is in determining the success or failure of major legislation. It's not just the scorekeeper; it's the gatekeeper.

"That," Rivlin says carefully, "is what I've known all along."

Peter Suderman is an associate editor at reason.