fastest growing form of intercity travel in the U.S.In 1997, Chinese-born entrepreneurs began regularly scheduled long-distance bus services that picked up passengers on the street. Tickets were priced so low that it was hard to figure how the operators could be breaking even, much less making a profit. Faced with declining market share, Greyhound and Peter Pan imitated the Chinatown model by teaming up to create a new venture called BoltBus. Then Coach USA got into the game with Megabus. Today, “curbside” buses—lines that begin and end their routes at the sidewalk as opposed to a traditional station—make up the
But over the past two years, the government has forced 27 bus companies based in Chinatown to close. The regulatory clampdown was fueled by a government study that found curbside carriers were disproportionately killing their passengers. Released by the National Transportation Safety Board, a federal agency, the study concluded that curbside bus companies were “seven times” more likely to be involved in an accident with at least one fatality than conventional bus operators. That finding was reported by The New York Times, the Los Angeles Times, Businessweek, USA Today, the New York Daily News, WNYC, and Reuters, among others. Although the study did not single out Chinatown bus companies the headline in Businessweek read, “Chinatown Buses Death Rate Said Seven Times That of Others.”
The study is bogus. Not only is the “seven times” finding incorrect, the entire report is a mangle of inaccurate charts and numbers that tell us virtually nothing meaningful about bus safety. There’s no evidence that curbside or Chinatown buses are any less safe than any other kind of bus.
How did the study authors figure curbside bus companies are “seven times” more prone to fatal accidents? For starters, they counted 37 accidents during the study period involving curbside buses in which there was at least one fatality. When I rebuilt the study data and contacted the companies involved, I found that, in 30 of those 37 accidents, curbside buses were not involved. In fact, 24 of those 30 misclassified cases involved Greyhound’s conventional bus fleet. (Greyhound’s curbside subsidiary BoltBus had no fatal accidents during the study period.)
The National Transportation Safety Board denied my requests for the study data, even though it was a taxpayer-funded report with an impact on policy. After my Freedom of Information Act request also failed to return the information following a six-month wait, I began reconstructing the study data from other sources.
Proceeding on the time-honored hunch that people who are hiding something have reason to do so, I generated a list of the 37 fatal crashes using a database obtained from a federal contractor that collects nationwide accident data. I analyzed that data with help of Aaron Brown, a quantitative analyst with the hedge fund AQR Capital Management. Brown was the first to point out major flaws in the NTSB’s methodology in an article published by Minyanville.com, accusing the study authors of “statistical malpractice.” I also consulted with Ed George, a professor of statistics and department chair at the University of Pennsylvania’s Wharton business school, who examined the study for the purposes of this article.
“When I first read the NTSB report, I thought this is just terrible statistics,” says Brown. “But it goes way beyond that. It’s almost as if someone took some random data and shook it together.”
The NTSB study grew out of a horrific March 2011 bus crash in the Bronx that killed 15 people. The accident, involving a bus company called World Wide Travel, was both a tragedy and an anomaly. From 2001 to 2011, there were an average of 34 fatal intercity/cross-country bus accidents each year. During the same period, there were an average of 23,000 fatal passenger car accidents annually. For every mile traveled, passengers are three times as likely to die when riding in a car than in a bus (of any sort), according to data obtained from the National Highway Traffic Safety Administration. [*]
One reason the World Wide Travel crash led to a broad study of industry safety is that it was a "Chinatown bus," according to a slew of media reports (see here, here, and here). Today, companies owned and operated by Chinese immigrants make up only a fraction of the industry, but “Chinatown bus” is often misused as a blanket term for all curbside carriers, particularly when safety is in question. The owner of the company wasn’t Chinese, nor was the bus driver. But when the accident occurred the bus was destined for New York City’s Chinatown to drop off its passengers, so it was a “Chinatown bus.”
Enter Sen. Charles Schumer (D-NY), who had been scrambling for evidence that Chinatown buses were unsafe going back to 2005. The senator had told the New York Post at the time, “My daughter goes to college in Boston, and many of her friends ride these buses, and they said they were worried about them.” After the World Wide Travel crash, Schumer held a press conference calling the safety record of these “low-cost tour bus” companies “alarming” and demanded more regulation. Schumer and Rep. Nydia Velazquez (D-NY) sent a joint letter to the NTSB requesting that it conduct a broad study of bus safety.
Velazquez and Schumer also made clear what they wanted from the study. “There is ample evidence,” the lawmakers wrote the NTSB, “that the incident involving World Wide Tours [sic] is not an isolated incident but rather just one example of an industry that, in many cases, is operating outside the bounds of city, state and federal transportation safety guidelines.”
After six months, the NTSB released its report on October 31, 2011. Among the lead findings was the one that curbside bus companies were seven times more likely to be involved in a fatal accident than conventional bus operators—“an amazing statistic,” noted Sen. Schumer at a press conference announcing the report.
The "seven times" finding caught the attention of Aaron Brown, who has a background in statistics and applied math. He knew something was wrong. “The figure was very hard to accept,” says Brown. “It takes thousands of data points to reliably establish a number like seven times in this sort of study.” Realizing that fatal bus accidents are relatively rare, Brown doubted the researchers could have gathered enough data to publish such a number without major qualification.
So he obtained a copy of the study. “In situations like this, usually I find that there’s an asterisk in the study and the researchers are making a more guarded statement that’s been brushed aside by newspaper reporters,” says Brown. “But in this case, the report said the same thing as the news accounts.”