Mass Transit

Government Assault on the Chinatown Bus Industry Fueled By Bogus Federal Study

An influential report by the National Transportation Safety Board used the wrong data and committed "statistical malpractice."

|

Customers wait for a bus that will take them from New York City to Tampa for $90. ||| Credit: Jim Epstein
Credit: Jim Epstein

In 1997, Chinese-born entrepreneurs began regularly scheduled long-distance bus services that picked up passengers on the street. Tickets were priced so low that it was hard to figure how the operators could be breaking even, much less making a profit. Faced with declining market share, Greyhound and Peter Pan imitated the Chinatown model by teaming up to create a new venture called BoltBus. Then Coach USA got into the game with Megabus. Today, "curbside" buses—lines that begin and end their routes at the sidewalk as opposed to a traditional station—make up the fastest growing form of intercity travel in the U.S.

But over the past two years, the government has forced 27 bus companies based in Chinatown to close. The regulatory clampdown was fueled by a government study that found curbside carriers were disproportionately killing their passengers. Released by the National Transportation Safety Board, a federal agency, the study concluded that curbside bus companies were "seven times" more likely to be involved in an accident with at least one fatality than conventional bus operators. That finding was reported by The New York Times, the Los Angeles TimesBusinessweekUSA Today, the New York Daily NewsWNYC, and Reuters, among others. Although the study did not single out Chinatown bus companies the headline in Businessweek read, "Chinatown Buses Death Rate Said Seven Times That of Others."

The study is bogus. Not only is the "seven times" finding incorrect, the entire report is a mangle of inaccurate charts and numbers that tell us virtually nothing meaningful about bus safety. There's no evidence that curbside or Chinatown buses are any less safe than any other kind of bus.

How did the study authors figure curbside bus companies are "seven times" more prone to fatal accidents? For starters, they counted 37 accidents during the study period involving curbside buses in which there was at least one fatality. When I rebuilt the study data and contacted the companies involved, I found that, in 30 of those 37 accidents, curbside buses were not involved. In fact, 24 of those 30 misclassified cases involved Greyhound's conventional bus fleet. (Greyhound's curbside subsidiary BoltBus had no fatal accidents during the study period.)

The National Transportation Safety Board denied my requests for the study data, even though it was a taxpayer-funded report with an impact on policy. After my Freedom of Information Act request also failed to return the information following a six-month wait, I began reconstructing the study data from other sources.

Proceeding on the time-honored hunch that people who are hiding something have reason to do so, I generated a list of the 37 fatal crashes using a database obtained from a federal contractor that collects nationwide accident data. I analyzed that data with help of Aaron Brown, a quantitative analyst with the hedge fund AQR Capital Management. Brown was the first to point out major flaws in the NTSB's methodology in an article published by Minyanville.com, accusing the study authors of "statistical malpractice." I also consulted with Ed George, a professor of statistics and department chair at the University of Pennsylvania's Wharton business school, who examined the study for the purposes of this article.

A Chinatown bus headed from New York City to Washington, D.C. ||| Jim Epstein
Jim Epstein

"When I first read the NTSB report, I thought this is just terrible statistics," says Brown. "But it goes way beyond that. It's almost as if someone took some random data and shook it together."

The NTSB study grew out of a horrific March 2011 bus crash in the Bronx that killed 15 people. The accident, involving a bus company called World Wide Travel, was both a tragedy and an anomaly. From 2001 to 2011, there were an average of 34 fatal intercity/cross-country bus accidents each year. During the same period, there were an average of 23,000 fatal passenger car accidents annually. For every mile traveled, passengers are three times as likely to die when riding in a car than in a bus (of any sort), according to data obtained from the National Highway Traffic Safety Administration. [*]

One reason the World Wide Travel crash led to a broad study of industry safety is that it was a "Chinatown bus," according to a slew of media reports (see here, here, and here). Today, companies owned and operated by Chinese immigrants make up only a fraction of the industry, but "Chinatown bus" is often misused as a blanket term for all curbside carriers, particularly when safety is in question. The owner of the company wasn't Chinese, nor was the bus driver. But when the accident occurred the bus was destined for New York City's Chinatown to drop off its passengers, so it was a "Chinatown bus."

Enter Sen. Charles Schumer (D-NY), who had been scrambling for evidence that Chinatown buses were unsafe going back to 2005. The senator had told the New York Post at the time, "My daughter goes to college in Boston, and many of her friends ride these buses, and they said they were worried about them." After the World Wide Travel crash, Schumer held a press conference calling the safety record of these "low-cost tour bus" companies "alarming" and demanded more regulation. Schumer and Rep. Nydia Velazquez (D-NY) sent a joint letter to the NTSB requesting that it conduct a broad study of bus safety.

NTSB Chairman Deborah Hersman, Sen. Charles Schumer, and Rep. Nydia Velazquez announce the results of the NTSB's study of curbside busing, October 31, 2011. |||

Velazquez and Schumer also made clear what they wanted from the study. "There is ample evidence," the lawmakers wrote the NTSB, "that the incident involving World Wide Tours [sic] is not an isolated incident but rather just one example of an industry that, in many cases, is operating outside the bounds of city, state and federal transportation safety guidelines."

After six months, the NTSB released its report on October 31, 2011. Among the lead findings was the one that curbside bus companies were seven times more likely to be involved in a fatal accident than conventional bus operators—"an amazing statistic," noted Sen. Schumer at a press conference announcing the report.

The "seven times" finding caught the attention of Aaron Brown, who has a background in statistics and applied math. He knew something was wrong. "The figure was very hard to accept," says Brown. "It takes thousands of data points to reliably establish a number like seven times in this sort of study." Realizing that fatal bus accidents are relatively rare, Brown doubted the researchers could have gathered enough data to publish such a number without major qualification.

So he obtained a copy of the study. "In situations like this, usually I find that there's an asterisk in the study and the researchers are making a more guarded statement that's been brushed aside by newspaper reporters," says Brown. "But in this case, the report said the same thing as the news accounts."

In a public docket accessible online, there is a list of the 71 companies that the NTSB classified as "curbside" and the 51 companies classified as "conventional."

These lists are jaw dropping. Greyhound and Peter Pan—the two most iconic conventional bus lines—were

A woman and her son wait for a bus from New York City to Tampa ||| Jim Epstein
Jim Epstein

categorized as "curbside" carriers.[**] It's as if a major study of the restaurant industry had classified McDonald's and Burger King as leading outdoor food trucks. Also, lesser-known carriers like Martz Trailways and Fullington Trailways are on the list. I spoke with officials at both those companies, who confirmed they are conventional bus carriers without any curbside lines.

On the "conventional" carrier list, Hampton Jitney is listed. Apparently the study authors never took part in the New York City summer ritual of getting picked up on the streets of the Upper East Side for a weekend on Long Island's beaches. (A call to Hampton Jitney confirmed that the company has no lines that pick up or drop off at a conventional station.)

The largest company on the conventional list is New Jersey Transit, a statewide public transit system, with its 2,172 buses. If public transit systems meet the study criteria, why stop with New Jersey Transit?[***]

The NTSB report states that during the study period these 71 curbside companies had 37 accidents with at least one fatality, and with a total of 52 fatalities.[****] I correlated the list of companies with federal accident data and came up with almost the same results: 37 fatal accidents and 51 fatalities. (My list is available here.)

Then I called every "curbside" company on the list that had experienced a fatal accident, including Greyhound, which was responsible for 24 of those accidents. Nearly every company I reached was not a curbside operator. I found that 30 of the 37 accidents that the NTSB classified as involving curbside buses did not involve a curbside bus. This alone invalidates almost all the study's findings. But that's just the beginning.

The study reported that curbside carriers had a fatal accident rate of 1.4 per 100 buses, while conventional carriers had a rate of 0.2 per 100 buses. Since 1.4 is seven times greater than 0.2, that's how "seven times" more got reported. Since the numerator 1.4 comes from the number of fatal accidents tallied at 37, we already know the calculation is wrong. But what about those "100 buses" the study put in the denominator?

Passengers wait for a bus from New York City to Washington, D.C. ||| Jim Epstein
Jim Epstein

Some press write-ups (see USA Today, Reuters, and the Los Angeles Times) naturally assumed that the study meant that curbside buses were seven times more prone to fatal accidents. To arrive at that figure, study authors would have had to add together all the buses operated by curbside companies. In fact, had the NTSB calculated the results in this way, the data would have shown essentially no difference between the fatal crash rates of curbside buses and conventional buses, even assuming the study had not mistakenly attributed those 30 extra accidents to curbside buses.

But as Aaron Brown first surmised and as the NTSB confirmed in an email, the study authors took a different approach which was misleading. They calculated the fatal accident rate for each bus company and then averaged together the company rates without taking into consideration the size of each company.

This would not have been such a problem had the number of buses operated by each company been about the same. But that wasn't at all the case.

In practice, Greyhound—let's say for a moment that the study was right in calling it a curbside bus company—had 1,515 buses and 24 accidents. Another bus company on the list, Sky Horse Bus Tour, had one bus and one fatal accident. The two companies were given equal weight. As a counterfactual, let's assume that Sky Horse's single accident hadn't occurred, but Greyhound had had 1,515 fatal accidents instead of 24. The NTSB would have come up with the same "seven times" finding. It's as if a rookie baseball player with three at bats and one hit received the same ranking as a starter with 600 at bats and 200 hits.

Even if the NTSB's method of ignoring company size told us something—and even if the accident data weren't wrong—the NTSB's "seven times" finding would still have little meaning because it doesn't achieve what researchers call statistical significance.

NTSB, "Report on Curbside Motorcoach Safety," p.45, fig. 9|||

The chart used to arrive at the "seven times" finding is pictured to the right. There are two vertical lines in the middle of each measure called error bars. They're drawn according to a "95% confidence interval," a standard measure of statistical significance. The bars completely overlap. This means the results could have easily occurred purely by chance. A research journal with any standards would have flagged this finding as inconclusive and not fit for publication. Instead, the NTSB promoted this number to reporters without mentioning how little it actually means.

"The key to statistical analysis is that it is innocent until proven guilty," says University of Pennsylvania Wharton School statistician Ed George, who examined the NTSB study for the purposes of this article. "You would start with the assumption that there's no difference in the safety rating of curbside and conventional bus companies. Then you look for persuasive evidence otherwise. The error bars overlap in this chart, so there is not persuasive evidence."

Other problems with the NTSB study abound. The agency had no data on miles traveled, generally a key measure in any analysis of transportation safety. And the study is derived from a federal data set known for its errors and omissions because it relies on local law enforcement agencies to voluntarily report data only every two years. The study acknowledges these limitations, but the press release didn't mention them.

When I emailed NTSB press officer Eric Weiss with all the problems with the study and asked for an explanation, he offered this response: "The NTSB stands by its report."

Ironically, the regulatory crackdown that followed the release of this dreadful study may lead to more highway deaths. The shrinking of the industry has caused bus ticket prices to spike considerably in the last year. That means fewer travelers will be enticed to leave at home a far greater threat to their safety, which is sitting in their driveways.

[*] Intercity buses are even safer than the "three times" statistic implies, since it was derived from data that includes school buses and transit buses. Seventy-two percent of fatal bus accidents involve school buses and transit buses, and only 3.4 percent involve intercity buses, according to a study by the University of Michigan Transportation Research Institute.

[**] As mentioned, Greyhound and Peter Pan do have a jointly owned subsidiary, BoltBus, but it had a fleet size of just 92 buses in 2011. The NTSB's list of curbside carriers states that Greyhound has a fleet size of 1,515 buses. According to company press officer Tim Stokes, the 1,515 number may include BoltBus, but primarily refers to the parent company, which is not a curbside carrier.

[***] If New Jersey Transit, a public agency, meets the study criteria as a conventional bus company, others that should have been included are the RTD bus system in Denver, the Massachusetts Bay Transit Authority, which serves the Boston area, and Metro, which buses commuters between Washington, D.C., Virginia, and Maryland.

[****] The study data does not include the March 12, 2011 World Wide Travel accident because "due to lags in data reporting it was not yet a part of the database," according to the NTSB.

NEXT: Tanning Beds Could Soon Come With Warning Labels Thanks to the FDA

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. Even if the seven times figure were correct, so what? Without knowing the actual risk, that figure is meaningless. If I have a one in a million chance of dying on a greyhound bus trip and a seven in a million chance of dying on a Chinatown bus trip, the Chinatown bus is strictly speaking seven times more likely to produce a casualty. But since the risk was so small to begin with, even a seven fold increase is not significant.

    1. Start working at home with Google! It’s by-far the best job Ive had. Last Wednesday I got a brand new BMW since getting a check for $6474 this – 4 weeks past. I began this 8-months ago and immediately was bringin home at least $77 per hour. I work through this link, http://www.Mojo50.com

    2. While I agree in principle, if your risk of dying on a bus trip were one in a million, much less seven in a million, it would make busses much riskier than cars. You may still consider that insignificant, of course, but I think it would affect most people’s calculations. After all, at least some people take car safety seriously, and the difference between a safe car and a less safe car is two orders of magnitude less than your example difference between bus rates.

      1. up to I looked at the bank draft that said $5552, I be certain that my mom in-law truley making money parttime at there labtop.. there brothers friend has been doing this 4 only about 17 months and just now paid for the morgage on there mini mansion and got a great Volkswagen Golf GTI. read more at wow65.com
        (Go to site and open “Home” for details)

  2. What a surprise that Chuckles Schumer is involved in this.

  3. These fucking foreign types come here and just think they can start a business and be free?

    1. It’s difficult to notice, but at the bottom of the poem at the statue of liberty it says (in small print), “PSYCH, SUCKERS”

  4. Must-read of the day. As a frequent user, and supporter, of curbside buses, I thank you for this. F*ck Chuck Schumer.

  5. Very, very, good takedown of some very, very bad research. Good job, Mr. Epstein.

    1. Seconded. Great job and great writing.

      I’d almost (almost!) be willing to pay more in taxes to have every single government study subjected to the same kind of review, with the proviso that the agencies responsible for bad studies had their budgets chopped in proportion. 10 bad studies out of 100, 10% budget hit. I bet it would be closer to 100 bad studies out of 100.

      1. Great research, not so great copy editing. “Average” is singular, and the last sentence in the article has a spurious comma which implies that sitting in your driveway is more dangerous than traveling. Plus a few other mistakes of that nature.

  6. How much $$ did Greyhound put into this “study”? How much influence did they exert over the results?

    I bet if you dug in, you would find “traditional” bus services had some part in this. They have to protect their monopoly because their overhead is so much higher than the curbside buses’.

    1. Greyhound and Amtrack. Why the hell take the train to New York, when you can take the Funwa for $30 and get there just as quick?

      1. Novelty.

        1. That is about it. You either have money to burn or you just like trains.

          1. Yep. Megabus is like 2 dollars if you book it in advance.

            Or it was a couple years ago.

          2. Basically – except for a few profitable commuter lines most of Amtracks ridership seems to be old retirees who like trains.

            1. Even the profitable commuter lines probably aren’t actually profitable. Amtrak likes to count maintenance as a capital expense, among other irregularities that would get a non government accountant jailed. But, remember, Highspeed rail is the future!

      2. Bar car.

        1. Good point. But you can bring your own booze on the bus. And it is a lot cheaper than what Amtrak charges.

          1. But you can bring your own booze on the bus.

            Wouldn’t that violate public consumption and open container laws?

            1. Maybe. But I have never had an issue with it. Just bring a flask and some mixer. No one bothers you.

              1. Mixer?

                Pussy.

    2. How could it be proved if there were?

      Direct donations by corporations are limited, so you would have to trace the money through PACs.

      I wonder if Greyhound’s workforce is unionized. It could be the reason the curbside busses were able to offer steep discounts is because their drivers aren’t unionized and greyhound’s are.

      Also the public transit systems that these busses sometimes compete with are probably unionized.

      1. I don’t think Greyhound is unionized. There was a huge kerfuffle years ago when they reorganized and started paying their drivers minimum wage.

  7. Great article!
    This is what journalism should really be about. Real research uncovering government lies and corruption and rent-seeking corportation using inside political connections to suppress competition. Bravo!

  8. Forget the press officer, contact the board members directly.

  9. Greyhound is a statistically insignificant killer.

    Best Chinatown bus experience: a lady asked the driver where we were and he shouted back “SHU UP, SHU UP!”

    1. Hi Raston Bot, That’s nothing! The other day I was at the bank, trying to cash my welfare check from Government Almighty, and this Chinese lady, she’s changing her Yen or Ming-Mang or Confusious Deutch-Marks or Ying-Yang or whatever it is that they have in her native UN-AMERICAN homeland, to Our Precious American Dollars that Are Derived from Our Precious Bodily Fluids, but I diverge? In Any case, she wants American dollars, and they give her less than she’s expecting, she says, more or less. “What gives?!?! Yetsterday I gotta so-and-so, now I get SOOO much less than so-and-so!!!” ? They says, “Fluctuations”. She says, “Well, FLUCK you white people too, then!!!”

  10. Chuck Schumer dishonest? Unpossible.

  11. Government-funded studies should have to be subjected to peer review before they can be published or used as a basis for policy.

    If we had this simple change it would make it much harder for rent-seeking interests to manipulate policy.

    They should be subjected to the same independent review process that any other scientific study has to go through.

    1. Unfortunately, peer review doesn’t help much (although it would have caught this one). It’s not someone validating the data and calculations, it’s some people in the field reading it over for general quality and significance. Plus, it’s done by peers, so government transportation employees at the Federal Highway Administration would review NTSB, and vice versa. They’re not likely to expose each other.

  12. I remember when those cheap Chinatown buses got popular, there was a supposed Mafia (Tong?) war over who ran what lines..

    http://www.washingtontimes.com…../?page=all

    1. Probably was. I don’t care what Chinese Mafia family runs it. I just want my $25 one way to New York.

      1. Why would you want a one way ticket to NYC? Or any ticket there?

  13. There are lies, damn lies and statistics.

    1. Nope. That’s not what happened here. They didn’t choose a pie graph instead of a bar graph to make it look worse, they just straight up pulled the numbers out of their ass. Please don’t try to cheapen what is nothing short of amazing reporting that we rarely see anymore.

  14. Reporting insignificant differences is the worst malpractice. If we could only get Suderman to stop doing it.

  15. My daughter is going to college in Philly in the fall, buy she’s also a big fan of NYC and Broadway. I have no problem with her grabbing the Chinese bus for $17.

  16. How has there not yet been a joke about Asians being bad drivers?

    1. I’m pretty sure Chuck Schumer banned those jokes.

  17. “The NTSB stands by its report.”

    translation; “The lobbyist’s check cleared, and you aren’t somebody scary with a national audience.”

    It is hard to believe that the NTSB is worth the oil that will be necessary to fry it in Hell.

  18. Major credit to Jim Epstein and Reason. Thanks again for the very best in journalism.

  19. til I saw the draft ov $6125, I did not believe that…my… sister could trully erning money part time on their apple laptop.. there friend brother haz done this for only about eight months and resantly cleard the morgage on their house and bought a new Chevrolet. we looked here,… http://www.up444.com

  20. So the same morons who use “fatalities per passenger-mile” for airlines can’t use the same meter stick for buses? Morons!

  21. You don’t seem interested in knowing why the NTSB put out such a report. While yes, it’s true that in part it’s because they’re lazy and inept, that’s not the whole story. What differentiates Greyhound (which they surely knew was NOT a curbside carrier) and NJ Transit as opposed to Chinatown buses? *Ding*Ding*Ding* You got it! UNIONS. The whole thing was an excuse to crack down on a more efficient and less expensive free market capitalist solution that was making life more difficult for union-dominated bus companies. The NTSB may have been perfectly capable of rigorous statistical analysis, but that wasn’t the point. The point was making the data show that they had a justification to crack down on Chinatown bus to help out their union friends.

  22. I wonder why Mr. Epstein didn’t sue for violation of FOIA? Failing to respond for six months… sounds like an easy case and attorneys’ fees are available. Nothing is better than sticking it to some branch of government for willfully violating the open-records act. Bloomberg has been doing quite a bit of this.

    The report is shameful. But unsurprising.

  23. Should not have to redo the study. Let me choose which bus I want to ride. Let me decide if I even care about safety – perhaps I’m rather invested in making this trip, safety be damned – perhaps I think I can spot a threat as well or better than a professor.

    The State does have a role in facilitating commerce and safety is part of it, within reason. Yet in my opinion, the regulations get weird and counter-productive precisely when the regulators start thinking they are doing someone a favor. How is doing something I never asked for, and with my money, doing me a favor? I believe I may be able to help my Country; my country can’t help me, beyond by just being my country of relative peace and freedom.

Please to post comments

Comments are closed.