Success Academy Posted Its Latest Test Scores. The Results Are Astounding.

New York's largest charter network outperformed traditional public schools in wealthy zip codes.

New York released its annual test scores this week, and Success Academy, the city's rapidly expanding charter school network, posted remarkable results. Again.

Success Academy schools did well in English—68 percent of students were proficient, compared with 30 percent in the city over all—but in math, the scores were astonishing. Ninety-three percent of Success Academy test-takers were proficient in math, compared with 35 percent citywide.

To put that into perspective, of the 1,282 public schools tested, just 12[*] were part of the Success Academy network, or 1 percent of the total. Yet 5 out of the 10 schools that scored highest in math were part of the Success Academy network. Of the 20 schools that did best in math, 9 were part of the Success Academy network. All twelve schools in the network were ranked in the top 40 for math. Results of this sort were unheard of before Success Academy arrived on the scene.

It's clear that Success Academy trounces the vast majority of public schools, but the rankings appear to show it dominating all but the very best. Are Success schools also markedly better than the other schools in the top five percent in New York City?

They are.

I asked statistician Aaron Brown to take a look at the data. He pointed out that a problem with the rankings is that "the chosen metric is so close to 100 percent for top schools that a big difference in ranking can represent a small difference in kids." So it could be that the quality of the education is really no better at Success Academy than at the other top schools.

To clarify what these results really mean, Brown crunched the numbers by putting all the Success Academy students, plus all the kids who go to the top five percent of other schools, in one pot. Then he took a random sample of 3,065 kids to see how they did compared to the 3,065 kids from Success Academy. This exercise is designed to answer the question: What if those Success Academy kids had been distributed among all of the top schools in the city? Would they have done better or worse in math?

Brown found that they would have performed significantly worse: 310 fewer kids would have gotten a level 4 on the exam, the top score. And 203 more kids would have received a 1 or a 2, which are considered to be failing grades. That would have been a tremendous waste of potential.

Of course, this assumes that all of these schools are drawing from the same pool. That's difficult to determine with any certainty, but we know that Success Academy students, who are selected through a random lottery, tend to come from disadvantaged backgrounds. Seventy-seven percent of its students qualify for free or reduced school lunch. Ninety-two percent of test takers were black or Hispanic.

It's fun to thumb through the data and see how Success schools trounced their counterparts in wealthy zip codes.

Two-bedroom apartments near Brooklyn's P.S. 321 tend to sell for more than a million dollars, in part because parents are desperate to get their kids into the famed Park Slope elementary school. At P.S. 321, 82 percent of test takes were proficient in math this year. That's respectable—56th highest in the city—but well below all 12 Success Academy schools.

At P.S. 87, which is P.S. 321's counterpart on Manhattan's Upper West Side (except the kids are even richer), 80 percent of students were proficient in math. That earned the school 67th place citywide, or well below every Success Academy school.

In an email, Brown noted selection bias might still partially explain Success' scores:

Any school will look good if its students are the brightest poor kids with the most parental support, and even affluent neighborhoods have some challenging kids and problematic parent situations. Still the overwhelming statistical performance of Success Academies versus even the best other New York public schools sets a high hurdle for anyone who wants to attribute the results to selection bias or test defects.

Let's not forget that when Mayor Bill de Blasio (D) was running for office, he told a teachers-union crowd that Success Academy's founder and CEO, Eva Moskowitz, must "stop being tolerated, enabled, supported."

Last year, I profiled Renee Lopez, a struggling student who was slipping through the cracks at a traditional public school. Then she won a coveted spot at Success Academy and got the help she needed.

For more on Success Academy's political fight with the mayor and the teachers union, watch Reason TV's profile of Eva Moskowitz, which I put together with Nick Gillespie:

[*] There are 32 schools in the Success Academy network, but the state combined 7 middle schools with their accompanying elementary schools in its data reporting. Twelve more schools haven't yet reached grade level to participate in the testing, and one school is a high school, which means students there take a different set of tests.