In an article in this week’s New York Times Magazine, author Paul Tough notifies supporters of market-based reform that they cannot simply dismiss the "no excuses" maxim when it is convenient. He cites two recent examples of charter schools (the Bruce Randolph School in Denver, CO, and the Urban Prep Academy in Chicago) that were criticized for their low overall performance. Both schools have been defended publicly by "pro-reform" types (the former by Jonathan Alter; the latter by the school’s founder, Tim King), arguing that comparisons of school performance must be valid – that is, the schools’ test scores must be compared with those of similar neighborhood schools.

For example, Tim King notes that, while his school does have a very low proficiency rate – 17 percent – his students are mostly poor African-Americans, whose scores should be compared with those of peers in nearby schools. Paul Tough’s rejoinder is to proclaim that statements like these represent the "very same excuses for failure that the education reform movement was founded to oppose." His basic argument is that a 17 percent pass rate is not good enough, regardless of where a school is located or how disadvantaged are its students, and that pointing to the low performance of comparable schools is really just shedding the "no excuses" mantra when it serves one’s purposes.

Without a doubt, the sentiment behind this argument is noble, not only because it calls out hypocrisy, but because it epitomizes the mantra that "all children can achieve." In this extreme form, however, it also carries a problematic implication: Virtually every piece of high-quality education research, so often cited by market-based reformers to advance the cause, is also built around such "excuses."

Among the most important guiding principles in education research – at least most of the work that receives heavy attention in our education policy debates – is that student performance on tests (or other outcomes) must be assessed by comparing it with that of similar students in similar schools and districts. That is why, for example, high-quality charter studies take great pains to ensure that all measurable student characteristics and other meaningful variables are controlled for, using sophisticated statistical models and, when possible, random assignment.

These techniques entail the very same principle – accounting for poverty, race and other factors – that Mr. Tough regards as an "excuse." But the sad fact is that, by his logic, there are only a minuscule number of successful schools in the nation’s most impoverished areas, whether charter or any other type. Indeed, the research is very clear that, on average, poor children enter school with an academic disadvantage, and that most schools don’t have either the resources or the know-how the to close this gap. Some of the nation’s best researchers are working to capture differences in student and school performance, precisely to pinpoint the things that may help to accelerate learning in such schools. Is all of their research just making excuses when it attempts to account for these demographic differences?

One might object to this line of argument by noting that most of these studies focus on growth, rather than the absolute performance levels (e.g., proficiency rates) that Mr. King and Mr. Alter discuss. But I would assert that the very concept of growth itself is threatened by this extreme "no excuses" perspective. If we cannot defend or praise a given school so long as its performance remains low, then lower-scoring schools that seem to produce rapid test score growth are, at best, less of a failure than those that don’t. So, in this sense, anyone who praises KIPP as a success, based on their students’ rapid testing progress, is tolerating failure and making excuses.

This same logic applies to the research on teacher effects. The value-added literature is entirely predicated on comparing students’ achievement gains between classrooms with similar students attending similar schools. Put simply, the models set different growth "expectations" for different students, based on how students with similar characteristics typically perform. If we follow Mr. Tough’s argument faithfully, it would be little more than an "excuse" to use variables such as income, race, and prior achievement to set these expectations. Thus, a huge number of low-performing students would just have to be regarded as abject failures, as would their teachers and schools. And the entire value-added enterprise is based on "excuses," because it sets "expectations" for testing growth by controlling for student characteristics.

Look, "no excuses" is a noble philosophy, but in this extreme form, it assumes an ideological character that borders on the absurd (see here for another example). If a school’s founder cannot point out that his low pass rates are comparable to those of similar schools in his area, then we have reached a point in our education debate where most of how we assess performance, and the great empirical research that does so, is founded on tolerating failure. Those who endorse this perspective must think three times before they use any of the research studies – of charter schools, teacher effectiveness, etc. – to advocate for their policy preferences.

I think there’s room for both views. Poverty should never be an excuse for failing to improve terrible schools, but there's no shame in acknowledging that low-scoring schools are dealing with persistent, powerful obstacles. Those making rapid progress can still be praised and defended, even if their absolute performance levels are lower than those of schools in more affluent areas. And, finally, it is important - indeed, imperative - to make proper comparisons in the field of policy analysis, as well as in policy debates. There's a difference between excuses and explanations.

- Matt Di Carlo