Waiting For Methodology Man
I finally saw the entire “Waiting for Superman” movie last weekend, in a mostly-empty Georgetown theatre. I went with my mother, not just because she’s a great public school teacher, but also because I needed someone to comfort me while I watched.
We both had strong reactions to dozens of things about the film, and you almost have to admire the chutzpah. It is about education – with a primary focus on teachers – and includes sit-down interviews with superintendents, parents, students, businessmen, economists, and journalists, but not one teacher.
Given all the attention that has already been lavished on it, I’ll discuss just one other thing that struck me, one which I keep hearing elsewhere. There is exactly one sentence in the whole film in which director Davis Guggenheim addresses the research on charter school effects beyond the anecdotal evidence that dominates his narrative. He notes, “Only one in five charters is excellent," with the implication that these charters show that it can be done.
He is presumably referring to the CREDO study released last year, which is the largest (15 states plus D.C.) and arguably the most overplayed charter analysis in history (for other multi-state studies showing no charter effects, see here, here, here, and here). The CREDO authors understandably framed their results in a “media-friendly” manner – by reporting the percentage of charters that did better than comparable regular public schools (17 percent), along with the proportion that did worse (37 percent).
My first point is that 17 percent is equivalent to one in six, not one in five. But beyond that, some charter advocates have taken the remarkable step of turning the finding that twice as many charters do worse than regular publics into “evidence” that the former should be expanded. The rationale is, as Guggenheim puts it, that these “one in five” charters are “excellent," and if we can increase that proportion, we can fix our public education system. There is only one problem: That’s not what the study says. Guggenheim is either deliberately misleading his viewers or, more likely, just doesn’t know what he’s talking about.