Revisiting The CREDO Charter School Analysis

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Most people involved in education policy know exactly what you mean when you refer to “the CREDO study." I can’t prove this, but suspect it may be the most frequently mentioned research report over the past two years (it was released in 2009).

For those who haven’t heard of it (or have forgotten), this report, done by the Center for Research on Education Outcomes (CREDO), which is based at Stanford University, was a comparison of charter schools and regular public schools in 15 states and the District of Columbia. Put simply, the researchers matched up real charter school students with fictional amalgamations of statistically similar students in the same area (the CREDO team called them “virtual twins”), and compared charter school students’ performance (in terms of test score gains) to that of their “twins." The “take home” finding – the one that everybody talks about – was that, among the charter schools included in the analysis, 17 percent had students who did better on the whole than their public school twins, in 37 percent they did worse, and in 46 percent there was no statistical difference. Results varied a bit by student subgroup and over time.

There are good reasons why this analysis is mentioned so often. For one thing, it remains the largest study of charter school performance to date, and despite some criticism that the "matching" technique biased charter effects downward, it was also a well done large-scale study (for a few other good multi-state charter studies, see here, here, and here). Nevertheless, as is so often the case, the manner in which its findings are discussed and understood sometimes reflect a few key errors of interpretation. Given that it still gets attention in major media outlets, as well as the fact that the CREDO team continues to release new state-specific reports (the latest one is from Pennsylvania), it makes sense to quickly clear up three of the most common misinterpretations.

The New Layoff Formula Project

In a previous post about seniority-based layoffs, I argued that, although seniority may not be the optimal primary factor upon which to base layoff decisions, we do not yet have an acceptable alternative in most places—one that would permit the “quality-based” layoffs that we often hear mentioned. In short, I am completely receptive to other layoff criteria, but insofar as new teacher evaluation systems are still in the design phase in most places, states and districts might want to think twice before chucking a longstanding criterion that has (at least some) evidence of validity before they have a workable replacement.

The New Teacher Project (TNTP) recently released a short policy brief outlining a proposed alternative. Let’s take a quick look at what they have to offer.

The Un-American Foundations Of Our Education Debate

Being from Spain, one of the first things that struck me as odd about the U.S. education debate was the ubiquitous depiction of “bad teachers” as the villains of education and “great teachers” as its saviors. Aside from the fact that this view is simplistic, the punish/praise-teachers chorus seemed particularly off-key—but I wasn’t sure why. I think I may have figured it out. I think that it may be un-American.

Let me explain. This is a nation that is supposed to be built around specific core values, such as individual effort, hard work, and taking responsibility for one’s own actions. If so, isn’t the fixation on teachers—to the seeming exclusion of students and parents—an indirect rejection of basic American principles?

This is not a discussion of what the good/bad teacher doctrine misses —we know it misses numerous dimensions of the education enterprise—but rather, what this doctrine assumes and how these assumptions conflict with the values that one expects most Americans to hold.

One problem with the narrow focus on teachers is that it views students exclusively as passive recipients of their own learning. Not to get too technical here, this goes back to a central question in the social sciences: namely, agency versus structure. Agency refers to the capacity of individuals to act independently and make their own choices. Structure refers to the conditions that shape and perhaps limit the range of alternative choices that are available. Western culture tends to favor agency over structure as an explanation for actions, a view which one would think would run particularly deep in the U.S.

The High Cost Of Closing Public Libraries

Government budget cuts, at all levels, can have tragic effects. It will take us a long time to recover from the damage the current cuts have done and will do. There are many vital public services – such as health care, aid to the homeless, and schools – that we must do our utmost to protect. But, at least for me, there are few cuts more bothersome than the closing of public libraries.

Sadly, these closings are happening all over the nation, including New York, Ohio, Michigan and elsewhere.

At the same time, use of libraries has been increasing for years. In 2008, according to the Institute of Museum and Library Services, the average person visited a public library 5.1 times, an increase of almost 20 percent since 1999. Of course, this use is not equally distributed – some people visit regularly, while others not at all.

In part, this is because many low-income Americans rely on libraries, not only for books and periodicals, but as their primary source of internet access. As a result, the number of computers in public libraries has almost doubled since 2000.

Let’s do some simple, illustrative math here.

A Big Fish In A Small Causal Pond

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

In three previous posts, I discussed what I’ve begun to call the “trifecta” of teacher-focused education reform talking points:

In many respects, this “trifecta” is driving the current education debate. You would have trouble finding many education reform articles, reports, or speeches that don’t use at least one of these arguments.

Indeed, they are guiding principles behind much of the Obama Administration’s education agenda, as well as the philosophies of high-profile market-based reformers, such as Joel Klein and Michelle Rhee. The talking points have undeniable appeal. They imply, deliberately or otherwise, that policies focused on improving teacher quality in and of themselves can take us a very long way - not all the way, but perhaps most of the way - towards solving all of our education problems.

This is a fantasy.

Straight Up, Between The Lines

Read carefully between the lines in Rick Hess’ recent blog post, “Can the Common Core Coalition Keeps [sic] Its Finlandophiles in Check?"

Predicting a “fifty-fifty chance that the Common Core effort will dissolve into an ideological clash," Hess writes that in “one short document, the Shankerites managed to do much to undermine the loose confederation that had supported the Common Core." He also lumps a broad spectrum of signatories into one supposedly errant educational faction. People such as former U.S. Surgeon General M. Joycelyn Elders, former U.S. Secretary of Education Richard Riley, Reagan appointee Checker Finn, George H.W. Bush appointee Charlie Kolb, George W. Bush appointee Susan B. Neuman, former Lockheed Martin CEO Norman Augustine, and the National Council on Teacher Quality’s Kate Walsh — all are labeled as being “a slew of left-leaning academics and consultants, dotted with my pal Checker Finn and a few long-retired Republican governors”—and the whole crew is charged with being “Finlandophiles." God forbid.

What’s going on here? What have we wrought with the Albert Shanker Institute’s "A Call for Common Content?"

I think Hess is doing more than cooking up a soup of crocodile tears and polemics. He’s clearly uncomfortable with the direction that the Common Core State Standards (CCSS) will take education in this country, but he's not willing to take a clear position either way. Given his dilemma, attacking a sound strategy for implementing the standards seems like little more than undermining them without the political risk of having to register a truly “straight up” objection. And this is not the first time he has attempted to evoke tensions among potential supporters of the Common Core standards. I cannot help but suspect that he has made up his mind, but can't quite bring himself to say so.

College For All, Profit For Some

The ideal of "College for All”—usually interpreted as meaning the acquisition of a four-year degree—is every bit as noble as it is unattainable, at least judging from actual graduation rates. It is within this tension that for-profit colleges wish to live—a kind of pseudo knight in shining armor riding gallantly into the battle for equal opportunity. But too many for-profit colleges (a.k.a., career colleges) are not solving educational issues. Rather, they are perpetuating inequalities and obscuring the fact that what is preached (e.g., “College for All”) has nothing to do with what gets achieved.

Many have pointed out that, by enshrining a path so few end up traveling (to say nothing of completing), we may be doing a great disservice to our youth. This argument is loud and clear; what may not be totally obvious is the variegated ways in which this constitutes a disservice. By idealizing the B.A./B.S. path, not only are we discouraging young people from exploring equally valid post high-school options, but we inadvertently may have also made them more vulnerable to the allure of disreputable for-profit colleges and/or encouraged for-profits to exploit this vulnerability.

As a matter of fact, one consequence (unintended, I am sure) of the “College for All” ideal may have been to widen the niche for for-profit career colleges. I am hardly the first to point out that the worst career colleges sell fake dreams by arm-twisting and sweet-talking potential students into taking out unsustainable—often federally-subsidized—loans for products of uncertain value. For-profit colleges did not create this dream. We did. They have only done what we would expect a for-profit entity to do: Exploit it.

Teacher Quality Is Not A Policy

I often hear the following argument: Improving teacher quality is more cost-effective than other options, such as reducing class size (see here, for example). I am all for evaluating policy alternatives based on their costs relative to their benefits, even though we tend to define the benefits side of the equation very narrowly - in terms of test score gains.

But “improving teacher quality” cannot yet be included in a concrete costs/benefits comparison with class size or anything else. It is not an actual policy. At best, it is a category of policy options, all of which are focused on recruitment, preparation, retention, improvement, and dismissal of teachers. When people invoke it, they are presumably referring to the fact that teachers vary widely in their test-based effectiveness. Yes, teachers matter, but altering the quality distribution is whole different ballgame from measuring it overall. It’s actually a whole different sport.

I think it is reasonable to speculate that we might get more bang for our buck if we could somehow get substantially better teachers, rather than more of them, as would be necessary to reduce class sizes. But the sad, often unstated truth about teacher quality is that there is very little evidence, at least as yet, that public policy can be used to improve it, whether cost-effectively or otherwise