Education

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

Please check out our two other posts (here and here), which present summaries and discussions of the actual evidence on the relationship between unions and test scores.

For years, some people have been determined to blame teachers’ unions for all that ails public education in America. This issue has been around a long time (see here and here), but, given the tenor of the current debate, it seems to bear rehashing.  According to this view, teachers unions negatively affect student achievement primarily through the mechanism of the collective bargaining agreement, or contract. These contracts are thought to include “harmful” provisions, such as seniority-based layoffs and unified salary schedules that give raises based on experience and education rather than performance.

But a fairly large proportion of public school teachers are not covered under legally-binding contracts.  In fact, there are ten states in which there are no legally binding K-12 teacher contracts at all (AL, AZ, AR, GA, LA, MS, NC, SC, TX, and VA). Districts in a few of these states have entered into what are called “meet and confer” agreements about salary, benefits, and other working conditions, but administrators have the right to break these agreements at will. For all intents and purposes, these states are free of many of the alleged “negative union effects."

Here’s a simple proposition: If teacher union contracts are the problem, then we should expect to see higher achievement outcomes in the ten states where there are no binding teacher contracts.

So, let’s take a quick look at how states with no contracts compare with the states that have them.

Those who publicly advocate for the kind of education policies put forth in "Waiting for Superman" are now seeing the equivalent of a letter-high fastball down the middle. They can wait on it and crank it out, using the buzz created by a major motion picture to advance the movie’s/campaign’s arguments at face value. I'm a little late on this (at least by blog standards), but over at Fordham’s Flypaper blog, Mike Petrilli saw this fastball, yet instead of unloading, he sacrifice bunted the runner into scoring position for the good of the team.

Responding to an interview in which Davis Guggenheim, the film’s director, claims that charter schools have "cracked the code" on how to educate even the poorest kids, Petrilli warns against the hubris of thinking that we are anywhere beyond first steps when it comes to fixing urban schools. He points out that charters like KIPP benefit from selection effects (more motivated and informed parents seek out the schools), and that the degree to which these schools have actually "closed the gap" between poor and affluent schools has been somewhat oversold. Petrilli also notes that while some of these schools seem to have "cracked the code," there is still little idea of how to expand them to serve more than a tiny minority of poor kids.

Thoughtful comments like these should remind those of us who care about expanding quality education that, although we may have canyon-sized differences between us on what needs to be done (Petrilli claims that those who disagree with him are trying to "strangle" reforms "in their crib"), there may be a few important respects in which we are closer than we may appear. Still (in addition to the crib-strangling allegations), I would take issue with one of Petrilli’s central points – that charters like KIPP may have "cracked the code," and the main problem now is how to scale them up. From my perspective, the "code" is specific policies and practices that produce results. And on this front, we’re practically still using decoder rings from cereal boxes.

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City's schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains. 

The media campaign surrounding “Waiting for Superman," which has already drawn considerable coverage, only promises to get bigger. While I would argue – at least in theory – that increased coverage of education is a good thing, it also means that this is a critically important (and, in some respects, dangerous) time. Millions of new people will be tuning in, believing that they are hearing serious discussions about the state of public education in America and “what the research says” about how it can be improved.

It’s therefore a sure bet that what I’ve called the “teacher effects talking point” will be making regular appearances. It goes something like this: Teachers are the most important schooling factor influencing student achievement. This argument provides much of the empirical backbone for the current push toward changes in teacher personnel policies. It is an important finding based on high-quality research, one with clear policy implications. It is also potentially very misleading.

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.