Shanker Blog

Persistently Low-Performing Incentives

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City's schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains. 

Research Wars

Weeks before the fact, a Sept. 29 forum sponsored by the Economic Policy Institute and the National Education Policy Center has sparked some interesting debate over at the National Journal. The event, centered around the recent book, Think Tank Research Quality: Lessons for Policy Makers, the Media and the Public, is an effort to separate "the junk research from the science."

The crux of the debate is whether the recent explosion of self-published reports by various educational think tanks has helped or hindered the effort to improve the quality of educational research. (Full disclosure: The Albert Shanker Institute is often called a "think tank" and we frequently self-publish.) The push and pull of dueling experts and conflicting reports, say some, has turned education research into a political football—moved down the field by one faction, only to be punted all the way to the other end by a rival faction—each citing "research" as their guide.

"My research says this works and that doesn’t," can always be countered by, "Oh yeah, well my research says that works and this doesn’t." There are even arguments about what "what works" means because, except for performance on standardized tests, our goals remain diverse, decentralized and subject to local control. As a result, public education is plagued by trial and error policies that rise and fall, district by district and state by state, like some sort of crazed popularity contest.

Teacher Quality On The Red Carpet; Accuracy Swept Under The Rug

The media campaign surrounding “Waiting for Superman," which has already drawn considerable coverage, only promises to get bigger. While I would argue – at least in theory – that increased coverage of education is a good thing, it also means that this is a critically important (and, in some respects, dangerous) time. Millions of new people will be tuning in, believing that they are hearing serious discussions about the state of public education in America and “what the research says” about how it can be improved.

It’s therefore a sure bet that what I’ve called the “teacher effects talking point” will be making regular appearances. It goes something like this: Teachers are the most important schooling factor influencing student achievement. This argument provides much of the empirical backbone for the current push toward changes in teacher personnel policies. It is an important finding based on high-quality research, one with clear policy implications. It is also potentially very misleading.

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.

More Than One Way Of Winning

Our guest author today is James R. Stone, professor and director of the National Research Center for Career & Technical Education at the University of Louisville.

In recent years, a consensus has emerged among education researchers and policymakers that all students should graduate from high school both "college- and career-ready." President Obama has made this part of his education agenda. And numerous advocacy organizations have championed the notion. But what does the phrase actually mean?

"College-ready" usually means not needing remedial courses once in college, and "career-ready" is usually equated with college-ready. High standards and expectations are the means recommended to prepare college-ready graduates. This means rigorous courses aligned with standards, and tests to ensure that students meet those standards. Presumably, career-readiness comes with the same requirements. The evidence contradicts the rhetoric, however. Paul Barton at ETS, Peter Cappelli at the Wharton School, and other labor market experts argue that being prepared for college is not the same as being prepared for a successful transition into the workforce.

Perhaps we ought to consider an alternative framework that more clearly defines what college- and career-ready means.

Extra Time: More From The Magazine's Education Poll

** Also posted here on "Valerie Strauss’ Answer Sheet" in the Washington Post.

A recent education poll conducted by Time Magazine has gotten a lot of attention. Many of the questions are worded so badly that the results are rather meaningless. The question on merit pay, for example, defines the practice as “paying teachers according to their effectiveness” (who would oppose that, if it could be accurately measured?). Other questions are very interesting, such as the one asking whether respondents would pay higher taxes to improve public schools (56 percent would). Or the finding that, when asked what will “improve student achievement the most," more than twice as many people choose “more involved parents” (54 percent) over “more effective teachers” (24 percent).

But, as is sometimes the case, a few of the survey’s most interesting results were not included in the published article, which highlighted only 11 out of 40-50 or so total questions (the full set of results is available here). Here are three or four unpublished items that caught my eye (the sample size is 1,000, with a margin of error of +/- 3 percent):

Labor Day In Hell

The new Albert Shanker Institute-supported report, The Global State of Workers’ Rights: Free Labor in a Hostile World, released on Labor Day by the human rights organization Freedom House, has received some notable attention in the press, both here and around the world. One photo essay in Foreign Policy, titled "Labor Day in Hell," illustrates 14 of the worst-offending nations, among them Belarus, North Korea, and Sudan (see the screenshot below).

Indeed, the report, which examined the state of labor rights in the world for the year 2009, found serious violations of workers’ freedoms in all parts of the world except Western Europe.　Countries were ranked on a five-category scale of Free, Mostly Free, Partly Free, Repressive, and Very Repressive.

The United States was rated as Mostly Free—the same rank accorded to Bolivia, Mongolia, Romania, and Zambia—less free than all of Western Europe and such nations as Australia, Canada, Chile, South Africa, and South Korea. As the report notes, although American law recognizes core labor rights, the U.S. political environment is "distinctly hostile to unions, collective bargaining, and labor protest." So not Hell, but not Heaven either.

Public Apples, Private Oranges: A More Ripened Look

In a previous post, I criticized articles in the USA Today and elsewhere (all citing data from the conservative Cato Institute), which claimed that federal government workers earn almost twice as much as private sector employees (including salary and benefits). I argued that en masse comparisons of public and private sector workers don’t tell us much, since the jobs that comprise the two sectors are very different.

For a more useful comparison, we need to understand not only that most public sector workers are professionals, but also that they tend to be more experienced, and more quickly promoted, than the typical private sector employee. For example, a lead research scientist will earn more than his or her staff scientists, whether they are working in the public or the private sector. So, if public sector employees in a given occupation tend to be more experienced or have more authority or responsibilities, they will appear “overpaid” even though they are not.

So, how does the public/private wage gap look when we compare professionals in the two sectors by both occupation and experience/responsibilities?

Green Shoots At The Grassroots

How can unions regain strength in a political and economic environment that has been hostile for decades? What can unions accomplish for working people in the dismal current economy?

These are tough questions that unionists grapple with every day – not just on Labor Day – and there’s probably no simple answer. One line of thinking is the coalition-oriented view that unions must embrace a "social movement" approach, and connect with other progressive groups that focus on "social identity, the environment, and globalization" (see here). Indeed, according to a recent article in The Nation, unions and environmentalists in New England are doing just that, and enjoying some success. Groups whose primary focus is teaching people how to save energy have joined with unions and community groups in coalitions that seek both to promote environmental stewardship and to create "high road" green jobs. According to activists, these will be good union jobs in sustainable, green industries. By recognizing shared interests and overlapping constituencies, they maintain, traditional tensions between unions and environmental groups have been overcome to the benefit of both.

This social movement model is founded on three essentials: "deep coalitions, policy research, and political action." It’s an approach in which the article’s author, Amy Dean, has a wealth of experience, and which she describes in a book she recently co-authored. (Full disclosure: The Albert Shanker Institute provided some support to Ms. Dean for the writing of this book.)

So does social movement unionism really blaze a grassroots trail to a union renaissance? That’s impossible to say with any certainty, but I have a few related points.

Are Value-Added Models Objective?

In recent discussions about teacher evaluation, some people try to distinguish between "subjective" measures (such as principal and peer observations) and "objective" measures (usually referring to value-added estimates of teachers’ effects on student test scores).

In practical usage, objectivity refers to the relative absence of bias from human judgment ("pure" objectivity being unattainable). Value-added models are called "objective" because they use standardized testing data and a single tool for analyzing them: All students in a given grade/subject take the same test and all teachers’ "effects" in a given district or state are estimated by the same model. Put differently, all teachers are treated the same (at least those 25 percent or so who teach grades and subjects that are tested), and human judgment is relatively absent.

By this standard, are value-added models objective? No. And it is somewhat misleading to suggest that they are.

Free Labor In A Hostile World

Our guest author today is Arch Puddington, director of research at Freedom House. The Global State of Workers’ Rights: Free Labor in a Hostile World, the Albert Shanker Institute-supported report he cites below, is available here. A "Map of Workers’ Rights," depicting its findings is here. 

This month marks the thirtieth anniversary of the founding of Solidarity, the independent trade union movement that played so crucial a role in the collapse of Communist rule in Poland and ultimately everywhere else where it held sway. Solidarity emerged from a series of spontaneous strikes called by workers at the shipbuilding yards of Poland’s Baltic coast cities. It quickly spread throughout the country, pulling in workers from steel works, textile mills, and coal mines. Soon, the working class was joined by the intellectual opposition, a loose movement of academics and former student activists that had been gathering momentum as the corruption of the Communist system became increasingly apparent. 

Solidarity thus quickly evolved into a broad movement for democracy, with a free-wheeling press, a diplomatic apparatus, and close ties to Poland’s influential Catholic Church. It was, however, the support of Poland’s huge working class that ensured Solidarity’s staying power. Where Communist regimes had faced down opposition stirrings among students and intellectuals in the past, it had never been confronted by an adversary as large, disciplined, and well-organized as Solidarity came to be.  

It’s worth mentioning during this U.S. Labor Day period that U.S. unions, led by individuals such as AFL-CIO President Lane Kirkland and AFT President Al Shanker (from whom this blog is named), among many others, were Solidarity’s staunchest supporters in the U.S.