Education Reform

When It Comes To How We Use Evidence, Is Education Reform The New Welfare Reform?

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

In the mid-1990s, after a long and contentious debate, the U.S. Congress passed the Personal Responsibility and Work Opportunity Reconciliation Act of 1996, which President Clinton signed into law. It is usually called the “Welfare Reform Act," as it effectively ended the Aid to Families with Dependent Children (AFDC) program (which is what most people mean when they say “welfare," even though it was [and its successor is] only a tiny part of our welfare state). Established during the New Deal, AFDC was mostly designed to give assistance to needy young children (it was later expanded to include support for their parents/caretakers as well).

In place of AFDC was a new program – Temporary Assistance for Needy Families (TANF). TANF gave block grants to states, which were directed to design their own “welfare” programs. Although the states were given considerable leeway, their new programs were to have two basic features: first, for welfare recipients to receive benefits, they had to be working; and second, there was to be a time limit on benefits, usually 3-5 years over a lifetime, after which individuals were no longer eligible for cash assistance (states could exempt a proportion of their caseload from these requirements). The general idea was that time limits and work requirements would “break the cycle of poverty”; recipients would be motivated (read: forced) to work, and in doing so, would acquire the experience and confidence necessary for a bootstrap-esque transformation.

There are several similarities between the bipartisan welfare reform movement of the 1990s and the general thrust of the education reform movement happening today. For example, there is the reliance on market-based mechanisms to “cure” longstanding problems, and the unusually strong liberal-conservative alliance of the proponents. Nevertheless, while calling education reform “the new welfare reform” might be a good soundbyte, it would also take the analogy way too far.

My intention here is not to draw a direct parallel between the two movements in terms of how they approach their respective problems (poverty/unemployment and student achievement), but rather in how we evaluate their success in doing so. In other words, I am concerned that the manner in which we assess the success or failure of education reform in our public debate will proceed using the same flawed and misguided methods that were used by many for welfare reform.

What Do Teachers Really Think About Education Reform?

There has recently been a lot of talk about teachers’ views on education policy. Many teachers have been quite vocal in their opposition to certain policies (also here) and many more have expressed their views democratically – through their unions – especially in states where teachers have collective bargaining rights.

We should listen carefully to these views, but it’s also important to bear in mind that there are millions of public school teachers out there, with a wide variety of opinions on any particular education policy, and not all of their voices might be getting through.

So, the question remains: How do most teachers feel about the current wave of education policy reforms spreading throughout states and districts, including (but not at all limited to) merit pay, eliminating tenure and incorporating test-based measures into teacher evaluations?

The logical mechanism by which we might learn more about teachers’ views on these policies is, of course, a survey. Unfortunately, useful national surveys are quite rare. In order to get accurate estimates, you need an unusually large number of teachers to take the survey (a deliberate "oversample"), and they must be randomly polled (lest there be selection bias). In my last post, I suggested that states/districts conduct their own teacher surveys.  In the meantime, some national evidence is already available, and if the data make one thing clear, it’s that we need more. When it comes to supporting or opposing different policies, teachers’ opinions, like everyone’s, depend a great deal on the details.

To Understand The Impact Of Teacher-Focused Reforms, Pay Attention To Teachers

You don’ t need to be a policy analyst to know that huge changes in education are happening at the state- and local-levels right now – teacher performance pay, the restriction of teachers’ collective bargaining rights, the incorporation of heavily-weighted growth model estimates in teacher evaluations, the elimination of tenure, etc. Like many, I am concerned about the possible consequences of some of these new policies (particularly about their details), as well as about the apparent lack of serious efforts to monitor them.

Our “traditional” gauge of “what works” – cross-sectional test score gains – is totally inadequate, even under ideal circumstances. Even assuming high quality tests that are closely aligned to what has been taught, raw test scores alone cannot account for changes in the student population over time and are subject to measurement error. There is also no way to know whether fluctuations in test scores (even fluctuations that are real) are the result of any particular policy (or lack thereof).

Needless to say, test scores can (and will) play some role, but I for one would like to see more states and districts commissioning reputable, independent researchers to perform thorough, longitudinal analyses of their assessment data (which would at least mitigate the measurement issues). Even so, there is really no way to know how these new, high-stakes test-based policies will influence the validity of testing data, and, as I have argued elsewhere, we should not expect large, immediate testing gains even if policies are working well. If we rely on these data as our only yardstick of how various policies are working, we will be getting a picture that is critically incomplete and potentially biased.

What are the options? Well, we can’t solve all the measurement and causality issues mentioned above, but insofar as the policy changes are focused on teacher quality, it makes sense to evaluate them in part by looking at teacher behavior and characteristics, particularly in those states with new legislation. Here’s a few suggestions.

Mixed Messages

Today is National Teacher Appreciation Day, as well as National Teacher Appreciation Week.  In various ways, millions of people are thanking their teachers for having made a difference in their lives, including President Obama, who held an official function at the White House today honoring the National Teacher of the Year.

But a couple of other things are happening today as well.

The Un-American Foundations Of Our Education Debate

Being from Spain, one of the first things that struck me as odd about the U.S. education debate was the ubiquitous depiction of “bad teachers” as the villains of education and “great teachers” as its saviors. Aside from the fact that this view is simplistic, the punish/praise-teachers chorus seemed particularly off-key—but I wasn’t sure why. I think I may have figured it out. I think that it may be un-American.

Let me explain. This is a nation that is supposed to be built around specific core values, such as individual effort, hard work, and taking responsibility for one’s own actions. If so, isn’t the fixation on teachers—to the seeming exclusion of students and parents—an indirect rejection of basic American principles?

This is not a discussion of what the good/bad teacher doctrine misses —we know it misses numerous dimensions of the education enterprise—but rather, what this doctrine assumes and how these assumptions conflict with the values that one expects most Americans to hold.

One problem with the narrow focus on teachers is that it views students exclusively as passive recipients of their own learning. Not to get too technical here, this goes back to a central question in the social sciences: namely, agency versus structure. Agency refers to the capacity of individuals to act independently and make their own choices. Structure refers to the conditions that shape and perhaps limit the range of alternative choices that are available. Western culture tends to favor agency over structure as an explanation for actions, a view which one would think would run particularly deep in the U.S.

A Big Fish In A Small Causal Pond

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

In three previous posts, I discussed what I’ve begun to call the “trifecta” of teacher-focused education reform talking points:

• Teachers are the most important (in-school) factor affecting achievement;
• Firing the bottom 5-10 percent of teachers will increase our scores to the level of high-performing nations; and
• Providing children with three, four, or five consecutive “top” teachers in a row would close the achievement gap.

In many respects, this “trifecta” is driving the current education debate. You would have trouble finding many education reform articles, reports, or speeches that don’t use at least one of these arguments.

Indeed, they are guiding principles behind much of the Obama Administration’s education agenda, as well as the philosophies of high-profile market-based reformers, such as Joel Klein and Michelle Rhee. The talking points have undeniable appeal. They imply, deliberately or otherwise, that policies focused on improving teacher quality in and of themselves can take us a very long way - not all the way, but perhaps most of the way - towards solving all of our education problems.

This is a fantasy.

Teacher Quality Is Not A Policy

I often hear the following argument: Improving teacher quality is more cost-effective than other options, such as reducing class size (see here, for example). I am all for evaluating policy alternatives based on their costs relative to their benefits, even though we tend to define the benefits side of the equation very narrowly - in terms of test score gains.

But “improving teacher quality” cannot yet be included in a concrete costs/benefits comparison with class size or anything else. It is not an actual policy. At best, it is a category of policy options, all of which are focused on recruitment, preparation, retention, improvement, and dismissal of teachers. When people invoke it, they are presumably referring to the fact that teachers vary widely in their test-based effectiveness. Yes, teachers matter, but altering the quality distribution is whole different ballgame from measuring it overall. It’s actually a whole different sport.

I think it is reasonable to speculate that we might get more bang for our buck if we could somehow get substantially better teachers, rather than more of them, as would be necessary to reduce class sizes. But the sad, often unstated truth about teacher quality is that there is very little evidence, at least as yet, that public policy can be used to improve it, whether cost-effectively or otherwise

How Many Teachers Does It Take To Close An Achievement Gap?

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Over the weekend, New York Times columnist Nick Kristof made a persuasive argument that teachers should be paid more. In making his case, he also put forth a point that you’ve probably heard before: “One Los Angeles study found that having a teacher from the 25 percent most effective group of teachers for four years in a row would be enough to eliminate the black-white achievement gap."

This is an instance of what we might call the "X consecutive teachers” argument (sometimes it’s three, sometimes four or five). It is often invoked to support, directly or indirectly, specific policy prescriptions, such as merit pay, ending tenure, or, in this case, higher salaries (see also here and here). To his credit, Kristof’s use of the argument is on the cautious side, but there are plenty of examples in which it used as evidence supporting particular policies.

Actually, the day after the column ran, in a 60 Minutes segment featuring “The Equity Project," a charter school that pays its teachers $125,000 a year, the school’s principal was asked how he planned to narrow the achievement gap with his school. His reply was: “The difference between a great teacher and a mediocre or poor teacher is several grade levels of achievement in a given year. A school that focuses all of its energy and its resources on fantastic teaching can bridge the achievement gap."

Indeed, it is among the most common arguments in our education policy debate today.  In reality, however, it is little more than a stylistic riff on empirical research findings, and a rough one at that. It is not at all useful when it comes to choosing between different policy options.

A Wisconsin Moment For Our Education Policy Debate

There is an obvious, albeit somewhat uncomfortable connection between what’s happening in Wisconsin and what’s been happening in education policy discussions.

A remarkably high proportion of the discussion is focused – implicitly or explicitly – on the presumed role of teachers’ unions. The public is told that our school systems are failing, and that teachers’ unions are at least partially to blame because they protect bad teachers and block “needed” reforms such as merit pay. In this storyline, unions are faceless villains that put the interests of adults above those of children.

Wisconsin represents a threat to this perspective in at least three important manners.

Lyndon Johnson: Education Reformer

Underlying virtually all contemporary education policy debates is the question of poverty. Certainly, high poverty and inequality do not mean we shouldn’t improve schools. On the other hand, the standpoint of some in the debate today evolved from an inarguable, commendable notion (poor kids can learn too) to an ideological brick wall, behind which those who dare speak poverty’s name are accused of “making excuses."

Anyone who reads history (or who has lived through it) knows that the tension between poverty and equality of educational opportunity is nothing new, nor is the debate about how to address them.  For example, these same issues arose during the campaign to pass the Elementary and Secondary Education Act (ESEA) in 1965, as President Lyndon Johnson cajoled everyone he could to push the bill through Congress.

Unlike most of the political leaders who are the self-proclaimed education reformers of today, Johnson had been a teacher. His teaching experience, at a small segregated school for Mexican Americans in the impoverished town of Cotulla, Texas, convinced him that poverty and educational inequality must be tackled in tandem.

Full disclosure: I have a minor obsession with Lyndon Johnson (it feels good to say that out loud), and I have read all of the released transcripts of the phone calls and White House meetings that LBJ recorded. In one of these conversations—on March 6, 1965—Johnson is speaking with Hubert Humphrey, his newly-inaugurated Vice-President, who spent much of his term serving as LBJ’s liaison to Congress. Johnson’s deep belief in quality public education as a key to reducing poverty comes across in this conversation.