The Rise and Fall of the Teacher Evaluation Reform Empire

Teacher evaluation reform during the late 2000s and 2010s was one of the fastest and widespread education policy changes in recent history. Thanks mostly to Race to the Top and ESEA “waivers,” over a period of about 10 years, the vast majority of the nation’s school districts installed new teacher evaluations. These new systems were quite different from their predecessors in terms of design, with 3-5 (rather than dichotomous) rating categories incorporating multiple measures (including some based on student testing results). And, in many states, there were varying degrees of rewards and/or consequences tied to the ratings (Steinberg and Donaldson 2016).

A recent working paper offers what is to date the most sweeping assessment of the impact of teacher evaluation reform on student outcomes, with data from 44 states and D.C. As usual, I would encourage you to read the whole paper (here's an earlier ungated version released in late 2021). It is terrific work by a great team of researchers (Joshua Bleiberg, Eric Brunner, Erika Harbatkin, Matthew Kraft, and Matthew Springer), and I’m going to describe the findings only superficially. We’ll get into a little more detail below, but the long and short of it is that evaluation reform had no statistically detectable aggregate effect on student test scores or attainment (i.e., graduation or college enrollment).

This timely analysis, in combination with the research on evaluations over the past few years, provides an opportunity to look back on this enormous reform effort, and whether and how states and districts might move forward.

What Teachers Want From PD

Teacher professional development (PD) often gets a bad rap. As a former educator of nearly a decade, I can count on one hand how many successful PD sessions I attended over the years. I can also remember how impactful those select few sessions were to my teaching craft.

According to the 2020 International Literacy Association (ILA) survey for teachers, higher education professionals, literacy consultants, and school administrators, PD was identified as a top priority for improving instruction. Participants also stated that PD needs more focus and attention to support teachers in their ongoing professional growth. Teachers understand the need for PD, but often feel unfulfilled by the experience.

So, I wondered—what do teachers really want from PD? After reviewing several articles and studies from the perspective of primary and secondary teachers, six common trends emerged.

Evidence In Education: Is Supply Meeting Teachers' Demand

In today’s public schools, teachers and administrators are constantly pressured to implement new reforms and initiatives, most of which claim to be research- or science-based. Schools are often viewed as the recipients of new policies and new knowledge generated by research. In So Much Reform, So Little ChangeCharles Payne states “Best Practice discourse lends itself to decontextualized thinking, reducing the problem of urban schooling to a cognitive one: if only our teachers and principals knew how to do it in the Big City...we should spend some time thinking with school people about what those reasons might be rather than just issuing more exhortations from on high” (p. 63). Including educators in the conversation is crucial for a successful rollout of any evidence-based intervention. Additionally, in order to best support our teachers, it is imperative we develop a system for them to have access to evidence-informed practices that are contextualized and can be tailored to their varying needs. 

While there are many problems with the “research to practice” approach, one barrier is that schools are frequently not given enough training, resources, and support to make sense of educational research. Educators need evidence, innovation, and new approaches; after all, much of their job is to constantly adapt their ways of teaching to meet the unique and changing needs of their students. But right now, too many educators are doing this work without the necessary tools to be successful. In this post I would like to sketch out a tool that could improve access to and applicability of research. By access, I refer not only to the ability of finding and downloading journal articles and other sources, but also to having the knowledge to discern questionable research. According to Paula J. Stanovich and Keith E. Stanovich (2003), a failure to discern good and bad research can cause teachers to try new methods that are not strongly supported by peer-reviewed research. Applicability refers to the extent to which research has practical implications for teachers.

Research And Evidence Can Help Guide Teachers During The Pandemic

This post is part of our series entitled Teaching and Learning During a Pandemic, in which we invite guest authors to reflect on the challenges of the Coronavirus pandemic for teaching and learning. Our guests today are Sara Kerr, Vice President of Education Policy Implementation at Results for America, and Nate Schwartz, Professor of Practice at Brown Universitys Annenberg Institute for School Reform. Other posts in the series are compiled here.

Teachers are used to playing many different roles, but this year they are facing the most complex challenges of their careers. They are being asked to be public health experts. Tech support specialists. Social workers to families reeling from the effects of layoffs and illness. Masters of distance learning and trauma-responsive educational practices. And they are being asked to take on these new responsibilities against a backdrop of rising COVID-19 cases in many parts of the country, looming budget cuts for many school districts, and a hyper-polarized political debate over the return to school.

To make any of this possible, educators need to be armed with the best available science, data, and evidence, not only about the operational challenges of reopening that have dominated the news cycle but also about how to to meet the increasingly complex social-emotional and academic needs of students and their families. They dont have time to sift through decades of academic papers for answers. Fortunately, the nations education researchers are eager and ready to help.

School Organizational Practices And The Challenges Of Remote Teaching During A Pandemic

This post is part of our series entitled Teaching and Learning During a Pandemic, in which we invite guest authors to reflect on the challenges of the Coronavirus pandemic for teaching and learning. Our guests today are Matthew A. Kraft, Associate Professor of Education and Economics at Brown University and Research Director at Upbeat, and Nicole S. Simon, director in the Office of K-16 Initiatives at the City University of New York. Other posts in the series are compiled here.

The COVID-19 pandemic has shuttered schools across the United States, upending traditional approaches to education. The health threats posed by the Coronavirus, a sudden shift to remote teaching, and added caretaking responsibilities at home have created a uniquely stressful and demanding context for teachers’ work. Major concerns exist about teachers’ wellbeing during the pandemic and their ability to successfully deliver instruction remotely. Teachers have also expressed apprehension about their willingness to return to the classroom when schools are able to reopen. Even more troubling are projections of substantial student learning loss and the likelihood that differential access to technology and learning supports at home are exacerbating longstanding achievement gaps along racial and socio-economic lines. 

We developed the “Teaching From Home Survey” for Upbeat to support districts in better understanding and responding to teachers’ experience in working remotely. Between April 27 and May 26, 2020, a diverse sample of 7,195 teachers working across nine southern, midwestern and eastern states answered the survey. 78% of teachers responded to the survey, including teachers working across 8 districts and 3 charter school networks in rural, suburban, and urban areas. Our analyses complement and extend recent findings by smaller, nationally representative surveys by USA Today, Educators for Excellence, Ed Week, and RAND. The large and diverse sample of respondents allow us to explore how teachers’ experiences working remotely differ across both individual and school characteristics.

Teaching During School Shutdowns Should Be A Team Sport

This post is part of our series entitled Teaching and Learning During a Pandemic, in which we invite guest authors to reflect on the challenges of the Coronavirus pandemic for teaching and learning. Our guest today is Susan Moore Johnson, the Jerome T. Murphy Research Professor at the Harvard Graduate School of Education. Other posts in the series are compiled here.

When schools suddenly closed in March and moved to online instruction, I wondered how I would have responded if I'd still been a high school English teacher. I imagined having to prepare a series of engaging Ted Talks with follow-up Q&As. But having talked with many administrators and teachers, I’ve realized that good online schooling during the pandemic is a team sport not a solo performance. It calls for careful preparation and coordination among many players. Just as Covid-19 has revealed hidden shortcomings in our society, it has exposed the limitations of compartmentalized schools that continue to rise or fall on the skills, autonomy and self-reliance of individual teachers.

As teachers faced the sudden reality of online teaching, they had many pressing questions: Are my students safe and confident or are they at risk, hungry, and fearful? Am I responsible for finding students who don’t show up online? What kind of schedule provides meaningful routines with necessary flexibility? How can I create social learning experiences for students who are isolated at home? What can I do to help students who fall behind? How can we meet the special learning needs of students who rely on one-to-one support? How can I fairly grade students’ progress when I can’t provide extra help to those who need it?  

In many schools, teachers struggled with such questions alone. Without a reliable forum where they could explore and resolve urgent problems with others, individuals did their best. Some convened their classes occasionally for live meetings, so students to could see one another and talk about how things were going for them. Many prepared weekly work packets for parents to pick up at the school or they posted assignments online—typically math problems or reading comprehension questions—for students to complete and upload for grading. Some relied on web-based educational resources, including short lectures by presenters their students had never seen. Many teachers were dismayed to realize that their repertoire of instructional practices had been drastically reduced to a few barren components. Meanwhile students within the same school might have either engaging or tedious learning experiences, depending on who their teacher was.

Our Profession Requires Hope, Now And Ever Since

This post is part of our series entitled Teaching and Learning During a Pandemic, in which we invite guest authors to reflect on the challenges of the Coronavirus pandemic for teaching and learning. Our contributor today is José Luis Vilson, a math educator for a middle school in the Inwood/Washington Heights neighborhood of New York City. He publishes regularly on his own site. Otherposts in the series are compiled here. 

On most mornings, I tweet a good morning message intended to share my intentions for the day and the work ahead of me. People usually receive it well because they understand I’ve worked at a school that serves marginalized students in underserved communities for the better part of 15 years. On occasion, I have to remind the occasional tweeter how important this context is in the midst of my more optimistic tweets. In many people’s minds, rage and fury are necessary accouterments for activists where positivity and smiles look like tools of the apparat. To express any form of affirmative outlook is to betray the ideals of disruption to the status quo. 

Yet, teaching in our most dire contexts necessitates hope, and this is no more evident than in what we’ve dubbed “remote learning.” In New York City, we’ve now entered two months of correspondence with peers and students through the Internet and called this process schooling. For years, we understood school as compulsory, inequitable, and vital to the very environment that created these conditions. More learning begets unlearning. More education presumably leads to more engaged citizens who would create a better world for their children than we did for ours. Many of us follow this principle in systems deeply antagonistic to these goals. 

When our government asked the nation’s largest public-school system to flip our entire system, we did so dutifully. Yet most of us knew that such a transition would exacerbate the already entrenched inequities in our system.

Why School Climate Matters For Teachers And Students

Our guest authors today are Mathew A. Kraft, associate professor of education and economics at Brown University, and Grace T. Falken, a research program associate at Brown’s Annenberg Institute. This article originally appeared in the May 2020 issue of The State Education Standard, the journal of the National Association of State Boards of Education.

Over the past decade, education reformers have focused much of their attention on raising teacher quality. This makes sense, given the well-evidenced, large impacts teachers have on student outcomes and the wide variation in teacher effectiveness, even within the same school (Goldhaber 2015Jackson et al. 2014). Yet this focus on individual teachers has caused policymakers to lose sight of the importance of the organizational contexts in which teachers work and students learn. 

The quality of a school’s teaching staff is greater than the sum of its parts. School environments can enable teachers to perform to their fullest potential or undercut their efforts to do so. 

When we think of work environments, we often envision physical features: school facilities, instructional resources, and the surrounding neighborhood. State and district policies that shape curriculum standards, class size, and compensation also come to mind. These things matter, but so do school climate factors that are less easily observed or measured. Teachers’ day-to-day experiences are influenced most directly by the culture and interpersonal environment of their schools.

Where Al Shanker Stood: How To Harass A Fulbright Teacher

In this column, originally published in the New York Times on November 3, 1985, Al Shanker argues that teachers need to be treated respectfully, as the dedicated professionals they are.

This is the story of a man who offended the powers that be. It has nothing to do with the Medicis, the Borgias or Henry the Eighth but a lot to do with the way too many of our schools are administered. Bert (not his real name) has been a New York City social studies teachers for many years, first in junior high and then in high school. He's also a perennial student. In addition to holding two master's degrees, he's spent most of his summers taking solid academic courses at places like Columbia, Temple, Princeton, Rensselaer. Over the years he's won three Fulbright scholarships for study in India, Israel and, this past summer, in Korea.

There's nothing parochial about Bert's  interests. He's done work in human relations, psychology, urban problems, Ottoman and Korean culture, the history of anti-Semitism and of slavery, law, school administration, American history and a whole college catalogue of other subjects. In 1980 he was cited for his "outstanding  professional participation" in a summer human rights workshop at Skidmore College.

Most important, there's nothing of the ivory tower or the dilettante in all this. Bert brought what he had studied back into his classroom and into his professional relations with his colleagues. His principal once praised him for the "keen insights" he offered in a presentation to the entire faculty after a summer of study in Israel. Now he's preparing a new syllabus for his social studies department based on his summer in Korea. His chairmen have consistently praised his teaching ability. A recent observation report on one of his lessons concluded with, "Keep up the good work."

Sounds like the record of one of the top teachers in his school, right? Wrong! The punch line is that Bert was rated "unsatisfactory" by his principal at the end of the last school year.

Why Teacher Evaluation Reform Is Not A Failure

The RAND Corporation recently released an important report on the impact of the Gates Foundation’s “Intensive Partnerships for Effective Teaching” (IPET) initiative. IPET was a very thorough and well-funded attempt to improve teaching quality in schools in three districts and four charter management organizations (CMOs). The initiative was multi-faceted, but its centerpiece was the implementation of multi-measure teacher evaluation systems and the linking of ratings from those systems to professional development and high stakes personnel decisions, including compensation, tenure, and dismissal. This policy, particularly the inclusion in teacher evaluations of test-based productivity measures (e.g., value-added scores), has been among the most controversial issues in education policy throughout the past 10 years.

The report is extremely rich and there's a lot of interesting findings in there, so I would encourage everyone to read it themselves (at least the executive summary), but the headline finding was that the IPET had no discernible effect on student outcomes, namely test scores and graduation rates, in the districts that participated, vis-à-vis similar districts that did not. Given that IPET was so thoroughly designed and implemented, and that it was well-funded, it can potentially be viewed as a "best case scenario" test of the type of evaluation reform that most states have enacted. Accordingly, critics of these reforms, who typically focus their opposition on the high stakes use of evaluation measures, particularly value-added and other test-based measures, in these evaluations, have portrayed the findings as vindication of their opposition. 

This reaction has merit. The most important reason why is that evaluation reform was portrayed by advocates as a means to immediate and drastic improvements in student outcomes. This promise was misguided from the outset, and evaluation reform opponents are (and were) correct in pointing this out. At the same time, however, it would be wise not to dismiss evaluation reform as a whole, for several reasons, a few of which are discussed below.