Teacher Quality

Will Value-Added Reinforce The Walls Of The Egg-Crate School?

Our guest author today is Susan Moore Johnson, Jerome T. Murphy Research Professor in Education at Harvard Graduate School of Education. Johnson directs the Project on the Next Generation of Teachers, which examines how best to recruit, develop, and retain a strong teaching force.

Academic scholars are often dismayed when policymakers pass laws that disregard or misinterpret their research findings. The use of value-added methods (VAMS) in education policy is a case in point.

About a decade ago, researchers reported that teachers are the most important school-level factor in students’ learning, and that that their effectiveness varies widely within schools (McCaffrey, Koretz, Lockwood, & Hamilton 2004; Rivkin, Hanushek, & Kain 2005; Rockoff 2004). Many policymakers interpreted these findings to mean that teacher quality rests with the individual rather than the school and that, because some teachers are more effective than others, schools should concentrate on increasing their number of effective teachers.

Based on these assumptions, proponents of VAMS began to argue that schools could be improved substantially if they would only dismiss teachers with low VAMS ratings and replace them with teachers who have average or higher ratings (Hanushek 2009). Although panels of scholars warned against using VAMS to make high-stakes decisions because of their statistical limitations (American Statistical Association, 2014; National Research Council & National Academy of Education, 2010), policymakers in many states and districts moved quickly to do just that, requiring that VAMS scores be used as a substantial component in teacher evaluation.

Starting Closest To Home: The Importance Of Developing Teachers’ Understanding Of The Social Contexts Of Their Classrooms

Our guest author today is John Lane, a former teacher and instructional coach who is now working as a post-doctoral researcher at Michigan State University on a project that investigates the impact of social networks and mentorship on the mathematics instructional practices of beginning teachers. 

It may seem foolish now, but there was a time in this country when policymakers believed that reforms were self-executing. Legislatures and educational bureaucrats would articulate the terms of the policies and their vision for improving schools, and teachers and others close to schools would translate these visions into practice. In the meantime, over the past fifty years or so, researchers have been able to better understand the vast gulf between reformers’ ideals and teachers’ practice. In short, we have come to understand that improving teacher’s practice is more difficult than anyone imagined.

Policymakers, however, seem not to have gotten this message, or to have gotten it only partially. For the most part, they still follow a familiar script that reads that teachers either lack the skill or the will to enact reforms, or both. Consequently, reforms typically ratchet up accountability while also including some provision for teacher learning.

In what follows, I focus on the content of this learning and what it might take to achieve it. First, I discuss why teacher learning is complex and often challenging. Next, I discuss how teacher learning is typically organized, and the substance of what teachers currently learn. Specifically, I contend that in teachers’ typical learning opportunities, reforms are reduced to a set of strategies that “work” across settings, and in which the contexts of teaching become an unwanted entanglement. In this post, I argue that teachers would benefit from opportunities to learn about the social dynamics of classrooms -- it is those dynamics, after all, that affect their own reform efforts and teacher practice more broadly. I then offer some ideas about how teachers might be able to accomplish this.

Developing Workplaces Where Teachers Stay, Improve, And Succeed

** Republished here in the Washington Post

Our guest authors today are Matthew A. Kraft and John P. Papay. Kraft is an Assistant Professor of Education at Brown University. Papay is an Assistant Professor of Education and Economics at Brown University. In 2015, they received the American Educational Research Association Palmer O. Johnson Memorial Award for the research discussed in this essay. 

When you study education policy, the inevitable question about what you do for a living always gets the conversation going. Controversies over teachers unions, charter schools, and standardized testing provide plenty of fodder for lively debates. People often are eager to share their own experiences about individual teachers who profoundly shaped their lives or were less than inspiring.

A large body of research confirms this common experience – teachers have large effects on students’ learning, and some teachers are far more effective than others. What is largely absent in these conversations, and in the scholarly literature, is a recognition of how these teachers are also supported or constrained by the organizational contexts in which they teach.

The absence of an organizational perspective on teacher effectiveness leads to narrow dinner conversations and misinformed policy. We tend to ascribe teachers’ career decisions to the students they teach rather than the conditions in which they work. We treat teachers as if their effectiveness is mostly fixed, always portable, and independent of school context. As a result, we rarely complement personnel reforms with organizational reforms that could benefit both teachers and students.

New Teacher Attrition And The Recession

The Beginning Teacher Longitudinal Study (BTLS), a terrific project by the National Center for Education Statistics, tracks a nationally representative cohort of beginning teachers (those who started in 2007 or 2008) through their first five years and documents their turnover outcomes. Results from this survey have been trickling out every year (see our post here), but the most recent report presents outcomes from these teachers’ first four years.

The headline story, as reported in the Washington Post, was that roughly 17 percent of these new teachers had left the profession entirely within their first four years. A number of commenters, including the Post article, hastened to point out that the BTLS estimates are far lower than the “conventional wisdom” statistic that 40-50 percent leave the profession within the first five years (see here for more on this figure; also see Perda 2013 for a similar five-year estimate using longitudinal data). These findings are released within a political context where teacher attrition (somewhat strangely) has become a contentious political issue, one which advocates tend to interpret in a manner that supports their pre-existing beliefs about education policy.

Putting this source of contention aside, the BTLS results clearly show that new teacher attrition during these years was far lower than is often assumed, and certainly that teachers are not fleeing the classroom at a greater clip than in previous years. This is important, and cannot be "explained away" by any one factor, but we should still be careful about generalizing too strongly these findings beyond this particular time period, given that the BTLS cohort of teachers entered the classroom almost precisely at the time that the "great recession" began. This is, of course, not a new or original point – it was, for instance, mentioned briefly in the Post article. And it's hardly groundbreaking to note that labor market behavior does not occur in a vacuum. Still, given all the commentary about the BTLS results, it may be worth reviewing briefly.

Teaching = Thinking + Relationship

Our guest author today is Bryan Mascio, who taught for over ten years in New Hampshire, primarily working with students who had been unsuccessful in traditional school settings. Bryan is now a doctoral student at the Harvard Graduate School of Education, where he conducts research on the cognitive aspects of teaching, and works with schools to support teachers in improving relationships with their students.

Before I became a teacher I worked as a caretaker for a wide variety of animals. Transitioning from one profession to the other was quite instructive. When I trained dogs, for example, it was straightforward: When the dog sat on command I would give him praise and a treat. After enough training, anyone else could give the command and the dog would perform just as well and as predictably. When I worked with students, on the other hand, it was far more complex – we worked together in a relationship, with give and take as they learned and grew.  Regrettably, when I look at how we train teachers today, it reminds me more of my first profession than my second.

Teaching is far more than a mechanized set of actions. Our most masterful teachers aren’t just following scripts or using pre-packaged curricula. They are tailoring lessons, making professional judgments, and forging deep bonds with students – all of which is far more difficult to see or understand.  Teaching is a cognitive skill that has human relationships at its center. Unfortunately, we typically don't view teaching this way in the United States. As a result, we usually don't prepare teachers like (or for) this, we don’t evaluate them like this, and we don’t even study them like this. In our public discussion of education, we typically frame teaching as a collection of behaviors, and teachers as though they are simply technicians.  This doesn’t just create a demoralized workforce; it also leaves students in the care of well-meaning and hard-working teachers who are, nonetheless, largely unable to meet their students' individual needs – due either to lack of preparation for, or mandates that prevent, meeting them.

Measurement And Incentives In The USED Teacher Preparation Regulations

Late last year, the U.S. Department of Education (USED) released a set of regulations, the primary purpose of which is to require states to design formal systems of accountability for teacher preparation (TP) programs. Specifically, states are required to evaluate annually the programs operating within their boundaries, and assign performance ratings. Importantly, the regulations specify that programs receiving low ratings should face possible consequences, such as the loss of federal funding.

The USED regulations on TP accountability put forth several outcomes that states are to employ in their ratings, including: Student outcomes (e.g., test-based effectiveness of graduates); employment outcomes (e.g., placement/retention); and surveys (e.g., satisfaction among graduates/employers). USED proposes that states have their initial designs completed by the end of this year, and start generating ratings in 2017-18.

As was the case with the previous generation of teacher evaluations, teacher preparation is an area in which there is widespread agreement about the need for improvement. And formal high stakes accountability systems can (even should) be a part of that at some point. Right now, however, requiring all states to begin assigning performance ratings to schools, and imposing high stakes accountability for those ratings within a few years, is premature. The available measures have very serious problems, and the research on them is in its relative infancy. If we cannot reliably distinguish between programs in terms of their effectiveness, it is ill-advised to hold them formally accountable for that effectiveness. The primary rationale for the current focus on teacher quality and evaluations was established over decades of good research. We are nowhere near that point for TP programs. This is one of those circumstances in which the familiar refrain of “it’s imperfect but better than nothing” is false, and potentially dangerous.

Is The Social Side Of Education Touchy Feely?

That's right, measuring social and organizational aspects of schools is just... well, "touchy feely." We all intuitively grasp that social relations are important in our work environments, that having mentors on the job can make a world of difference, that knowing how to work with colleagues matters to the quality of the end product, that innovation and improvement relies on the sharing of ideas, that having a good relationship with supervisors influences both engagement and performance, and so on.

I could go on, but I don't have to; we all just know these things. But is there hard evidence, other than common sense and our personal experiences? Behaviors such as collaboration and interaction or qualities like trust are difficult to quantify. In the end, is it possible that they are just 'soft' and that, even if they’re important (and they are), they just don't belong in policy conversations?

Wrong.

In this post, I review three distinct methodological approaches that researchers have used to understand social-organizational aspects of schools. Specifically, I selected studies that examine the relationship between aspects of teachers' social-organizational environments and their students' achievement growth. I focus both on the methods and on the substantive findings. This is because I think some basic sense of how researchers look at complex constructs like trust or collegiality can deepen our understanding of this work and lead us to embrace its implications for policy and practice more fully.

Broadening The Educational Capability Conversation: Leveraging The Social Dimension

Our guest author today is James P. Spillane, Spencer T. and Ann W. Olin Professor in Learning and Organizational Change at the School of Education and Social Policy at Northwestern University. Spillane has published extensively on issues of education policy, policy implementation, school reform, and school leadership. His most recent books are Distributed Leadership in Practice and Diagnosis and Design for School Improvement. Learn more about Spillane's work at www.distributedleadership.org.

We are well into a new century – 15 years and counting! Yet, we continue to fixate on last century notions about human capability. Specifically, we still dwell mostly on the individual teacher or school leader, on investing in and developing their individual human capital so as to improve their productivity and in turn generating higher returns to the individual, school organization, school system, and society. The empirical evidence has established educational professionals' human capital is undoubtedly important for school and school-system productivity.* At the same time, however, by fixating primarily on human capital, we miss or undermine the significance and potential of social capital.

Social capital captures the idea that capability (and by extension productivity) is not simply an individual matter but also a social matter. In other words, in addition to individual capability, there are (often untapped) resources that reside in the relations among people within organizations, systems, or society – a social capability. These social relations can be a source of and a channel for crucial resources such as trust, information, expertise, materials, security, obligation, incentives, and so on - see Bryk & Schneider 2002; Coburn 2001; Daly, Moolenaar, Bolivar, & Burke 2010; Frank, Zhao, & Borman 2004; Frank, Zhao, Penuel, Ellefson, & Porter 2011; Louis, Marks, & Kruse 1996; Moolenaar, Karsten, Sleegers, & Daly 2014. In a given system or organization, social capital is much more than the aggregate of members' human capital.

The Smoke And The Fire From Evaluations Of Teach For America

A recent study by the always reliable research organization Mathematica takes a look at the characteristics and test-based effectiveness of Teach For America (TFA) teachers who were recruited as part of a $50 million federal “Investing in Innovation” grant, which is supporting a substantial scale-up of TFA’s presence in U.S. public schools.

The results of this study pertain to a small group of recruits (and comparison non-TFA teachers) from the first two years of the program – i.e., a sample of 156 PK-5 teachers (66 TFA and 90 non-TFA) in 36 schools spread throughout 10 states. What distinguishes the analysis methodologically is that it exploits the random assignment of students to teachers in these schools, which ensures that any measured differences between TFA and comparison teachers are not due to unobserved differences in the students they are assigned to teach.

The Mathematica researchers found, in short, that the estimated differences in the impact of TFA and comparison teachers on math and reading scores across all grades were modest in magnitude and not statistically discernible at any conventional level. There were, however, meaningful positive estimated differences in the earliest grades (PK-2), though they were only statistically significant in reading, and the coefficient in reading for grades 3-5 was negative (and not significant). Let’s take a quick look at these and other findings from this report and what they might mean.