New Report: Does Money Matter in Education? Second Edition

In 2012, we released a report entitled “Does Money Matter in Education?,” written by Rutgers Professor Bruce Baker. The report presented a thorough, balanced review of the rather sizable body of research on the relationship between K-12 education spending and outcomes. The motivation for this report was to address the highly contentious yet often painfully oversimplified tribal arguments regarding the impact of education spending and finance reforms, as well as provide an evidence-based guide for policymakers during a time of severe budgetary hardship. It remains our most viewed resource ever, by far.

Now, almost four years later, education spending in most states and localities is still in trouble. For example, state funding of education is lower in 2016 than it was in 2008 (prior to the recession) in 31 states (Leachman et al. 2016). Moreover, during this time, there has been a continuing effort to convince the public that how much we spend on schools doesn’t matter for outcomes, and that these spending cuts will do no harm.

As is almost always the case, the evidence on spending in education is far more nuanced and complex than our debates about it (on both “sides” of the issue). And this evidence has been building for decades, with significant advances since the release of our first “Does Money Matter?” report. For this reason, we have today released the second edition, updated by the author. The report is available here.

Evidence From A Teacher Evaluation Pilot Program In Chicago

The majority of U.S. states have adopted new teacher evaluation systems over the past 5-10 years. Although these new systems remain among the most contentious issues in education policy today, there is still only minimal evidence on their impact on student performance or other outcomes. This is largely because good research takes time.

A new article, published in the journal Education Finance and Policy, is among the handful of analyses examining the preliminary impact of teacher evaluation systems. The researchers, Matthew Steinberg and Lauren Sartain, take a look at the Excellence in Teaching Project (EITP), a pilot program carried out in Chicago Public Schools starting in the 2008-09 school year. A total of 44 elementary schools participated in EITP in the first year (cohort 1), while an additional 49 schools (cohort 2) implemented the new evaluation systems the following year (2009-10). Participating schools were randomly selected, which permits researchers to gauge the impact of the evaluations experimentally.

The results of this study are important in themselves, and they also suggest some more general points about new teacher evaluations and the building body of evidence surrounding them.

The Role Of Teacher Diversity In Improving The Academic Performance Of Students Of Color

Last month, the Albert Shanker Institute released a report on the state of teacher diversity, which garnered fair amount of press attention – see here, here, here, and here. (For a copy of the full report, see here.) This is the second of three posts, which are all drawn from a research review published in the report. The first post can be found here. Together, they help to explain why diversity in the teaching force—or lack thereof—should be  a major concern.

It has long been argued that there is a particular social and emotional benefit to children of color, and especially those children from high-poverty neighborhoods, from knowing—and being known and recognized by—people who look like themselves who are successful and in positions of authority. But there is also a growing body of evidence to suggest that students derive concrete academic benefits from having access to demographically similar teachers.

For example, in one important study, Stanford professor Thomas Dee reanalyzed test score data from Tennessee’s Project STAR class size experiment, still one of the largest U.S. studies to employ the random assignment of students and teachers. Dee found that a one-year same-race pairing of students and teachers significantly increased the math and reading test scores of both Black and White students by roughly 3 to 4 percentile points. These effects were even stronger for poor Black students in racially segregated schools (Dee, 2004).

Recent Evidence On The New Orleans School Reforms

A new study of New Orleans (NOLA) schools since Katrina, published by the Education Research Alliance (ERA), has caused a predictable stir in education circles (the results are discussed in broader strokes in this EdNext article, while the full paper is forthcoming). The study’s authors, Doug Harris and Matthew Larsen, compare testing outcomes before and after the hurricanes that hit the Gulf Coast in 2005, in districts that were affected by those storms. The basic idea, put simply, is to compare NOLA schools to those in other storm-affected districts, in order to assess the general impact of the drastic educational change undertaken in NOLA, using the other schools/districts as a kind of control group.

The results, in brief, indicate that: 1) aggregate testing results after the storms rose more quickly in NOLA vis-à-vis the comparison districts, with the difference in 2012 being equivalent to roughly 15 percentile points ; 2) there was, however, little discernible difference in the trajectories of NOLA students who returned after the storm and their peers in other storm-affected districts (though this latter group could only be followed for a short period, all of which occurred during these cohorts' middle school years). Harris and Larsen also address potential confounding factors, including population change and trauma, finding little or no evidence that these factors generate bias in their results.

The response to this study included the typical of mix of thoughtful, measured commentary and reactionary advocacy (from both “sides”). And, at this point, so much has been said and written about the study, and about New Orleans schools in general, that I am hesitant to join the chorus (I would recommend in particular this op-ed by Doug Harris, as well as his presentation at our recent event on New Orleans).

Where Al Shanker Stood: Union-District Partnerships

In this New York Times piece, which was published on March 9, 1986, Al Shanker discusses a study suggesting that union-district partnership, not confrontation, is the best way to enact and implement reforms that will improve schools.

In the last 25 years, teachers' unions have grown in size and influence. In the minds of many they represent an establishment just as much as the local board of education and the superintendent of schools. Many critics of our schools have been eager to portray teacher unions as supporters of educationally undesirable rules and procedures, such as seniority, which were borrowed from the industrial sector. They view teacher unions as fighting for these rules at any cost and using their bargaining powers to shoot down constructive change whenever it threatens to infringe on teachers' vested interests.

But an interesting new study gives us quite a different picture of the impact that teacher unions and collective bargaining have on the reform process. In preparing Teacher Unions, School Staffing and Reform, a Harvard Graduate School of Education research team led by Susan Moore Johnson analyzed 155 contracts chosen at random from a variety of school districts around the country. And, from June of 1984 to February, 1985, they did extensive, in-depth field work in 5 of the districts, where they examined documents, sat in on meetings and interviewed 187 teachers, principals, union leaders and central office administrators.

What emerges is a valuable insight into the dynamics and complexity of the reform process, why some proposals work and why others fall flat. Though new programs tend to be formulated in legislative chambers or in governors' mansions, the key to success, the authors conclude, is what happens on the district level, within the individual collective bargaining unit. And some interesting patterns emerge.

The Story Behind The Story: Social Capital And The Vista Unified School District

Our guest author today is Devin Vodicka, superintendent of Vista Unified, a California school district serving over 22,000 students that was recently accepted into the League of Innovative Schools. Dr. Vodicka participates in numerous state and national leadership groups, including the Superintendents Technical Working Group of the U.S. Education Department .

Transforming a school district is challenging and complex work, often requiring shifts in paradigms, historical perspective, and maintaining or improving performance. Here, I’d like to share how we approached change at Vista Unified School District (VUSD) and to describe the significant transformation we’ve been undergoing, driven by data, focused on relationships, and based in deep partnerships. Although Vista has been hard at work over many years, this particular chapter starts in July of 2012 when I was hired.  

When I became superintendent, the district was facing numerous challenges: Declining enrollment, financial difficulties, strained labor relations, significant turnover in the management ranks, and unresolved lawsuits were all areas in need of attention. The school board charged me and my team with transforming the district, which serves large numbers of linguistically, culturally, and economically diverse students. While there is still significant room for improvement, much has changed in the past three years, generally trending in a positive direction. Below is the story of how we did it.

The Magic Of Multiple Measures

Our guest author today is Cara Jackson, Assistant Director of Research and Evaluation at the Urban Teacher Center.

Teacher evaluation has become a contentious issue in U.S.  Some observers see the primary purpose of these reforms as the identification and removal of ineffective teachers; the popular media as well as politicians and education reform advocates have all played a role in the framing of teacher evaluation as such.  But, while removal of ineffective teachers was a criterion under Race to the Top, so too was the creation of evaluation systems to be used for teacher development and support.

I think most people would agree that teacher development and improvement should be the primary purpose, as argued here.  Some empirical evidence supports the efficacy of evaluation for this purpose (see here).  And given the sheer number of teachers we need, declining enrollment in teacher preparation programs, and the difficulty disadvantaged schools have retaining teachers, school principals are probably none too enthusiastic about dismissing teachers, as discussed here.

Of course, to achieve the ambitious goal of improving teaching practice, an evaluation system must be implemented well.  Fans of Harry Potter might remember when Dolores Umbridge from the Ministry of Magic takes over as High Inquisitor at Hogwarts and conducted “inspections” of Hogwart’s teachers in Book 5 of J.K. Rowling’s series.  These inspections pretty much demonstrate how not to approach classroom observations: she dictates the timing, fails to provide any of indication of what aspects of teaching practice she will be evaluating, interrupts lessons with pointed questions and comments, and evidently does no pre- or post-conferencing with the teachers. 

Where Al Shanker Stood: Policymaking And Innovation

In this piece, which was published in the New York Times on December 24, 1995, Al Shanker uses a creative analogy to argue that policies require experimentation and refinement before they are brought to scale, and that some reformers mistake this process for rigidity and "stifling innovation."

A couple of weeks ago, the New York Times food section ran an article about a French bread that you can make with a food processor (November 22, 1995). The article claimed that the baguette was as delicious as the kind you buy in a good bakery. I was skeptical. I have made bread for my family and friends for a number of years, and I know that a good French loaf is a real accomplishment. I had no trouble believing that the bread would be quick and easy. But delicious? Nevertheless, I tried the recipe for Thanksgiving. It was terrific!

Though making the bread was as painless as the article said, the process by which Charles van Over, a chef and restaurateur, arrived at the recipe was anything but simple. Van Over experimented over a period of several years in order to get a bread with the best possible texture, flavor, and crust - and a recipe that could be made with predictable results by other cooks. It occurred to me as I read the article that there might be some lessons for school reformers in van Over's systematic efforts to perfect his recipe for a food processor baguette.

New School Climate Tool Facilitates Early Intervention On Social-Emotional Issues: Bullying And Suicide Prevention

Our guest author today is Dr. Alvin Larson, director of research and evaluation at Meriden Public Schools, a district that serves about 8,900 students in Meriden, CT. Dr. Larson holds a B.A. in Sociology, M. Ed., M.S. in Educational Research and a Ph.D. in Educational Psychology. The intervention described below was made possible with support from Meriden's community, leadership and education professionals.

For the most part, students' social-emotional concerns start small; if left untreated, though, they can become severe and difficult to manage. Inappropriate behaviors are not only harmful to the student who exhibits them; they can also serve to increase the social bruising of his/her peers and can be detrimental to the climate of the entire school. The problem is that many of these bruises are not directly observable – or not until they become scars. School psychologists and counselors are familiar with bruised students who act out overtly, but some research suggests that 4.3% of our students carry social-emotional scars of which counselors are unaware (Larson, AERA 2014). To develop a more preventative approach, foster pro-social attitudes and a positive school climate, we need to be able to identify and support the students with hidden bruises as well as intervene with pre-bullies early in their school careers.

Since 2011, Connecticut’s Local Education Agencies (LEAs) have been required to purchase or develop a student school climate survey. The rationale for this is that anti-social attitudes and a negative school climate are associated with lower academic achievement, current behavior problems, as well as future criminal behaviors (DeLisi et al 2013; Hawkins et al 2000) and suicide ideation (King et al 2001). There are hundreds of anonymous school climate surveys, but none of them was designed to provide the kind of information that we need to help individual students.

Do We Know How To Hold Teacher Preparation Programs Accountable?

This piece is co-authored by Cory Koedel and Matthew Di Carlo. Koedel is an Associate Professor of Economics and Public Policy at the University of Missouri, Columbia.

The United States Department of Education (USED) has proposed regulations requiring states to hold teacher preparation programs accountable for the performance of their graduates. According to the proposal, states must begin assigning ratings to each program within the next 2-3 years, based on outcomes such as graduates’ “value-added” to student test scores, their classroom observation scores, how long they stay in teaching, whether they teach in high-needs schools, and surveys of their principals’ satisfaction.

In the long term, we are very receptive to, and indeed optimistic about, the idea of outcomes-based accountability for teacher preparation programs (TPPs). In the short to medium term, however, we contend that the evidence base underlying the USED regulations is nowhere near sufficient to guide a national effort toward high-stakes TPP accountability.

This is a situation in which the familiar refrain of “it’s imperfect but better than nothing” is false, and rushing into nationwide design and implementation could be quite harmful.