Skip to:

School Effects

  • How Relationships Drive School Improvement—And Actionable Data Foster Strong Relationships

    Written on April 20, 2017

    Our guest authors today are Elaine Allensworth, Molly Gordon and Lucinda Fickel. Allensworth is Lewis-Sebring Director of the University of Chicago Consortium on School Research; Gordon is Senior Research Analyst at the University of Chicago Consortium on School Research; and Fickel is Associate Director of Policy at the University of Chicago Urban Education Institute. Elaine Allensworth explores this topic further in Teaching in Context: The Social Side of Education Reform edited by Esther Quintero (Harvard Education Press: 2017). 

    As researchers at the UChicago Consortium on School Research, we believe in using data to support school improvement, such as data on students’ performance in school (attendance, grades, behavior, test scores), surveys of students and teachers on their school experiences. But data does nothing on its own. In the quarter-century that our organization has been conducting research on Chicago Public Schools, one factor has emerged time and time again as vital both for making good use of data, and the key element in school improvement: relationships.

    Squishy and amorphous as it might initially sound, there is actually solid empirical grounding not only about the importance of relationships for student learning, but also about the organizational factors that foster strong relationships. In 2010, the Consortium published Organizing Schools for Improvement, which drew on a decade of administrative and survey data to examine a framework called the 5Essentials (Bryk et al. 2010). The book details findings that elementary/middle schools strong on the 5Essentials—strong leaders, professional capacity, parent-community ties, instructional guidance, and a student-centered learning climate—were highly likely to improve, while others showed little change or fell behind.

    READ MORE
  • Do Subgroup Accountability Measures Affect School Ratings Systems?

    Written on October 28, 2016

    The school accountability provisions of No Child Left Behind (NCLB) institutionalized a focus on the (test-based) performance of student subgroups, such as English language learners, racial and ethnic groups, and students eligible for free- and reduced-price lunch (FRL). The idea was to shine a spotlight on achievement gaps in the U.S., and to hold schools accountable for serving all students.

    This was a laudable goal, and disaggregating data by student subgroups is a wise policy, as there is much to learn from such comparisons. Unfortunately, however, NCLB also institutionalized the poor measurement of school performance, and so-called subgroup accountability was not immune. The problem, which we’ve discussed here many times, is that test-based accountability systems in the U.S. tend to interpret how highly students score as a measure of school performance, when it is largely a function of factors out of schools' control, such as student background. In other words, schools (or subgroups of those students) may exhibit higher average scores or proficiency rates simply because their students entered the schools at higher levels, regardless of how effective the school may be in raising scores. Although NCLB’s successor, the Every Student Succeeds Act (ESSA), perpetuates many of these misinterpretations, it still represents some limited progress, as it encourages greater reliance on growth-based measures, which look at how quickly students progress while they attend a school, rather than how highly they score in any given year (see here for more on this).

    Yet this evolution, slow though it may be, presents a somewhat unique challenge for the inclusion of subgroup-based measures in formal school accountability systems. That is, if we stipulate that growth model estimates are the best available test-based way to measure school (rather than student) performance, how should accountability systems apply these models to traditionally lower scoring student subgroups?

    READ MORE
  • Thinking About Tests While Rethinking Test-Based Accountability

    Written on August 5, 2016

    Earlier this week, per the late summer ritual, New York State released its testing results for the 2015-2016 school year. New York City (NYC), always the most closely watched set of results in the state, showed a 7.6 percentage point increase in its ELA proficiency rate, along with a 1.2 percentage point increase in its math rate. These increases were roughly equivalent to the statewide changes.

    City officials were quick to pounce on the results, which were called “historic,” and “pure hard evidence” that the city’s new education policies are working. This interpretation, while standard in the U.S. education debate, is, of course, inappropriate for many reasons, all of which we’ve discussed here countless times and will not detail again (see here). Suffice it to say that even under the best of circumstances these changes in proficiency rates are only very tentative evidence that students improved their performance over time, to say nothing of whether that improvement was due to a specific policy or set of policies.

    Still, the results represent good news. A larger proportion of NYC students are scoring proficient in math and ELA than did last year. Real improvement is slow and sustained, and this is improvement. In addition, the proficiency rate in NYC is now on par with the statewide rate, which is unprecedented. There are, however, a couple of additional issues with these results that are worth discussing quickly.

    READ MORE
  • A Small But Meaningful Change In Florida's School Grades System

    Written on July 28, 2016

    Beginning in the late 1990s, Florida became one of the first states to assign performance ratings to public schools. The purpose of these ratings, which are in the form of A-F grades, is to communicate to the public “how schools are performing relative to state standards.” For elementary and middle schools, the grades are based entirely on standardized testing results.

    We have written extensively here about Florida’s school grading system (see here for just one example), and have used it to illustrate features that can be found in most other states’ school ratings. The primary issue is the heavy reliance that states place on how highly students score on tests, which tells you more about the students the schools serve than about how well they serve those students – i.e., it conflates school and student performance. Put simply, some schools exhibit lower absolute testing performance levels than do other schools, largely because their students enter performing at lower levels. As a result, schools in poorer neighborhoods tend to receive lower grades, even though many of these schools are very successful in helping their students make fast progress during their few short years of attendance.

    Although virtually every states’ school rating system has this same basic structure to varying degrees, Florida’s system warrants special attention, as it was one of the first in the nation and has been widely touted and copied (as well as researched -- see our policy brief for a review of this evidence). It is also noteworthy because it contains a couple of interesting features, one of which exacerbates the aforementioned conflation of student and school performance in a largely unnoticed manner. But, this feature, discussed below, has just been changed by the Florida Department of Education (FLDOE). This correction merits discussion, as it may be a sign of improvement in how policymakers think about these systems.

    READ MORE
  • Caring School Leadership

    Written on March 31, 2016

    Our guest authors today are Mark A. Smylie, professor emeritus at the University of Illinois-Chicago, Joseph Murphy, professor at Peabody College, Vanderbilt University, and Karen Seashore Louis, professor at the University of Minnesota.  Their research concerns school organization, leadership, and improvement.  This blog post is based on an article titled “Caring School Leadership: A Multi-Disciplinary, Cross-Occupational Model” which will be published later this year in the American Journal of Education.

    From our years of studying school leadership and reform, working with practicing educators, and participating in education policy development, we have come to the conclusion that caring lies at the heart of effective schooling and good school leadership.  In this time of intense academic pressures, accountability policies, and top-down approaches to reform, however, the concept of caring has been neglected, overshadowed by attention to more “objective”, task-oriented aspects of school organization and leadership (Cassidy & Bates, 2005; Richert, 1994 (pp.109-118); Rooney, 2015).  This, we contend, is a serious problem for both students and teachers.

    In this blog, we share some of our recent thinking about what caring school leadership is and why it is important. We draw on empirical and theoretical literatures from education and from disciplines outside education, particularly research on human service occupations such as health care, social services, and the ministry. And we present a model of caring school leadership. Our ideas were developed with principals in mind, but they apply to any educator engaged in school leadership work. We focus on students as the primary beneficiaries of caring. It should be noted that, as we argue for the importance of caring in schools, we do not mean to diminish the importance of academic achievement nor the need to care for staff and the community. We consider managing mutually-reinforcing combinations of caring support and academic press a central function of school leadership.

    READ MORE
  • The Narrative Of School Failure And Why We Must Pay Attention To Segregation In Educational Policy

    Written on January 28, 2016

    Our guest authors today are Kara S. Finnigan, Associate Professor at the Warner School of Education of the University of Rochester, and Jennifer Jellison Holme, Associate Professor at the University of Texas at Austin.  Finnigan and Holme have published several articles and briefs on the issue of school integration including articles in press in Teachers College Record and Educational Law and Policy Review as well as a research brief for the National Coalition on School Diversity. This is the first of a two-part blog series on this topic.

    Imagine that you wake up one morning with a dull pain in your tooth. You take ibuprofen, apply an ice pack, and try to continue as if things are normal.  But as the pain continues to grow over the next few days, you realize that deep down there is a problem – and you are reminded of this every so often when you bite down and feel a shooting pain.  Eventually, you can’t take it any longer and get an x-ray at the dentist’s office, only to find out that what was originally a small problem has spread throughout the whole tooth and you need a root canal.  Now you wish you hadn’t waited so long.

    Why are we talking about a root canal in a blog post about education? As we thought about how to convey the way we see the situation with low-performing schools, this analogy seemed to capture our point. Most of us can relate to what happens when we overlook a problem with our teeth, and yet we don’t pay attention to what can happen when we overlook the underlying problems that affect educational systems.

    In this blog post, we argue that school segregation by race and poverty is one of the underlying causes of school failure, and that it has been largely overlooked in federal and state educational policy in recent decades.

    READ MORE
  • Trust: The Foundation Of Student Achievement

    Written on May 21, 2015

    When sharing with me the results of some tests, my doctor once said, "You are a scientist, you know a single piece of data can't provide all the answers or suffice to make a diagnosis. We can't look at a single number in isolation, we need to look at all results in combination." Was my doctor suggesting that I ignore that piece of information we had? No. Was my doctor deemphasizing the result? No. He simply said that we needed additional evidence to make informed decisions. This is, of course, correct.

    In education, however, it is frequently implied or even stated directly that the bottom line when it comes to school performance is student test scores, whereas any other outcomes, such as cooperation between staff or a supportive learning environment, are ultimately "soft" and, at best, of secondary importance. This test-based, individual-focused position is viewed as serious, rigorous, and data driven. Deviation from it -- e.g., equal emphasis on additional, systemic aspects of schools and the people in them -- is sometimes derided as an evidence-free mindset. Now, granted, few people are “purely” in one camp or the other. Most probably see themselves as pragmatists, and, as such, somewhere in between: Test scores are probably not all that matters, but since the rest seems so difficult to measure, we might as well focus on "hard data" and hope for the best.

    Why this narrow focus on individual measures such as student test scores or teacher quality? I am sure there are many reasons but one is probably lack of familiarity with the growing research showing that we must go beyond the individual teacher and student and examine the social-organizational aspects of schools, which are associated (most likely causally) with student achievement. In other words, all the factors skeptics and pragmatists might think are a distraction and/or a luxury, are actually relevant for the one thing we all care about: Student achievement. Moreover, increasing focus on these factors might actually help us understand what’s really important: Not simply whether testing results went up or down, but why or why not.

    READ MORE
  • The Status Fallacy: New York State Edition

    Written on March 5, 2015

    A recent New York Times story addresses directly New York Governor Andrew Cuomo’s suggestion, in his annual “State of the State” speech, that New York schools are in a state of crisis and "need dramatic reform." The article’s general conclusion is that the “data suggest otherwise.”

    There are a bunch of important points raised in the article, but most of the piece is really just discussing student rather than school performance. Simple statistics about how highly students score on tests – i.e., “status measures” – tell you virtually nothing about the effectiveness of the schools those students attend, since, among other reasons, they don’t account for the fact that many students enter the system at low levels. How much students in a school know in a given year is very different from how much they learned over the course of that year.

    I (and many others) have written about this “status fallacy” dozens of times (see our resources page), not because I enjoy repeating myself (I don’t), but rather because I am continually amazed just how insidious it is, and how much of an impact it has on education policy and debate in the U.S. And it feels like every time I see signs that things might be changing for the better, there is an incident, such as Governor Cuomo’s speech, that makes me question how much progress there really has been at the highest levels.

    READ MORE
  • Turning Conflict Into Trust Improves Schools And Student Learning

    Written on March 3, 2015

    Our guest author today is Greg Anrig, vice president of policy and programs at The Century Foundation and author of Beyond the Education Wars: Evidence That Collaboration Builds Effective Schools.

    In recent years, a number of studies (discussed below; also see here and here) have shown that effective public schools are built on strong collaborative relationships, including those between administrators and teachers. These findings have helped to accelerate a movement toward constructing such partnerships in public schools across the U.S. However, the growing research and expanding innovations aimed at nurturing collaboration have largely been neglected by both mainstream media and the policy community.

    Studies that explore the question of what makes successful schools work never find a silver bullet, but they do consistently pinpoint commonalities in how those schools operate. The University of Chicago's Consortium on Chicago School Research produced the most compelling research of this type, published in a book called Organizing Schools for Improvement. The consortium gathered demographic and test data, and conducted extensive surveys of stakeholders, in more than 400 Chicago elementary schools from 1990 to 2005. That treasure trove of information enabled the consortium to identify with a high degree of confidence the organizational characteristics and practices associated with schools that produced above-average improvement in student outcomes.

    The most crucial finding was that the most effective schools, based on test score improvement over time after controlling for demographic factors, had developed an unusually high degree of "relational trust" among their administrators, teachers, and parents.

    READ MORE
  • The Persistent Misidentification Of "Low Performing Schools"

    Written on February 3, 2015

    In education, we hear the terms “failing school” and “low-performing school” quite frequently. Usually, they are used in soundbyte-style catchphrases such as, “We can’t keep students trapped in ‘failing schools.’” Sometimes, however, they are used to refer to a specific group of schools in a given state or district that are identified as “failing” or “low-performing” as part of a state or federal law or program (e.g., waivers, SIG). There is, of course, interstate variation in these policies, but one common definition is that schools are “failing/low-performing” if their proficiency rates are in the bottom five percent statewide.

    Putting aside the (important) issues with judging schools based solely on standardized testing results, low proficiency rates (or low average scores) tell you virtually nothing about whether or not a school is “failing.” As we’ve discussed here many times, students enter their schools performing at different levels, and schools cannot control the students they serve, only how much progress those students make while they’re in attendance (see here for more).

    From this perspective, then, there may be many schools that are labeled “failing” or “low performing” but are actually of above average effectiveness in raising test scores. And, making things worse, virtually all of these will be schools that serve the most disadvantaged students. If that’s true, it’s difficult to think of anything more ill-advised than closing these schools, or even labeling them as “low performing.” Let’s take a quick, illustrative look at this possibility using the “bottom five percent” criterion, and data from Colorado in 2013-14 (note that this simple analysis is similar to what I did in this post, but this one is a little more specific; also see Glazerman and Potamites 2011; Ladd and Lauen 2010; and especially Chingos and West 2015).

    READ MORE

Pages

Subscribe to School Effects

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.