Skip to:

School Effects

  • Where Do Achievement Gaps Come From?

    Written on August 10, 2017

    For almost two decades now, educational accountability policy in the U.S. has included a focus on the performance of student subgroups, such as those defined by race and ethnicity, income, or special education status. The (very sensible) logic behind this focus is the simple fact that aggregate performance measures, whether at the state-, district-, or school levels, often mask large gaps between subgroups.

    Yet one of the unintended consequences of this subgroup focus has been confusion among both policymakers and the public as to how to interpret and use subgroup indicators in formal school accountability systems, particularly when those indicators are expressed as simple “achievement gaps” or “gap closing” measures. This is not only because achievement gaps can narrow for undesirable reasons and widen for desirable reasons, but also because many gaps exist prior to entry into the school (or district). If, for instance, a large Hispanic/White achievement gap for a given cohort exists at the start of kindergarten, it is misleading and potentially damaging to hold a school accountable for the persistence of that gap in later grades – particularly in cases where public policy has failed to provide the extra resources and supports that might help lower-performing students make accelerated achievement gains every year. In addition, the coarseness of current educational variables, particularly those usually used as income proxies, limits the detail and utility of some subgroup measures.

    A helpful and timely little analysis by David Figlio and Krzystof Karbownik, published by the Brookings Institution, addresses some of these issues, and the findings have clear policy implications.

    READ MORE
  • For Florida's School Grading System, A Smart Change With Unexpected Effects

    Written on July 13, 2017

    Last year, we discussed a small but potentially meaningful change that Florida made to its school grading system, one that might have attenuated a long-standing tendency of its student “gains” measures, by design, to favor schools that serve more advantaged students. Unfortunately, this result doesn’t seem to have been achieved.

    Prior to 2014-15, one of the criteria by which Florida students could be counted as having “made gains” was scoring as proficient or better in two consecutive years, without having dropped a level (e.g., from advanced to proficient). Put simply, this meant that students scoring above the proficiency threshold would be counted as making “gains,” even if they in fact made only average or even below average progress, so long as they stayed above the line. As a result of this somewhat crude “growth” measure, schools serving large proportions of students scoring above the proficiency line (i.e., schools in affluent neighborhoods) were virtually guaranteed to receive strong “gains” scores. Such “double counting” in the “gains” measures likely contributed to a very strong relationship between schools’ grades and their students’ socio-economic status (as gauged, albeit roughly, by subsidized lunch eligibility rates).

    Florida, to its credit, changed this “double counting” rule effective in 2014-15. Students who score as proficient in two consecutive years are no longer automatically counted as making “gains.” They must also exhibit some score growth in order to receive the designation.

    READ MORE
  • Improving Accountability Measurement Under ESSA

    Written on May 25, 2017

    Despite the recent repeal of federal guidelines for states’ compliance with the Every Student Succeeds Act (ESSA), states are steadily submitting their proposals, and they are rightfully receiving some attention. The policies in these proposals will have far-reaching consequences for the future of school accountability (among many other types of policies), as well as, of course, for educators and students in U.S. public schools.

    There are plenty of positive signs in these proposals, which are indicative of progress in the role of proper measurement in school accountability policy. It is important to recognize this progress, but impossible not to see that ESSA perpetuates long-standing measurement problems that were institutionalized under No Child Left Behind (NCLB). These issues, particularly the ongoing failure to distinguish between student and school performance, continue to dominate accountability policy to this day. Part of the confusion stems from the fact that school and student performance are not independent of each other. For example, a test score, by itself, gauges student performance, but it also reflects, at least in part, school effectiveness (i.e., the score might have been higher or lower had the student attended a different school).

    Both student and school performance measures have an important role to play in accountability, but distinguishing between them is crucial. States’ ESSA proposals make the distinction in some respects but not in others. The result may end up being accountability systems that, while better than those under NCLB, are still severely hampered by improper inference and misaligned incentives. Let’s take a look at some of the key areas where we find these issues manifested.

    READ MORE
  • How Relationships Drive School Improvement—And Actionable Data Foster Strong Relationships

    Written on April 20, 2017

    Our guest authors today are Elaine Allensworth, Molly Gordon and Lucinda Fickel. Allensworth is Lewis-Sebring Director of the University of Chicago Consortium on School Research; Gordon is Senior Research Analyst at the University of Chicago Consortium on School Research; and Fickel is Associate Director of Policy at the University of Chicago Urban Education Institute. Elaine Allensworth explores this topic further in Teaching in Context: The Social Side of Education Reform edited by Esther Quintero (Harvard Education Press: 2017). 

    As researchers at the UChicago Consortium on School Research, we believe in using data to support school improvement, such as data on students’ performance in school (attendance, grades, behavior, test scores), surveys of students and teachers on their school experiences. But data does nothing on its own. In the quarter-century that our organization has been conducting research on Chicago Public Schools, one factor has emerged time and time again as vital both for making good use of data, and the key element in school improvement: relationships.

    Squishy and amorphous as it might initially sound, there is actually solid empirical grounding not only about the importance of relationships for student learning, but also about the organizational factors that foster strong relationships. In 2010, the Consortium published Organizing Schools for Improvement, which drew on a decade of administrative and survey data to examine a framework called the 5Essentials (Bryk et al. 2010). The book details findings that elementary/middle schools strong on the 5Essentials—strong leaders, professional capacity, parent-community ties, instructional guidance, and a student-centered learning climate—were highly likely to improve, while others showed little change or fell behind.

    READ MORE
  • Do Subgroup Accountability Measures Affect School Ratings Systems?

    Written on October 28, 2016

    The school accountability provisions of No Child Left Behind (NCLB) institutionalized a focus on the (test-based) performance of student subgroups, such as English language learners, racial and ethnic groups, and students eligible for free- and reduced-price lunch (FRL). The idea was to shine a spotlight on achievement gaps in the U.S., and to hold schools accountable for serving all students.

    This was a laudable goal, and disaggregating data by student subgroups is a wise policy, as there is much to learn from such comparisons. Unfortunately, however, NCLB also institutionalized the poor measurement of school performance, and so-called subgroup accountability was not immune. The problem, which we’ve discussed here many times, is that test-based accountability systems in the U.S. tend to interpret how highly students score as a measure of school performance, when it is largely a function of factors out of schools' control, such as student background. In other words, schools (or subgroups of those students) may exhibit higher average scores or proficiency rates simply because their students entered the schools at higher levels, regardless of how effective the school may be in raising scores. Although NCLB’s successor, the Every Student Succeeds Act (ESSA), perpetuates many of these misinterpretations, it still represents some limited progress, as it encourages greater reliance on growth-based measures, which look at how quickly students progress while they attend a school, rather than how highly they score in any given year (see here for more on this).

    Yet this evolution, slow though it may be, presents a somewhat unique challenge for the inclusion of subgroup-based measures in formal school accountability systems. That is, if we stipulate that growth model estimates are the best available test-based way to measure school (rather than student) performance, how should accountability systems apply these models to traditionally lower scoring student subgroups?

    READ MORE
  • Thinking About Tests While Rethinking Test-Based Accountability

    Written on August 5, 2016

    Earlier this week, per the late summer ritual, New York State released its testing results for the 2015-2016 school year. New York City (NYC), always the most closely watched set of results in the state, showed a 7.6 percentage point increase in its ELA proficiency rate, along with a 1.2 percentage point increase in its math rate. These increases were roughly equivalent to the statewide changes.

    City officials were quick to pounce on the results, which were called “historic,” and “pure hard evidence” that the city’s new education policies are working. This interpretation, while standard in the U.S. education debate, is, of course, inappropriate for many reasons, all of which we’ve discussed here countless times and will not detail again (see here). Suffice it to say that even under the best of circumstances these changes in proficiency rates are only very tentative evidence that students improved their performance over time, to say nothing of whether that improvement was due to a specific policy or set of policies.

    Still, the results represent good news. A larger proportion of NYC students are scoring proficient in math and ELA than did last year. Real improvement is slow and sustained, and this is improvement. In addition, the proficiency rate in NYC is now on par with the statewide rate, which is unprecedented. There are, however, a couple of additional issues with these results that are worth discussing quickly.

    READ MORE
  • A Small But Meaningful Change In Florida's School Grades System

    Written on July 28, 2016

    Beginning in the late 1990s, Florida became one of the first states to assign performance ratings to public schools. The purpose of these ratings, which are in the form of A-F grades, is to communicate to the public “how schools are performing relative to state standards.” For elementary and middle schools, the grades are based entirely on standardized testing results.

    We have written extensively here about Florida’s school grading system (see here for just one example), and have used it to illustrate features that can be found in most other states’ school ratings. The primary issue is the heavy reliance that states place on how highly students score on tests, which tells you more about the students the schools serve than about how well they serve those students – i.e., it conflates school and student performance. Put simply, some schools exhibit lower absolute testing performance levels than do other schools, largely because their students enter performing at lower levels. As a result, schools in poorer neighborhoods tend to receive lower grades, even though many of these schools are very successful in helping their students make fast progress during their few short years of attendance.

    Although virtually every states’ school rating system has this same basic structure to varying degrees, Florida’s system warrants special attention, as it was one of the first in the nation and has been widely touted and copied (as well as researched -- see our policy brief for a review of this evidence). It is also noteworthy because it contains a couple of interesting features, one of which exacerbates the aforementioned conflation of student and school performance in a largely unnoticed manner. But, this feature, discussed below, has just been changed by the Florida Department of Education (FLDOE). This correction merits discussion, as it may be a sign of improvement in how policymakers think about these systems.

    READ MORE
  • Caring School Leadership

    Written on March 31, 2016

    Our guest authors today are Mark A. Smylie, professor emeritus at the University of Illinois-Chicago, Joseph Murphy, professor at Peabody College, Vanderbilt University, and Karen Seashore Louis, professor at the University of Minnesota.  Their research concerns school organization, leadership, and improvement.  This blog post is based on an article titled “Caring School Leadership: A Multi-Disciplinary, Cross-Occupational Model” which will be published later this year in the American Journal of Education.

    From our years of studying school leadership and reform, working with practicing educators, and participating in education policy development, we have come to the conclusion that caring lies at the heart of effective schooling and good school leadership.  In this time of intense academic pressures, accountability policies, and top-down approaches to reform, however, the concept of caring has been neglected, overshadowed by attention to more “objective”, task-oriented aspects of school organization and leadership (Cassidy & Bates, 2005; Richert, 1994 (pp.109-118); Rooney, 2015).  This, we contend, is a serious problem for both students and teachers.

    In this blog, we share some of our recent thinking about what caring school leadership is and why it is important. We draw on empirical and theoretical literatures from education and from disciplines outside education, particularly research on human service occupations such as health care, social services, and the ministry. And we present a model of caring school leadership. Our ideas were developed with principals in mind, but they apply to any educator engaged in school leadership work. We focus on students as the primary beneficiaries of caring. It should be noted that, as we argue for the importance of caring in schools, we do not mean to diminish the importance of academic achievement nor the need to care for staff and the community. We consider managing mutually-reinforcing combinations of caring support and academic press a central function of school leadership.

    READ MORE
  • The Narrative Of School Failure And Why We Must Pay Attention To Segregation In Educational Policy

    Written on January 28, 2016

    Our guest authors today are Kara S. Finnigan, Associate Professor at the Warner School of Education of the University of Rochester, and Jennifer Jellison Holme, Associate Professor at the University of Texas at Austin.  Finnigan and Holme have published several articles and briefs on the issue of school integration including articles in press in Teachers College Record and Educational Law and Policy Review as well as a research brief for the National Coalition on School Diversity. This is the first of a two-part blog series on this topic.

    Imagine that you wake up one morning with a dull pain in your tooth. You take ibuprofen, apply an ice pack, and try to continue as if things are normal.  But as the pain continues to grow over the next few days, you realize that deep down there is a problem – and you are reminded of this every so often when you bite down and feel a shooting pain.  Eventually, you can’t take it any longer and get an x-ray at the dentist’s office, only to find out that what was originally a small problem has spread throughout the whole tooth and you need a root canal.  Now you wish you hadn’t waited so long.

    Why are we talking about a root canal in a blog post about education? As we thought about how to convey the way we see the situation with low-performing schools, this analogy seemed to capture our point. Most of us can relate to what happens when we overlook a problem with our teeth, and yet we don’t pay attention to what can happen when we overlook the underlying problems that affect educational systems.

    In this blog post, we argue that school segregation by race and poverty is one of the underlying causes of school failure, and that it has been largely overlooked in federal and state educational policy in recent decades.

    READ MORE
  • Trust: The Foundation Of Student Achievement

    Written on May 21, 2015

    When sharing with me the results of some tests, my doctor once said, "You are a scientist, you know a single piece of data can't provide all the answers or suffice to make a diagnosis. We can't look at a single number in isolation, we need to look at all results in combination." Was my doctor suggesting that I ignore that piece of information we had? No. Was my doctor deemphasizing the result? No. He simply said that we needed additional evidence to make informed decisions. This is, of course, correct.

    In education, however, it is frequently implied or even stated directly that the bottom line when it comes to school performance is student test scores, whereas any other outcomes, such as cooperation between staff or a supportive learning environment, are ultimately "soft" and, at best, of secondary importance. This test-based, individual-focused position is viewed as serious, rigorous, and data driven. Deviation from it -- e.g., equal emphasis on additional, systemic aspects of schools and the people in them -- is sometimes derided as an evidence-free mindset. Now, granted, few people are “purely” in one camp or the other. Most probably see themselves as pragmatists, and, as such, somewhere in between: Test scores are probably not all that matters, but since the rest seems so difficult to measure, we might as well focus on "hard data" and hope for the best.

    Why this narrow focus on individual measures such as student test scores or teacher quality? I am sure there are many reasons but one is probably lack of familiarity with the growing research showing that we must go beyond the individual teacher and student and examine the social-organizational aspects of schools, which are associated (most likely causally) with student achievement. In other words, all the factors skeptics and pragmatists might think are a distraction and/or a luxury, are actually relevant for the one thing we all care about: Student achievement. Moreover, increasing focus on these factors might actually help us understand what’s really important: Not simply whether testing results went up or down, but why or why not.

    READ MORE

Pages

Subscribe to School Effects

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.