Accountability

How Non-Zero Tolerance Policies Better Support Our Students: Part II

As I discussed in a previous post, one of the most controversial approaches to school discipline in the U.S. is the use of zero-tolerance policies. These policies include exclusionary practices, such as office referrals and suspension, which remove students from their classroom and isolate them from the school community. Zero tolerance policies in schools have been shown to have a detrimental effect on all students, particularly Black and Brown students. Skiba et al (2011), for instance, wrote about how these punitive methods cause students to miss critical instructional time and feel less connected to their teachers and peers. 

Zero tolerance policies are embedded in high-stakes accountability structures. As White (2020) states, these policies overly focus on student behavior and the idea that individual hard work is the best way to promote high test scores. They do not foster a sense of community- and relationship- building. While policymakers had positive intentions in promoting a more rigorous and equalitarian experience for students—laying out each infraction and punishment with the intention of applying discipline uniformly across student groups—that is not what has happened. According to the aforementioned research, Black and Brown students were still punished more harshly for the same infractions than were their peers. Thus, the negative consequences of these policies have far outweighed the benefits. 

Many schools are implementing alternative methods of discipline that stress the importance of taking proactive measures to reduce exclusionary practices. In the previous post, I focused on the importance of restorative justice policies as a strong strategy to support children and their development. But there are also multiple alternative models that have been shown to be effective among students of varying ages and demographics. These models focus on relationship development, and staff training, which I will discuss below. Specifically, the three other models include: School Wide Positive Behavior Support (SWPBS), Monarch Room, and Inclusive Skill-Building Learning Approach (ISLA). In short, the idea that zero tolerance is the only approach is unsupportable.

The Uncertain Short-Term Future Of School Growth Models

Over the past 20 years, public schools in the U.S. have come to rely more and more on standardized tests, and the COVID-19 pandemic has halted the flow of these data. This is hardly among the most important disruptions that teachers, parents, and students have endured over the past year or so. But one of the corollaries of skipping a year (or more) of testing is its implications for estimating growth models, which are statistical approaches for assessing the association between students' testing progress and those students' teachers, schools, or districts. 

This type of information, used properly, is always potentially useful, but it may be particularly timely right now, as we seek to understand how the COVID-19 pandemic affected educational outcomes, and, perhaps, how those outcomes varied by different peri-pandemic approaches to schooling. This includes the extent to which there were meaningful differences by student subgroup (e.g., low-income students who may have had more issues with virtual schooling). 

To be clear, the question of when states should resume testing should be evaluated based on what’s best for schools and students, and in my view this decision should not include consideration of any impact on accountability systems (the latest development is that states will not be allowed to cancel testing entirely but may be allowed to curtail it). In either case, though, the fate of growth models over the next couple of years is highly uncertain. The models rely on tracking student test scores over time, and so skipping a year (and maybe even more) is obviously a potential problem. A new working paper takes a first step toward assessing the short-term feasibility of growth estimates (specifically school and district scores). But this analysis also provides a good context for a deeper discussion of how we use (and sometimes misuse) testing data in education policy.

Social Identity Development In The Age Of Accountability

According to a recent NPR article, the “majority of parents” do not talk to their children about social identity, which refers to group membership based on characteristics such as religion, gender, national origin, race, family makeup, and socio-economic status. The article presents results from a report, co-published by The Sesame Workshop and NORC at the University of Chicago, called the Identity Matters Study. The study includes survey responses from 6,000 parents about their children’s sense of identity at home and in the classroom, as well as results from a second survey of 1,046 educators’ perspectives on identity development in school. 

Many readers were quick to respond that NPR’s headline was misleading, pointing out that the wording should have been, “White parents rarely, if ever, discuss ethnicity, gender, class or other identity categories with their kids…” This objection has merit. The report does show parent responses by race for just five survey questions, but the data confirm that White parents are far less likely than Black parents to talk about identity with students, with 6 percent of White parents and 22 percent of Black parents answering that they often talk about race. Nevertheless, the study concludes that, overall, 60 percent of parents rarely or never talk about race, ethnicity, or social class with their children. In the second survey, which queried educators, the study found that one third of teachers had a student affected by a negative comment targeting their social identity, but that most teachers feel unprepared or uncomfortable when it comes to navigating conversations on the matter. 

It is clear that we need more comprehensive survey data, including more questions broken down by race and other demographic indicators, in order to have a better understanding of how children are developing a sense of social identity at home. In the meantime, what we can gather from this article is that many students, particularly White students, do not develop an awareness of their own complex social identity until they get to school, and that this awareness often comes via negative comments.

How can schools do better?

The False Choice Of Growth Versus Proficiency

Tennessee is considering changing its school accountability system such that schools have the choice of having their test-based performance judged by either status (how highly students score) or growth (how much progress students make over the course of the year). In other words, if schools do poorly on one measure, they are judged by the other (apparently, Texas already has a similar system in place).

As we’ve discussed here many times in the past, status measures, such as proficiency rates, are poor measures of school performance, since some students, particularly those living in poverty, enter their schools far behind their more affluent peers. As a result, schools serving larger proportions of poor students will exhibit lower scores and proficiency rates, even if they are very effective in compelling progress from their students. That is why growth models, which focus on individual student gains on over time, are a superior measure of school performance per se.

This so-called “growth versus proficiency” debate has resurfaced several times over the years, and it was particularly prevalent during the time when states were submitting proposals for their accountability systems during reauthorization of the Elementary and Secondary Education Act. The policy that came out of these discussions was generally promising, as many states moved at least somewhat toward weighting growth model estimates more heavily. 

At the same time, however, it is important to mention that the “growth versus proficiency” debate sometimes implies that states must choose between these two types of indicators. This is misleading. And the Tennessee proposal is a very interesting context for discussing this, since they are essentially using these two types of measures interchangeably. The reality, of course, is that both types of measures transmit valuable but different information, and both have a potentially useful role to play in accountability systems.

Tests Worth Teaching To

Our guest authors today are Chester E. Finn, Jr. and Andrew E. Scanlan. Finn is a distinguished senior fellow and president emeritus at the Thomas B. Fordham Institute and a senior fellow at Stanford University’s Hoover Institution. Scanlan is a research and policy associate at the Thomas B. Fordham Institute.

This year, some 165,000 American educators are teaching Advanced Placement (AP) classes—a veritable army, mobilized to serve some three million students as they embark on coursework leading to the AP program’s rigorous three-hour exams each May. As we explore in our new book, Learning in the Fast Lane: The Past, Present and Future of Advanced Placement, preparing these young people to succeed on the tests (scored from 1 to 5, with 3 or better deemed “qualifying”) is a major instructional objective for teachers as well as for the students (and their families) who recognize the program’s potential to significantly enhance their post-secondary prospects.

For AP teachers, one might suppose that this objective would be vexing—yet another end-of-year exam that will constrain their curricular choices, stunt their classroom autonomy, and turn their pupils into cram-and-memorize machines rather than eager, deeper learners, creative thinkers, and inquisitive intellectuals.

One might also suppose that the AP program, as it has infiltrated 70 percent of U.S. public (and half of private) high schools, would be vulnerable to the anti-testing resentments and revolts of recent years. These have been largely driven by government-imposed school accountability regimes that are mostly based on the scores kids get on state-mandated assessments, especially in math and English. That’s led many schools to press teachers to devote more hours to “test prep,” minimize time spent on other subjects, and neglect topics that aren’t included in state standards (and therefore won’t be tested). It’s not unreasonable, then, to expect resistance to AP as well.

The Offline Implications Of The Research About Online Charter Schools

It’s rare to find an educational intervention with as unambiguous a research track record as online charter schools. Now, to be clear, it’s not a large body of research by any stretch, its conclusions may change in time, and the online charter sub-sector remains relatively small and concentrated in a few states. For now, though, the results seem incredibly bad (Zimmer et al. 2009; Woodworth et al. 2015). In virtually every state where these schools have been studied, across virtually all student subgroups, and in both reading and math, the estimated impact of online charter schools on student testing performance is negative and large in magnitude.

Predictably, and not without justification, those who oppose charter schools in general are particularly vehement when it comes to online charter schools – they should, according to many of these folks, be closed down, even outlawed. Charter school supporters, on the other hand, tend to acknowledge the negative results (to their credit) but make less drastic suggestions, such as greater oversight, including selective closure, and stricter authorizing practices.

Regardless of your opinion on what to do about online charter schools’ poor (test-based) results, they are truly an interesting phenomenon for a few reasons.

Why Teacher Evaluation Reform Is Not A Failure

The RAND Corporation recently released an important report on the impact of the Gates Foundation’s “Intensive Partnerships for Effective Teaching” (IPET) initiative. IPET was a very thorough and well-funded attempt to improve teaching quality in schools in three districts and four charter management organizations (CMOs). The initiative was multi-faceted, but its centerpiece was the implementation of multi-measure teacher evaluation systems and the linking of ratings from those systems to professional development and high stakes personnel decisions, including compensation, tenure, and dismissal. This policy, particularly the inclusion in teacher evaluations of test-based productivity measures (e.g., value-added scores), has been among the most controversial issues in education policy throughout the past 10 years.

The report is extremely rich and there's a lot of interesting findings in there, so I would encourage everyone to read it themselves (at least the executive summary), but the headline finding was that the IPET had no discernible effect on student outcomes, namely test scores and graduation rates, in the districts that participated, vis-à-vis similar districts that did not. Given that IPET was so thoroughly designed and implemented, and that it was well-funded, it can potentially be viewed as a "best case scenario" test of the type of evaluation reform that most states have enacted. Accordingly, critics of these reforms, who typically focus their opposition on the high stakes use of evaluation measures, particularly value-added and other test-based measures, in these evaluations, have portrayed the findings as vindication of their opposition. 

This reaction has merit. The most important reason why is that evaluation reform was portrayed by advocates as a means to immediate and drastic improvements in student outcomes. This promise was misguided from the outset, and evaluation reform opponents are (and were) correct in pointing this out. At the same time, however, it would be wise not to dismiss evaluation reform as a whole, for several reasons, a few of which are discussed below.

The Theory And Practice Of School Closures

The idea of closing “low performing schools” has undeniable appeal, at least in theory. The basic notion is that some schools are so dysfunctional that they cannot be saved and may be doing irreparable harm to their students every day they are open. Thus, it is argued, closing such schools and sending their students elsewhere is the best option – even if students end up in “average” schools, proponents argue, they will be better off.

Such closures are very controversial, however, and for good reason. For one thing, given adequate time and resources, schools may improve – i.e., there are less drastic interventions that might be equally (or more) effective as a way to help students. Moreover, closing a school represents a disruption in students’ lives (and often, by the way, to the larger community). In this sense, any closure must offer cumulative positive effects sufficient to offset an initial negative effect. Much depends on how and why schools are identified for closure, and the quality of the schools that displaced students attend. In practice, then, closure is a fairly risky policy, both educationally and (perhaps especially) politically. This disconnect between the appeal of theoretical school closures and the actual risks, in practice, may help explain why U.S. educational policy has been designed such that many schools operate at some risk of closure, but relatively few ever end up shutting their doors.

Despite the always contentious debates about the risks and merits of closing “low performing schools,” there has not been a tremendous amount of strong evidence about effects (in part because such closures have been somewhat rare). A new report by the Center for Research on Education Outcomes (CREDO) helps fill the gap, using a very large dataset to examine the test-based impact of school closures (among other things). The results speak directly to the closure debate, in both specific and general terms, but interpreting them is complicated by the fact that this analysis evaluates what is at best a policy done poorly.

Where Do Achievement Gaps Come From?

For almost two decades now, educational accountability policy in the U.S. has included a focus on the performance of student subgroups, such as those defined by race and ethnicity, income, or special education status. The (very sensible) logic behind this focus is the simple fact that aggregate performance measures, whether at the state-, district-, or school levels, often mask large gaps between subgroups.

Yet one of the unintended consequences of this subgroup focus has been confusion among both policymakers and the public as to how to interpret and use subgroup indicators in formal school accountability systems, particularly when those indicators are expressed as simple “achievement gaps” or “gap closing” measures. This is not only because achievement gaps can narrow for undesirable reasons and widen for desirable reasons, but also because many gaps exist prior to entry into the school (or district). If, for instance, a large Hispanic/White achievement gap for a given cohort exists at the start of kindergarten, it is misleading and potentially damaging to hold a school accountable for the persistence of that gap in later grades – particularly in cases where public policy has failed to provide the extra resources and supports that might help lower-performing students make accelerated achievement gains every year. In addition, the coarseness of current educational variables, particularly those usually used as income proxies, limits the detail and utility of some subgroup measures.

A helpful and timely little analysis by David Figlio and Krzystof Karbownik, published by the Brookings Institution, addresses some of these issues, and the findings have clear policy implications.