Atlanta: Bellwether Or Whistleblower For Test-Driven Reform?

Early in the life of No Child Left Behind, one amateur but insightful futurist on the Shanker Institute Board remarked to me: "Well, if you tie teacher pay, labeling failing schools, and evaluations of teachers and principals all to student test results—guess what?—you’ll get student test results. But some 20, years down the road when these kids get out of high school, we may discover they don’t know anything."

The quip did not necessarily suggest that we were headed for massive cheating scandals. Nor did it mean that students should never be assessed to find out how well they were learning what had been taught. It was just a warning that the incentives to produce score results would produce them —one way or another—and whether or not they stood for any true reflection on learning. Meaning, in this case, that a system that defines success narrowly in terms of test score gains will, at minimum, invite exaggerated claims and, at worst, encourage corruption.

An important report was released this spring that should bring some U. S. education "reformers" up short as they pursue policies based on test-based incentives. Instead, Incentives and Test-Based Accountability in Education, by the National Research Council (NRC), was received as a blip on their screens. A serious research review, the report looked at "15 test-based incentive programs, including large scale policies of NCLB, its predecessors, and state high school exit exams as well as a number of experiments and programs carried out in the United States and other countries." Its conclusion: "Despite using them [test-based incentives] for several decades, policymakers and educators do not yet know how to consistently generate positive effects on achievement and to improve education."

In other words, given the methods we are now using to grant performance pay, design evaluation plans, or fix low performing schools, these incentives don’t work. Moreover, looking at recent education history, they haven’t worked for quite a long time.

Not surprisingly, the NRC report relates its findings to the long-standing social science truism of "Campbell’s Law":

The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

It should take neither social science axioms nor creative insight for us to predict cheating on tests, given their preeminent role in all sorts of educational (and non-educational) decisions these days. Thus, it’s very sad—but not that surprising—that serious inquiries are underway about possible cheating in Atlanta, Washington, D.C., Los Angeles, Baltimore, and elsewhere. Indeed, Atlanta seems to be one of worst cases, thus far (if far from the only one), and we can expect to hear a lot more details in the months ahead.

According to a June 30 report commissioned by Georgia Governor Nathan Deal, widespread cheating was found in 44 of the 56 Atlanta schools it examined—about 79 percent of the schools in the district. While serious challenges to report’s details are expected, one conclusion seems sound: that "a culture of fear and a conspiracy of silence infected this school system and kept many teachers from speaking freely about misconduct." As in Atlanta, doubts about the seriousness of the Washington, D.C. school system’s own inquiry into cheating seem to have convinced the U.S. Department of Education to take a closer look. In Baltimore, Superintendent Andres Alonzo says he plans to aggressively investigate cheating allegations. In Los Angeles, Superintendent John Deasy moved to close a chain of six charter schools, after its founder was reported to have ordered principals and teachers to use actual test questions to prepare students for the 2010 state exams.

The approach of the Georgia Governor’s report was systematic and intensive. It’s authors closely analyzed erasure rates in Atlanta schools as compared with rates statewide, examined every school where such rates warranted suspicion, and also interviewed staff —those who admitted to cheating, those who acknowledged it but did not participate, and whistleblowers, many of whom suffered reprisals.

Let’s look at a single school. Parks Middle School became a featured darling of the district under principal Christopher Waller. The school was recognized as a superstar achiever by the Annie E. Casey Foundation, which "interceded and contributed $10,000 to Waller" when he threatened to leave the school in 2006. At Parks, Waller offered incentive payments to selected school staff members. It was the only middle school to make 100 percent of its targets in 2008. It also was touted with "highest honors" from the district, and Waller himself "received the Atlanta Family Award, resulting in several thousand dollars going to him personally."

For more puff on Parks and Waller, see Annie E. Casey’s 2007 "Beating the Odds at Atlanta’s Parks Middle School," by Sarah Torian, or her 2008 update. In it, Waller attributes his success at recruiting organizational partners to the involvement of the Casey Foundation. The 2007 publication lists partners that include After School all Stars, Communities in Schools, Digital connection-One Economy, Georgia State University, Hands on Atlanta, Mendez Foundation, the Ministers’ Alliance, Music Matters, Pittsburgh Community Improvement Association, Salvation Army College of Officer Training, and United Way.

If the Governor’s report is right, the reality at Parks stood in startling contrast to this glowing picture of success. Running excerpts from its findings look something like this: "Cheating occurred on the CRCT [Georgia’s Criterion Referenced Competency Test] at Parks Middle School in 2006, 2007, 2008, 2009 and 2010 . . . The cheating started when Principal Christopher Waller began at Parks and recruited two teachers to change answers in 2006. . . Teachers felt as if they had no option but to do what Principal Waller directed them to do..."

The report also reveals that teachers who did not go along with Waller were ostracized, offered promotions in return for cooperation, or did not have their contracts renewed. While it appears that what happened at Parks Middle School was an extreme case, state investigators found that similar, harmful pressures ran through the system—favoritism, intimidation, ostracism, and removal of teachers being the most notable.

The main point is that where cheating occurs, no one knows what the kids can really do, at least not on the tests. Right now, it’s unclear whether we can even reach agreement on what students are supposed to have learned, much less design a better system to assess how well they have learned it. But if we don’t find other, broader, richer, and more nuanced indicators of value to tell us what kind of an education our students are getting, test-based reformers may well get an education system that resembles the tests to which they attach so much weight. The more we rely on these tests for all kinds of decisions, the more they are likely to change the nature of education itself, and not in ways that will be good for future generations of students.

- Eugenia Kemble