Shanker Blog

Atlanta: Bellwether Or Whistleblower For Test-Driven Reform?

Early in the life of No Child Left Behind, one amateur but insightful futurist on the Shanker Institute Board remarked to me: "Well, if you tie teacher pay, labeling failing schools, and evaluations of teachers and principals all to student test results—guess what?—you’ll get student test results. But some 20, years down the road when these kids get out of high school, we may discover they don’t know anything."

The quip did not necessarily suggest that we were headed for massive cheating scandals. Nor did it mean that students should never be assessed to find out how well they were learning what had been taught. It was just a warning that the incentives to produce score results would produce them —one way or another—and whether or not they stood for any true reflection on learning. Meaning, in this case, that a system that defines success narrowly in terms of test score gains will, at minimum, invite exaggerated claims and, at worst, encourage corruption.

An important report was released this spring that should bring some U. S. education "reformers" up short as they pursue policies based on test-based incentives. Instead, Incentives and Test-Based Accountability in Education, by the National Research Council (NRC), was received as a blip on their screens. A serious research review, the report looked at "15 test-based incentive programs, including large scale policies of NCLB, its predecessors, and state high school exit exams as well as a number of experiments and programs carried out in the United States and other countries." Its conclusion: "Despite using them [test-based incentives] for several decades, policymakers and educators do not yet know how to consistently generate positive effects on achievement and to improve education."

In other words, given the methods we are now using to grant performance pay, design evaluation plans, or fix low performing schools, these incentives don’t work. Moreover, looking at recent education history, they haven’t worked for quite a long time.

A "Decent Work" Solution To America's Jobs Crisis

Decent work? Some days, it sounds like an oxymoron, doesn’t it? It also brings to mind an old saying, favored by the AFL-CIO’s late president, Lane Kirkland, that if work were so great, the rich would have kept it for themselves.

But the truth is that work is one of life’s realities. For most people, it is the sole source of income. Work also can bring great personal satisfaction. Whether self-employed or working for a large multinational corporation, we all aspire to jobs that are interesting, safe, and pay a good wage with benefits – a job that can support a family, with something left over. Even these days, when people are happy to have ANY job, we still want THAT kind of a job: Decent work at decent pay.

But "decent work" is much more than a daydream – it is a concrete social and economic policy issue that is at the heart of a decade-long campaign by a major United Nations agency, the International Labor Organization, (ILO). Since 1999, the ILO, with support from member governments as well as employer and labor representatives, has pushed the "Decent Work Agenda". This document declares that "work is central to people's well-being." Not only does work provide income, it can bring about broad "social and economic advancement" and strengthen "individuals, their families and communities", in other words, "decent work" creates "upward mobility" or as Americans often put it, "raises all boats."

But these broader "social and economic" gains don’t come with just any work, the ILO argues.

Teacher Evaluations: Don't Begin Assembly Until You Have All The Parts

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Over the past year or two, roughly 15-20 states have passed or are considering legislation calling for the overhaul of teacher evaluation. The central feature of most of these laws is a mandate to incorporate measures of student test score growth, in most cases specifying a minimum percentage of a teacher’s total score that must consist of these estimates.

There’s some variation across states, but the percentages are all quite high. For example, Florida and Colorado both require that at least 50 percent of an evaluation must be based on growth measures, while New York mandates a minimum of 40 percent. These laws also vary in terms of other specifics, such as the degree to which the growth measure proportion must be based on state tests (rather than other assessments), how much flexibility districts have in designing their systems, and how teachers in untested grades and subjects are evaluated. But they all share that defining feature of mandating a minimum proportion – or “weight” – that must be attached to a test-based estimate of teacher effects (at least for those teachers in tested grades and subjects).

Unfortunately, this is typical of the misguided manner in which many lawmakers (and the advocates advising them) have approached the difficult task of overhauling teacher evaluation systems. For instance, I have discussed previously the failure of most systems to account for random error. The weighting issue is another important example, and it violates a basic rule of designing performance assessment systems: You should exercise extreme caution in pre-deciding the importance of any one component until you know what the other components will be. Put simply, you should have all the parts in front of you before you begin the assembly process.

Peer Effects And Attrition In High-Profile Charter Schools

An article in last week’s New York Times tells the story of child who was accepted (via lottery) into the highly-acclaimed Harlem Success Academy (HSA), a charter school in New York City. The boy’s mother was thrilled, saying she felt like she had just gotten her son a tuition-free spot in an elite private school. From the very first day of kindergarten, however, her child was in trouble. Sometimes he was sent home early; other times he was forced to stay late and “practice walking the hallways” as punishment for acting out. During his third week, he was suspended for three days.

Shortly thereafter, the mother, who had been corresponding with the principal and others about these incidents, received an e-mail message from HSA founder Eva Moskowitz. Moskowitz told her that, at this school, it is “extremely important that children feel successful," and that HSA, with its nine-hour days, during which children are “being constantly asked to focus and concentrate," can sometimes “overwhelm children and be a bad environment." The mother understood this to be a veiled threat of sorts, but was not upset at the time. Indeed, she credits HSA staff with helping her to find a regular public school for her child to attend. Happily, her son eventually ended up doing very well at his new school.

It’s very important to remember that this is really only one side of the story. It’s also an anecdote, and there is no way to tell how widespread this practice might be at HSA, or at charter schools in general. I retell it here because it helps to illustrate a difficult-to-measure “advantage” that some charter schools have when compared with regular neighborhood schools – the peer effects of attrition without replacement.

Wisconsin: Will It Be "Cool Hand Luke" Or "Norma Rae"?

As the implications of Wisconsin Governor Scott Walker’s attack on collective bargaining begin to sink in, some local officials have eagerly embraced one possibility opened up by the new anti-bargaining law: replacing union workers with convict labor.

This is not a new idea, at least not in Racine County. Last summer, budget problems led the county to try to replace unionized seasonal workers with prison labor. Teamsters Local 43 sued, arguing that the move violated the union contract. The judge sided with the union, but changes in the state’s collective bargaining law since that time have altered the legal picture, and Racine County administrators are taking another look at the idea.

How has the new law changed things? Not only did it strip unionized workers of their right to negotiate over health care and retirement issues, it also removed their contractual rights to their jobs – in the sense that they can no longer claim that certain jobs fall within the scope of the union contract and should be filled by union workers. This gives state and local officials the ability to hire private contract workers and even prison inmates to take those positions.

This is a "win-win" situation, according to Racine County Executive Jim Ladwig. While conceding that the idea is unpopular, he argued that "once people see things are still running smoothly, running efficiently, a lot of the fears will be alleviated." While the prisoners do not get paid for their work, they may earn time off their sentences, he said.

The Faulty Logic Of Using Student Surveys In Accountability Systems

In a recent post, I discussed the questionable value of student survey data to inform teacher evaluation models. Not only is there little research support for such surveys, but the very framing of the idea often reflects faulty reasoning.

A quote from a recent Educators 4 Excellence white paper helps to illustrate the point:

For a system that aims to serve students, young people’s interests are far too often pushed aside. Students’ voices should be at the forefront of the education debate today, especially when it comes to determining the effectiveness of their teacher.

This sounds noble… but seriously, why should students’ opinions be "at the forefront of the education debate"? Are students’ needs better served when we ask students what they need directly? Research on this is explicit: no, not really.

The Implications Of An Extreme "No Excuses" Perspective

In an article in this week’s New York Times Magazine, author Paul Tough notifies supporters of market-based reform that they cannot simply dismiss the "no excuses" maxim when it is convenient. He cites two recent examples of charter schools (the Bruce Randolph School in Denver, CO, and the Urban Prep Academy in Chicago) that were criticized for their low overall performance. Both schools have been defended publicly by "pro-reform" types (the former by Jonathan Alter; the latter by the school’s founder, Tim King), arguing that comparisons of school performance must be valid – that is, the schools’ test scores must be compared with those of similar neighborhood schools.

For example, Tim King notes that, while his school does have a very low proficiency rate – 17 percent – his students are mostly poor African-Americans, whose scores should be compared with those of peers in nearby schools. Paul Tough’s rejoinder is to proclaim that statements like these represent the "very same excuses for failure that the education reform movement was founded to oppose." His basic argument is that a 17 percent pass rate is not good enough, regardless of where a school is located or how disadvantaged are its students, and that pointing to the low performance of comparable schools is really just shedding the "no excuses" mantra when it serves one’s purposes.

Without a doubt, the sentiment behind this argument is noble, not only because it calls out hypocrisy, but because it epitomizes the mantra that "all children can achieve." In this extreme form, however, it also carries a problematic implication: Virtually every piece of high-quality education research, so often cited by market-based reformers to advance the cause, is also built around such "excuses."

Can We Make Voting Like Tweeting?

A recent Brookings Institution forum on new social media and the re-invigoration of democracy got me thinking about whether and how Twitter and Facebook could successfully increase political participation, specifically voter turnout. Voter turnout is one of the most important indicators of a healthy democracy and – as many have noted – U.S. voter participation rates are remarkably low.

It does not surprise me that people don’t see the immediate gains of voting. Going to the polls on election day entails individual costs (e.g., time, figuring out polling locations), while the benefits are essentially collective and weakly dependent on the vote of any one individual. Thus, people may find that it’s in their interest not to bother (Downs 1957 is the classic work on this). This rational approach conflicts with a more normative (even moral) understanding of democracy and civic behavior – e.g., we know we should all vote; it’s as much our responsibility as our right.

In a much less academic vein, although many U.S. citizens are free-riders when it comes to voting, it appears that Americans love to give their detailed opinions on all kinds of things. For example, why are Americans, who are so enthusiastic and industrious when it comes to writing lengthy product reviews, indolent when they are asked (once every four years) to voice their political views? How can we make voting as compelling as writing an online review? And can social media help in this endeavor?

A 'Summary Opinion' Of The Hoxby NYC Charter School Study

Almost two years ago, a report on New York City charter schools rocked the education policy world. It was written by Hoover Institution scholar Caroline Hoxby with co-authors Sonali Murarka and Jenny Kang. Their primary finding was that:

On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the “Scarsdale-Harlem achievement gap” [the difference in scores between students in Harlem and those in the affluent NYC suburb] in math, and 66 percent of the achievement gap in English.

The headline-grabbing conclusion was uncritically repeated by most major news outlets, including the New York Post, which called the charter effects “off the charts," and the NY Daily News, which announced that, from that day forward, anyone who opposed charter schools was “fighting to block thousands of children from getting superior educations." A week or two later, Mayor Michael Bloomberg specifically cited the study in announcing that he was moving to expand the number of NYC charter schools. Even today, the report is often mentioned as primary evidence favoring the efficacy of charter schools.

I would like to revisit this study, but not as a means to relitigate the “do charters work?" debate. Indeed, I have argued previously that we spend too much time debating whether charter schools “work," and too little time asking why some few are successful. Instead, my purpose is to illustrate an important research maxim: Even well-designed, sophisticated analyses with important conclusions can be compromised by a misleading presentation of results.

What Do We Do When Second Graders Think Math Is Not For Girls?

Although the past several generations have seen declining gender inequalities in educational attainment, gender-based differences in the fields of study we choose seem to persist (see here). For example, the percentage of women obtaining degrees in the science, technology, engineering, and mathematics (STEM) fields has remained exceedingly static in the last few decades (see here).

In trying to explain this persistent trend, some conclude that (1) women are not as interested in these fields, and/or that (2) women just aren’t as good as men in these domains. But how would one tell whether these explanations are right or wrong?

One problem is that people share specific, culturally based ideas about what men and women are and should be. Numerous studies demonstrate that the dearth of women in STEM fields can be directly linked to negative associations regarding girls and the sciences, and especially girls and math ability (see here and here).