• Cheating, Honestly

    Whatever one thinks of the heavy reliance on standardized tests in U.S. public education, one of the things on which there is wide agreement is that cheating must be prevented, and investigated when there’s evidence it might have occurred.

    For anyone familiar with test-based accountability, recent cheating scandals in Atlanta, Washington, D.C., Philadelphia and elsewhere are unlikely to have been surprising. There has always been cheating, and it can take many forms, ranging from explicit answer-changing to subtle coaching on test day. One cannot say with any certainty how widespread cheating is, but there is every reason to believe that high-stakes testing increases the likelihood that it will happen. The first step toward addressing that problem is to recognize it.

    A district, state or nation that is unable or unwilling to acknowledge the possibility of cheating, do everything possible to prevent it, and face up to it when evidence suggests it has occurred, is ill-equipped to rely on test-based accountability policies. 

  • Selective Schools In New Orleans

    Charter schools in New Orleans, LA (NOLA) receive a great deal of attention, in no small part because they serve a larger proportion of public school students than do charters in any other major U.S. city. Less discussed, however, is the prevalence of NOLA’s “selective schools” (elsewhere, they are sometimes called “exam schools”). These schools maintain criteria for admission and/or retention, based on academic and other qualifications (often grades and/or standardized test scores).

    At least six of NOLA’s almost 90 public schools are selective – one high school, four (P)K-8 schools and one serving grades K-12. When you add up their total enrollment, around one in eight NOLA students attends one of these schools.*

    Although I couldn’t find recent summary data on the prevalence of selective schools in urban districts around the U.S., this is almost certainly an extremely high proportion (for instance, selective schools in New York City and Chicago, which are mostly secondary schools, serve only a tiny fraction of students in those cities).

  • Bahraini Repression Against Teachers And Health Care Workers

    The Kingdom of Bahrain, a small island nation in the Persian Gulf, is a perfect political stew, situated as it is at the confluence of political, religious, economic and international tensions simmering in the Persian Gulf.  A majority Shi’a Muslim country ruled for hundreds of years by Sunni tribal chieftains with family ties to Saudi Arabia, Bahrain is a dictatorship whose people have regularly demanded political reform and seen their aspirations crushed.

    Today, with strong support from the oil-rich Saudis, the Kingdom’s hard-line Al-Khalifa regime enjoys absolute powers, although the day-to-day political reality is often complex.

    Bahrain is also the home of the U.S. Navy’s Fifth Fleet, which patrols those volatile waters, largely to keep an eye on Iran. The U.S. considers the port a critical element of its military posture in the Gulf. This consideration drives U.S. policy toward Bahrain. The centuries-old Bahraini-Saudi connection has, predictably, deterred the U.S. and other democratic countries from applying significant pressure to the kingdoms’ rulers. Stability is the byword.

  • Creating A Valid Process For Using Teacher Value-Added Measures

    ** Reprinted here in the Washington Post

    Our guest author today is Douglas N. Harris, associate professor of economics and University Endowed Chair in Public Education at Tulane University in New Orleans. His latest book, Value-Added Measures in Education, provides an excellent, accessible review of the technical and practical issues surrounding these models. 

    Now that the election is over, the Obama Administration and policymakers nationally can return to governing.  Of all the education-related decisions that have to be made, the future of teacher evaluation has to be front and center.
    In particular, how should “value-added” measures be used in teacher evaluation? President Obama’s Race to the Top initiative expanded the use of these measures, which attempt to identify how much each teacher contributes to student test scores. In doing so, the initiative embraced and expanded the controversial reliance on standardized tests that started under President Bush’s No Child Left Behind.

    In many respects, The Race was well designed. It addresses an important problem - the vast majority of teachers report receiving limited quality feedback on instruction. As a competitive grants program, it was voluntary for states to participate (though involuntary for many districts within those states). The Administration also smartly embraced the idea of multiple measures of teacher performance.

    But they also made one decision that I think was a mistake.  They encouraged—or required, depending on your vantage point—states to lump value-added or other growth model estimates together with other measures. The raging debate since then has been over what percentage of teachers’ final ratings should be given to value-added versus the other measures. I believe there is a better way to approach this issue, one that focuses on teacher evaluations not as a measure, but rather as a process.

  • A Case Against Assigning Single Ratings To Schools

    The new breed of school rating systems, some of which are still getting off the ground, will co-exist with federal proficiency targets in many states, and they are (or will be) used for a variety of purposes, including closure, resource allocation and informing parents and the public (see our posts on the systems in INFLOHCONYC).*

    The approach that most states are using, in part due to the "ESEA flexibility" guidelines set by the U.S. Department of Education, is to combine different types of measures, often very crudely, into a single grade or categorical rating for each school. Administrators and media coverage usually characterize these ratings as measures of school performance - low-rated schools are called "low performing," while those receiving top ratings are characterized as "high performing." That's not accurate - or, at best, it's only partially true.

    Some of the indicators that comprise the ratings, such as proficiency rates, are best interpreted as (imperfectly) describing student performance on tests, whereas other measures, such as growth model estimates, make some attempt to isolate schools’ contribution to that performance. Both might have a role to play in accountability systems, but they're more or less appropriate depending on how you’re trying to use them.

    So, here’s my question: Why do we insist on throwing them all together into a single rating for each school? To illustrate why I think this question needs to be addressed, let’s take a quick look at four highly-simplified situations in which one might use ratings.

  • When You Hear Claims That Policies Are Working, Read The Fine Print

    When I point out that raw changes in state proficiency rates or NAEP scores are not valid evidence that a policy or set of policies is “working," I often get the following response: “Oh Matt, we can’t have a randomized trial or peer-reviewed article for everything. We have to make decisions and conclusions based on imperfect information sometimes."

    This statement is obviously true. In this case, however, it's also a straw man. There’s a huge middle ground between the highest-quality research and the kind of speculation that often drives our education debate. I’m not saying we always need experiments or highly complex analyses to guide policy decisions (though, in general, these are always preferred and sometimes required). The point, rather, is that we shouldn’t draw conclusions based on evidence that doesn't support those conclusions.

    This, unfortunately, happens all the time. In fact, many of the more prominent advocates in education today make their cases based largely on raw changes in outcomes immediately after (or sometimes even before) their preferred policies were implemented (also see hereherehereherehere, and here). In order to illustrate the monumental assumptions upon which these and similar claims ride, I thought it might be fun to break them down quickly, in a highly simplified fashion. So, here are the four “requirements” that must be met in order to attribute raw test score changes to a specific policy (note that most of this can be applied not only to claims that policies are working, but also to claims that they're not working because scores or rates are flat):

  • Are Stereotypes About Career And Technical Education Crumbling?

    The stereotypes, bias, and misunderstanding that have for many decades surrounded and isolated Career and Technical Education (CTE) may slowly be crumbling.  A recent report by the National Research Center for Career and Technical Education (NRCCTE) argues that traditional CTE typology -- the way in which CTE students are identified and classified -- is obsolete.  The distinctions between “CTE” students and “academic” students are no longer useful. Today, nearly all high school students, including the highest achieving academic-track students, enroll in some CTE courses.

    Moreover, a significant number of students complete “high intensity” CTE courses as well as academic courses, in patterns that cross SES lines. In order to understand the contemporary high school experience, these researchers argue, we need a new typology based on the reality of today’s classroom, students, and curricula.

    The October 2012 study, “A Typology for Understanding the Career and Technical Education Credit-taking Experience of High School Students," proposes a new, more nuanced classification system --  one the authors believe would more accurately capture the high school experience and needs of today’s students. The researchers argue that these long-overdue changes could alter experts’ views of what students actually study in high school, break down the obsolete conceptual barriers that currently divide CTE and academic curricula, and help educators work with students to devise the most appropriate pathways to academic and career success.

  • Value-Added, For The Record

    People often ask me for my “bottom line” on using value-added (or other growth model) estimates in teacher evaluations. I’ve written on this topic many times, and while I have in fact given my overall opinion a couple of times, I have avoided expressing it in a strong “yes or no” format. There's a reason for this, and I thought maybe I would write a short piece and explain myself.

    My first reaction to the queries about where I stand on value-added is a shot of appreciation that people are interested in my views, followed quickly by an acute rush of humility and reticence. I know think tank people aren’t supposed to say things like this, but when it comes to sweeping, big picture conclusions about the design of new evaluations, I’m not sure my personal opinion is particularly important.

    Frankly, given the importance of how people on the ground respond to these types of policies, as well as, of course, their knowledge of how schools operate, I would be more interested in the views of experienced, well-informed teachers and administrators than my own. And I am frequently taken aback by the unadulterated certainty I hear coming from advocates and others about this completely untested policy. That’s why I tend to focus on aspects such as design details and explaining the research – these are things I feel qualified to discuss.  (I also, by the way, acknowledge that it’s very easy for me to play armchair policy general when it's not my job or working conditions that might be on the line.)

    That said, here’s my general viewpoint, in two parts. First, my sense, based on the available evidence, is that value-added should be given a try in new teacher evaluations.

  • Describing, Explaining And Affecting Teacher Retention In D.C.

    The New Teacher Project (TNTP) has released a new report on teacher retention in D.C. Public Schools (DCPS). It is a spinoff of their “The Irreplaceables” report, which was released a few months ago, and which is discussed in this post. The four (unnamed) districts from that report are also used in this one, and their results are compared with those from DCPS.

    I want to look quickly at this new supplemental analysis, not to rehash the issues I raised about“The Irreplaceables," but rather because of DCPS’s potential importance as a field test site for a host of policy reform ideas – indeed, the majority of core market-based reform policies have been in place in D.C. for several years, including teacher evaluations in which test-based measures are the dominant component, automatic dismissals based on those ratings, large performance bonuses, mutual consent for excessed teachers and a huge charter sector. There are many people itching to render a sweeping verdict, positive or negative, on these reforms, most often based on pre-existing beliefs, rather than solid evidence.

    Although I will take issue with a couple of the conclusions offered in this report, I'm not going to review it systematically. I think research on retention is important, and it’s difficult to produce reports with original analysis, while very easy to pick them apart. Instead, I’m going to list a couple of findings in the report that I think are worth examining, mostly because they speak to larger issues.

  • The Educational Attainment Of Girls And Boys: Two Sides of the Same Coin

    Last month, Malala Yousafzai, a 14-year-old Pakistani girl, was shot in the head, in an attempted assassination by Taliban militants. Her “crime” was daring to advocate for girls’ education. In a New York Times column, Nicholas Kristof observes that we in the West find it “easy to dismiss such incidents as distant barbarities," and uses the example of sex trafficking to illustrate that we “have a blind spot for our own injustices." I agree. However, I am not sure we need to go so far to find domestic injustices.

    How about a close look within this very area: The education of girls (and boys) in the U.S.? Stories about how girls have surpassed boys in educational attainment have become common, and are often linked to statements about how boys are forgotten and/or lost. This rhetoric is troubling for several reasons. First, it can be read to imply a zero-sum equation; that is, that the educational advancement of girls is the cause of boys’ educational neglect. Second, stories about girls’ “successes” and boys’ “failures” may obscure more than they reveal.

    There are the "lost boys" of higher education and the "missing girls" of STEM. We worry about boys and reading and girls and math. Recurring questions include where are the women in technology? Or, are there enough novels that cater to boys? Women have sailed past men in obtaining college degrees but, importantly, continue to concentrate in different fields and need Ph.D.s to match men with bachelor’s in the workplace.

    When issues are addressed in this fragmented manner, it’s hard to tell if it’s girls or boys that we should be worrying about. Well, both and neither. What all these pieces of the puzzle really say is that – at least in this day, age, and nation – gender still matters.