The War On Error

The debate on the use of value-added models (VAM) in teacher evaluations has reached an impasse of sorts. Opponents of VAM use contend that the imprecision is too high for the measures to be used in evaluation; supporters argue that current systems are inadequate, that all measures entail error but this doesn’t preclude using the estimates. 

This back-and-forth may be missing the mark, and it is not particularly useful in the states and districts that are already moving ahead. The more salient issue, in my view, is less about the amount of error than about how it is dealt with when the estimates are used (along with other measures) in evaluation systems.

Teachers certainly understand that some level of imprecision is inherent in any evaluation method—indeed, many will tell you about colleagues who shouldn’t be in the classroom, but receive good evaluation ratings from principals year after year. Proponents of VAM often point to this tendency of current evaluation systems to give “false positive” ratings as a reason to push forward quickly. But moving so carelessly that we disregard the error in current VAM estimates—and possible methods to reduce its negative impacts—is no different than ignoring false positives in existing systems.

A Matter Of Time

Extended school time is an education reform option that seems to be gaining in popularity. President Obama gave his endorsement earlier this year, while districts such as DCPS have extended time legislation under consideration.

The idea is fairly simple: Make the school day and/or year longer, so kids will have more time to learn.  Unlike many of the policy proposals flying around these days, it’s an idea that actually has some basis in research. While, by itself, more time yields negligible improvements in achievement, there is some evidence (albeit mixed evidence) that additional time devoted to “academic learning” can have a positive effect, especially for students with low initial test scores. So, more time might have potential benefits (at least in terms of test scores), but the time must be used wisely.

Still, extending schools days/years, like all policy options, must of course be evaluated in terms of cost effectiveness.  Small increases, such as adding a few days to the school calendar, are inconsistently and minimally effective, while larger increases in school time are an expensive intervention that must be weighed against alternatives, as well as against the fact that states and districts are, facing a few more years of fiscal crisis, cutting other potentially effective programs.

Attention To Pay

The debate over how best to restructure teacher salary systems is older than I am—with good reason: Instructional salaries represent roughly 40 percent of current K-12 public school expenditures.  And some of the arguments for changing current salary structures make sense, at least in theory. 

For instance, there is a case for tying step increases (typically awarded according to years of service) to additional measures, such as strengthened evaluation systems and curriculum-linked professional development (as is the case in the recently-ratified Baltimore contract). These types of changes, if they are bargained and approved by teachers, could be of real benefit to all stakeholders.

At the same time, it’s unfortunate that some of the talking points used commonly by those who wish to overhaul teacher salary systems are rather misleading and oversimplified. Not only do they sometimes seem designed to inspire outrage against teachers, they also tend to obscure or ignore important facts about the relationship between teacher pay and teacher quality.  Three such arguments seem particularly pervasive.

The Teaching Experience

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

The topic of teacher experience is getting a lot of attention in education debates.  In part, this makes sense, since experience (years of service) does play several important roles in education policy, including teachers’ raises and transfer/layoff policies. 

Usually, experience is discussed in terms of its relationship to performance –whether more experienced teachers produce larger student test score gains than less experienced teachers.  There is a pretty impressive body of research on this topic, the findings of which are sometimes used to argue for policy changes that eliminate the role of experience in salary and other employment policies.  Proponents of these changes often argue that experience is only weakly related to performance, and therefore shouldn’t be used in determining salary and other conditions of work.  It is not unusual to hear people say that experience doesn’t matter at all.

As is often the case when empirical research finds its way into policy debates, the “weakly related” characterization of the findings on the experience/achievement relationship borders on oversimplification, while the claim that experience doesn’t matter is flat-out wrong.  The relationship is substantial but context-dependent, and blanket statements about it often hide as much as they reveal.

Learning Versus Punishment And Accountability

Our guest author today is Jeffrey Pfeffer, Thomas D. Dee II Professor of Organizational Behavior at the Stanford University Graduate School of Business. We find it intriguing, given the current obsession with “accountability” in education reform. It is reprinted with permission from Dr. Pfeffer’s blog, Rational Rants, found at http://www.jeffreypfeffer.com.

People seem to love to exact retribution on those who screw up—it satisfies some primitive sense of justice. For instance, research in experimental economics shows that people will voluntarily give up resources to punish others who have acted unfairly or inappropriately, even though such behavior costs those doing it and even in circumstances where there is going to be no future interaction to be affected by the signal sent through the punishment. In other words, people will mete out retribution even when such behavior is economically irrational.

Evasive Maneuvers

In a previous post, I showed how the majority of funding for education and other public services comes from state and local tax revenue, and that low-income families pay a disproportionate share of these taxes (as a percentage of income). 

One of the reasons why this is the case is that many corporations – especially the largest and most profitable – have managed to avoid paying most of the state taxes that they owe (45 states have some form of business tax).  State corporate income taxes (CIT) are levied on business profits – so, for the most part, it’s only the highest-income individuals who are liable (through the businesses they own) for corporate taxes (the top 10 percent wealthiest individuals own about 90 percent of all corporate stock).

A 2005 joint report by Citizens for Tax Justice and the Institute on Taxation and Economic Policy took a close look at state CIT payments by 252 Fortune 500 companies between 2001 and 2003. Their findings were astounding. These corporations were able to shelter roughly two-thirds of their actual profits from state taxation, while 71 of them paid not a penny in state taxes during at least one year between 2001 and 2003.  During the years they paid no taxes, these 71 companies reported $86 billion in profits to their shareholders.

"No Comment" Would Have Been Better

Bruce Baker is a professor at Rutgers University who writes an informative blog called School Finance 101.  He presented some descriptive analysis of New Jersey charter schools in a post, and ended up being asked to comment on the data by a reporter.  The same reporter dutifully asked the New Jersey Charter Schools Association (NJCSA) to comment on the analysis. 

The NJCSA describes itself as “the leading statewide advocate for charter public schools in New Jersey and a principal source of public information about charter schools in the state.”  The organization issued the following response to Baker’s analysis:

The New Jersey Charter Schools Association seriously questions the credibility of this biased data. Rutgers University Professor Bruce Baker is closely aligned with teachers unions, which have been vocal opponents of charter schools and have a vested financial interest in their ultimate failure.

Baker is a member of the Think Tank Review Panel, which is bankrolled by the Great Lakes Center for Education Research and Practice. Great Lakes Center members include the National Education Association and the State Education Affiliate Associations in Illinois, Indiana, Michigan, Minnesota, Ohio and Wisconsin. Its chairman is Lu Battaglieri, the executive director of the Michigan Education Association.

There are now thousands of children on waiting lists for charters schools in New Jersey. This demand shows parents want the option of sending their children to these innovative schools and are satisfied with the results.

Note the stretch that they have to make to allege that Baker is “closely aligned” with teachers unions—he occasionally reviews papers for an organization that is partly funded by unions. There is no formal connection beyond that. Note also that the NJCSA statement “questions the credibility of [sic] this biased data”—meaning they doubt the credibility of data from the State of New Jersey, which Baker merely recasts as graphs and maps. There is not a shred of substance in this statement that addresses the data or Baker’s description of them. It’s pure guilt by association (and there’s not really even an association).

Michelle Rhee's Testing Legacy: An Open Question

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

Michelle Rhee’s resignation and departure have, predictably, provoked a flurry of conflicting reactions. Yet virtually all of them, from opponents and supporters alike, seem to assume that her tenure at the helm of the D.C. Public Schools (DCPS) helped to boost student test scores dramatically. She and D.C. Mayor Adrian Fenty made similar claims themselves in the Wall Street Journal (WSJ) just last week.

Hardly anybody, regardless of their opinion about Michelle Rhee, thinks that test scores alone are an adequate indicator of student success. But, in no small part because of her own emphasis on them, that is how this debate has unfolded. Her aim was to raise scores and, with few exceptions (also here and here), even those who objected to her “abrasive” style and controversial policies seem to believe that she succeeded wildly in the testing area.

This conclusion is premature. A review of the record shows that Michelle Rhee’s test score “legacy” is an open question. 

There are three main points to consider:

How Deep Is The Teacher Bench?

On most sports teams, coaches assess players in part by considering who is available to replace them. Teams with “deep benches” have more leeway in making personnel changes, because quality replacements are available.

The same goes for teaching. Those who aggressively wish to start firing larger numbers of teachers every year rely on an obvious but critical assumption (often unstated): that schools and districts can find better replacements.

In other words, it is both counterproductive (and very expensive) to fire teachers if you can’t replace them with a more effective alternative. Even those few commentators who have addressed this matter sometimes ignore another important fact: The teacher labor market is about to change dramatically, with a massive wave of retirements lasting 5-10 years. Thus, most current assumptions about the stability and quality of the applicant pool over this period may be unsupportable.

The numbers are a bit staggering.

Who Pays For Education?

In education debates, especially these days, there is endless talk about spending – how to spend money, what programs to cut, and how to increase the bang-to-buck ratio. This is not surprising: In 2007-08 (the last year for which national U.S. Census data are available), we spent almost $600 billion. That’s quite a figure, and we all have an interest in spending that money wisely.

What is sometimes surprising is how little we hear about how we get that money. Of course, we all know that our tax dollars fund our public schools, and most of us know that state and local revenue is the primary source of this funding (about 90 percent; on average, about half state and half local). Less commonly-known, however, is who pays these bills – who bears the largest share of the tax burden, relative to their income? At the federal level, taxation is largely progressive, which means that, on the whole, higher-income families pay a larger percentage of their earned income to the federal government than lower-income families. This is, very simply, due to the fact that higher income brackets are taxed at higher rates.

But when it comes to state and local taxes, the picture is different. The poorest families pay far more of their income than the richest (i.e., taxes are regressive). In other words, the money that funds public education is a burden disproportionately borne by poor and middle-income Americans. And the lower your income, the more of it you pay. Given this situation, combined with a fiscal crisis that threatens to linger for several years, the best solution – raising revenue through a more equitable system – may be the only one not on the table.