Skip to:

Revisiting The "5-10 Percent Solution"


The Report argues that economically it is better to fire a low Value Added teacher even after limited data and replace him/her with an average VA teacher. Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms. Chetty/Friedman/Rockoff paper You can't dismiss it when it uses 20 years of data.

I think it is important to note the following realities: 1. The tests to measure "value" are not worth the paper they are printed on. Reliability, validity, and relevance are absent. They do not measure what K-12 should be achieving. 2. Good teachers does not equal high test scores, or high 1-year increases. 3. We are talking about humans and learning, which are not linear or 2-dimensional or steady. They are complex and erratic, which is not accounted for when treated in a linear statistical model. 4. The language of "good teacher" and "bad teacher" based on test scores is offensive, new, simplistic, and toborrow from Sports Illustrated, a "Sign that the Apocalypse" is upon us, educationally-speaking. 5. Finally, it is sad. We know good assessment, good teaching, and what kids need to be successful(in every sense of the word) as adults. We prefer the fast, cheap, simple, numerical with a strong dose of scapegoating.

It's curious in reading the link from "calculation" to the paper by Hanushek that no where in the paper did a calculation appear. We're supposed to believe his conclusions based on his suggestion and his prior work (though it did mention the Gary, Indiana study). It really would have been nice to see the data that would support the reasonableness of his assumptions such as how a difference in teacher effectiveness based on test scores translated to a consistent response of student learning. Instead he gives an estimate and then goes on to say that estimate is a likely lower bound. Unfortunately, we are left with if you assume this and you assume that some students might have astounding gains while others will be permanently damaged. As you have clearly shown, it's difficult to demonstrate any reliability in the these VAM scores and a supporter of using VAM scores is left to arm wave that they are still useful. Unfortunately,reformers believe in VAMs despite the lack of reliability because they see the alternative as no accountability. As mentioned just this last weekend on Up With Chris Hayes, 99% of all teachers (supposedly) receive satisfactory ratings making current non VAM evaluations meaningless. If you're starting point is that teachers (or administrators) can't be trusted, even unreliable VAMs will be seen as an improvement. You suggest that "value-added scores will be at least modestly correlated with the non-test-based components of those evaluations". It's not clear what that would mean in practice. If there is not a great deal of variation in the non-test-based components for a number of teachers, but there is significant variation in the test-based scores, the test-based component of their evaluation is likely to dominate the evaluations. Add to this the possibility the value-added scores might be viewed as the only "objective" portion of the evaluation, these scores could end up being given more weight regardless of the actual legislation. As you say, "Expectations matter", but so do the assumptions of those policy makers. If your assumption is, irrespective of factors beyond a teacher's control, teachers are effective or they're not, that a poorly performing teacher in one setting might not become an effective teacher in another, that poorly performing teachers can't improve, or that you can reliably replace your dismissed teachers with ones that are at least average all the time your policies are just as likely do more damage than lead to these hoped for huge improvements. Just what should our expectations be if those 5-10 percent all happen to teach in the same schools.If you already allow there is a concern about the distribution of quality teachers in a district, will this policy improve this situation or make it worse (in practice, not just in theory?)


This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.