Skip to:

Are Value-Added Models Objective?

Comments

Clearly VAMs and classroom observations by principals and peers measure very DIFFERENT aspects of teaching. A clear explanation for what the VAM estimates capture and what classroom observations represent is needed, indeed. They are both meaningful and useful and they are NOT THE SAME. A policy maker may have preferences for the measures to implement in a school district and may put higher weight on one or the other depending on her priorities - that is subjective! You are raising important issues to be considered when adopting VAMs. Yes, these arguments are valid in general. But the crucial point of your objective/subjective argument is whether you apply the same metric absent from human judgment to make meaningful inferences. Using the exact same evaluation protocol, principals and peers can rate a particular teacher differently according of their own understanding of the questions and their own perception of the observed teacher behavior. There can be a thousand principals or external/peer evaluators in a district. This is where the human judgment enters the measurement and that’s why classroom observations are more likely to be subjective. Similarly, letter grades (As, Bs, Cs, etc.) and GPAs measures are student learning metrics but they vary across teachers and schools. For one teacher in one school, a particular essay can be worth of A+, while the same essay for another teacher (within the same or other school) can find it worth of B-. This is an example of metric subject to human judgment and does not permit meaningful comparisons across teachers and schools. In that respect, it is more likely to be subjective. Now if a policy maker/researcher/someone is interested in evocative comparisons across students, teachers, and schools within a district or a state, one would need to use the most objective measures one can conceive prone to the least human judgment. Student test scores are an example. Rather than comparing the number of A students in a district, a more meaningful measure should use a common test score. Yes, there are many issues with tests. However, tests are more likely to be objective indicators of what they are designed to test (specific items such as fractions or broader problem solving skills). One of the main reasons for not including a writing section in standardized achievement tests is to eliminate the need for humans to grade those, thus the extensive use of multiple choice type questions. You are absolutely right that we need to be careful with the language – a VAM estimate of a teacher fixed effect is not equivalent to the broader notion of teacher effectiveness. And paying attention to issues of VAM choice and implementation is extremely important. But all of your arguments are not making VAMs less objective.

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.