A recent poll on education attitudes from Gallup and Phi Delta Kappan got a lot of attention, including a mention on ABC’s "This Week with Christian Amanpour," which devoted most of its show to education yesterday. They flashed results for one of the poll’s questions, showing that 72 percent of Americans believe that "each teacher should be paid on the basis of the quality of his or her work," rather than on a "standard-scale basis."
Anyone who knows anything about survey methodology knows that responses to questions can vary dramatically with different wordings (death tax, anyone?). The wording of this Gallup/PDK question, of course, presumes that the "quality of work" among teachers might be measured accurately. The term "teacher quality" is thrown around constantly in education circles, and in practice, it is usually used in the context of teachers’ effects on students’ test scores (as estimated by various classes of "value-added" models).
But let’s say the Gallup/PDK poll asked respondents if "each teacher should be paid on the basis of their estimated effect on their students’ standardized test scores, relative to other teachers." Think the results would be different? Of course. This doesn’t necessarily say anything about the "merit" of the compensation argument, so to speak, nor does it suggest that survey questions should always emphasize perfect accuracy over clarity (which would also create bias of a different sort). But has anyone looked around recently and seen just how many powerful words, such as "quality," are routinely used to refer to standardized test score-related measures? I made a tentative list.