Skip to:

What Is A Standard Deviation?

Comments

While I understand the approach of standardizing scores for research purposes, I agree with those who argue it's important to translate back the differences into # of additional questions answered correctly. This is particularly important in those states that use Student Growth Percentiles. Most people fail to appreciate that in the middle part of the distribution, where there are lots of students, a difference in 1 question answered correctly bumps the SGP up dramatically... And indeed there might be absolutely no difference between an SGP of say 45-55 in number of items answered correctly because say 10 percent of respondents have the median # of items answered correctly...

Matt, Very interesting post. I think its really important to explain standard deviations to a non-statistical audience. Two responses to your post and people's comments 1. Using standard deviations to compare between populations is a potentially risky endeavor. Since standard deviation is based on the variance, a mean difference in a population with less variance will seem to have a larger effect size than the same difference in a population with greater variance. So an intervention with a more homogenous population may seem to have a greater effect than a more diverse population. 2. There's no way to directly convert standard deviations into the number of correct responses. Most standardized tests don't use the raw percent correct to calculate scores. Rather, statistical models to estimate student scores based on their response patterns. So depending on what questions they answered correctly students will get different scores.

Good stuff, Matt. One of my great frustrations is how easily so much education research converts S.D.'s into some "real" measure that isn't "real" at all. "X months/years of learning" is a particularly popular one, but I would argue it distracts from the actual outcome, which is a change in the number of questions answered correctly on a test. If I were king of Dataland, I would forbid this practice without, at the very least, first telling us what was really being converted into S.D.s -- in the case above, telling us how many questions on a test (or even raw score points) leads to a change from a z-score of 0 to 1. Mark

Matt, Mark -- Can any of you comment on exactly what we're talking about when one teacher has a higher value add score than another? For example, on a 65 question state multiple choice exam, does a "high performing" teacher get his/her students to get 57 questions correct, while a "low performing" will on average get his/her students to score....53 correct? 50? 40? How many questions are we talking about? Of course it's different for different states and depends on the test, but can either of you give a general ballpark of what we're talking about here?

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.