Education

What Is A Standard Deviation?

Anyone who follows education policy debates might hear the term “standard deviation” fairly often. Most people have at least some idea of what it means, but I thought it might be useful to lay out a quick, (hopefully) clear explanation, since it’s useful for the proper interpretation of education data and research (as well as that in other fields).

Many outcomes or measures, such as height or blood pressure, assume what’s called a “normal distribution." Simply put, this means that such measures tend to cluster around the mean (or average), and taper off in both directions the further one moves away from the mean (due to its shape, this is often called a “bell curve”). In practice, and especially when samples are small, distributions are imperfect -- e.g., the bell is messy or a bit skewed to one side -- but in general, with many measures, there is clustering around the average.

Let’s use test scores as our example. Suppose we have a group of 1,000 students who take a test (scored 0-20). A simulated score distribution is presented in the figure below (called a "histogram").

New York City: The Mississippi Of The Twenty-First Century?

Last month saw the publication of a new report, New York State’s Extreme School Segregation, produced by UCLA’s highly regarded Civil Rights Project. It confirmed what New York educators have suspected for some time: our schools are now the most racially segregated schools in the United States. New York’s African-American and Latino students experience “the highest concentration in intensely-segregated public schools (less than 10% white enrollment), the lowest exposure to white students, and the most uneven distribution with white students across schools."

Driving the statewide numbers were schools in New York City, particularly charter schools. Inside New York City, “the vast majority of the charter schools were intensely segregated," the report concluded, significantly worse in this regard “than the record for public schools."

New York State’s Extreme School Segregation provides a window into the intersection of race and class in the city’s schools. As a rule, the city’s racially integrated schools are middle class, in which middle-class white, Asian, African-American and Latino students all experience the educational benefits of racial diversity. By contrast, the city’s racially segregated public schools are generally segregated by both race and class: extreme school segregation involves high concentrations of African-American and Latino students living in poverty.

What Is Implicit Bias, And How Might It Affect Teachers And Students? (Part I)

This is the first in a series of three posts about implicit bias. Here are the second and third parts.

The research on implicit bias both fascinates and disturbs people. It’s pretty cool to realize that many everyday mental processes happen so quickly as to be imperceptible. But the fact that they are so automatic, and therefore outside of our conscious control, can be harder to stomach.

In other words, the invisible mental shortcuts that allow us to function can be quite problematic – and a real barrier to social equality and fairness – in contexts where careful thinking and decision-making are necessary. Accumulating evidence reveals that “implicit biases” are linked to discriminatory outcomes ranging from the seemingly mundane, such as poorer quality interactions, to the highly consequential, such as constrained employment opportunities and a decreased likelihood of receiving life-saving emergency medical treatments.

Two excellent questions about implicit bias came up during our last Good Schools Seminar on "Creating Safe and Supportive Schools."

When Growth Isn't Really Growth, Part Two

Last year, we published a post that included a very simple graphical illustration of what changes in cross-sectional proficiency rates or scores actually tell us about schools’ test-based effectiveness (basically nothing).

In reality, year-to-year changes in cross-sectional average rates or scores may reflect "real" improvement, at least to some degree, but, especially when measured at the school- or grade-level, they tend to be mostly error/imprecision (e.g., changes in the composition of the samples taking the test, measurement error and serious issues with converting scores to rates using cutpoints). This is why changes in scores often conflict with more rigorous indicators that employ longitudinal data.

In the aforementioned post, however, I wanted to show what the changes meant even if most of these issues disappeared magically. In this one, I would like to extend this very simple illustration, as doing so will hopefully help shed a bit more light on the common (though mistaken) assumption that effective schools or policies should generate perpetual rate/score increases.

Estimated Versus Actual Days Of Learning In Charter School Studies

One of the purely presentational aspects that separates the new “generation” of CREDO charter school analyses from the old is that the more recent reports convert estimated effect sizes from standard deviations into a “days of learning” metric. You can find similar approaches in other reports and papers as well.

I am very supportive of efforts to make interpretation easier for those who aren’t accustomed to thinking in terms of standard deviations, so I like the basic motivation behind this. I do have concerns about this particular conversion -- specifically, that it overstates things a bit -- but I don’t want to get into that issue. If we just take CREDO’s “days of learning” conversion at face value, my primary, far more simple reaction to hearing that a given charter school sector's impact is equivalent to a given number of additional "days of learning" is to wonder: Does this charter sector actually offer additional “days of learning," in the form of longer school days and/or years?

This matters to me because I (and many others) have long advocated moving past the charter versus regular public school “horserace” and trying to figure out why some charters seem to do very well and others do not. Additional time is one of the more compelling observable possibilities, and while they're not perfectly comparable, it fits nicely with the "days of learning" expression of effect sizes. Take New York City charter schools, for example.

Valuing Home Languages Sets The Foundation For Early Learning

Our guest author today is Candis Grover, the Literacy & Spanish Content Manager at ReadyRosie.com, an online resource that models interactive oral language development activities that parents and caregivers of young children can do to encourage learning.

Many advocates, policymakers, and researchers now recognize that a strong start requires more than just a year of pre-K. Research shows that promoting children’s success starts with helping parents recognize the importance of loving interactions and “conversations” with their babies.

The above statement, which is taken from a recent report, Subprime Learning: Early Education in America since the Great Recession, emphasizes the role of parents as the earliest investors in the academic success of their children. This same report states that more than one in five of these families speaks a primary language other than English, and that this statistic could reach 40 percent by 2030. Despite the magnitude of these numbers, the Subprime Learning report asserts that the research on dual language learners has been largely ignored by those developing early childhood education policies and programs.

When Checking Under The Hood Of Overall Test Score Increases, Use Multiple Tools

When looking at changes in testing results between years, many people are (justifiably) interested in comparing those changes for different student subgroups, such as those defined by race/ethnicity or income (subsidized lunch eligibility). The basic idea is to see whether increases are shared between traditionally advantaged and disadvantaged groups (and, often, to monitor achievement gaps).

Sometimes, people take this a step further by using the subgroup breakdowns as a crude check on whether cross-sectional score changes are due to changes in the sample of students taking the test. The logic is as follows: If the increases are found when comparing advantaged and more disadvantaged cohorts, then an overall increase cannot be attributed to a change in the backgrounds of students taking the test, as the subgroups exhibited the same pattern. (For reasons discussed here many times before, this is a severely limited approach.)

Whether testing data are cross-sectional or longitudinal, these subgroup breakdowns are certainly important and necessary, but it's wise to keep in mind that standard variables, such as eligibility for free and reduced-price lunches (FRL), are imperfect proxies for student background (actually, FRL rates aren't even such a great proxy for income). In fact, one might reach different conclusions depending on which variables are chosen. To illustrate this, let’s take a look at results from the Trial Urban District Assessment (TUDA) for the District of Columbia Public Schools between 2011 and 2013, in which there was a large overall score change that received a great deal of media attention, and break the changes down by different characteristics.

Select Your Conclusions, Apply Data

The recent release of the National Assessment of Educational Progress (NAEP) and the companion Trial Urban District Assessment (TUDA) was predictably exploited by advocates to argue for their policy preferences. This is a blatant misuse of the data for many reasons that I have discussed here many times before, and I will not repeat them.

I do, however, want to very quickly illustrate the emptiness of this pseudo-empirical approach – finding cross-sectional cohort increases in states/districts that have recently acted policies you support, and then using the increases as evidence that the policies “work." For example, the recent TUDA results for the District of Columbia Public Schools (DCPS), where scores increased in all four grade/subject combinations, were immediately seized upon supporters of the reforms that have been enacted by DCPS as clear-cut evidence of the policy triumph. The celebrators included the usual advocates, but also DCPS Chancellor Kaya Henderson and the U.S. Secretary of Education Arne Duncan (there was even a brief mention by President Obama in his State of The Union speech).

My immediate reaction to this bad evidence was simple (though perhaps slightly juvenile) – find a district that had similar results under a different policy environment. It was, as usual, pretty easy: Los Angeles Unified School District (LAUSD).

Recovering One Of The Midwest’s Best Ideas

* Reprinted here in the Washington Post

Our guest author today is Dr. Conor P. Williams, a proud product of Michigan’s public schools, and currently a Senior Researcher in the New America Foundation’s Early Education Initiative. Follow him on Twitter: @conorpwilliams

President Obama sent a veritable drawerful of his cabinet to Detroit last fall (and Vice President Joe Biden led a similar visit last month). While the Tigers were headed for the postseason, the big shots weren’t in town for a glimpse of quality baseball. Attorney General Eric Holder, National Economic Council Director Gene Sperling, HUD Secretary Shaun Donovan, and Transportation Secretary Anthony Foxx were in the Motor City to brainstorm with state and local leaders on ways to use federal resources to spark -- and hopefully speed -- Detroit’s economic recovery.

While there are flickers of economic revival in the city, it’s hard to imagine that this conversation was wide-ranging enough to break the spiral. Is there an easy long-term recovery to be found in Detroit—or are its considerable problems the product of a fatally flawed economic development plan? There’s ample evidence for the latter.

Changing the city’s course will require much more than budgetary tweaks. It’s going to take a comprehensive rethinking of the area’s approach to education and economic opportunities. It’s going to require starting with the youngest Detroiters—and building a lasting foundation for economic growth.

Matching Up Teacher Value-Added Between Different Tests

The U.S. Department of Education has released a very short, readable report on the comparability of value-added estimates using two different tests in Indiana – one of them norm-referenced (the Measures of Academic Progress test, or MAP), and the other criterion-referenced (the Indiana Statewide Testing for Educational Progress Plus, or ISTEP+, which is also the state’s official test for NCLB purposes).

The research design here is straightforward – fourth and fifth grade students in 46 schools across 10 districts in Indiana took both tests, their teachers’ value-added scores were calculated, and the scores were compared. Since both sets of scores were based on the same students and teachers, this is allows a direct comparison of how teachers’ value-added estimates compare between these two tests. The results are not surprising, and they square with similar prior studies (see here, here, here, for example): The estimates based on the two tests are moderately correlated. Depending on the grade/subject, they are between 0.4 and 0.7. If you’re not used to interpreting correlation coefficients, consider that only around one-third of teachers were in the same quintile (fifth) on both tests, and another 40 or so percent were one quintile higher or lower. So, most teachers were within a quartile, about a quarter of teachers moved two or more quintiles, and a small percentage moved from top to bottom or vice-versa.

Although, as mentioned above, these findings are in line with prior research, it is worth remembering why this “instability” occurs (and what can be done about it).