Shanker Blog

Gender Pay Gaps And Educational Achievement Gaps

There is currently an ongoing rhetorical war of sorts over the gender wage gap. One “side” makes the common argument that women earn around 75 cents on the male dollar (see here, for example).

Others assert that the gender gap is a myth, or that it is so small as to be unimportant.

Often, these types of exchanges are enough to exasperate the casual observer, and inspire claims such as “statistics can be made to say anything." In truth, however, the controversy over the gender gap is a good example of how descriptive statistics, by themselves, say nothing. What matters is how they’re interpreted.

Moreover, the manner in which one must interpret various statistics on the gender gap applies almost perfectly to the achievement gaps that are so often mentioned in education debates.

The Education Reform Movement: Reset Or Redo?

Our guest author today is Dr. Clifford B. Janey, former superintendent for the Newark Public Schools, District of Columbia Public Schools, and Rochester City School District. He is currently a Senior Weismann Fellow at the Bankstreet College of Education in New York City, and a Shanker Institute board member.

For too many students, families, and communities, the high school diploma represents either a dream deferred or a broken contract between citizens and the stewards of America's modern democracy. With the reform movement’s unrelenting focus on testing and its win/lose consequences for students and staff, the high school diploma, which should signify college and work readiness, has lost its value.

Not including the over seven thousand students who drop out of high school daily, the gap between the percentage of those who graduate and their readiness for college success will continue to worsen the social and income inequalities in life. Recent studies report that America has the highest number of people (46.2 million) living in poverty since data collection began in 1959. While poverty and its conditions have been unforgiving, policy makers and education reformers have largely ignored this reality. Rebuttals to this argument are interesting, but, without fundamental change, the predictable growth within the ranks of poverty will continue.

A framework within which solutions will thrive requires a redo of the national reform focus, not merely a reset of existing efforts—including teacher evaluation systems, closing low performing schools (and opening up new ones that are at best marginally better), and increasing the opportunity for mayoral control (which still commands attention but with little assurance of transparency).

Reviving Labor After Wisconsin: Unions Need A New Approach Emphasizing Civil Rights

Our guest authors today are Richard D. Kahlenberg, a senior fellow at The Century Foundation, and Moshe Z. Marvit, a civil rights attorney. They are authors of the book Why Labor Organizing Should Be a Civil Right: Rebuilding a Middle-Class Democracy By Enhancing Worker Voice.

Conservatives are calling the failure of public sector unions to recall Wisconsin Governor Scott Walker labor’s “Waterloo.”  Just as private sector unionism has declined from a third of the workforce in the 1950s to less than 7 percent today, Charles Krauthammer writes in the Washington Post, “Tuesday, June 5, 2012, will be remembered as the beginning of the long decline of the public-sector union."

This forecast seems an exercise in hyperbole (many voters don’t think recall elections are an appropriate response to policy disputes) but the setback for labor was indeed serious.  One of the lessons of Wisconsin is that if public sector unions want to survive, they need to find ways to help revive trade unionism in the private sector.

The fates of the two sectors are deeply intertwined.

The Landmark Case Of Us V. Them

Patrick Riccards, CEO of the education advocacy group ConnCAN, has published a short piece on his personal blog in which he decries the “vicious and fact-free attacks” in education debates.

The post lists a bunch of “if/then” statements to illustrate how market-based reform policy positions are attacked on personal grounds, such as, “If one provides philanthropic support to improve public schools, then one must be a profiteer looking to make personal fortunes off public education." He summarizes the situation with a shot of his own: “Yes, there are no attacks that are too vicious or too devoid of fact for the defenders of the status quo." What of his fellow reformers? They “simply have to stand and take the attacks and the vitriol, no matter how ridiculous."

Mr. Riccards is dead right that name-calling, ascription of base motives, and the abuse of empirical evidence are rampant in education debates. I myself have criticized the unfairness of several of his “if/then” statements, including the accusations of profiteering and equating policy views with being “anti-teacher."

But anyone who thinks that this behavior is concentrated on one “side” or the other must be wearing blinders.

Teachers: Pressing The Right Buttons

The majority of social science research does not explicitly dwell on how we go from situation A to situation B. Instead, most social scientists focus on associations between different outcomes. This “static” approach has advantages but also limitations. Looking at associations might reveal that teachers who experience condition A are twice as likely to leave their schools than teachers who experience condition B. But what does this knowledge tell us about how to move from condition A to condition B? In many cases, very little.

Many social science findings are not easily “actionable” for policy purposes precisely because they say nothing about processes or sequences of events and activities unfolding over time, and in context. While conventional quantitative research provides indications of what works — on average — across large samples, a look at processes reveals how factors or events (situated in time and space) are associated with each other. This kind of research provides the detail that we need, not just to understand the world, but to do so in a way that is useful and enables us to act on it constructively.

Although this kind of work is rare, every now then a quantitative study showing “process sensitivity” sees the light of day. This is the case of a recent paper by Morgan and colleagues (2010) examining how the events that teachers experience routinely affect their commitment to remain in the profession.

A Game Of Inches

One of the more telling episodes in education I’ve seen over the past couple of years was a little dispute over Michelle Rhee’s testing record that flared up last year. Alan Ginsburg, a retired U.S. Department of Education official, released an informal report in which he presented the NAEP cohort changes that occurred during the first two years of Michelle Rhee’s tenure (2007-2009), and compared them with those during the superintendencies of her two predecessors.

Ginsburg concluded that the increases under Chancellor Rhee, though positive, were less rapid than in previous years (2000 to 2007 in math, 2003 to 2007 in reading). Soon thereafter, Paul Peterson, director of Harvard’s Program on Educational Leadership and Governance, published an article in Education Next that disputed Ginsburg’s findings. Peterson found that increases under Rhee amounted to roughly three scale score points per year, compared with around 1-1.5 points annually between 2000 and 2007 (the actual amounts varied by subject and grade).

Both articles were generally cautious in tone and in their conclusions about the actual causes of the testing trends. The technical details of the two reports – who’s “wrong” or “right” - are not important for this post (especially since more recent NAEP results have since been released). More interesting was how people reacted - and didn’t react - to the dueling analyses.

We Should Only Hold Schools Accountable For Outcomes They Can Control

Let’s say we were trying to evaluate a teacher’s performance for this academic year, and part of that evaluation would use students’ test scores (if you object to using test scores this way, put that aside for a moment). We checked the data and reached two conclusions. First, we found that her students made fantastic progress this year. Second, we also saw that the students’ scores were still quite a bit lower than their peers’ in the district. Which measure should we use to evaluate this teacher?

Would we consider judging her even partially based on the latter – students’ average scores? Of course not. Those students made huge progress, and the only reason their absolute performance levels are relatively low is because they were low at the beginning of the year. This teacher could not control the fact that she was assigned lower-scoring students. All she can do is make sure that they improve. That’s why no teacher evaluation system places any importance on students’ absolute performance, instead focusing on growth (and, of course, non-test measures). In fact, growth models control for absolute performance (prior year’s test scores) so it doesn't bias the results.

If we would never judge teachers based on absolute performance, why are we judging schools that way? Why does virtually every school/district rating system place some emphasis – often the primary emphasis – on absolute performance?

Three Important Distinctions In How We Talk About Test Scores

In education discussions and articles, people (myself included) often say “achievement” when referring to test scores, or “student learning” when talking about changes in those scores. These words reflect implicit judgments to some degree (e.g., that the test scores actually measure learning or achievement). Every once in a while, it’s useful to remind ourselves that scores from even the best student assessments are imperfect measures of learning. But this is so widely understood - certainly in the education policy world, and I would say among the public as well - that the euphemisms are generally tolerated.

And then there are a few common terms or phrases that, in my personal opinion, are not so harmless. I’d like to quickly discuss three of them (all of which I’ve talked about before). All three appear many times every day in newspapers, blogs, and regular discussions. To criticize their use may seem like semantic nitpicking to some people, but I would argue that these distinctions are substantively important and may not be so widely-acknowledged, especially among people who aren’t heavily engaged in education policy (e.g., average newspaper readers).

So, here they are, in no particular order.

Herding FCATs

About a week ago, Florida officials went into crisis mode after revealing that the proficiency rate on the state’s writing test (FCAT) dropped from 81 percent to 27 percent among fourth graders, with similarly large drops in the other two grades in which the test is administered (eighth and tenth). The panic was almost immediate. For one thing, performance on the writing FCAT is counted in the state’s school and district ratings. Many schools would end up with lower grades and could therefore face punitive measures.

Understandably, a huge uproar was also heard from parents and community members. How could student performance decrease so dramatically? There was so much blame going around that it was difficult to keep track – the targets included the test itself, the phase-in of the state’s new writing standards, and test-based accountability in general.

Despite all this heated back-and-forth, many people seem to have overlooked one very important, widely-applicable lesson here: That proficiency rates, which are not "scores," are often extremely sensitive to where you set the bar.

Quality Control In Charter School Research

There's a fairly large body of research showing that charter schools vary widely in test-based performance relative to regular public schools, both by location as well as subgroup. Yet, you'll often hear people point out that the highest-quality evidence suggests otherwise (see here, here and here) - i.e., that there are a handful of studies using experimental methods (randomized controlled trials, or RCTs) and these analyses generally find stronger, more uniform positive charter impacts.

Sometimes, this argument is used to imply that the evidence, as a whole, clearly favors charters, and, perhaps by extension, that many of the rigorous non-experimental charter studies - those using sophisticated techniques to control for differences between students - would lead to different conclusions were they RCTs.*

Though these latter assertions are based on a valid point about the power of experimental studies (the few of which we have are often ignored in the debate over charters), they are dubiously overstated for a couple of reasons, discussed below. But a new report from the (indispensable) organization Mathematica addresses the issue head on, by directly comparing estimates of charter school effects that come from an experimental analysis with those from non-experimental analyses of the same group of schools.

The researchers find that there are differences in the results, but many are not statistically significant and those that are don't usually alter the conclusions. This is an important (and somewhat rare) study, one that does not, of course, settle the issue, but does provide some additional tentative support for the use of strong non-experimental charter research in policy decisions.