The Proportionality Principle In Teacher Evaluations

Our guest author today is Cory Koedel, Assistant Professor of Economics at the University of Missouri.

In a 2012 post on this blog, Dr. Di Carlo reviewed an article that I coauthored with colleagues Mark Ehlert, Eric Parsons and Michael Podgursky. The initial article (full version here, or for a shorter, less-technical version, see here) argues for the policy value of growth models that are designed to force comparisons to be between schools and teachers in observationally-similar circumstances.

The discussion is couched within the context of achieving three key policy objectives that we associate with the adoption of more-rigorous educational evaluation systems: (1) improving system-wide instruction by providing useful performance signals to schools and teachers; (2) eliciting optimal effort from school personnel; and (3) ensuring that current labor-market inequities between advantaged and disadvantaged schools are not exacerbated by the introduction of the new systems.

We argue that a model that forces comparisons to be between equally-circumstanced schools and teachers – which we describe as a “proportional” model – is best-suited to achieve these policy objectives. The conceptual appeal of the proportional approach is that it fully levels the playing field between high- and low-poverty schools. In contrast, some other growth models have been shown to produce estimates that are consistently associated with the characteristics of students being served (e.g., Student Growth Percentiles).

Expectations For Student Performance Under NCLB Waivers

A recent story in the Chicago Tribune notes that Illinois’ NCLB waiver plan sets lower targets for certain student subgroups, including minority and low-income students. This, according to the article, means that “Illinois students of different backgrounds no longer will be held to the same standards," and goes on to quote advocates who are concerned that this amounts to lower expectations for traditionally lower-scoring groups of children.

The argument that expectations should not vary by student characteristics is, of course, valid and important. Nevertheless, as Chad Aldeman notes, the policy of setting different targets for different groups of students has been legally required since the enactment of NCLB, under which states must “give credit to lower-performing groups that demonstrate progress." This was supposed to ensure, albeit with exceedingly crude measures, that schools weren't punished due to the students they serve, and how far behind were those students upon entry into the schools.

I would take that a step further by adding two additional points. The first is quite obvious, and is mentioned briefly in the Tribune article, but too often is obscured in these kinds of conversations: Neither NCLB nor the waivers actually hold students to different standards. The cut scores above which students are deemed “proficient," somewhat arbitrary though they may be, do not vary by student subgroup, or by any other factor within a given state. All students are held to the same exact standard.

Performance Measurement In Healthcare And Education

A recent story in the New York Times reports that, according to an Obama Administration-commissioned panel, the measures being used to evaluate the performance of healthcare providers are unfairly penalizing those that serve larger proportions of disadvantaged patients (thanks to Mike Petrilli for sending me the article). For example, if you’re grading hospitals based on simple, unadjusted re-admittance rates, it might appear as if hospitals serving high poverty populations are doing worse -- even if the quality of their service is excellent -- since readmissions are more likely for patients who can’t afford medication, or aren’t able to take off from work, or don’t have home support systems.

The panel recommended adjusting the performance measures, which, for instance, are used for Medicare reimbursement, using variables such as patient income and education, as this would provide a more fair accountability system – one that does not penalize healthcare institutions and their personnel for factors that are out of their control.

There are of course very strong, very obvious parallels here to education accountability policy, in which schools are judged in part based on raw proficiency rates that make no attempt to account for differences in the populations of students in different schools. The comparison also reveals an important feature of formal accountability systems in other policy fields.

Can Early Language Development Promote Children's Psychological Wellbeing?

We know oral language is young children's door into the world of knowledge and ideas, the foundation for reading, and the bedrock of all academic learning. But, can language also protect young kids against behavioral problems?

A number of studies have identified a co-occurrence of language delays and behavioral maladjustment, an association that remains after controlling for socio-demographic characteristics and academic achievement (here and here). However, most research on the issue has been cross-sectional and correlational making it hard to establish whether behavioral issues cause language delays, language delays cause behavioral issues, or another factor is responsible for both.

A recent paper by Marc Bornstein, Chun-Shin Hahn, and Joan Suwalsky (2013) was able to shed some light on these questions concluding that "language competencies in early childhood keep behavioral adjustment problems at bay." This is important given the fact that minority children raised in poverty tend to have smaller than average vocabularies and are also overrepresented in pre-K expulsions and suspensions.

An Education Hearing I'd Like To See

At the end of February, the District of Columbia Council’s Education Committee held its annual hearing on the performance of the District’s Public Schools (DCPS). The hearing (full video is available here) lasted over four hours, and included discussion on a variety of topics, but there was, inevitably, a block of time devoted to the discussion of DCPS testing results (and these questions were the focus of the news coverage).

These exchanges between Council members and DCPS Chancellor Kaya Henderson focused particularly on the low-stakes Trial Urban District Assessment (TUDA).* Though it was all very constructive and not even remotely hostile, it’s fair to say that Ms. Henderson was grilled quite a bit (as is often the case at these kinds of hearings). Unfortunately, the arguments from both sides of the dais were fraught with the typical misinterpretations of TUDA, and I could not get past how tragic it is to see legislators question the superintendent of a large urban school district based on a misinterpretation of what the data mean - and to hear that superintendent respond based on the same flawed premises.

But what I really kept thinking -- as I have before in similar contexts -- was how effective Chancellor Henderson could have been in answering the Council’s questions had she chosen to interpret the data properly (and I still hold out hope that this will become the norm some day). So, let’s take a quick look at a few major arguments that were raised during the hearing, and how they might have been answered.

"Show Me What Democracy Looks Like"

Our guest author today is John McCrann, a Math teacher and experiential educator at Harvest Collegiate High School in New York City. John is a member of the America Achieves Fellowship, Youth Opportunities Program, and Teacher Leader Study Group. He tweets at @JohnTroutMcCran.

New York City’s third through eighth graders are in the middle of state tests, and many of our city’s citizens have taken strong positions on the value (or lack thereof) of these assessments.  The protests, arguments and activism surrounding these tests remind me of a day when I was a substitute civics teacher during summer school.  “I need help," Charlotte said as she approached my desk, “what is democracy?"

On that day, my mind flashed to a scene I witnessed outside the White House in the spring of 2003.  On one side of the fence, protestors shouted: “Show me what democracy looks like! This is what democracy looks like!”  On the other side worked an administration who had invaded another country in an effort to “expand democracy." Passionate, bright people on both sides of that fence believed in the idea that Charlotte was asking about, but came to very different conclusions about how to enact the concept. 

Why A Diverse Teaching Force?

This is the third in a series of three posts about implicit bias. Here are the first and second parts.

The arguments for increasing the representation of people of color in teaching are often based around two broad rationales. First is the idea that, in a diverse, democratic society, teachers of color can serve as important role models for all children. The second idea is that teachers of color are particularly well suited to teaching students of color because they possess an inherent understanding of the culture and backgrounds of these learners.

I can think of at least two additional pro-diversity arguments that are relevant here, not only for schools but also for the broader landscape of work organizations. First, diversity can increase everyone's sense of "fitting in" in a given setting; social belonging is a basic human need that can in turn predict a wide range of favorable outcomes. Second, diversity can do more than offer role models. Repeated exposure to male pre-K teachers or black, female high school principals can challenge and expand our thinking about who is or is not  suited to certain tasks – and even the nature of those jobs and the skills required to do them. This is important to the much broader goal of fairness and equality because it contributes to disrupting strong stereotypic associations present in our culture that too often limit opportunities for people of color and women.

As I noted the first two posts of my implicit bias series (here and here), intergroup contact is one of the best researched means of reducing explicit (here and here) and unconscious (racial, gender) bias (here and here). This post explains why and how faculty diversity can act as an institution-level "de-biasing" policy or strategy.

The Wonder In Language

Our guest author today is Daniela O'Neill, Professor of Psychology at the University of Waterloo in Waterloo, Ontario. You can learn more about her work here.

In a wonderful book, Narratives from the Crib, a little two-year-old girl’s talk to herself in her crib before going to sleep was recorded by her parents and carefully transcribed by child language researchers, who then explored and wrote about the many interesting things captured in this self-talk.

Narratives in the Crib is a collection of the work of these scholars. Emily was the name of the little girl, and her talk was a fascinating window into her mind – into what she was wondering about, thinking about and trying to understand. Many years ago, when I was “listening” to Emily talk as I read the book, a little word caught my attention, because she used it a lot – it was the little word maybe.

Why did it catch my attention? Because, at the time, I’d been thinking about three- and four-year-olds’ understanding of themselves in time – that is, their understanding that they have a “past-self," a “present-self” and a “future-self," and that these are all connected in time. When children reach three- to four years old, there appears to be a pretty big shift in understanding of this concept, one which coincides, for example, with children beginning to understand and use words like “yesterday” and “tomorrow."

Will the SAT Overhaul Help Achieve Equity?

The College Board, the organization behind the SAT, acknowledges that historically its tests have been biased in favor of the children of wealthy, well educated elites – those who live in the best zip codes, are surrounded by books, go to the best regarded schools (both public and private), enjoy summer enrichment programs, and can avail themselves of as much tutoring and SAT test-prep coaching as they need. That’s why, early last month, College Board president David Coleman announced that the SAT would undergo significant changes, with the aim of making it more fair and equitable for disadvantaged students.

Among the key changes, which are expected to take effect in 2016, are: the democratization of access to test-prep courses (by trying to make them less necessary and entering into an agreement with the Khan Academy to offer free, online practice problems*); ensuring that every exam include a reading passage from one of the nation’s “founding documents," such as the Declaration of Independence or the Bill of Rights, or from one of the important discussions of such texts, such as the Rev. Dr. Martin Luther King Jr.'s “Letter From Birmingham Jail”; and replacing “arcane 'SAT words' (‘depreciatory,’ ‘membranous’)," with words that are more “commonly used in college courses, such as ‘synthesis’ and ‘empirical.’” (See here.)

Will this help? Well, maybe, but the SAT’s long held -- but always elusive -- mission to help identify and reward merit, rather than just privilege, will only be met insofar as its creators can be sure that all students have had an equal opportunity to learn these particular vocabulary words and have read these particular “founding documents” and texts. That is, it comes down to a question of curriculum.

What Is Implicit Bias, And How Might It Affect Teachers And Students? (Part II - Solutions)

This is the second in a series of three posts about implicit bias. Here are the first and third parts.

In my first post on this topic, I argued that teachers are better positioned than, say, doctors or judges, to learn specifics about the individuals they serve. This strategy – called “individuating” – has proven to be effective in reducing implicit biases (related to race, gender, ethnicity, etc.). This post offers additional thoughts on how we might support teachers' orientation to get to know their students. Second, I discuss additional strategies that have been proven to be effective in mitigating the effects of implicit biases.

A couple of weeks ago, a colleague asked a great question during the Shanker Institute’s Good Schools Seminar on "Creating Safe and Supportive Schools." His question was motivated by a presentation on implicit bias by Kirwan Institute director Sharon Davies. The question was: Wouldn’t you expect more conscious, systematic decision-making (and fewer automatic, snap judgments) from teachers who, after all, see their students everyday and get to know them well? (See here, minute 50:55.)

As I related in the previous post, individuating (or learning about the particulars of a person, his/her interests, skills, family, etc.) can be a very effective "de-biasing" tool.* So, how might we leverage and support teachers' natural inclination to get to know students well? How might a potential de-biasing intervention build on this feature of teachers' work?

Subscribe to Education