Education

Educational Unilateralism

In New York City this week, a special "plenary summit" of the UN General Assembly met to encourage the world to step up support for the Millennium Development Goals (MDG) first okayed by the UN in 2000. These eight goals – which include slashing poverty, combating disease, fighting hunger, protecting the environment, and boosting education – had a 2015 target date for their achievement. Ten years on, the summit reviewed progress and urged participants to speed up the pace.While the eradication of disease and hunger was named as the key priority, the nations of the world also recognized the crucial importance of education. Goal 2 focuses on the right of all kids to at least a primary school education. Goal 3 promotes the right of girls to have the same access to education as boys – a major problem in much of the developing world. 

Although the U.S. is the world’s largest donor country, surveys show that few Americans have heard about it. President Obama, who during his campaign pledged to fund a $2 billion Global Fund for Education, has done little – what with the financial crisis and debates over both the means and ends of foreign assistance programs getting in the way. In the meanwhile, critics call MDG little more than a laundry list of needs, with no real strategy on how to achieve them. Still, the goals are well worth reading, if only as a reflection of what the world believes (at least on paper) are the rock-bottom problems facing humanity in the 21^st century.

Persistently Low-Performing Incentives

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City's schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains. 

Research Wars

Weeks before the fact, a Sept. 29 forum sponsored by the Economic Policy Institute and the National Education Policy Center has sparked some interesting debate over at the National Journal. The event, centered around the recent book, Think Tank Research Quality: Lessons for Policy Makers, the Media and the Public, is an effort to separate "the junk research from the science."

The crux of the debate is whether the recent explosion of self-published reports by various educational think tanks has helped or hindered the effort to improve the quality of educational research. (Full disclosure: The Albert Shanker Institute is often called a "think tank" and we frequently self-publish.) The push and pull of dueling experts and conflicting reports, say some, has turned education research into a political football—moved down the field by one faction, only to be punted all the way to the other end by a rival faction—each citing "research" as their guide.

"My research says this works and that doesn’t," can always be countered by, "Oh yeah, well my research says that works and this doesn’t." There are even arguments about what "what works" means because, except for performance on standardized tests, our goals remain diverse, decentralized and subject to local control. As a result, public education is plagued by trial and error policies that rise and fall, district by district and state by state, like some sort of crazed popularity contest.

Teacher Quality On The Red Carpet; Accuracy Swept Under The Rug

The media campaign surrounding “Waiting for Superman," which has already drawn considerable coverage, only promises to get bigger. While I would argue – at least in theory – that increased coverage of education is a good thing, it also means that this is a critically important (and, in some respects, dangerous) time. Millions of new people will be tuning in, believing that they are hearing serious discussions about the state of public education in America and “what the research says” about how it can be improved.

It’s therefore a sure bet that what I’ve called the “teacher effects talking point” will be making regular appearances. It goes something like this: Teachers are the most important schooling factor influencing student achievement. This argument provides much of the empirical backbone for the current push toward changes in teacher personnel policies. It is an important finding based on high-quality research, one with clear policy implications. It is also potentially very misleading.

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.

More Than One Way Of Winning

Our guest author today is James R. Stone, professor and director of the National Research Center for Career & Technical Education at the University of Louisville.

In recent years, a consensus has emerged among education researchers and policymakers that all students should graduate from high school both "college- and career-ready." President Obama has made this part of his education agenda. And numerous advocacy organizations have championed the notion. But what does the phrase actually mean?

"College-ready" usually means not needing remedial courses once in college, and "career-ready" is usually equated with college-ready. High standards and expectations are the means recommended to prepare college-ready graduates. This means rigorous courses aligned with standards, and tests to ensure that students meet those standards. Presumably, career-readiness comes with the same requirements. The evidence contradicts the rhetoric, however. Paul Barton at ETS, Peter Cappelli at the Wharton School, and other labor market experts argue that being prepared for college is not the same as being prepared for a successful transition into the workforce.

Perhaps we ought to consider an alternative framework that more clearly defines what college- and career-ready means.

Extra Time: More From The Magazine's Education Poll

** Also posted here on "Valerie Strauss’ Answer Sheet" in the Washington Post.

A recent education poll conducted by Time Magazine has gotten a lot of attention. Many of the questions are worded so badly that the results are rather meaningless. The question on merit pay, for example, defines the practice as “paying teachers according to their effectiveness” (who would oppose that, if it could be accurately measured?). Other questions are very interesting, such as the one asking whether respondents would pay higher taxes to improve public schools (56 percent would). Or the finding that, when asked what will “improve student achievement the most," more than twice as many people choose “more involved parents” (54 percent) over “more effective teachers” (24 percent).

But, as is sometimes the case, a few of the survey’s most interesting results were not included in the published article, which highlighted only 11 out of 40-50 or so total questions (the full set of results is available here). Here are three or four unpublished items that caught my eye (the sample size is 1,000, with a margin of error of +/- 3 percent):

Are Value-Added Models Objective?

In recent discussions about teacher evaluation, some people try to distinguish between "subjective" measures (such as principal and peer observations) and "objective" measures (usually referring to value-added estimates of teachers’ effects on student test scores).

In practical usage, objectivity refers to the relative absence of bias from human judgment ("pure" objectivity being unattainable). Value-added models are called "objective" because they use standardized testing data and a single tool for analyzing them: All students in a given grade/subject take the same test and all teachers’ "effects" in a given district or state are estimated by the same model. Put differently, all teachers are treated the same (at least those 25 percent or so who teach grades and subjects that are tested), and human judgment is relatively absent.

By this standard, are value-added models objective? No. And it is somewhat misleading to suggest that they are.

Does Language Shape Thought?

Do the words we use frame the thoughts that we have? And, if so, does the language we speak affect how we think?

It turns out that linguists and cognitive scientists have been going back and forth on this issue for years. There is a fascinating article on the subject in last weekend’s New York Times Magazine (which I’m only now getting around to reading). It’s a piece by Guy Deutscher, an honorary research fellow at the School of Languages, Linguistics and Cultures at the University of Manchester, and author of a forthcoming book, Through the Language Glass: Why the World Looks Different in Other Languages.

First popularized in the 1940s, the Sapir-Whorf hypothesis of linguistic relativity "seduced a whole generation into believing that our mother tongue restricts what we are able to think." The lack of a future and past tense in a given language, for example, was supposed to limit some speakers’ ability to comprehend the concepts of future and past.

Although such ethnocentric and romantic aspects of the theory have been totally discredited, new research does suggest that language can have an effect on both thought and perception. For example, it has recently "been demonstrated in a series of ingenious experiments that we even perceive colors through the lens of our mother tongue." As it turns out, different languages "carve up the spectrum of visible light" in different ways, with, for instance, many languages considering blue and green to be variations of the same color. And, astonishingly, "our brains are trained to exaggerate the distance between shades of color if these have different names in our language." So, "as strange as it may sound, our experience of a Chagall painting actually depends to some extent on whether our language has a word for blue."

Help The Economy: Put A Teacher's Aide In Every Classroom

Economist Robert Shiller (co-creator of the Case-Shiller Home Price Index, an essential tool for investors and economists) has an interesting idea for stimulating the economy: Put a teacher’s aide in every classroom.

Why? As reported by the Wall Street Journal, "Not only would it employ millions, but it would be good for the children," who would benefit from "the extra attention of another person."

Shiller is regarded as one of the most important economists today. The Arthur M. Okun professor of economics at Yale University and professor of finance at the Yale School of Management, he forewarned about both the dot.com bust and the housing bubble. For years, he criticized the so-called efficient markets model of economics, which many today cite as a key driver of the policies that led to the financial crisis. He is also the author of many books, including, Irrational Exuberance in 2000, which warned that the peaking real estate and stock markets were in bubble territory.

Shiller is worried about today’s economy. He estimates that the likelihood of a double-dip recession is growing and that we are "teetering" on the brink of a dangerous deflationary spiral. What to do?

The Test-Based Language Of Education

A recent poll on education attitudes from Gallup and Phi Delta Kappan got a lot of attention, including a mention on ABC’s "This Week with Christian Amanpour," which devoted most of its show to education yesterday. They flashed results for one of the poll’s questions, showing that 72 percent of Americans believe that "each teacher should be paid on the basis of the quality of his or her work," rather than on a "standard-scale basis."

Anyone who knows anything about survey methodology knows that responses to questions can vary dramatically with different wordings (death tax, anyone?). The wording of this Gallup/PDK question, of course, presumes that the "quality of work" among teachers might be measured accurately. The term "teacher quality" is thrown around constantly in education circles, and in practice, it is usually used in the context of teachers’ effects on students’ test scores (as estimated by various classes of "value-added" models).

But let’s say the Gallup/PDK poll asked respondents if "each teacher should be paid on the basis of their estimated effect on their students’ standardized test scores, relative to other teachers." Think the results would be different? Of course. This doesn’t necessarily say anything about the "merit" of the compensation argument, so to speak, nor does it suggest that survey questions should always emphasize perfect accuracy over clarity (which would also create bias of a different sort). But has anyone looked around recently and seen just how many powerful words, such as "quality," are routinely used to refer to standardized test score-related measures? I made a tentative list.