• Technology In Education: An Answer In Search Of A Problem?

    In a recent blog post, Larry Cuban muses about the enthusiasm of some superintendents, school board members, parents, and pundits for expensive, new technologies, such as “iPads, tablets, and 1:1 laptops."

    Without any clear evidence, they spend massively on the newest technology, expecting that “these devices will motivate students to work harder, gain more knowledge and skills, and be engaged in schooling." They believe such devices can help students develop the skills they will need in a 21st century labor market—and hope they will somehow help to narrow the achievement gap that has been widening between rich and poor.

    But, argues Cuban, for those school leaders “who want to provide credible answers to the inevitable question that decision-makers ask about the effectiveness of new devices, they might consider a prior question. What is the pressing or important problem to which an iPad is the solution?"

    Good question. Now, good enough? I am not so sure. It still implicitly assumes an iPad must be a solution to some-thing in education.

  • Dispatches From The Nexus Of Bad Research And Bad Journalism

    In a recent story, the New York Daily News uses the recently-released teacher data reports (TDRs) to “prove” that the city’s charter school teachers are better than their counterparts in regular public schools. The headline announces boldly: New York City charter schools have a higher percentage of better teachers than public schools (it has since been changed to: "Charters outshine public schools").

    Taking things even further, within the article itself, the reporters note, “The newly released records indicate charters have higher performing teachers than regular public schools."

    So, not only are they equating words like “better” with value-added scores, but they’re obviously comfortable drawing conclusions about these traits based on the TDR data.

    The article is a pretty remarkable display of both poor journalism and poor research. The reporters not only attempted to do something they couldn’t do, but they did it badly to boot. It’s unfortunate to have to waste one’s time addressing this kind of thing, but, no matter your opinion on charter schools, it's a good example of how not to use the data that the Daily News and other newspapers released to the public.

  • The Charter School Authorization Theory

    Anyone who wants to start a charter school must of course receive permission, and there are laws and policies governing how such permission is granted. In some states, multiple entities (mostly districts) serve as charter authorizers, whereas in others, there is only one or very few. For example, in California there are almost 300 entities that can authorize schools, almost all of them school districts. In contrast, in Arizona, a state board makes all the decisions.

    The conventional wisdom among many charter advocates is that the performance of charter schools depends a great deal on the “quality” of authorization policies – how those who grant (or don’t renew) charters make their decisions. This is often the response when supporters are confronted with the fact that charter results are varied but tend to be, on average, no better or worse than those of regular public schools. They argue that some authorization policies are better than others, i.e., bad processes allow some poorly-designed schools start, while failing to close others.

    This argument makes sense on the surface, but there seems to be scant evidence on whether and how authorization policies influence charter performance. From that perspective, the authorizer argument might seem a bit like tautology – i.e., there are bad schools because authorizers allow bad schools to open, and fail to close them. As I am not particularly well-versed in this area, I thought I would look into this a little bit.

  • The ‘Snob’ Debate: Making High School Matter For Non-College-Bound Students

    Our guest author today is James R. Stone, professor and director of the National Research Center for Career & Technical Education at the University of Louisville.

    The current debate about “college for all” centers on a recent speech made by President Obama in Troy, MI, in which he argued that all young people should get at least some post-high school education or training. Republican presidential primary candidate Rick Santorum, in a misreading of Obama’s remarks, responded with a focus on four-year degrees alone—suggesting, among other things, that four-year college degrees are overrated and that the president’s emphasis on college devalued working people without such degrees. The political chatter around this particular back-and-forth continues, but the issue of “college for all” has rightly raised some serious issues about the content and direction of U.S. education policy both at the high school and post-secondary levels.

    Statistics seem to show that the college-educated  graduates of four-year institutions earn more money and experience less unemployment than their non-college-educated peers. This has fueled the argument is that college is the surest path—perhaps the only path—into the middle class. But the argument confuses correlation with causality. What if every U.S. citizen obtained a community college or university degree? Would that really do anything to alter wage rates at Starbucks, or increase salaries for home healthcare aides (an occupation projected to enjoy the highest demand over the next decade)? Of course not.

  • Why Stop In Early Childhood? Two-Generation Strategies To Improve Educational Outcomes

    In education research, it is now widely accepted that ages 0 to 5 are crucial years for child development. In addition, there is a growing body of evidence demonstrating that children perform better behaviorally and academically in families with stable employment and rising incomes, families with stable employment and those where parents themselves are improving their own educational levels.

    Although it’s clear that increasing parents’ human capital protects and enhances the investments made in their children, "few programs have addressed the postsecondary education and training needs of low-income parents" (p. 2) through comprehensive, family-(child- and parent-) centered strategies.*

    I learned about some remarkable exceptions at a recent New America Foundation discussion on innovations in child care and early learning. Four providers from around the country were asked to describe their programs, all largely focused on helping parents achieve the kind of economic stability needed to support their children’s educational attainment.**

  • Revisiting The "5-10 Percent Solution"

    In a post over a year ago, I discussed the common argument that dismissing the “bottom 5-10 percent" of teachers would increase U.S. test scores to the level of high-performing nations. This argument is based on a calculation by economist Eric Hanushek, which suggests that dismissing the lowest-scoring teachers based on their math value-added scores would, over a period of around ten years  (when the first cohort of students would have gone through the schooling system without the “bottom” teachers), increase U.S. math scores dramatically – perhaps to the level of high-performing nations such as Canada or Finland.*

    This argument is, to say the least, controversial, and it invokes the full spectrum of reactions. In my opinion, it's best seen as a policy-relevant illustration of the wide variation in test-based teacher effects, one that might suggest a potential of a course of action but can't really tell us how it will turn out in practice. To highlight this point, I want to take a look at one issue mentioned in that previous post – that is, how the instability of value-added scores over time (which Hanushek’s simulation doesn’t address directly) might affect the projected benefits of this type of intervention, and how this is turn might modulate one's view of the huge projected benefits.

    One (admittedly crude) way to do this is to use the newly-released New York City value-added data, and look at 2010 outcomes for the “bottom 10 percent” of math teachers in 2009.

  • A Look At The Education Programs Of The Gates Foundation

    Our guest author today is Ken Libby, a graduate student studying educational foundations, policy and practice at the University of Colorado at Boulder.

    The Bill & Melinda Gates Foundation is the largest philanthropic organization involved in public education. Their flexible capital allows the foundation to change course, experiment and take on tasks that would be problematic for other organizations.

    Although the foundation’s education programs have been the subject of both praise and controversy, one area in which they deserve a great deal of credit is transparency. Unlike most other foundations, which provide a bare minimum, time-lagged account of their activities, Gates not only provides a description of each grant on its annually-filed IRS 990-PF forms, but it also maintains a continually updated list of grants posted on the foundation’s website. This nearly real-time outlet provides the public with information about grants months before the foundation is required to do so.

    The purpose of this post is to provide descriptive information about programmatic support and changes between 2008 and 2010. These are the three years for which information is currently available.

  • Apprenticeships: A Rigorous And Tested Training Model For Workers And Management

    Our guest author today is Robert I. Lerman, Institute Fellow at the Urban Institute and Professor of Economics at American University. Professor Lerman conducts research and policy analyses on employment, income support and youth development, especially as they affect low-income populations. He served on the National Academy of Sciences panel examining the U.S. post-secondary education and training system for the workplace.

     

    In a recent Washington Post article, Peter Whoriskey points out the striking paradox of serious worker shortages at a time of high unemployment.  His analysis is one of many indicating the difficulties faced by manufacturing firms in hiring enough workers with adequate occupational skills.  As a result, many firms are having serious problems meeting the demand for their products, putting on long shifts, and turning down orders.

    The article cites a survey of manufacturers indicating that as many as 600,000 jobs are going unfilled.  The skilled jobs going begging include machinists, welders, and machine operators -- jobs that pay good wages.  So what happened?

  • Ready, Aim, Hire: Predicting The Future Performance Of Teacher Candidates

    In a previous post, I discussed the idea of “attracting the best candidates” to teaching by reviewing the research on the association between pre-service characteristics and future performance (usually defined in terms of teachers’ estimated effect on test scores once they get into the classroom). In general, this body of work indicates that, while far from futile, it’s extremely difficult to predict who will be an “effective” teacher based on their paper traits, including those that are typically used to define “top candidates," such as the selectivity of the undergraduate institutions they attend, certification test scores and GPA (see here, here, here and here, for examples).

    There is some very limited evidence that other, “non-traditional” measures might help. For example, a working paper, released last year, found a statistically discernible, fairly strong association between first-year math value-added and an index constructed from surveys administered to Teach for America candidates. There was, however, no association in reading (note that the sample was small), and no relationships in either subject found during these teachers’ second years.*

    A recently-published paper – which appears in the peer-reviewed journal Education Finance and Policy, originally released as working paper in 2008 –  represents another step forward in this area. The analysis, presented by the respected quartet of Jonah Rockoff, Brian Jacob, Thomas Kane, and Douglas Staiger (RJKS), attempts to look beyond the set of characteristics that researchers are typically constrained (by data availability) to examine.

    In short, the results do reveal some meaningful, potentially policy-relevant associations between pre-service characteristics and future outcomes. From a more general perspective, however, they are also a testament to the difficulties inherent in predicting who will be a good teacher based on observable traits.

  • Reign Of Error: The Publication Of Teacher Data Reports In New York City

    Late last week and over the weekend, New York City newspapers, including the New York Times and Wall Street Journal, published the value-added scores (teacher data reports) for thousands of the city’s teachers. Prior to this release, I and others argued that the newspapers should present margins of error along with the estimates. To their credit, both papers did so.

    In the Times’ version, for example, each individual teacher’s value-added score (converted to a percentile rank) is presented graphically, for math and reading, in both 2010 and over a teacher’s “career” (averaged across previous years), along with the margins of error. In addition, both papers provided descriptions and warnings about the imprecision in the results. So, while the decision to publish was still, in my personal view, a terrible mistake, the papers at least make a good faith attempt to highlight the imprecision.

    That said, they also published data from the city that use teachers’ value-added scores to label them as one of five categories: low, below average, average, above average or high. The Times did this only at the school level (i.e., the percent of each school’s teachers that are “above average” or “high”), while the Journal actually labeled each individual teacher. Presumably, most people who view the databases, particularly the Journal's, will rely heavily on these categorical ratings, as they are easier to understand than percentile ranks surrounded by error margins. The inherent problems with these ratings are what I’d like to discuss, as they illustrate important concepts about estimation error and what can be done about it.