K-12 Education

Is The Social Side Of Education Touchy Feely?

That's right, measuring social and organizational aspects of schools is just... well, "touchy feely." We all intuitively grasp that social relations are important in our work environments, that having mentors on the job can make a world of difference, that knowing how to work with colleagues matters to the quality of the end product, that innovation and improvement relies on the sharing of ideas, that having a good relationship with supervisors influences both engagement and performance, and so on.

I could go on, but I don't have to; we all just know these things. But is there hard evidence, other than common sense and our personal experiences? Behaviors such as collaboration and interaction or qualities like trust are difficult to quantify. In the end, is it possible that they are just 'soft' and that, even if they’re important (and they are), they just don't belong in policy conversations?

Wrong.

In this post, I review three distinct methodological approaches that researchers have used to understand social-organizational aspects of schools. Specifically, I selected studies that examine the relationship between aspects of teachers' social-organizational environments and their students' achievement growth. I focus both on the methods and on the substantive findings. This is because I think some basic sense of how researchers look at complex constructs like trust or collegiality can deepen our understanding of this work and lead us to embrace its implications for policy and practice more fully.

Broadening The Educational Capability Conversation: Leveraging The Social Dimension

Our guest author today is James P. Spillane, Spencer T. and Ann W. Olin Professor in Learning and Organizational Change at the School of Education and Social Policy at Northwestern University. Spillane has published extensively on issues of education policy, policy implementation, school reform, and school leadership. His most recent books are Distributed Leadership in Practice and Diagnosis and Design for School Improvement. Learn more about Spillane's work at www.distributedleadership.org.

We are well into a new century – 15 years and counting! Yet, we continue to fixate on last century notions about human capability. Specifically, we still dwell mostly on the individual teacher or school leader, on investing in and developing their individual human capital so as to improve their productivity and in turn generating higher returns to the individual, school organization, school system, and society. The empirical evidence has established educational professionals' human capital is undoubtedly important for school and school-system productivity.* At the same time, however, by fixating primarily on human capital, we miss or undermine the significance and potential of social capital.

Social capital captures the idea that capability (and by extension productivity) is not simply an individual matter but also a social matter. In other words, in addition to individual capability, there are (often untapped) resources that reside in the relations among people within organizations, systems, or society – a social capability. These social relations can be a source of and a channel for crucial resources such as trust, information, expertise, materials, security, obligation, incentives, and so on - see Bryk & Schneider 2002; Coburn 2001; Daly, Moolenaar, Bolivar, & Burke 2010; Frank, Zhao, & Borman 2004; Frank, Zhao, Penuel, Ellefson, & Porter 2011; Louis, Marks, & Kruse 1996; Moolenaar, Karsten, Sleegers, & Daly 2014. In a given system or organization, social capital is much more than the aggregate of members' human capital.

Teacher Turnover At Success Academy Charter Schools

A recent New York Times article about the Success Academies, a large chain of New York City charter schools, focuses a great deal on the long working hours and heavy stress faced by teachers at these schools. The article reports that three Success Academy (SA) schools had teacher turnover rates above 50 percent. Officials from the network, however, dispute these figures, which they say are inflated by the fact that many teachers who leave SA schools simply transfer to other SA schools (i.e., they are counted falsely as leaving SA when they are in fact staying within the network).

In fact, SA officials claim that, when one account for these intra-network transfers, their true turnover rate across all their schools ("attrition from the network," in the article) between June 2013 and June 2014 was 17 percent, which is far lower than many critics suggest. Now, on the one hand, these ongoing debates about teacher turnover at SA schools, which have been occurring regularly for years, are a little strange. It is clear that SA teachers work unusually long hours in high stress, tightly regulated environments, and do so for salaries that are lower than those offered by most other professional jobs with similar working conditions. It is, therefore, hardly surprising that turnover would be high; indeed, high teacher churn, like student mobility, is in many respects part of the model of schools such as the Success Academies (and, of course, some turnover, such as that among poorly performing teachers or those who are not a good fit for their schools, can be beneficial).

On the other hand, however, SA officials are making an empirical claim about turnover at their schools, one that includes an interesting and somewhat unusual angle (intra-network mobility). And this claim is very easy to examine with teacher-level data that we happen to have available via a public records request. So, let’s take a quick look at turnover at SA between 2012-13 and 2013-14 (the latest year-to-year transition we have).

Charter Schools, Special Education Students, And Test-Based Accountability

Opponents often argue that charter schools tend to serve a disproportionately low number of special education students. And, while there may be exceptions and certainly a great deal of variation, that argument is essentially accurate. Regardless of why this is the case (and there is plenty of contentious debate about that), some charter school supporters have acknowledged that it may be a problem insofar as charters are viewed as a large scale alternative to regular public schools.

For example, Robin Lake, writing for the Center for Reinventing Public Education, takes issue with her fellow charter supporters who assert that “we cannot expect every school to be all things to every child.” She argues instead that schools, regardless of their governance structures, should never “send the soft message that kids with significant differences are not welcome,” or treat them as if “they are somebody else’s problem.” Rather, Ms. Lake calls upon charter school operators to take up the banner of serving the most vulnerable and challenging students and “work for systemic special education solutions.”

These are, needless to say, noble thoughts, with which many charter opponents and supporters can agree. Still, there is a somewhat more technocratic but perhaps more actionable issue lurking beneath the surface here: Put simply, until test-based accountability systems in the U.S. are redesigned such that they stop penalizing schools for the students they serve, rather than their effectiveness in serving those students, there will be a rather strong disincentive for charters to focus aggressively on serving special education students. Moreover, whatever accountability disadvantage may be faced by regular public schools that serve higher proportions of special education students pales in comparison with that faced by all schools, charter and regular public, located in higher-poverty areas. In this sense, then, addressing this problem is something that charter supporters and opponents should be doing together.

The Education Policy Glossary

Like most policy fields, education is full of jargon. There are countless acronyms, terms and phrases that may hold little meaning for the average citizen, but are used routinely in education circles. Moreover, there are just as many words and phrases that carry a different meaning in education than they do in regular conversation.

We at the Shanker Institute have started a new project to help people, inside and outside the field, to understand the language of education policy. Accordingly, we have assembled the first installment of an education policy glossary that indicates what people in education typically mean, intentionally or unintentionally, when they use certain words and phrases.

We hope that this will encourage more people to engage in the public discourse, and to improve understanding and consistency among those of us who are already participating. The glossary is below.

Lessons And Directions From The CREDO Urban Charter School Study

Last week, CREDO, a Stanford University research organization that focuses mostly on charter schools, released an analysis of the test-based effectiveness of charter schools in “urban areas” – that is, charters located in cities located within in 42 urban areas throughout 22 states. The math and reading testing data used in the analysis are from the 2006-07 to 2010-11 school years.

In short, the researchers find that, across all areas included, charters’ estimated impacts on test scores, vis-à-vis the regular public schools to which they are compared, are positive and statistically discernible. The magnitude of the overall estimated effect is somewhat modest in reading, and larger in math. In both cases, as always, results vary substantially by location, with very large and positive impacts in some places and negative impacts in others.

These “horse race” charter school studies are certainly worthwhile, and their findings have useful policy implications. In another sense, however, the public’s relentless focus on the “bottom line” of these analyses is tantamount to asking continually a question ("do charter schools boost test scores?") to which we already know the answer (some do, some do not). This approach is somewhat inconsistent with the whole idea of charter schools, and with harvesting what is their largest potential contribution to U.S. public education. But there are also a few more specific issues and findings in this report that merit a little bit of further discussion, and we’ll start with those.

Recent Trends In The Sources Of Public Education Revenue

Every year, the U.S. Census Bureau issues a report on the overall state of public education finances in the U.S. There is usually a roughly 2-3 year lag on the data – for example, the latest report applies to the 2011-12 fiscal year – but the report and accompanying data are a good way to keep an eye on the general education finance situation both in individual states as well as nationwide, particularly among those of us who are somewhat casual followers (though it bears keeping in mind that these data do not include many charter schools).

One of the more interesting trends in recent years is the breakdown of total revenue by source. As most people know, U.S. public school systems are funded by a combination of federal, state and local revenue. Today, although states vary considerably in the configuration of these three sources, on the whole, most funding comes from state and local revenue, with a smaller but still significant contribution from federal government sources (total revenue in 2011-12 was about $595 billion).

But there has been some volatility in these relative contributions over the past few years (at least the past few years for which data are available). The graph below presents the percent of total elementary/secondary education revenue from federal, state and local sources between 1989-90 and 2011-12.

The Big Story About Gender Gaps In Test Scores

The OECD recently published a report about differences in test scores between boys and girls on the Programme for International Student Assessment (PISA), which is a test of 15 year olds conducted every three years in multiple subjects. The main summary finding is that, in most nations, girls are significantly less likely than boys to score below the “proficient” threshold in all three subjects (math, reading and science). The report also includes survey items and other outcomes.

First, it is interesting to me how discussions of these gender gaps differ from those about gaps between income or ethnicity groups. Specifically, when we talk about gender gaps, we interpret them properly – as gaps in measured performance between groups of students. Any discussion of gaps between groups defined in terms of income or ethnicity, on the other hand, are almost always framed in terms of school performance.

This is partially because schools in the U.S. are segregated by income and ethnicity, but not really by gender, and also because some folks have a tendency to overestimate the degree to which income- and ethnicity-based achievement gaps stem from systematic variation in schooling inputs, whereas in reality they are more a function of non-school factors (though, of course, schools matter, and differences in school quality reinforce the non-school-based impact). That said, returning to the findings of this report, I was slightly concerned with how, in some cases, they were reported in the media.

The Smoke And The Fire From Evaluations Of Teach For America

A recent study by the always reliable research organization Mathematica takes a look at the characteristics and test-based effectiveness of Teach For America (TFA) teachers who were recruited as part of a $50 million federal “Investing in Innovation” grant, which is supporting a substantial scale-up of TFA’s presence in U.S. public schools.

The results of this study pertain to a small group of recruits (and comparison non-TFA teachers) from the first two years of the program – i.e., a sample of 156 PK-5 teachers (66 TFA and 90 non-TFA) in 36 schools spread throughout 10 states. What distinguishes the analysis methodologically is that it exploits the random assignment of students to teachers in these schools, which ensures that any measured differences between TFA and comparison teachers are not due to unobserved differences in the students they are assigned to teach.

The Mathematica researchers found, in short, that the estimated differences in the impact of TFA and comparison teachers on math and reading scores across all grades were modest in magnitude and not statistically discernible at any conventional level. There were, however, meaningful positive estimated differences in the earliest grades (PK-2), though they were only statistically significant in reading, and the coefficient in reading for grades 3-5 was negative (and not significant). Let’s take a quick look at these and other findings from this report and what they might mean.