The Offline Implications Of The Research About Online Charter Schools

It’s rare to find an educational intervention with as unambiguous a research track record as online charter schools. Now, to be clear, it’s not a large body of research by any stretch, its conclusions may change in time, and the online charter sub-sector remains relatively small and concentrated in a few states. For now, though, the results seem incredibly bad (Zimmer et al. 2009; Woodworth et al. 2015). In virtually every state where these schools have been studied, across virtually all student subgroups, and in both reading and math, the estimated impact of online charter schools on student testing performance is negative and large in magnitude.

Predictably, and not without justification, those who oppose charter schools in general are particularly vehement when it comes to online charter schools – they should, according to many of these folks, be closed down, even outlawed. Charter school supporters, on the other hand, tend to acknowledge the negative results (to their credit) but make less drastic suggestions, such as greater oversight, including selective closure, and stricter authorizing practices.

Regardless of your opinion on what to do about online charter schools’ poor (test-based) results, they are truly an interesting phenomenon for a few reasons.

The first reason is probably the most boring, but it deserves quick mention, and it is the methodological issue of whether online charter school students are different from their peers who attend “brick and mortar” schools. More specifically, to what degree might online charter students differ from their peers in ways that affect testing performance, but that are not captured by available variables? This is, of course, an issue in most studies of school effects, but it seems particularly salient in the case of online charters.

For example, some students attend online schools due to disability or illness, or because they live in remote areas. For these parents and students, online charters may be the best choice – or even the only choice – regardless of testing results. Some parents also use online schools because they want to homeschool their children. Regardless of the reasons (and there are plenty), online schooling is an unusual choice for parents and students, and it’s very difficult to state with any confidence that the students who attend these schools aren’t dissimilar from their peers in ways that might affect the results of empirical studies. 

My view on this (and this is just my opinion) is that these types of unobserved differences may indeed affect the results of test-based evaluations, but that the net effect is likely to be moderate at best. In other words, I don't think it would "explain away" the results (I'm actually not entirely convinced about whether the bias would be upward or downward). But it is definitely still worth bearing in mind.

Second, as someone who is always more concerned about why rather than whether charters perform better or worse than comparable regular public schools, online charter schools are quite fascinating. In traditional charter school studies (i.e., comparisons of “brick and mortar” charter and regular public schools), it is, in my opinion, most useful to look for differences in policy and practice that are associated with better or worse performance, and use that information to improve all schools. In the case of online charter schools, the policy difference is clear and quite drastic: instruction is delivered online instead of in person. So, most basically, if we accept the current state of the empirical research, what is it about this educational medium that seems to produce inferior (test-based) results overall? 

On the one hand, the grouchy old man in me wants to say that good, old-fashioned sitting in a room and listening to someone who knows what they’re talking about is, for most students, more effective than even well-structured education at home in front of a computer screen. Human interaction (including that with other students) has its benefits.

On the other hand, there is a great deal of variation within the online charter school sub-sector in terms of approach (Gill et al. 2015). That includes how lessons are delivered (video, text), whether students are required to be online at specific times, the amount of interactivity with and feedback from instructors, and so on. Some schools even combine online and in-person instruction. In other words, I'd be just as interested in looking for associations within the online charter sub-sector as I would be in making comparisons between sub-sectors (i.e., between online and "brick and mortar" schools).

Is it possible that there is a “right way” to deliver education online, one that could produce results on par with the traditional method? I would speculate that: 1) there is such a way (or, more accurately, multiple approaches that work for different students); but 2) these approaches would only be more effective than traditional schooling for a tiny subgroup of students; and 3) the students for whom it would be effective aren’t always (or even often) the ones who select into online charter schools. 

But that is just speculation. The main point here is that connecting variation in policies and practices with variation in results is an important tool for learning about effective instruction, and the concept of online schooling is nothing if not a clear and unique variation in policy and practice. From this perspective, finding the “right ways” to do online instruction would not necessarily serve as a justification for more online schools, but it could certainly offer policy-relevant information, perhaps important information, that could benefit all schools. Whether that opportunity is worth the bad results is, of course, a serious problem.

A third and final interesting point about the negative evidence on online charter schools is that this is an example of a situation in which testing evidence is quite clear, but has not (yet) had much of an impact on policy. The reality is that enrollment in online charter schools is growing, though, again, that growth is not uniform across states (Woodworth et al. 2015). 

To charter school opponents, this is no doubt an example of a double standard – charter school supporters, they might argue, tout their testing results when they are positive but remain silent when that evidence is negative. There may certainly be some of that tribalism going on here, though, again, many charter supporters acknowledge the need to do something about this situation (and, by the way, the selective evidence approach is most definitely not confined to any one “side” in the education policy debate).

But there’s also a different take on this seemingly incongruous juxtaposition of poor results and a growing sector, one that isn't even remotely original but might appeal across the education ideology spectrum. And that is: test results don’t tell parents everything that they need to know, or even most of what they want to know, about school quality and fit. 

It’s not clear how many parents of online charter students are aware of the research, and it is possible that some of these parents would change their minds were they made aware of it. Regardless, though, parents choose schools for many reasons that have nothing to do with test scores, and it is plausible to think that parents and students who choose online schools are even more likely to have educational needs and wants to which standardized test scores are not particularly pertinent. 

This speaks to a fundamental feature of school choice theory (and advocacy). The idea of school choice, in crude terms, is that parents know what’s best for their children, and so schools that are effective will survive, while ineffective schools will be deselected via non-enrollment. In this way, competition will improve performance in all sectors. 

Putting aside all the complications underlying this theory, which thoughtful choice advocates understand and acknowledge, what is the role of outcome-based research (and accountability) in a school choice system? If the available evidence indicates that a given school or type of school is having a large negative effect on meaningful outcomes, but parents continue to choose that option, what is the responsibility of policymakers (and advocates) in that situation?

For school choice “purists,” for lack of a better term, the testing results may not carry much weight (though, again, a few seem to put forth testing evidence when it suits them). Without a robust system of multi-sectoral choice, the potential competitive benefits of choice may not arise. There is also the (somewhat circular but still fair) argument that parents of online charter school students pay taxes, and if they want online schools, they should have that choice.

But is it responsible for policymakers to take this position when it comes to online charter schools, given the current evidence? I am inclined to say no, but not without hesitation (not only because it is a fundamentally ethical and political question, but also because I could so easily question the premises behind it, most notably the fact that it depends completely on estimates from imperfect models that use one type of data). One somewhat ironic reality of U.S. education policy is that test scores are the law of the land, but even the most egregious offenders can be difficult to prosecute, and that is not always a bad thing.