“A decade of mediocre results is our control group”
The title of this blog post—“a decade of mediocre results is our control group”—is based on a response we received from a district superintendent responding to our survey seeking examples of success in blended learning. Here’s the context: a couple of months ago we launched a survey to find examples of blended learning success in traditional public school districts. We were seeking examples based on student achievement as determined by assessments, course grades, or other similar measures. In the survey we asked if the district had comparison data, for example with a control group.
The Randolph Central School District in western New York State was among those who filled out the survey. In her response to the question about a control group or comparison data, Randolph Superintendent Kimberly Moritz wrote “There is no control group of students who did not take part in blended learning but a decade of mediocre results without blended learning could certainly be considered a control group.”
Randolph’s results since implementing blended learning in its elementary grades have been impressive. Scores on state assessments have improved significantly, and the school’s ranking in a respected regional ranking of schools and districts has risen sharply.
It is likely that we will be profiling Randolph’s efforts and results when we release the first case studies of success with blended learning, so I won’t go into much more detail here. But what we see in Randolph and other districts that we are interviewing for the proof points project is a valuable counterpoint to the headline from a recent report, Personalized Instruction: New Interest, Old Rhetoric, Limited Results, and the Need for a New Direction for Computer-Mediated Learning from the National Education Policy Center (NEPC). Similar to some other reports from NEPC, the full report includes valuable information that is obscured by the opening tenor of the report, which set the tone for subsequent media reports. The report adds to the body of knowledge demonstrating that the use of education technology alone shows no significant difference in improving student achievement, because how the technology is implemented is so important. (To be clear, as we have discussed previously this finding does not discount the benefits that online courses can provide for student access.)
From the report:
“…there are several ways that these systems [of personalized instruction] can be implemented in the classroom. We are just beginning to experiment with and evaluate different implementation models—and the data show that implementation models matter. How a system is integrated into classroom routines and structures strongly mediates the outcomes for students.” (emphasis added)
That is exactly right, and we see it as we study blended and personalized learning programs in various schools. Administrators in Randolph Central School District appear to have done several things that are consistent with other successful implementations, including that they:
- Set a clear goal at the start, which was to improve student achievement in elementary school math and reading as measured by state assessments.
- Figured out how to use digital content and a technology platform to help teachers better understand their students, including using multiple formative assessments throughout the school year to create “fluid ability groups,” ending the “random path” that students had been taking through math throughout the school year and across grades.
- Created a set of “non-negotiables” about how data would be used in the school to change instruction.
- Repeated communications about the change to multiple audiences with “boorish redundancy.”
Two facts about this implementation are equally true (and apply to many other blended learning efforts), but stressing one and not the other creates a misleading story:
- The use of technology was not nearly enough on its own to create change; yet
- The changes could not have been made without using technology.
The same technology could be implemented with very different results. If it was simply added to existing classrooms without thoughtful planning and careful application, it would not likely result in similar gains.
Perhaps every form of educational technology should come with a version of the EPA new car gasoline efficiency statement: your mileage may vary.