Collecting data isn't as easy as it seems

This is our first post from Lauren Vashaw, one of the Keeping Pace researchers. Every year around this time the Keeping Pace research team is deep into data collection and analysis. We strive to report the most accurate data, and even though every year the tracking of online learning data improves, data gaps and inaccuracies persist. It’s not unusual that we find multiple sources of data that appear to contradict one another. Even as we appreciate the improvements in available data, we are often reminded of the challenges that surround researching a field for which data are sparse.

A recent event highlighted many of these issues.  We had one source, from a state education agency, which provided a set of online enrollment numbers, and another source, from a newspaper article in that state, that quoted a different enrollment number. The especially intriguing aspect of this was that the newspaper article appeared to be quoting the same education agency source that we had. When we tracked the numbers back to the agency report, and asked the agency about the apparent discrepancy, our request triggered an internal investigation within the agency to determine the correct numbers. At the time of our writing this blog post, the question has not been resolved.

We appreciate the efforts by the education agency to understand the numbers, and don’t mean to suggest that the personnel there are incapable of or unwilling to find the right numbers. Instead, this event simply highlights the challenges that many educators, policymakers, and researchers have when trying to understand the online learning landscape. Some state agencies are trying to track numbers, and are challenged by the lack of data collection mechanisms; other agencies aren’t making any concerted effort to do so. Even those that are attempting data collection are rarely getting information from single-district programs. . . but that’s a subject for another post.