A few months ago, I ran across a comment from a K-12 educator talking about students who had fallen behind during the pandemic. I can’t seem to track down the comment, but it was quite positive and gung-ho, claiming that the school system knew what needed to be done to help these students catch up. Maybe I’m just a surly negative-minded grinch, but I didn’t believe it. After all, there are lots of students who were already falling behind before the pandemic, and that pattern has gone on for a long time. It seems clear to me that school systems have not in fact shown that they know what is needed to help students catch up.
The problem is real. Harry Anthony Patrinos, Emiliana Vegas, Rohan Carter-Rau provide an overview of the studies done so far about learning loss in the first two years of the COVID pandemic in “An Analysis of COVID-19 Student Learning Loss” (May 2022, World Bank Policy Research Working Paper 10033). A short readable overview of the study by the authors is available here.
This paper is a survey of existing research. The authors did a search for research papers from around the world that offered estimates of COVID-related learning loss for K-12 students. They hunted down 61 papers, but some of those just compared before-and-after test scores, sometimes with very small samples or with no effort to adjust for potentially confounding variables. For example, if schools serving certain groups of students were more or less likely to close during the pandemic, or if the families of certain groups of students reacted differently to the pandemic, such differences should be taken into account. Thus, the authors focused mainly on the 35 studies with large sample sizes that made some effort to sort out other factors. They write:
Our final database consists of 35 robust studies and reports documenting learning loss,
representing data from 20 countries … Most studies (32) find evidence of learning
loss. Of the 35 studies reporting learning loss, 27 reported findings in a comparable effect size format. As shown in Figure 1, most studies found learning losses ranging from 0.25 to 0.12 SD. In five studies, learning losses were even greater. The average learning loss across these studies is 0.17 standard deviation – which equates to over half a school year of learning loss.The studies consistently find heterogeneous effects of learning loss by socio-economic status, past academic learning, and subject of learning. In our review, 20 studies examined learning loss by socio-economic status. Of these, 15 find greater learning loss among students or schools with lower socio-economic status, while 5 fail to find a statistically significant difference. Many studies have also found learning loss to be worse for students who had struggled academically prior to the pandemic. …
Learning losses are significant in countries reporting losses through descriptive statistics. For example, adolescent girls’ literacy and numeracy scores declined by more than 6 percent in Bangladesh (Amin et al 2021). In India, the share of grade 3 children in government schools who can perform simple subtraction decreased from 24 percent in 2018 to only 16 percent in 2020 and the share who can read a grade 2 level text decreased from 19 percent in 2018 to 10 percent in 2020 (ASER 2021a). In Pakistan, the proportion of children in classes 1-5 who can read a story declined from 24 percent in 2019 to 22 percent in 2021 (ASER 2021b). There are reported learning losses in one college in Sri Lanka (Sayeejan and Nithlavarnan 2018). In Uganda, the percentage of learners rated proficient in literacy in English and numeracy in 2021 dropped by 5 percent and
13 percent from that of 2018 (NAPE 2021). In Canada, grade 2 and 3 students reading assessments declined by 4 to 5 points (Georgiou 2021). In the Republic of Korea, there was a significant decrease in scores in for medical school students (Kim et al. 2021).
The McKinsey consulting firm provides some additional background in its report, “How COVID-19 caused a global learning crisis” (April 2022). Here’s a figure showing the patterns of school shutdowns around the world.
The McKinsey authors describe the global patterns in this way:
- High-performing systems, with relatively high levels of pre-COVID-19 performance, where students may be about one to five months behind due to the pandemic (for example, North America and Europe, where students are, on average, fourmonths behind).
- Low-income prepandemic-challenged systems, with very low levels of pre-COVID-19 learning, where students may be about three to eight months behind due to the pandemic (for example, sub-Saharan Africa, where students are on average six months behind).
- Pandemic-affected middle-income systems, with moderate levels of pre-COVID-19 learning, where students may be nine to 15 months behind (for example, Latin America and South Asia, where students are, on average, 12 months behind).
These changes may not seem huge, but they are. For example, consider for a moment what it would be worth if, on average, every student had accomplished the equivalent of an additional year of academic learning upon graduation. For education reform policymakers, this kind of outcome would be like hearing angels sing. Now consider the alternative, where on average students are a half-year or a year behind.
The pandemic-related learning loss should be considered a national catastrophe, with an aggressive response that goes well beyond getting children back to in-person learning. Curriculums need adjusting, because so many students won’t be ready for what they would normally have been taught. There should be plans and funding for longer school days and summer sessions for the next several years, especially for students who were already falling behind even before the pandemic. Small-group and in-person tutoring is one method that does seem to help students catch up, and a dramatic expansion of such programs–bringing in volunteers from retirees to parents to college and high school students, who can operate both in-person and on-line, is badly needed. But my surly, negative-minded grinchy self suspects that we are just going to muddle through the pandemic learning loss issues, without dramatic changes, and that will be a costly and long-term mistake.
Interested readers may also want to check out an overview article discussing these studies and others in the Economist magazine (July 7, 2022).
