Academic Impact and Recovery Conceptualization and Results Damian Betebenner — Center for Assessment Reidy Interactive Lecture Series Portsmouth, New Hampshire September 21st, 2022

COVID-19 Pandemic • The COVID-19 Pandemic lead to, arguably, the largest educational disruption in the history of the United States. • Two-and-a-half years into the pandemic and the impacts of the pandemic are still being investigated. • The pandemic impacted students academically, emotionally, physically, and financially. • In the brief overview I’ll focus on academic impact and discuss what is meant by academic impact and recovery as well as present results compiled across multiple states.

What is Academic Impact? • The pandemic and all the ensuing disruptions functioned/ functions as an “academic headwind”, impeding (in general) the academic progress of students. • Headwinds impede progress in two ways: ▪ They slow one’s rate of progress (speedometer) ▪ And by slowing one’s rate of progress they lead to less distance being travelled (odometer). • In education these two impediments manifest as: ▪ Decrease in student growth (i.e. decrease in velocity = deceleration). ▪ Decrease in student attainment.

Learning acceleration/deceleration • Summarization of assessment data (either for diagnostic or accountability purposes) emphasizes two types of data results: Status (i.e. student attainment) and Growth (i.e., student academic progress) • With the pandemic, emphasis has changed to look at academic impact • We emphasize that Impact is synonymous with decleration and Recovery is synonymous with acceleration ▪ Deceleration, by definition, is the change in (i.e. decrease) velocity (i.e. growth) ▪ Acceleration, by definition, is the change in (i.e., increase) velocity (i.e., growth ▪ To understand whether and the extent to which recovery is occurring, one must understand the magnitude of impact.

Assessment Data • To investigate impact we utilized two primary sources of data • State summative assessment data • State English Language Proficiency Data • We investigated academic impact by using two complementary analyses: • Change in attainment: Equi-percentile scale score change (all grades) • Change in growth: Baseline referenced student growth percentile change (grades with growth)

2020-2021: Academic Impact

2020-2021: Academic Impact

2020-2021: A Year of Disruptions

2020-2021: Multi-State Impact Pandemic Related Academic Impact in Indiana 10

2020-2021: Multi-State Impact Pandemic Related Academic Impact in Indiana 11

2020-2021: Multi-State Impact (WIDA-ACCESS)

Academic Recovery 2021-2022 • Education during the 2021-2022 academic year was much closer to normal for students. • With school closer to normal, an explicit goal was to try and help students recover from losses incurred during the initial phase of the pandemic. • Our primary goal with analyses in 2021-2022 is to accurately distinguish between three situations: • Deterioration: Continued decreased rates of learning • Stabilization: A return to normal rates of learning • Recovery: Acceleration to above normal rates of learning sufficient to catch students up from last year’s impacts.

2021-2022: Recovery (or not)

Recovery • We are in the middle of compiling 2022 results across multiple states looking at academic recovery • Most results thus far for states suggest a return to typical rates of learning which suggests something between stabilization bordering on recovery to stabilization bordering on deterioration. • This does not mean students are returning to levels of achievement they would have reached under typical learning scenarios • Acceleration of learning necessary to get students to recover will be difficult to produce without dramatic interventions

Cautions going forward • Maintaining scales to pre-pandemic years will increasingly become challenging for states. • Two views of returning to “normal” • Is the system returning to normal. • Are students impacted by the pandemic returning to normal. • System can return to “normal” without student returning to normal — students eventually leave the system. • Status comparison increasingly become misleading: Grade 3 students in 2022-2023 were in kindergarten in 2019-2020 —

State Leaders • Darin Nielsen, Utah Department of Education • Robert Lee, Massachusetts Department of Elementary and Secondary Education • Charity Flores, Indiana Department of Education

About this copy

Academic Impact and Recovery — Center for Assessment Reidy Interactive Lecture Series, September 2022.

This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.