COVID-19 Academic Impact Analysis

Topics of Discussion Agenda • Summary of Findings • Academic Impact & Recovery Time • Specific Areas of Concern • IDOE Response & Action Plan

Summary of Findings: Three Major Takeaways How Much - To what extent were students impacted academically by the COVID-19 pandemic? • The academic impacts of the pandemic were substantial, ranging from moderate to very large. • 2021 ILEARN data indicates substantial academic impact across grades 3 to 8 in both ELA and Mathematics. • 2021 WIDA-ACCESS data confirms large academic impacts for most English Language Learners. • In the isolated contexts where modest to no impacts were observed, questions about the efficacy of education pre-COVID are a concern.

Summary of Findings: Three Major Takeaways In What - In which content areas were students academically impacted? • ILEARN Indiana students experienced large to very large academic impacts in math and moderate to large academic impacts in English Language Arts. • WIDA-ACCESS Most Indiana English Language Learners experienced large to very large academic impacts in English Language Proficiency.

Summary of Findings: Three Major Takeaways Who - Which demographic/academic student groups experienced academic impact? • In Grades 3 to 8, academic impacts were of comparable size within content area. • Student demographic subgroups (e.g., ethnicity) experienced comparable impact by grade and content area. • Student academic subgroups experienced different amounts of academic impact. English Language Learners experienced different amounts of impact depending upon grade. • Corporation and school impacts were highly variable with most corporations and schools experiencing substantial academic impacts.

Academic Impact & Recovery Time How Much? Size of impact is commensurate with length of recovery & academic support • Recovery is defined as getting back to where students or groups of students would be under normal learning conditions. • Recovery does not imply getting back to proficiency. Getting to proficiency would be a higher bar than recovery for pre-pandemic, non-proficient students to meet.

Academic Impact & Recovery Time How Much? Three Categories • “Large” academic impact implies academic recovery exceeding one- year and likely requiring supplemental academic support. • “Moderate” academic impact implies academic recovery of up to one- year and likely requiring some supplemental academic support. • “Modest or no” academic impact implies little deviation from what was observed historically. Caution in interpreting this group should be exercised (discussed later).

Data Sources Three sources of data used to assess academic impact: • ILEARN: ELA and Mathematics using both student academic growth and student attainment. • WIDA-ACCESS: English Language Proficiency exam using both academic growth and student attainment. • NWEA-MAP: ELA and Mathematics assessments with less recent data for Fall 2020 and Winter 2021 (Grades 1 to 10) • Currently no data for high school students or student data in other content areas.

ILEARN Academic Impact Results Academic Impact Grade ELA Mathematics 3 Moderate/Large Large 4 Moderate/Large Large 5 Moderate/Large Large 6 Moderate/Large Large 7 Moderate/Large Large 8 Moderate/Large Large

ILEARN Academic Impact Results Academic Impact Ethnicity ELA Mathematics American Indian Moderate/Large Large Asian Large Large Black Large Large Hispanic Large Large Multiracial Moderate/Large Large Native Hawaiian or PI Moderate/Large Large White Moderate/Large Large

ILEARN Academic Impact Results Academic Impact English Learner Status ELA Mathematics English Learner Large Large Not an English Moderate/Large Large Learner

ILEARN Academic Impact Results Academic Impact Special Education Status ELA Mathematics General Education Moderate/Large Large Special Education Moderate/Large Moderate

ILEARN Academic Impact Results Academic Impact Gender ELA Mathematics Female Moderate/Large Large Male Moderate/Large Large

ILEARN Academic Impact Results Academic Impact Socio-economic Status ELA Mathematics No Free/Reduced Moderate/Large Large Price Lunch Free/Reduced Price Large Large Lunch

WIDA-ACCESS Academic Impact Results Academic Impact Grade Composite 1 Large 2 Large 3 Large 4 Large 5 Moderate 6 Moderate/Large 7 Moderate 8 Moderate 9 Modest or none 10 Modest or none 11 Modest or none 12 Modest or none

Specific Areas of Concern Type 1: Issues due to COVID-19 • Academic impact in mathematics is consistently large and across all grades. • Academic impacts in ELA, though not as large as math, are considerable. • These impacts likely extending across all K-12 grades and to other content areas. • Rates of learning required for recovery are unprecedented at the scale required.

Specific Areas of Concern Type 2: Pre-existing issues identified due to COVID • There are some large groups of students demonstrating little to no academic impact. • Little to no academic impact for large groups likely brings to light ineffectual pre-pandemic performance. • High school WIDA-ACCESS growth shows no decline during the pandemic. • Lowest achievers in some grades and content areas showed very little decline during the pandemic.

IDOE Response & Action Plan Multi-Year Recovery Effort • Roles & Responsibilities of IDOE staff • Supports • Interventions • Summary of Action Plan • What currently exists? • What needs to be created?

Thank You

Appendix: Detailed Results ILEARN 2019 to 2021: Grade by Content Area Attainment - Percent Proficient ELA Mathematics Grade Percent Percent Difference Percent Percent Difference Proficient Proficient 2019 to Proficient Proficient 2019 to

Appendix: Detailed Results ILEARN: 2019 to 2021: Grade by Content Area Growth - Median SGP ELA Mathematics Grade Median Median Difference Median Median Difference SGP SGP 2019 to SGP SGP 2019 to

Appendix: Detailed Results ILEARN 2019 to 2021: Ethnicity by Content Area Attainment - Percent Proficient ELA Mathematics Ethnicity Percent Percent Difference Percent Percent Difference Proficient Proficient 2019 to Proficient Proficient 2019 to American 44% 37.3% -6.7% 42.9% 31.3% -11.6% Indian Asian 63.8% 56.7% -7.1% 67.9% 56.5% -11.4% Black 24.7% 17.4% -7.3% 22.5% 11.6% -10.9% Hispanic 35.5% 27.3% -8.2% 34.4% 22.0% -12.4% Multiracial 43.3% 36.4% -6.9% 41.7% 30.5% -11.2% Native 42.4% 32.7% -9.7% 46.1% 29.1% -17.0% Hawaiian or PI White 54.2% 47.1% -7.1% 54.6% 44.4% -12.2%

Appendix: Detailed Results ILEARN 2019 to 2021: Ethnicity by Content Area Growth - Median SGP ELA Mathematics Ethnicity Median Median Difference Median Median Difference SGP SGP 2019 to SGP SGP 2019 to American 49.5 40.0 -9.5 49.0 30.0 -19.0 Indian Asian 66.0 56.0 -13.0 68.0 44.0 -24.0 Black 42.0 30.0 -12.0 43.0 21.0 -22.0 Hispanic 51.0 37.0 -14.0 49.0 28.0 -21.0 Multiracial 47.0 37.0 -10.0 47.0 27.0 -20.0 Native 50.0 42.0 -8.0 57.0 35.0 -22.0 Hawaiian or PI White 51.0 41.0 -10.0 51.0 33.0 -18.0

Appendix: Detailed Results ILEARN 2019 to 2021: Free/Reduced Lunch Status by Content Area Attainment - Percent Proficient ELA Mathematics Socio- Economic Percent Percent Difference Percent Percent Difference Status Proficient Proficient 2019 to Proficient Proficient 2019 to Paid Lunch 61.9% 54.0% -7.9% 61.7% 50.9% -10.8% Free or 33.5% 25.9% -7.6% 33.4% 22.0% -11.4% Reduced Price Lunch

Appendix: Detailed Results ILEARN 2019 to 2021: Free/Reduced Lunch Status by Content Area Growth - Median SGP ELA Mathematics Socio- Economic Median Median Difference Median Median Difference Status SGP SGP 2019 to SGP SGP 2019 to Paid Lunch 54 45 -9 54 35 -19 Free or 46 34 -12 46 26 -20 Reduced Price Lunch

Appendix: Detailed Results ILEARN 2019 to 2021: Special Education Status by Content Area Attainment - Percent Proficient ELA Mathematics Special Education Percent Percent Difference Percent Percent Difference Status Proficient Proficient 2019 to Proficient Proficient 2019 to General 53.8% 45.6% -8.2% 53.1% 41.1% -12.0% Education Special 15.7% 12.9% -2.8% 18.6% 14.1% -4.5% Education

Appendix: Detailed Results ILEARN 2019 to 2021: Special Education Status by Content Area Growth - Median SGP ELA Mathematics Special Education Median Median Difference Median Median Difference Status SGP SGP 2019 to SGP SGP 2019 to General 52.0 41.0 -11.0 52.0 31.0 -21.0 Education Special 41.0 31.0 -10.0 36.0 29.0 -7.0 Education

Appendix: Detailed Results ILEARN 2019 to 2021: English Learner Status by Content Area Attainment - Percent Proficient ELA Mathematics English Learner Percent Percent Difference Percent Percent Difference Status Proficient Proficient 2019 to Proficient Proficient 2019 to English 16.3% 11.7% -4.6% 23.2% 14.2% -9.0% Learner Not-an- 49.9% 42.9% -7.0% 49.3% 38.9% -10.4% English Learner

Appendix: Detailed Results ILEARN 2019 to 2021: English Learner Status by Content Area Growth - Median SGP ELA Mathematics English Learner Median Median Difference Median Median Difference Status SGP SGP 2019 to SGP SGP 2019 to English 49.0 35.0 -14.0 49.0 30.0 -19.0 Learner Not-an- 50.0 40.0 -10.0 50.0 31.0 -19.0 English Learner

Appendix: Detailed Results ILEARN 2019 to 2021: Gender by Content Area Attainment - Percent Proficient ELA Mathematics Gender Percent Percent Difference Percent Percent Difference Proficient Proficient 2019 to Proficient Proficient 2019 to Female 52.9% 44.8% -8.1% 47.2% 35.0% -12.2% Male 43.2% 36.3% -6.9% 48.4% 38.8% -9.6%

Appendix: Detailed Results ILEARN 2019 to 2021: Gender by Content Area Growth - Median SGP ELA Mathematics Gender Median Median Difference Median Median Difference SGP SGP 2019 to SGP SGP 2019 to Female 52 42 -10 53 31 -22 Male 48 36 -12 47 30 -17

About this copy

COVID-19 Academic Impact Analysis — Indiana Department of Education, July 2, 2021.

This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.