Overview of COVID-19 Academic Impact Study

Introduction Components • Provide context of COVID-19 in Indiana & nationally • Identify state & federal expectations • Acknowledge “pre-existing” conditions • Indicate purpose/s of study • Understand academic impact of COVID-19 to provide intentional supports & resources • NOT accountability

Research Questions • Question #1: Who are the students in Indiana? • Overall description of student population. • Overall description of students for whom assessment data has been collected. Who do we not have data? • Question #2: To what extent did COVID-19 disruptions impact student academic performance & learning outcomes? • Overall academic impact • Disparities in academic impact • Content area academic impact (E/La & math; English language proficiency) • Question #3: What is the anticipated academic recovery time based on the experienced COVID-19 disruptions?

Question #1: discuss Adam’s work to ensure “apples to apples” comparisons in study
Question #2: reinforce that this question will address state-level; publishing information on corporation level has strong potential to turn into Accountability

Research Questions Areas of Consideration • Student racial/ethnic subgroup • Special education status • English learner status • Poverty status • Community type (urban, suburban, rural) • School type (traditional public, charter, nonpublic/choice) • Mode of instruction • Student attendance

Are there areas not included here that leadership would like to consider?

Data Sources Components • Assessment data • ILEARN for E/La & math (grades 3-8) • WIDA ACCESS 2.0 for English learners (K-12) • NWEA MAP interim for E/La & math (grades 1-10) • Population data • Ed-Fi Attendance & Student Demographic data • Poverty index data • Technology data • Annual technology plan

Data Sources Notes on NWEA MAP data • Received data from Fall 2020-2021 and Winter 2020-2021 administration windows • Data consisted primarily of language arts and mathematics data for Grades • Statewide participation approached/exceeded 40% in all grades except 9 & 10 Notes on High School data • Limited interim assessment data makes it difficult to conduct sufficient analysis of grades 9-12 • Changes in ISTEP+ 10 administration introduce problems for comparison of status & growth.

NOTE: data limitations at high school level

NWEA Statewide Participation Grade E/La Math E/La Math Level Fall 2020 Fall 2020 Winter 2020 Winter 2020 State 38.38% 38.03% 37.91% 37.58%

Methodology Components • Consultation with Indiana Technical Advisory Committee • Measurement of academic impact • Status: comparison of proficiency level from year to year • Growth: comparison of growth (cohort referenced and baseline) from year to year • Measurement of recovery time • No Impact • Minor Impact • Moderate Impact • Significant Impact

Methodology Measurement of Academic Impact – NWEA Interim Assessment • Consider status & growth comparisons of the following: • Fall 19/20 to Fall 20/21 • Fall 20/21 to Winter 20/21 • Winter 19/20 to Winter 20/21 • Fall 20/21 to Spring 20/21 • Winter 20/21 to Spring 20/21 • Spring 18/19 to Spring 20/21

Methodology Measurement of Academic Impact – Summative Assessment • Consider status & growth comparisons of the following: • Spring 2019 to Spring 2021 Measurement of Academic Impact – WIDA ACCESS 2.0 Assessment • Consider status & growth comparisons of the following: • Winter 2020 to Winter 2021

Findings Components • Enrollment, attendance, and mode of instruction, broken out by sub- topics • Experienced learning loss, broken out by sub-topics • Disparities in learning loss, broken out by sub-topics • Content area learning loss, broken out by sub-topics • Assigned recovery time “impact” statuses, broken out by sub-topics

Conclusion & Recommendations Components • Summary of COVID-19 academic impact • Policy & support interventions • Theory of action—identify purpose/priorities/goals for accelerating learning • Divide into short-term/immediate need & long-term strategy based on establishment of “baseline” and current “impact” category • Ensure equity considerations of “pre-existing conditions” are considered in any strategic planning

For policy & support interventions: cross-departmental collaboration to develop and create appropriate response to ensure support is provided to school corporations
While we will have the ability to look at data at the corporation level, we strongly recommend that such information be used for internal/deliberative purposes only—we do not want this report to be used as a replacement for accountability in this “off” year

Timeline Components • May/June 2021 • Receive summative assessment & WIDA ACCESS 2.0 assessment results in May/June • Conduct status & growth analyses on summative and EL proficiency assessment results • June/July 2021 • Finalize report & share findings with DOE leadership & relevant DOE teams to align findings with DOE supports and guidance • August 2021 • Share report with policymakers/stakeholders • Publish report & present findings/DOE’s strategic response

Communication Questions • Does this research plan align with the goals and priorities held by leadership for the “learning loss” study? • Are there any areas of concern with this research plan? Are there any areas that are missing from this research plan? • How does this study fit into the overall mission of leadership? • Which divisions/directors need to be included in review of findings & development of strategic response plan

About this copy

Overview of COVID-19 Academic Impact Study — Indiana Department of Education, May 1, 2021.

This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.

• Talk page
• Download PDF
• Original file
• All talks