COVID-19 Academic Impact in Indiana
Damian Betebenner, Adam Van Iwaarden & Nathan Dadey
Center for Assessment
Indiana TAC Meeting
October 29, 2021
Update on Academic Impact Analysis and Report
• We last met in late July and discussed initial findings.
• The July Board of Education presentation indicated overall academic impact
was moderate/large in ELA and large in mathematics.
• Both status-based analyses as well as growth-based analyses support these
findings.
• Since then we have finalized results associated with:
• ILEARN spring 2021 summative assessment
• WIDA-ACCESS 2021 summative assessment
• Today we want to share results associated with the analyses as well as some
supplementary results requested by the TAC during the previous meeting.
Pandemic Related Academic Impact in Indiana 2
How do we quantify academic impact using test data
• Historical consecutive and skip-year growth norms were created for ILEARN and ACCESS in
May, 2021 for use with 2021 ILEARN and ACCESS data.
• We can use test data in two ways to quantify the impact of this headwind:
• Best way: Quantify how much the rate of learning (i.e., growth) slowed:
• ILEARN: Use historical skip-year growth norms to quantify growth and compare to pre-COVID growth.
• ACCESS: Use historical consecutive-year growth norms and compare to pre-COVID growth
• NWEA-MAP: Provides growth data based upon historical norms but we currently don’t have historical growth to
compare to.
• Next best way: Look at how much 2021 status changed:
• ILEARN: Use 2019 status (or some adjustment thereof) as a proxy.
• ACCESS: Use 2020 status (or some adjustment thereof) as a proxy.
• Adjustments include imputation for missing data as well as propensity score matching for changes in
demographic characteristics of the population.
• Indiana participation rates were normal, with participation exceeding 95% in all grades
and content areas.
Pandemic Related Academic Impact in Indiana 3
Review of results: ILEARN
• Academic impacts in mathematics were larger than those in ELA,
though impacts were large in both content areas.
• Academic impacts were consistent in size across grades.
• Across demographic subgroups, academic impacts were fairly uniform.
Pandemic Related Academic Impact in Indiana 4
Review of results: WIDA-ACCESS
• For English Language Learners taking the WIDA-ACCESS English
• Academic impacts were large for students in the elementary grades.
• Academic impacts were moderate for students in middle school
grades.
• Academic impacts were modest to none for students in the high
school grades.
Pandemic Related Academic Impact in Indiana 5
Questions
Arizona State Board of Education Meeting, October 25th, 2021
ILEARN Attainment: 2019 to 2021 Percent Proficient
ELA Mathematics
Grade
Percent Percent Difference Percent Percent Difference
Proficient Proficient 2019 to 2021 Proficient Proficient 2019 to 2021
Pandemic Related Academic Impact in Indiana 7
ILEARN Attainment: 2019 to 2021 Mean Scale Score
Difference Gamma Effect Size: ELA
Decile SS 2019 gamma.ss.ela.g3 gamma.ss.ela.g4 gamma.ss.ela.g5 gamma.ss.ela.g6 gamma.ss.ela.g7 gamma.ss.ela.g8
Pandemic Related Academic Impact in Indiana 8
ILEARN Attainment: 2019 to 2021 Mean Scale Score
Difference Gamma Effect Size: Mathematics
2019 SS Decile gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math.
Pandemic Related Academic Impact in Indiana 9
ILEARN Growth: 2019 to 2021 Student Growth
ELA Mathematics
Grade
Median SGP Median SGP Difference 2019 Median SGP Median SGP Difference 2019
2019 2021 to 2021 2019 2021 to 2021
Pandemic Related Academic Impact in Indiana 10
ILEARN Growth: 2019 to 2021 SGP Difference Gamma Effect
Size: ELA
Decile SS 2019 gamma.sgp.ela.g5 gamma.sgp.ela.g6 gamma.sgp.ela.g7 gamma.sgp.ela.g8
Pandemic Related Academic Impact in Indiana 11
ILEARN Growth: 2019 to 2021 SGP Difference Gamma Effect
Size: Mathematics
Decile SS 2019 gamma.sgp.math.g5 gamma.sgp.math.g6 gamma.sgp.math.g7 gamma.sgp.math.g8
Pandemic Related Academic Impact in Indiana 12
ILEARN Attainment: 2019 to 2021 Percent Proficient by
Ethnicity
ELA Mathematics
Ethnicity
Percent Percent Difference 2019 Percent Percent Difference 2019
Proficient 2019 Proficient to 2021 Proficient 2019 Proficient 2021 to 2021
American Indian 44% 37.3% -6.7% 42.9% 31.3% -11.6%
Asian 63.8% 56.7% -7.1% 67.9% 56.5% -11.4%
Black 24.7% 17.4% -7.3% 22.5% 11.6% -10.9%
Hispanic 35.5% 27.3% -8.2% 34.4% 22.0% -12.4%
Multiracial 43.3% 36.4% -6.9% 41.7% 30.5% -11.2%
Native Hawaiian 42.4% 32.7% -9.7% 46.1% 29.1% -17.0%
or PI
White 54.2% 47.1% -7.1% 54.6% 44.4% -12.2%
Pandemic Related Academic Impact in Indiana 13
ILEARN Growth: 2019 to 2021 Student Growth by Ethnicity
ELA Mathematics
Ethnicity Median SGP Median Difference 2019 Median Median Difference 2019
2019 SGP to 2021 SGP SGP to 2021
American Indian 49.5 40.0 -9.5 49.0 30.0 -19.0
Asian 66.0 56.0 -13.0 68.0 44.0 -24.0
Black 42.0 30.0 -12.0 43.0 21.0 -22.0
Hispanic 51.0 37.0 -14.0 49.0 28.0 -21.0
Multiracial 47.0 37.0 -10.0 47.0 27.0 -20.0
Native Hawaiian 50.0 42.0 -8.0 57.0 35.0 -22.0
or PI
White 51.0 41.0 -10.0 51.0 33.0 -18.0
Pandemic Related Academic Impact in Indiana 14
ILEARN Attainment: 2019 to 2021 Student Attainment by
Special Education Status
ELA Mathematics
Special
Education Median Median Difference Median Median Difference
Status SGP SGP 2019 to SGP SGP 2019 to
General 52.0 41.0 -11.0 52.0 31.0 -21.0
Education
Special 41.0 31.0 -10.0 36.0 29.0 -7.0
Education
Pandemic Related Academic Impact in Indiana 15
ILEARN Growth: 2019 to 2021 Student Growth by Special
Education Status
ELA Mathematics
Special
Education Median Median Difference Median Median Difference
Status SGP SGP 2019 to SGP SGP 2019 to
General 52.0 41.0 -11.0 52.0 31.0 -21.0
Education
Special 41.0 31.0 -10.0 36.0 29.0 -7.0
Education
Pandemic Related Academic Impact in Indiana 16
About this copy
COVID-19 Academic Impact in Indiana — Indiana Department of Education, January 27, 2022.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.