COVID-19 Academic Impact in Indiana
Damian Betebenner, Adam Van Iwaarden & Nathan Dadey
Center for Assessment
Indiana TAC Meeting
July 26, 2021
How do we quantify academic impact using test data
• The pandemic was like a “headwind” that (presumably) slowed down the learning
of students due to a variety of factors.
• We can use test data in two ways to quantify the impact of this headwind:
• Best way: Quantify how much the rate of learning (i.e., growth) slowed:
• ILEARN: Use historical skip-year growth norms to quantify growth and compare to pre-COVID growth.
• ACCESS: Use historical consecutive-year growth norms and compare to pre-COVID growth
• NWEA-MAP: Provides growth data based upon historical norms but we currently don’t have historical
growth to compare to.
• Next best way: Look at how much 2021 status changed:
• ILEARN: Use 2019 status (or some adjustment thereof) as a proxy.
• ACCESS: Use 2020 status (or some adjustment thereof) as a proxy.
• Adjustments include (potentially) imputation for missing data as well as propensity score matching for
• NOTE: When providing 2021 data, users will (correctly) want to compare their
data to 2019 (or 2020 with ACCESS). The task is to make sure that such a
comparison is an apples-to-apples comparison.
Pandemic Related Academic Impact in Indiana 2
How do we quantify academic impact using test data
• Historical consecutive and skip-year growth norms were created for ILEARN and
ACCESS in May, 2021 for use with 2021 ILEARN and ACCESS data.
• We can use test data in two ways to quantify the impact of this headwind:
• Best way: Quantify how much the rate of learning (i.e., growth) slowed:
• ILEARN: Use historical skip-year growth norms to quantify growth and compare to pre-COVID growth.
• ACCESS: Use historical consecutive-year growth norms and compare to pre-COVID growth
• NWEA-MAP: Provides growth data based upon historical norms but we currently don’t have historical
growth to compare to.
• Next best way: Look at how much 2021 status changed:
• ILEARN: Use 2019 status (or some adjustment thereof) as a proxy.
• ACCESS: Use 2020 status (or some adjustment thereof) as a proxy.
• Adjustments include (potentially) imputation for missing data as well as propensity score matching for
• NOTE: When providing 2021 data, users will (correctly) want to compare their
data to 2019 (or 2020 with ACCESS). The task is to make sure that such a
comparison is an apples-to-apples comparison.
Pandemic Related Academic Impact in Indiana 3
Results
• As the Board of Education presentation indicated, overall academic impact
was moderate/large in ELA and large in mathematics.
• Both status-based analyses as well as growth-based analyses support these
findings.
• The following tables illustrate numerically what these impacts were.
• For rough comparison, the effect size impacts we are seeing for
achievement are roughly 1 to 5 times as large as what was reported for
Hurricane Katrina in Louisiana depending upon grade and content area.
(Effect sizes of -0.06 and -0.17 are associated with Katrina).
• Groups experiencing minimal impact should be scrutinized closely with
regard to pre-pandemic level of educational efficacy.
Pandemic Related Academic Impact in Indiana 4
ILEARN Attainment: 2019 to 2021 Percent Proficient
ELA Mathematics
Grade
Percent Percent Difference Percent Percent Difference
Proficient Proficient 2019 to 2021 Proficient Proficient 2019 to 2021
Pandemic Related Academic Impact in Indiana 5
ILEARN Attainment: 2019 to 2021 Mean Scale Score
Difference Gamma Effect Size: ELA
Decile SS 2019 gamma.ss.ela.g3 gamma.ss.ela.g4 gamma.ss.ela.g5 gamma.ss.ela.g6 gamma.ss.ela.g7 gamma.ss.ela.g8
Pandemic Related Academic Impact in Indiana 6
ILEARN Attainment: 2019 to 2021 Mean Scale Score
Difference Gamma Effect Size: Mathematics
2019 SS Decile gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math. gamma.ss.math.
Pandemic Related Academic Impact in Indiana 7
ILEARN Growth: 2019 to 2021 Student Growth
ELA Mathematics
Grade
Median SGP Median SGP Difference 2019 Median SGP Median SGP Difference 2019
2019 2021 to 2021 2019 2021 to 2021
Pandemic Related Academic Impact in Indiana 8
ILEARN Growth: 2019 to 2021 SGP Difference Gamma
Effect Size: ELA
Decile SS 2019 gamma.sgp.ela.g5 gamma.sgp.ela.g6 gamma.sgp.ela.g7 gamma.sgp.ela.g8
Pandemic Related Academic Impact in Indiana 9
ILEARN Growth: 2019 to 2021 SGP Difference Gamma
Effect Size: Mathematics
Decile SS 2019 gamma.sgp.math.g5 gamma.sgp.math.g6 gamma.sgp.math.g7 gamma.sgp.math.g8
Pandemic Related Academic Impact in Indiana 10
ILEARN Attainment: 2019 to 2021 Percent Proficient by
Ethnicity
ELA Mathematics
Ethnicity
Percent Percent Difference 2019 Percent Percent Difference 2019
Proficient 2019 Proficient to 2021 Proficient 2019 Proficient 2021 to 2021
American Indian 44% 37.3% -6.7% 42.9% 31.3% -11.6%
Asian 63.8% 56.7% -7.1% 67.9% 56.5% -11.4%
Black 24.7% 17.4% -7.3% 22.5% 11.6% -10.9%
Hispanic 35.5% 27.3% -8.2% 34.4% 22.0% -12.4%
Multiracial 43.3% 36.4% -6.9% 41.7% 30.5% -11.2%
Native Hawaiian 42.4% 32.7% -9.7% 46.1% 29.1% -17.0%
or PI
White 54.2% 47.1% -7.1% 54.6% 44.4% -12.2%
Pandemic Related Academic Impact in Indiana 11
ILEARN Growth: 2019 to 2021 Student Growth by
Ethnicity
ELA Mathematics
Ethnicity
Median SGP Median Difference 2019 Median Median Difference 2019
2019 SGP to 2021 SGP SGP to 2021
American Indian 49.5 40.0 -9.5 49.0 30.0 -19.0
Asian 66.0 56.0 -13.0 68.0 44.0 -24.0
Black 42.0 30.0 -12.0 43.0 21.0 -22.0
Hispanic 51.0 37.0 -14.0 49.0 28.0 -21.0
Multiracial 47.0 37.0 -10.0 47.0 27.0 -20.0
Native Hawaiian 50.0 42.0 -8.0 57.0 35.0 -22.0
or PI
White 51.0 41.0 -10.0 51.0 33.0 -18.0
Pandemic Related Academic Impact in Indiana 12
ILEARN Attainment: 2019 to 2021 Student Attainment
by Special Education Status
ELA Mathematics
Special
Education Median Median Difference Median Median Difference
Status SGP SGP 2019 to SGP SGP 2019 to
General 52.0 41.0 -11.0 52.0 31.0 -21.0
Education
Special 41.0 31.0 -10.0 36.0 29.0 -7.0
Education
Pandemic Related Academic Impact in Indiana 13
ILEARN Growth: 2019 to 2021 Student Growth by
Special Education Status
ELA Mathematics
Special
Education Median Median Difference Median Median Difference
Status SGP SGP 2019 to SGP SGP 2019 to
General 52.0 41.0 -11.0 52.0 31.0 -21.0
Education
Special 41.0 31.0 -10.0 36.0 29.0 -7.0
Education
Pandemic Related Academic Impact in Indiana 14
Performance by initial level of achievement
• A consistent finding put forward from interim assessment reports and
picked up by educational pundits is that it is the most ”vulnerable”
who are experiencing the biggest impacts.
• Based solely upon demographic subgroup analysis, that finding was
not supported.
• To dig deeper we regularly look at SGP distributions by prior score
decile. For cohort referenced SGPs we expect a uniform distribution.
For baseline referenced SGPs, deviations from uniform indicate
impact.
Pandemic Related Academic Impact in Indiana 15
Student Growth Percentile Goodness−of−Fit Descriptives
2021 ELA SGP, Grade 5 (N = 75,778)
SGP Deciles by Prior Achievement Level (ELA)
Above Proficiency (18.2%)
Prior Achievement Level
At Proficiency (28.5%)
Approaching Proficiency (23.4%)
Below Proficiency (29.9%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [5110,5221) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [5737,5750) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.0 88.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[5060,5362) 11.12 13.67 12.00 12.39 10.78 9.08 8.48 7.77 7.60 7.10 (9.9%) 100
2nd/[5362,5393) 16.06 14.96 13.42 10.41 9.38 8.17 7.75 6.58 7.01 6.26 (10.1%)
Prior Scale Score Decile/Range *
3rd/[5393,5416) 15.99 14.24 12.45 9.69 10.05 8.24 7.77 7.44 7.18 6.96 (9.9%)
Empirical SGP Distribution
4th/[5416,5436) 15.99 14.71 11.54 9.63 10.12 7.37 8.64 7.46 6.97 7.57 (10%)
5th/[5436,5454) 16.89 13.91 11.52 9.91 9.02 8.35 8.02 7.58 6.95 7.86 (10%)
6th/[5454,5471) 16.85 12.18 11.05 10.34 9.46 9.22 8.40 7.31 7.39 7.79 (9.6%)
7th/[5471,5489) 15.56 12.12 12.04 10.49 9.13 9.58 8.38 7.45 7.39 7.85 (9.9%)
8th/[5489,5510) 14.82 12.53 11.52 10.18 9.42 9.24 8.51 8.34 7.53 7.93 (10.2%)
9th/[5510,5538) 14.82 12.41 11.19 9.97 9.86 9.08 9.03 8.14 8.00 7.50 (10.2%)
10th/[5538,5760] 13.24 11.11 11.08 10.51 9.28 8.81 10.09 9.00 9.07 7.80 (10.1%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 ELA SGP, Grade 6 (N = 77,131)
SGP Deciles by Prior Achievement Level (ELA)
Above Proficiency (20.1%)
Prior Achievement Level
At Proficiency (26%)
Approaching Proficiency (24.3%)
Below Proficiency (29.6%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [5130,5233) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [5752,5769) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 96.2 N = 26
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[5090,5384) 14.01 14.46 13.33 11.29 8.87 9.15 8.41 6.79 6.79 6.90 (9.9%) 100
2nd/[5384,5419) 16.76 15.22 11.91 10.33 9.81 8.48 7.45 6.87 6.62 6.56 (10.1%)
Prior Scale Score Decile/Range *
3rd/[5419,5444) 15.39 13.93 11.98 10.91 9.85 8.74 8.16 7.19 6.98 6.88 (9.6%)
Empirical SGP Distribution
4th/[5444,5466) 15.97 13.92 10.89 11.27 10.20 9.32 6.98 7.42 7.09 6.94 (10.2%)
5th/[5466,5485) 16.21 12.64 11.03 11.58 9.70 8.74 7.97 7.39 7.23 7.52 (9.9%)
6th/[5485,5504) 15.53 13.13 10.89 11.28 8.82 8.15 8.30 8.03 7.83 8.04 (10.2%)
7th/[5504,5524) 16.16 13.00 11.34 10.21 9.18 8.35 7.55 8.10 7.67 8.43 (10%)
8th/[5524,5547) 15.84 12.81 11.62 9.27 9.04 9.10 8.37 8.16 7.27 8.53 (10.1%)
9th/[5547,5578) 14.89 12.10 10.79 10.21 8.82 9.16 8.38 8.17 8.29 9.19 (10.1%)
10th/[5578,5810] 15.05 11.24 9.97 8.58 9.14 8.26 8.28 9.03 9.54 10.93 (10%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 ELA SGP, Grade 7 (N = 78,682)
SGP Deciles by Prior Achievement Level (ELA)
Above Proficiency (15.6%)
Prior Achievement Level
At Proficiency (32.4%)
Approaching Proficiency (24.1%)
Below Proficiency (27.8%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [5130,5227) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [5801,5824) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 26
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[5110,5411) 12.47 13.27 12.60 11.02 10.52 9.77 8.70 8.09 7.14 6.42 (9.9%) 100
2nd/[5411,5450) 14.29 12.91 12.81 10.84 9.98 9.26 8.94 6.99 7.21 6.77 (10%)
Prior Scale Score Decile/Range *
3rd/[5450,5477) 15.92 13.14 12.30 10.34 9.90 8.92 8.90 7.38 6.62 6.58 (10%)
Empirical SGP Distribution
4th/[5477,5499) 16.64 14.62 12.23 10.67 10.00 8.33 7.98 7.01 6.73 5.78 (9.9%)
5th/[5499,5520) 16.58 13.78 12.10 10.11 9.50 8.65 8.49 7.75 6.48 6.57 (10.2%)
6th/[5520,5539) 15.96 13.07 11.53 9.76 9.89 9.55 8.90 7.69 7.12 6.52 (9.6%)
7th/[5539,5560) 15.58 11.96 11.20 10.01 10.06 9.22 8.76 8.52 7.98 6.72 (10.3%)
8th/[5560,5583) 15.58 12.76 11.30 10.48 9.85 8.92 8.49 8.10 7.70 6.82 (10.1%)
9th/[5583,5613) 15.75 12.45 11.09 10.15 9.46 8.96 9.42 8.04 7.55 7.12 (9.9%)
10th/[5613,5850] 14.14 11.46 10.74 9.10 10.06 9.26 9.38 9.21 9.16 7.48 (10.2%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 ELA SGP, Grade 8 (N = 78,305)
SGP Deciles by Prior Achievement Level (ELA)
Above Proficiency (18.1%)
Prior Achievement Level
At Proficiency (30.5%)
Approaching Proficiency (25.8%)
Below Proficiency (25.7%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [5150,5260) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [5824,5844) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[5130,5442) 14.67 15.13 12.46 12.02 9.63 8.73 7.85 6.93 6.24 6.34 (9.9%) 100
2nd/[5442,5477) 17.74 15.00 12.56 11.29 9.59 7.89 7.59 6.87 5.49 5.98 (10%)
Prior Scale Score Decile/Range *
3rd/[5477,5501) 19.03 14.76 11.92 10.23 9.19 8.28 7.01 6.72 6.19 6.67 (9.9%)
Empirical SGP Distribution
4th/[5501,5522) 17.99 14.17 11.58 10.62 8.81 8.54 7.60 7.18 6.32 7.20 (10.2%)
5th/[5522,5541) 17.02 13.61 11.54 10.46 8.87 8.54 8.18 7.88 7.01 6.88 (9.9%)
6th/[5541,5559) 15.27 12.65 10.96 9.76 8.94 9.07 8.70 9.02 7.90 7.73 (9.7%)
7th/[5559,5578) 14.25 11.32 10.76 9.63 9.69 8.89 9.35 8.49 8.89 8.74 (10.1%)
8th/[5578,5599) 13.24 11.09 9.51 9.41 9.29 9.47 9.81 8.88 9.80 9.51 (10.2%)
9th/[5599,5626) 11.69 10.81 9.06 9.82 9.51 9.46 9.56 9.49 9.59 11.01 (10.2%)
10th/[5626,5865] 10.30 9.61 9.51 9.54 9.70 9.31 9.86 9.01 10.39 12.77 (10%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 Mathematics SGP, Grade 5 (N = 75,724)
SGP Deciles by Prior Achievement Level (Mathematics)
Above Proficiency (26%)
Prior Achievement Level
At Proficiency (33.3%)
Approaching Proficiency (18.7%)
Below Proficiency (22%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [6110,6177) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [6740,6758) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 40.0 12.0 48.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[6104,6342) 22.94 19.25 13.42 10.84 9.08 7.56 6.65 4.60 3.37 2.29 (9.9%) 100
2nd/[6342,6376) 20.83 16.25 13.15 12.72 10.02 8.34 6.45 5.01 4.31 2.91 (9.8%)
Prior Scale Score Decile/Range *
3rd/[6376,6402) 20.94 17.28 12.60 12.16 9.02 7.68 6.64 5.22 4.95 3.50 (10.2%)
Empirical SGP Distribution
4th/[6402,6423) 20.99 15.98 12.56 11.22 8.74 8.52 7.43 5.94 5.05 3.56 (9.8%)
5th/[6423,6442) 22.13 15.21 11.82 10.93 9.00 8.25 7.37 6.17 5.02 4.12 (10%)
6th/[6442,6460) 21.14 13.16 11.57 11.42 9.60 8.46 8.18 6.29 5.75 4.42 (9.7%)
7th/[6460,6479) 19.50 13.91 12.40 9.85 10.73 9.38 7.38 6.91 5.47 4.47 (10.1%)
8th/[6479,6501) 18.95 14.78 12.03 10.68 10.30 9.21 8.18 6.18 5.82 3.85 (10.2%)
9th/[6501,6531) 20.17 14.81 12.68 10.81 9.76 8.77 7.90 6.93 5.06 3.11 (10%)
10th/[6531,6730] 17.07 14.99 13.68 12.26 10.61 9.65 8.39 6.21 4.90 2.24 (10.2%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 Mathematics SGP, Grade 6 (N = 77,079)
SGP Deciles by Prior Achievement Level (Mathematics)
Above Proficiency (21.1%)
Prior Achievement Level
At Proficiency (33.3%)
Approaching Proficiency (20.8%)
Below Proficiency (24.7%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [6110,6113) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range)
Highest OSS: [6806,6825) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 7.7 3.8 88.5 N = 26
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[6100,6380) 12.23 15.07 14.75 14.09 9.94 9.55 8.56 6.77 5.07 3.98 (9.9%) 100
2nd/[6380,6416) 17.63 16.94 15.18 12.51 9.40 8.57 6.40 5.58 4.48 3.31 (10%)
Prior Scale Score Decile/Range *
3rd/[6416,6441) 21.84 17.66 13.49 10.51 8.42 7.86 6.90 5.67 4.48 3.19 (9.6%)
Empirical SGP Distribution
4th/[6441,6463) 24.24 17.35 13.62 11.01 9.06 7.02 6.06 4.65 3.49 3.49 (10.3%)
5th/[6463,6482) 25.61 17.40 12.49 10.75 7.97 7.37 5.81 4.75 4.47 3.37 (9.9%)
6th/[6482,6501) 26.30 17.47 12.21 10.45 8.10 6.93 6.06 5.17 4.27 3.05 (10.2%)
7th/[6501,6520) 25.24 17.07 12.50 9.64 8.21 7.00 6.11 5.60 4.46 4.18 (9.7%)
8th/[6520,6543) 24.72 16.89 11.62 9.85 8.03 7.66 7.02 5.47 4.76 3.99 (10%)
9th/[6543,6574) 23.96 16.07 11.06 9.55 7.89 7.24 7.20 5.87 5.81 5.34 (10.3%)
10th/[6574,6800] 21.58 13.95 10.55 9.16 9.32 7.79 7.96 6.43 6.34 6.92 (10%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 Mathematics SGP, Grade 7 (N = 78,561)
SGP Deciles by Prior Achievement Level (Mathematics)
Above Proficiency (22.8%)
Prior Achievement Level
At Proficiency (25.8%)
Approaching Proficiency (25.5%)
Below Proficiency (26%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [6120,6203) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 23
Extremes (Range)
Highest OSS: [6812,6829) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 8.0 8.0 12.0 72.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[6110,6395) 9.29 13.78 15.12 13.22 11.99 10.15 9.64 7.85 5.65 3.31 (9.9%) 100
2nd/[6395,6435) 12.44 14.60 14.95 12.81 10.66 9.40 8.89 6.75 5.96 3.53 (10%)
Prior Scale Score Decile/Range *
3rd/[6435,6463) 14.63 14.82 14.48 11.70 10.73 10.23 9.00 6.42 4.94 3.04 (9.9%)
Empirical SGP Distribution
4th/[6463,6486) 16.51 16.90 13.98 11.88 11.31 9.31 7.36 5.42 4.44 2.88 (10%)
5th/[6486,6506) 19.04 17.18 14.18 12.21 10.22 8.60 6.57 5.23 3.81 2.96 (9.7%)
6th/[6506,6527) 19.40 17.79 15.42 11.56 9.30 7.74 6.08 5.30 4.20 3.21 (10.3%)
7th/[6527,6548) 21.51 17.83 13.83 10.33 9.54 8.16 6.03 5.20 4.34 3.24 (9.8%)
8th/[6548,6573) 22.28 16.67 13.26 10.57 8.80 8.24 6.92 5.62 4.43 3.22 (10.2%)
9th/[6573,6607) 21.80 14.90 12.05 10.09 9.51 8.49 7.46 5.88 5.34 4.48 (9.9%)
10th/[6607,6850] 19.18 12.43 10.51 9.27 9.23 8.23 8.14 7.65 7.63 7.73 (10.2%)
*Prior score deciles can be uneven depending upon the prior score distribution
Student Growth Percentile Goodness−of−Fit Descriptives
2021 Mathematics SGP, Grade 8 (N = 78,228)
SGP Deciles by Prior Achievement Level (Mathematics)
Above Proficiency (21%)
Prior Achievement Level
At Proficiency (26.2%)
Approaching Proficiency (24.1%)
Below Proficiency (28.7%)
Median Student Growth Percentile
Ceiling / Floor Effects Test
Student Growth Percentile Range
Current Observed Score
5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [6120,6225) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 24
Extremes (Range)
Highest OSS: [6858,6883) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.0 12.0 16.0 60.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[6110,6411) 9.10 14.03 13.22 13.48 12.25 11.75 9.57 7.00 5.50 4.10 (10%) 100
2nd/[6411,6459) 12.49 15.34 14.52 12.14 10.73 10.75 7.92 6.30 4.93 4.88 (9.9%)
Prior Scale Score Decile/Range *
3rd/[6459,6491) 15.17 16.43 13.68 12.15 10.97 8.76 7.18 5.40 5.26 4.99 (9.9%)
Empirical SGP Distribution
4th/[6491,6516) 17.20 17.36 14.62 11.44 9.30 8.26 6.70 5.33 5.13 4.67 (9.9%)
5th/[6516,6539) 20.81 17.98 12.39 10.35 8.69 7.51 6.62 6.23 5.36 4.06 (10.3%)
6th/[6539,6559) 23.53 17.16 11.69 9.43 8.67 7.32 6.18 6.50 5.23 4.29 (9.6%)
7th/[6559,6581) 23.98 15.08 12.11 10.26 8.64 7.92 6.47 5.71 5.14 4.71 (10.1%)
8th/[6581,6607) 23.14 15.20 11.07 10.45 8.87 7.82 6.92 6.07 5.62 4.85 (10.1%)
9th/[6607,6640) 23.17 14.58 10.48 9.63 8.69 8.00 7.13 6.77 6.11 5.44 (10.2%)
10th/[6640,6870] 17.51 12.79 10.53 10.32 9.49 9.14 8.75 8.33 7.08 6.06 (10%)
*Prior score deciles can be uneven depending upon the prior score distribution
School level impact
• We are currently in the process of finalizing school level impact
analyses and want to share some of the initial looks at these results.
• To calculate impact at the school level we calculate for each school:
Unadjusted Impact = Median SGP2019- Median SGP2021
• We adjust for regression to the mean for this difference score by
using pre-pandemic (2017 to 2019) school level median SGP scores.
• The following bubble plots illustrate academic impact for Indiana
Schools.
Pandemic Related Academic Impact in Indiana 16
School Level COVID Academic Impact
ILEARN: ELA By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 10.9% Higher 2019 Growth
School Size
50 Students
100 Students
16.5% 15% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: ELA By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 23.7% Higher 2019 Growth
School Size
50 Students
100 Students
32.9% 29.5% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: ELA By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 37.4% Higher 2019 Growth
School Size
50 Students
100 Students
33.5% 36.1% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: ELA By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 22.4% Higher 2019 Growth
School Size
50 Students
100 Students
13.5% 16.8% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: ELA By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 5.5% Higher 2019 Growth
School Size
50 Students
100 Students
3.5% 2.7% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: Mathematics By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 5.3% Higher 2019 Growth
School Size
50 Students
100 Students
11.1% 6.8% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: Mathematics By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 13.5% Higher 2019 Growth
School Size
50 Students
100 Students
16.9% 18.3% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: Mathematics By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 23.6% Higher 2019 Growth
School Size
50 Students
100 Students
25.1% 28.3% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: Mathematics By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 31.1% Higher 2019 Growth
School Size
50 Students
100 Students
32.4% 26% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
School Level COVID Academic Impact
ILEARN: Mathematics By 2019 Growth and Achievement
Higher 2019 Achievement Higher 2019 Achievement
Lower 2019 Growth 26.4% Higher 2019 Growth
School Size
50 Students
100 Students
14.5% 20.6% 250 Students
500 Students
2019 Percent Proficient
Adjusted
50 COVID Impact
Improvement (> 5)
40 Modest to None (−5 to 5)
Moderate (−15 to −5)
Large (−25 to −15)
Achievement
Severe (< −25)
Lower 2019 Growth Higher 2019 Growth
Lower 2019 Achievement Lower 2019 Achievement
Growth 2019 Median SGP
About this copy
COVID-19 Academic Impact in Indiana — Indiana Department of Education, July 26, 2021.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.