Illinois SGP Analyses for 2019
Damian W Betebenner
Adam Van Iwaarden
Chicago, Illinois
December 12th, 2019
Background
• 2019 Marked the 5th year of New Meridean (formerly
PARCC) test administration allowing student growth
calculation using two prior scores.
• The growth model designated to be used for
calculation was student growth percentiles (SGP).
• The Center for Assessment was contracted to perform
SGP analyses for 2019.
– Fall 2018 data analysis conducted in February 2019.
– Spring 2019 data analysis conducted in June - August 2019.
Illinois SGP Analyses for 2019 2
Background
• Analytic assumptions were originally determined in
2016 and have been the same in all following years.
– SGPs are calculated using theta values at both
the consortium and state level.
– SGP norm groups (grade/content area
progressions) with min 1,000 students
– No progressions with skipped years/grades
– No course repeat (test retake) norm groups
[Illinois SGP Analyses for 2019] 3
Background
SGPs are calculated based upon test results for
students taking the same tests at the same time.
• Grade level ELA tests simpler to model than test taking
patterns associated with Grade level Mathematics combined
with Algebra I, II, Geometry, Integrated Math I, II, and III.
• Norm groups were originally proposed in 2016. All potential
analyses were attempted, with cohorts with fewer than 1,000
students identified with a note in place of an SGP estimate.
[Illinois SGP Analyses for 2019] 4
Background
ELA Norm Groups:
• Spring 2017/Spring 2018 to Spring 2019:
– Grade 3 to 4, 3/4 to 5, 4/5 to 6, 5/6 to 7, 6/7 to 8
Mathematics Norm Groups:
• Spring 2017/Spring 2018 to Spring 2019:
– Grade 3 to 4, 3/4 to 5,4/ 5 to 6,5/ 6 to 7 and 6/7 to 8
[Illinois SGP Analyses for 2019] 5
Background
• SGPs, and AGP (adequate growth percentiles) are provided
to IL
• Scaled Score CSEMs provided by Pearson were incorporated
for calculation of SGP SEMs and measurement error
corrected (SIMEX adjusted) SGPs.
• Ranked SIMEX adjusted SGPs were provided to PARCC
members beginning in Spring 2017.
• Results are provided using both state and consortium level
norms.
[Illinois SGP Analyses for 2019] 6
Results
• SGP analyses successfully completed mid-July
• All state (and consortium) level analyses required
approximately 36 hours to complete.
[Illinois SGP Analyses for 2019] 7
Results
• SGPs are uniformly distributed and uncorrelated with prior
achievement: All students, regardless of starting point can
demonstrate high/low growth.
• Median SGP for consortium and state norm groups are 50.
• Consortium-based median SGPs (by state, grade and content
area) ranged from 35 to 60.
• Tests of SGP distribution uniformity and the existence of ceiling
effects are performed for each SGP norm-group analysis run.
• Goodness of fit results were generally quite good (i.e.,
uniform distribution of SGPs by prior achievement) with no
indications of ceiling/floor effects.
[Illinois SGP Analyses for 2019] 8
Results
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2018−2019.2 ELA SS SGP, Grade 4 (N = 135,033) 2018−2019.2 ELA SS SGP, Grade 5 (N = 139,260)
SGP Deciles by Prior Achievement Level (ELA SS) SGP Deciles by Prior Achievement Level (ELA SS)
Level 5 (3.4%)
Level 5 (7.6%)
Prior Achievement Level Prior Achievement Level
Level 4 (34%)
Level 4 (31.1%)
Level 3 (23.9%)
Level 3 (27.4%)
Level 2 (17.8%)
Level 2 (20%)
Level 1 (20.9%)
Level 1 (13.9%)
Median Student Growth Percentile Median Student Growth Percentile
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score 5 9
Current Observed Score 19 29 39 49 59 69 79 89 94 99
to to to to to to to to to to to to 5 9 to to to to to to to to to to
1 5 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [ 650 ] 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 996
Extremes (Range)
Lowest OSS: [ 650 ] 94.1 5.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 1054
Highest OSS: [ 850 ] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 18.1 81.9 N = 155 Extremes (Range) Highest OSS: [ 850 ] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.8 7.0 80.2 N = 86
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distr ibution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distr ibution
9 to to to to to to to to to 9
to 0 10 20 30 40 50 60 70 80 90 100 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[650,681) 9.71 10.03 10.25 10.33 10.21 9.55 9.66 9.84 9.90 10.52 (9.8%) 100 100
1st/[650,692) 9.86 10.22 10.06 9.93 9.98 9.99 9.88 9.85 9.83 10.40 (9.5%)
2nd/[681,699) 9.39 10.63 10.43 10.30 9.50 9.85 9.47 9.95 10.53 9.97 (9.7%) 2nd/[692,708) 9.46 10.05 9.96 10.09 9.83 10.05 9.73 10.16 9.84 10.83 (9.8%)
Prior Scale Score Decile/Range * Prior Scale Score Decile/Range *
3rd/[699,714) 9.43 10.00 9.47 9.53 9.77 10.45 10.61 9.92 9.92 10.90 (10.4%) 3rd/[708,720) 9.34 9.75 10.11 9.94 9.92 9.87 10.33 10.08 10.34 10.31 (10.1%)
Empirical SGP Distribution
Empirical SGP Distribution
4th/[714,725) 10.15 9.96 9.34 9.84 10.72 10.12 9.83 10.14 9.38 10.53 (8.8%) 4th/[720,730) 9.49 10.13 9.89 9.81 10.14 10.11 10.05 9.64 9.83 10.91 (10.2%)
5th/[725,736) 9.69 10.32 10.55 10.74 9.55 9.89 9.89 9.58 10.06 9.74 (10.4%) 5th/[730,739) 9.42 9.69 10.14 10.43 10.11 9.81 9.80 10.22 10.09 10.30 (9.5%)
6th/[736,747) 8.92 10.11 10.31 9.61 10.20 10.32 9.30 10.05 9.79 11.39 (10.3%) 6th/[739,748) 9.74 10.42 9.87 9.69 9.68 9.95 10.04 10.20 10.18 10.23 (9.7%)
7th/[747,757) 9.85 9.90 9.63 9.98 10.42 8.97 11.65 9.57 10.09 9.94 (9.2%) 7th/[748,758) 9.26 9.86 9.98 10.18 9.88 9.97 10.16 9.92 9.95 10.83 (11%)
8th/[757,771) 9.52 10.08 9.86 10.27 9.39 10.31 9.74 10.43 10.07 10.34 (10.5%) 8th/[758,769) 9.75 10.22 10.21 9.75 10.20 10.13 9.79 9.96 9.66 10.33 (9.9%)
9th/[771,788) 9.28 10.20 9.93 10.57 9.62 10.45 9.66 9.90 9.80 10.58 (10.5%) 9th/[769,785) 9.10 9.67 9.93 10.27 10.28 10.01 10.20 9.84 10.29 10.41 (10.2%)
10th/[788,850] 9.84 9.83 10.26 9.77 9.91 9.96 9.99 10.05 9.92 10.45 (10.4%) 10th/[785,850] 9.74 10.04 9.96 9.74 9.78 9.93 9.97 10.29 9.96 10.60 (10.1%)
*Prior score deciles can be une ven depending upon the pr ior score distr ibution *Prior score deciles can be une ven depending upon the pr ior score distr ibution
[Illinois SGP Analyses for 2019] 9
Results
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2018−2019.2 Mathematics SS SGP, Grade 4 (N = 135,167) 2018−2019.2 Mathematics SS SGP, Grade 5 (N = 139,286)
SGP Deciles by Prior Achievement Level (Mathematics SS) SGP Deciles by Prior Achievement Level (Mathematics SS)
Level 5 (8.3%) Level 5 (3.4%)
Prior Achievement Level
Level 4 (28.3%)
Prior Achievement Level
Level 4 (30%)
Level 3 (27.3%)
Level 3 (26.1%)
Level 2 (21.8%) Level 2 (25.5%)
Level 1 (13.7%) Level 1 (15.5%)
Median Student Growth Percentile Median Student Growth Percentile
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score 5 9
Current Observed Score 19 29 39 49 59 69 79 89 94 99
to to to to to to to to to to to to 5 9 to to to to to to to to to to
1 5 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [ 650 ] 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 831
Extremes (Range)
Lowest OSS: [ 650 ] 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 285
Highest OSS: [ 850 ] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 11.2 88.8 N = 170 Extremes (Range) Highest OSS: [ 850 ] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 9.4 13.3 77.3 N = 128
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distr ibution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distr ibution
9 to to to to to to to to to 9
to 0 10 20 30 40 50 60 70 80 90 100 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[650,693) 9.73 9.85 10.38 10.43 9.98 9.48 10.02 10.26 9.49 10.37 (9.6%) 100 100
1st/[650,692) 9.67 10.03 10.02 10.00 9.97 10.10 10.08 9.96 9.76 10.41 (9.6%)
2nd/[693,708) 9.34 10.55 9.85 9.76 10.48 9.90 9.60 9.72 10.24 10.57 (10.1%) 2nd/[692,705) 9.57 9.95 10.04 10.12 9.90 9.85 10.02 10.00 10.07 10.48 (10.2%)
Prior Scale Score Decile/Range * Prior Scale Score Decile/Range *
3rd/[708,719) 9.60 9.97 10.14 10.25 9.57 9.57 10.24 10.11 10.14 10.41 (9.5%) 3rd/[705,715) 9.28 9.89 9.80 9.96 10.01 9.94 9.95 10.10 10.14 10.93 (10.2%)
Empirical SGP Distribution
Empirical SGP Distribution
4th/[719,729) 10.41 9.42 9.68 9.80 10.51 10.38 9.39 9.90 9.48 11.04 (9.9%) 4th/[715,724) 9.73 10.16 10.33 10.08 9.57 9.83 10.05 9.93 9.86 10.47 (9.9%)
5th/[729,739) 9.09 9.97 10.77 9.66 10.00 10.07 10.42 9.70 10.46 9.86 (10.9%) 5th/[724,732) 9.66 10.00 9.96 9.92 10.44 9.93 10.06 9.67 9.88 10.47 (8.1%)
6th/[739,748) 9.85 10.70 8.63 10.63 9.63 9.49 10.14 10.06 10.05 10.81 (9.5%) 6th/[732,741) 9.40 10.20 9.67 10.16 9.91 10.01 10.10 10.22 10.07 10.25 (11.4%)
7th/[748,757) 9.65 9.44 10.39 9.98 10.24 11.05 9.14 9.79 9.64 10.67 (8.7%) 7th/[741,750) 9.39 9.57 10.02 9.80 9.98 10.34 10.00 9.53 10.54 10.83 (8.8%)
8th/[757,769) 9.57 10.16 10.01 9.34 10.24 10.43 10.52 9.49 10.08 10.16 (11.2%) 8th/[750,762) 9.78 10.09 10.35 10.00 9.62 9.65 9.78 10.47 9.75 10.52 (11.4%)
9th/[769,785) 9.51 9.80 10.20 10.72 9.49 9.81 9.75 10.08 9.97 10.66 (10.2%) 9th/[762,777) 9.36 9.91 9.71 9.60 10.13 10.13 10.16 9.91 10.13 10.96 (10.2%)
10th/[785,850] 9.40 10.27 9.78 9.86 10.25 9.89 10.31 10.02 9.94 10.28 (10.5%) 10th/[777,850] 9.46 10.05 10.07 10.20 10.24 9.98 9.96 9.89 9.92 10.23 (10.2%)
*Prior score deciles can be une ven depending upon the pr ior score distr ibution *Prior score deciles can be une ven depending upon the pr ior score distr ibution
[Illinois SGP Analyses for 2019] 10
Growth & Achievement Plots
[Illinois SGP Analyses for 2019] 11
Growth & Achievement Plots
[Illinois SGP Analyses for 2019] 12
Growth & Achievement Plots
[Illinois SGP Analyses for 2019] 13
Growth & Achievement Plots
[Illinois SGP Analyses for 2019] 14
About this copy
Illinois SGP Analyses for 2019 — Illinois Technical Advisory Committee, December 12, 2019.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.