SGP Calculation & Anomalies

Illinois Technical Advisory Committee · January 28, 2026

SGP Calculation & Anomalies — Illinois Technical Advisory Committee, January 28, 2026.
Published

January 28, 2026

Event
Technical advisory committee
Slides
16
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. SGP Calculation & Anomalies Damian Betebenner & Adam VanIwaarden Center for Assessment January 29th, 2026
  2. Overview
  3. SGP Background • An SGP is a conditional probability: SGP ≡ 100 ⋅ Pr(Current Score or Less | Prior Score(s)) • For example, an SGP of 90 implies that the probability of observing the current score or less given the prior score(s) is • Equivalently, an SGP of 90 implies that the probability of observing the current score or higher given the prior(s) is 0.1. • SGPs are calculated using non-parametric (b-spline based) quantile regression.
  4. SGP: Scale agnosticism • SGPs, being norm-referenced, are, in general, immune to assessment transitions: • As long as all students take the “same” assessment at the “same” time, we can calculate growth-norms associated with those students. • In addition, the analytic methodology behind SGP calculation is designed to be invariant to mono-tone transformations of scale. • In theory, if there is a mono-tone transformation of scale between two assessments, then SGPs will be invariant to that transformation. • SGPs have been successfully calculated across numerous assessment transitions (state transition to PARCC and SBAC, for example)
  5. SGP: Properties and Goodness-of-Fit • Given that SGPs are, by definition, conditional probabilities, they are independent (i.e., uncorrelated) with the conditioning variable - prior score(s). • Percentiles, by definition are uniformly distributed. One way to test whether the model fits the data is to test against uniformity. • Goodness-of-Fit plots exported as part of the SGP run help see to what extent observed SGPs deviate from the theoretic expectation.
  6. Goodness of Fit: Grade 4 and Grade 8 mathematics Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2024−2025.2 Math SGP, Grade 4 (N = 124,220) 2024−2025.2 Math SGP, Grade 8 (N = 127,694) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) Level 5 (6.7%) Level 5 (4%) Level 4 (22.8%) Prior Achievement Level Prior Achievement Level Level 4 (27.2%) Level 3 (24.1%) Level 3 (35.2%) Level 2 (23.9%) Level 2 (29.4%) Level 1 (18.1%) Level 1 (8.5%) Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [ −15 ] 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 73 Lowest OSS: [ −15 ] 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 69 Extremes (Range) Extremes (Range) Highest OSS: [ 3.9543 ] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 8.5 89.4 N = 47 Highest OSS: [4.94,5.702) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.0 19.4 25.8 51.6 N = 31 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[−15,−1.909) 10.15 9.42 11.87 10.47 9.04 9.88 10.59 10.04 8.83 9.72 (7.8%) 100 1st/[−15,−1.421) 10.05 9.57 10.06 10.08 9.96 10.46 9.96 9.97 9.44 10.45 (8.5%) 100 2nd/[−1.909,−1.376) 10.68 8.55 9.80 12.00 9.59 8.09 10.73 9.16 11.14 10.26 (10.3%) 2nd/[−1.421,−1.017) 9.67 10.83 9.81 9.94 9.87 9.75 9.85 10.11 9.93 10.22 (9.6%) Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * 3rd/[−1.376,−0.9414) 8.92 11.30 9.52 9.06 8.77 10.80 9.89 10.41 10.07 11.26 (11.8%) 3rd/[−1.017,−0.6991) 9.54 9.70 10.01 10.41 9.85 9.99 9.80 9.67 10.12 10.90 (10.3%) Empirical SGP Distribution Empirical SGP Distribution 4th/[−0.9414,−0.6491) 10.33 9.29 9.75 11.19 8.60 10.71 8.24 11.20 9.64 11.06 (9.1%) 4th/[−0.6991,−0.4373) 9.57 10.82 10.31 10.05 10.58 9.57 9.77 9.72 9.62 10.00 (9.4%) 5th/[−0.6491,−0.373) 8.98 11.40 10.39 9.21 11.00 9.97 10.74 9.60 9.13 9.58 (9.2%) 5th/[−0.4373,−0.1163) 9.94 10.05 9.57 9.86 9.72 10.17 9.83 10.19 9.90 10.77 (11.4%) 6th/[−0.373,−0.1046) 10.01 9.71 10.54 9.19 12.18 8.58 10.27 8.50 10.84 10.18 (9.1%) 6th/[−0.1163,0.1508) 9.15 9.26 10.50 10.22 9.82 10.06 10.55 10.00 10.17 10.27 (9.4%) 7th/[−0.1046,0.2544) 9.51 8.94 10.32 10.21 8.95 10.92 10.43 10.76 10.14 9.81 (11.6%) 7th/[0.1508,0.4625) 9.02 10.28 9.61 9.68 10.11 10.07 9.92 10.23 10.16 10.92 (10.2%) 8th/[0.2544,0.6333) 9.49 10.95 10.15 10.30 8.96 9.60 9.78 10.04 9.91 10.83 (10.7%) 8th/[0.4625,0.8228) 9.84 10.00 10.16 9.99 10.17 9.71 9.82 9.81 9.87 10.62 (10%) 9th/[0.6333,1.062) 9.47 11.62 8.40 9.99 11.39 10.29 8.51 9.67 10.07 10.59 (8.8%) 9th/[0.8228,1.415) 10.03 10.14 10.09 10.21 9.59 10.08 9.66 10.06 9.80 10.35 (11.1%) 10th/[1.062,15] 9.65 9.14 10.67 10.09 9.70 9.51 11.33 9.71 10.11 10.09 (11.7%) 10th/[1.415,15] 9.17 9.98 9.93 9.89 10.19 9.96 10.17 9.99 10.20 10.52 (10%) *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
  7. Goodness of Fit: Grade 8 mathematics
  8. Illinois SGP Calculation • If you look closely at the previous plots you will see prior score values (roughly) between -3 and 3 displayed • Historically (going back to PARCC) we have used theta values to calculate SGPs in Illinois. • The use of theta values began to obviate issues related to scale score rounding as well as LOSS/HOSS issues. • This is true of both cohort and baseline referenced SGPs.
  9. Anomalous Results at 850 • The above results show students with HOSS scores in (at least) 2025 grade 8 math and 2024 grade 7 math together with their SGPs. • Note the student with perfect scores in 3 consecutive years (2025, 2024, 2023) and a SGP of 34 (baseline SGP of 39)
  10. Anomalous Results at 850 • The issue (unfortunately) is that the 850 that is reported is not the same score used to calculate the SGP. Scale Score Count Minimum Theta Maximum Theta Grade 8 2025 850 580 3.683 15.0 Grade 7 2024 850 122 5.554 15.0 Grade 6 2023 850 51 4.035 15.0 All 3 Years 850 3
  11. Anomalous Results at 850 • For the 3 students with perfect math scores (850) in 2023, 2024 and 2025 Theta (2023, 2024, 2025) SGP SGP Baseline Student 1 4.0349; 5.5541; 3.838 34 39 Student 2 4.0349; 5.5541; 4.4067 57 54 Student 3 5.4898; 15; 4.0317 25 53
  12. Several Issues to Consider • Based upon my examination of these cases, there isn’t an issue with “calculations”. All the SGP calculations are as expected. • However, there are other issues that are problematic: • The reporting scale is not a strictly mono-tone transformation of the theta scale (many theta scores get mapped to 850). Thus, same scale score trajectories lead to different SGPs. • The multi-variate density is, obviously, extremely sparse at the extremes, so SGP accuracy and precision are questionable.
  13. Several Issues to Consider • Issues with ceiling effects on the test have long been an issue in SGP calculation. • Several states have adopted the policy of just assigning an SGP of 99 to any student getting a perfect score (e.g., 850). • That may obviate the issue at the HOSS, however, there are likely issues near the HOSS that aren’t “corrected”. • When an over-preponderance of students are assigned the HOSS, then low SGPs can result (indicating, correctly, the the probability of having a perfect current score is not “rare” given perfect prior scores) • However, that mathematical explanation does not translate into the correct message for a parent.
  14. Anomalous Results at 850 • The SGP of 25 is associated with a drop of almost 11 theta points (However the HOSS definition shows 850 in both years) • The message to a parent is that, “your child didn’t make a year’s worth of learning” (or something like that). Theta (2023, 2024, 2025) SGP SGP Baseline Student 1 4.0349; 5.5541; 3.838 34 39 Student 2 4.0349; 5.5541; 4.4067 57 54 Student 3 5.4898; 15; 4.0317 25 53
  15. Summary • At the ceiling of the IAR, there are issues with the accuracy and precision of the SGPs. • There are two intersecting issues: • Use of theta values for SGP calculation • Reporting of SGPs with scale scores • Probably the best way to over come these issues (as best they can from an analytic perspective) is: • Calculate SGPs using scaled scores (in addition to thetas just to have those too). • Investigate and deal with any emerging ceiling issues (lots of 850s in the analysis) by possibly assigning all students a 99. • More broadly, the question is, “What score decline at the top of the scale should we be comfortable signaling is worrisome?”