SGP: Precision, Interpretation & Utility
Kathleen Flannagan & Damian Betebenner
Massachusetts DESE & Center for Assessment
NEERO
Portsmouth, New Hampshire
April 25th, 2019
Background
• Precision/reliability of test scores has been an area
of active research since the beginning of
psychometrics.
• Beyond test scores, data derived from test scores is
also subject to investigations of precision/
reliability.
• In the past several years, the individual student
growth scores SGPs derived from test data have
been subject to research regarding precision/
reliability.
4/25/19 SGP: Precision, Interpretation & Utility 2
Background
• Sirici, Wells, and Keller (2016) recommended
abandoning SGPs for individual level reporting due
to precision/reliability concerns.
• Betebenner, DePascale, Marion, et al. found,
counter to Sirici, adequate precision/reliability in
SGPs for reporting purposes.
• This presentation provides SEMs for SGPs results
from a large scale testing to illustrate how
individual growth results can be used to
4/25/19 SGP: Precision, Interpretation & Utility 3
SGP SEM Calculation
• SGP SEMs are calculation using scaled score
CSEMs provided as part of the state’s assessment
data used for SGP calculation.
• Scale scored CSEMs are used to perturb the
observed scales so that 100 simulated SGPs are
calculated for each student.
• The 100 simulated SGPs are used to calculate a
standard error for each SGP reported.
4/25/19 SGP: Precision, Interpretation & Utility 4
SGP & SGP SEM Facts
• SGPs, being percentiles, are uniformly distributed.
• However, the distribution of observed SGPs based
upon a normal CSEM is normal.
• That is, a for a student with a true SGP of 50,
observed SGPs occur most often near 50.
4/25/19 SGP: Precision, Interpretation & Utility 5
Distribution of SGP with error
SGP with Measurement Error: SGP of 50, SE of 15
Probability of SGP Occurring
SGP
4/25/19 SGP: Precision, Interpretation & Utility 6
4/25/19 SGP: Precision, Interpretation & Utility 7
About this copy
SGP: Precision, Interpretation & Utility — Northeastern Educational Research Organization, April 23, 2019.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.