SGP: Precision, Interpretation & Utility

Northeastern Educational Research Organization · April 23, 2019

SGP: Precision, Interpretation & Utility — Northeastern Educational Research Organization, April 23, 2019.
Published

April 23, 2019

Event
Conference
Slides
7
Format
Converted from PowerPoint

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. SGP: Precision, Interpretation & Utility Kathleen Flannagan & Damian Betebenner Massachusetts DESE & Center for Assessment NEERO Portsmouth, New Hampshire April 25th, 2019
  2. Background • Precision/reliability of test scores has been an area of active research since the beginning of psychometrics. • Beyond test scores, data derived from test scores is also subject to investigations of precision/ reliability. • In the past several years, the individual student growth scores SGPs derived from test data have been subject to research regarding precision/ reliability. 4/25/19 SGP: Precision, Interpretation & Utility 2
  3. Background • Sirici, Wells, and Keller (2016) recommended abandoning SGPs for individual level reporting due to precision/reliability concerns. • Betebenner, DePascale, Marion, et al. found, counter to Sirici, adequate precision/reliability in SGPs for reporting purposes. • This presentation provides SEMs for SGPs results from a large scale testing to illustrate how individual growth results can be used to 4/25/19 SGP: Precision, Interpretation & Utility 3
  4. SGP SEM Calculation • SGP SEMs are calculation using scaled score CSEMs provided as part of the state’s assessment data used for SGP calculation. • Scale scored CSEMs are used to perturb the observed scales so that 100 simulated SGPs are calculated for each student. • The 100 simulated SGPs are used to calculate a standard error for each SGP reported. 4/25/19 SGP: Precision, Interpretation & Utility 4
  5. SGP & SGP SEM Facts • SGPs, being percentiles, are uniformly distributed. • However, the distribution of observed SGPs based upon a normal CSEM is normal. • That is, a for a student with a true SGP of 50, observed SGPs occur most often near 50. 4/25/19 SGP: Precision, Interpretation & Utility 5
  6. Distribution of SGP with error SGP with Measurement Error: SGP of 50, SE of 15 Probability of SGP Occurring SGP 4/25/19 SGP: Precision, Interpretation & Utility 6
  7. 4/25/19 SGP: Precision, Interpretation & Utility 7