SGP SEM for 2018 EOG/EOC
Testing
Damian Betebenner
Center for Assessment
Georgia TAC Meeting
Atlanta, Georgia
April 30th, 2019
Background
• During the November TAC meeting, Qi noted
discrepant SGP SEM values compared to previous
years.
• Though part of the normal Georgia analyses, SGP
SEM results are not a part of normal QC as they
are, in general, not used by Georgia.
• Upon review of the results and source code in the
SGP package, it was discovered that the SGP SEMs
were incorrectly calculated and were too small.
4/30/19 SGP SEM for 2018 EOG/EOC Testing 2
SGP SEM Calculation
• SGP SEMs are calculation using CSEMs provided
as part of the state’s assessment data used for SGP
calculation.
• Modifications were made to the SGP package in
2017 to accommodate states supplying error
variances instead of standard errors.
• Those changes led to the package simulating SGP
SEMs using CSEMs approximately half as large as
actual CSEMs.
• Recall the results from the November, 2018 TAC.
4/30/19 SGP SEM for 2018 EOG/EOC Testing 3
Estimate the Standard Errors of
the SGPs Richard Woods,
Georgia’s School Superintendent
“Educating Georgia’s Future”
gadoe.org
Figure 1: Individual SGP standard error over time (uncorrected SGP)
4/30/19 SGP SEM for 2018 EOG/EOC Testing 4
Estimate the Standard Errors of
the SGPs Richard Woods,
Georgia’s School Superintendent
“Educating Georgia’s Future”
gadoe.org
Figure 1: Individual SGP standard error over time (uncorrected SGP, cont’d)
4/30/19 SGP SEM for 2018 EOG/EOC Testing 5
Estimate the Standard Errors of
the SGPs Richard Woods,
Georgia’s School Superintendent
“Educating Georgia’s Future”
gadoe.org
Figure 1: Individual SGP standard error over time (uncorrected SGP, cont’d)
4/30/19 SGP SEM for 2018 EOG/EOC Testing 6
Estimate the Standard Errors of
the SGPs Richard Woods,
Georgia’s School Superintendent
“Educating Georgia’s Future”
gadoe.org
Figure 1: Individual SGP standard error over time (uncorrected SGP, cont’d)
4/30/19 SGP SEM for 2018 EOG/EOC Testing 7
Corrected Standard Errors of the SGPs
ELA 2018 Mathematics 2018 9th Grade Lit 2018 American Lit 2018
Frequency Frequency Frequency Frequency
SGP Standard Error SGP Standard Error SGP Standard Error SGP Standard Error
Coordinate Algebra 2018 Analytic Geometry 2018 Algebra I 2018 Geometry 2018
Frequency Frequency 4000 Frequency Frequency
SGP Standard Error SGP Standard Error SGP Standard Error SGP Standard Error
4/30/19 SGP SEM for 2018 EOG/EOC Testing 8
About this copy
Georgia TAC CSEM Presentation — Georgia Department of Education, April 30, 2019.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.