GEORGIA SUMMARY SGP RESULTS
CORRELATIONS BETWEEN GROWTH AND PRIOR ACHIEVEMENT
Damian W Betebenner & Adam VanIwaarden
National Center for the Improvement of Educational Assessment
Dover, New Hampshire December 15th, 2016
ISSUE: BACKGROUND
• Initial SGP analyses with preliminary 2016 ELA and Mathematics EOG
data were completed in late July. Preliminary EOC data was also available
• As part of validation of results, correlations between mean prior
achievement (mean of prior scale score standardized within grade) and
mean SGP (unadjusted and SIMEX adjusted) were examined at the school
level.
• In all but one content area, the correlations increased from 2015 to 2016.
• We want to explore whether these increases reflect a change in the
system or are a manifestation of the model (e.g., measurement error).
ANALYSIS: RESULTS
• From 2015 to 2016 there is a definite trend upward in correlations
between prior achievement and growth.
• Examination of data (see supplementary scatterplots) rule out outliers/
changes in marginal distributions as the reason.
• For example, non-random assignment of students to school by level of
achievement is one of the factors for measurement error bias. Yet no
marked changes in stratification of schools based upon achievement
was observed.
• The increasing correlations observed are synonymous with larger
increases in mean SGP for schools serving higher achieving students
(higher mean achievement) than their counterparts serving lower
achieving students.
ANALYSIS: RESULTS
• Examination of data (see supplementary scatterplots) rule out
outliers/changes in marginal distributions as the reason.
• For example, non-random assignment of students to school by level
of achievement is one of the factors for measurement error bias.
Yet no marked changes in stratification of schools based upon
achievement was observed.
• The increasing correlations observed are consistent with changes in
percent proficient for schools between 2015 and 2016. Schools
serving higher achieving students had, on average, higher rates of
improvement than schools serving lower achieving students (see
supplementary scatterplots)
ANALYSIS: RESULTS
• To investigate more broadly, we examined results associated with
assessment transitions in 7 states:
• 2 non-consortium affiliated states (including Georgia)
• 2 PARCC affiliated states
• 3 SBAC affiliated states
• All summary statistics calculated using a 15 student minimum.
CORRELATIONS:
MEAN PRIOR ACHIEVEMENT BY MEAN SGP
GA: SCHOOL LEVEL CORRELATIONS
SIMEX ADJUSTED/MEAN PRIOR ACHIEVEMENT 2014/2015/2016
2014 (CRCT/EOCT) 2015 (Milestones) 2016 (Milestones)
American Lit 0.28 (427) 0.18 (427) 0.5 (436)
Analytic Geometry 0.34 (426) 0.37 (436) 0.45 (346)
Biology 0.11(438) 0.21(449) 0.23 (447)
Coordinate Algebra 0.23 (649) 0.23 (725) 0.21(246)
Economics 0.09 (426) 0.05 (427) 0.29 (428)
ELA 0.38 (1,745) 0.3 (1,760) 0.44 (1.742)
Grade 9 Lit 0.22 (465) 0.25 (540) 0.36 (584)
Mathematics 0.35 (1,744) 0.38 (1,760) 0.42 (1,752)
Physical Science 0.22 (465) 0.25 (540) 0.27 (584)
Science 0.27 (1,747) 0.3 (1,760) 0.39 (1,752)
Social Studies 0.24 (1,747) 0.21(1,760) 0.34 (1,752)
US History 0.03 (423) 0.06 (426) 0.18 (431)
GA: SCHOOL LEVEL CORRELATIONS
MEAN SIMEX ADJUSTED SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA Social Studies Science
Grade 4 0.18/0.23/0.26 0.23/0.09/0.29 0.09/0.10/0.18 0.18/0.20/0.32
Grade 5 0.11/0.13/0.20 0.11/0.11/0.24 -0.05/-0.10/0.10 0.06/0.06/0.13
Grade 6 0.27/0.18/0.24 0.28/0.22/0.25 0.08/0.06/0.15 0.16/0.18/0.20
Grade 7 0.35/0.22/0.29 0.23/0.36/0.32 0.10/0.07/0.08 0.04/-0.01/0.06
Grade 8 0.23/0.23/0.30 0.38/0.22/0.34 0.00/-0.07/0.07 0.08/0.06/0.21
Entire School 0.35/0.38/0.42 0.38/0.30/0.44 0.24/0.21/0.34 0.27/0.30/0.39
NON-ALIGNED 1 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.14/0.12/0.02/0.03 0.31/0.23/0.09/0.10
Grade 5 0.13/0.07/0.12/0.13 0.31/0.22/0.19/0.19
Grade 6 0.13/0.03/0.03/0.09 0.13/0.15/0.18/0.06
Grade 7 0.26/0.18/0.14/0.13 0.24/0.22/0.17/0.09
Grade 8 —/0.26/0.15/0.18/ 0.17/0.19/0.17/0.12
Entire School 0.26/0.26/0.23/0.26 0.34/0.40/0.33/0.21
NON-ALIGNED 2 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.06/0.10/0.10 0.08/0.19/0.20
Grade 5 0.09/0.12/0.18 0.12/0.19/0.18
Grade 6 0.13/0.18/0.15 0.15/0.11/0.23
Grade 7 0.24/0.17/0.27 0.19/0.28/0.31
Grade 8 0.13/0.35/0.32 0.17/0.26/0.33
Entire School 0.27/0.33/0.34 0.24/0.35/0.39
PARCC 1 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.32/0.25/0.18 0.50/0.40/0.28
Grade 5 0.34/0.27/0.30 0.42/0.25/0.28
Grade 6 0.34/0.20/0.25 0.35/0.24/0.21
Grade 7 0.20/0.24/0.17 0.33/0.26/0.22
Grade 8 0.31/0.31/0.21 0.30/0.15/0.12
Grade 9 —/0.14/0.13
Entire School 0.43/0.33/0.31 0.58/0.29/0.29
PARCC 2 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.11/0.19/0.12 0.23/0.11/0.16
Grade 5 0.08/0.11/0.21 0.10/0.03/0.18
Grade 6 0.20/0.16/0.26 0.15/0.06/0.22
Grade 7 0.28/0.18/0.19 0.17/0.17/0.19
Grade 8 0.15/0.33/0.37 0.26/0.17/0.17
Grade 9 0.21/0.20/0.25
Grade 10 0.25/0.26/—
Entire School 0.34/0.25/0.39 0.33/0.24/0.29
PARCC 3 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.14/0.15/0.31 0.32/0.26/0.27
Grade 5 0.07/0.14/0.20 0.24/0.32/0.32
Grade 6 0.17/0.29/0.31 0.32/0.14/0.31
Grade 7 0.45/0.15/0.36 0.17/0.33/0.49
Grade 8 0.43/0.30/0.58 0.36/0.18/0.51
Grade 9 —/0.07/0.33
Entire School 0.43/0.37/0.53 0.32/0.25/0.45
SBAC 1 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.12/0.20/0.05 0.44/0.15/0.15
Grade 5 0.05/0.07/0.06 -0.05/-0.04/0.17
Grade 6 -0.05/0.15/0.14 0.04/0.23/0.11
Grade 7 -0.19/0.04/0.07 0.08/0.29/0.27
Grade 8 0.07/0.25/0.28 -0.10/0.22/0.12
Entire School 0.07/0.28/0.18 0.13/0.29/0.29
SBAC 2 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 0.14/0.02/0.15 0.17/0.15/0.19
Grade 5 0.24/0.08/0.04 0.34/0.14/0.17
Grade 6 0.25/0.05/0.08 0.41/0.14/0.11
Grade 7 0.15/0.03/-0.04 0.26/0.16/0.09
Grade 8 0.16/0.13/0.15 0.21/0.06/0.21
Entire School 0.40/0.16/0.18 0.32/0.22/0.26
SBAC 3 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE
Mathematics ELA
Grade 4 -0.05/0.14/0.00 0.10/-0.04/0.13
Grade 5 0.14/0.00/0.06 0.24/0.01/0.09
Grade 6 -0.10/0.21/0.21 0.32/0.11/0.11
Grade 7 0.22/0.23/0.25 0.30/0.25/0.35
Grade 8 0.10/0.28/0.11 0.09/0.26/0.30
Grade 9 0.36/0.21/0.20 0.30/0.07/0.20
Grade 10 0.25/0.08/0.26 0.20/0.18/-0.03
Grade 11 0.31/0.16/0.23 0.30/0.01/-0.02
Entire School 0.24/0.27/0.27 0.30/0.16/0.30
CORRELATIONS:
CURRENT ACHIEVEMENT BY PERCENT FREE/REDUCED
LUNCH
GA SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA Social Studies Science
Grade 4 -0.50/-0.77/-0.75 -0.56/-0.80/-0.78 -0.50/-0.72/-0.71 -0.53/-0.74/-0.73
Grade 5 -0.50/-0.72/-0.70 -0.58/-0.81/-0.79 -0.56/-0.73/-0.70 -0.56/-0.74/-0.72
Grade 6 -0.52/-0.70/-0.68 -0.53/-0.72/-0.71 -0.50/-0.61/-0.64 -0.49/-0.65/-0.64
Grade 7 -0.48/-0.72/-0.71 -0.51/-0.72/-0.74 -0.45/-0.64/-0.67 -0.47/-0.67/-0.69
Grade 8 -0.49/-0.67/-0.68 -0.55/-0.71/-0.69 -0.50/-0.62/-0.64 -0.50/-0.64/-0.51
Entire School -0.48/-0.69/-0.68 -0.52/-0.70/-0.68 -0.51/-0.65/-0.65 -0.52/-0.67/-0.67
PARCC I SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.68/-0.67/-0.70 -0.78/-0.71/-0.71
Grade 5 -0.57/-0.58/-0.66 -0.74/-0.63/-0.65
Grade 6 -0.48/-0.45/-0.49 -0.62/-0.44/-0.43
Grade 7 -0.52/-0.46/-0.48 -0.61/-0.43/-0.43
Grade 8 -0.57/-0.25/-0.34 -0.55/-0.46/-0.45
Entire School -0.73/-0.49/-0.51 -0.75/-0.57/-0.57
PARCC 2 SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.69/-0.58/-0.65 -0.72/-0.63/-0.63
Grade 5 -0.70/-0.64/-0.56 -0.70/-0.68/-0.57
Grade 6 -0.64/-0.64/-0.65 -0.64/-0.62/-0.56
Grade 7 -0.65/-0.55/-0.53 -0.63/-0.56/-0.51
Grade 8 -0.60/-0.37/-0.45 -0.68/-0.50/-0.53
Entire School -0.71/-0.64/-0.64 -0.73/-0.62/-0.62
PARCC 3 SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.73/-0.68/-0.40 -0.74/-0.69/-0.43
Grade 5 -0.76/-0.57/-0.52 -0.66/-0.67/-0.45
Grade 6 -0.73/-0.43/-0.46 -0.66/-0.48/-0.43
Grade 7 -0.53/-0.72/-0.23 -0.51/-0.52/-0.35
Grade 8 -0.43/-0.40/-0.43 -0.54/-0.54/-0.56
Entire School -0.66/-0.68/-0.39 -0.74/-0.62/-0.49
SBAC 1 SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.55/-0.57/-0.61 -0.66/-0.69/-0.69
Grade 5 -0.47/-0.60/-0.61 -0.63/-0.63/-0.59
Grade 6 -0.46/-0.60/-0.65 -0.48/-0.64/-0.63
Grade 7 -0.49/-0.40/-0.60 -0.52/-0.42/-0.58
Grade 8 -0.30/-0.57/-0.43 -0.39/-0.63/-0.44
Entire School -0.59/-0.65/-0.66 -0.71/-0.68/-0.68
SBAC 2 SCHOOL LEVEL CORRELATIONS
CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.42/-0.47/-0.54 -0.34/-0.41/-0.53
Grade 5 -0.45/-0.46/-0.55 -0.52/-0.54/-0.60
Grade 6 -0.36/-0.39/-0.43 -0.38/-0.43/-0.50
Grade 7 -0.41/-0.53/-0.48 -0.36/-0.44/-0.57
Grade 8 -0.48/-0.49/-0.53 -0.40/-0.49/-0.49
Entire School -0.50/-0.52/-0.56 -0.50/-0.54/-0.60
CORRELATIONS:
MEAN SGP AND PERCENT FREE/REDUCED LUNCH
GA SCHOOL LEVEL CORRELATIONS
MEAN SIMEX ADJUSTED SGP TO CURRENT PERCENT ED 2014/2015/2016
Mathematics ELA Social Studies Science
Grade 4 -0.17/-0.41/-0.36 -0.18/-0.19/-0.33 -0.14/-0.21/-0.34 -0.16/-0.20/-0.35
Grade 5 -0.07/-0.20/-0.17 -0.12/-0.15/-0.22 -0.11/-0.04/-0.15 -0.12/-0.10/-0.14
Grade 6 -0.28/-0.28/-0.25 -0.19/-0.22/-0.22 -0.14/-0.10/-0.21 -0.11/-0.18/-0.24
Grade 7 -0.13/-0.21/-0.30 -0.12/-0.30/-0.25 -0.05/-0.08/-0.17 -0.01/-0.06/-0.15
Grade 8 -0.09/-0.09/-0.16 -0.30/-0.19/-0.23 -0.11/-0.01/-0.05 -0.07/-0.05/-0.12
Entire School -0.14/-0.31/-0.29 -0.18/-0.18/-0.27 -0.17/-0.14/-0.26 -0.13/-0.16/-0.26
PARCC I SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.31/-0.29/-0.24 -0.53/-0.33/-0.26
Grade 5 -0.22/-0.09/-0.21 -0.40/-0.19/-0.10
Grade 6 -0.12/-0.07/-0.12 -0.39/-0.19/-0.10
Grade 7 -0.25/-0.08/0.00 -0.28/-0.04/0.02
Grade 8 -0.24/-0.01/-0.04 -0.11/0.00/0.00
Entire School -0.34/-0.13/-0.18 -0.49/-0.14/-0.07
SBAC 2 SCHOOL LEVEL CORRELATIONS
MEAN SGP TO PERCENT ED 2014/2015/2016
Mathematics ELA
Grade 4 -0.19/-0.18/-0.14 -0.09/-0.31/-0.18
Grade 5 -0.16/-0.20/-0.17 -0.31/-0.31/-0.16
Grade 6 -0.20/-0.07/-0.08 -0.16/-0.14/-0.05
Grade 7 -0.11/-0.22/-0.06 -0.07/-0.22/-0.09
Grade 8 -0.05/-0.16/-0.12 -0.03/-0.19/-0.20
Entire School -0.21/-0.24/-0.17 -0.20/-0.31/-0.20
CORRELATIONS:
MISCELLANEOUS CORRELATIONS
GA STUDENT LEVEL CORRELATIONS
PRIOR SCALE SCORE TO CURRENT SCALE SCORE 2014/2015/2016
Mathematics ELA Social Studies Science
Grade 4 0.79/0.81/0.84 0.78/0.78/0.82 0.76/0.77/0.77 0.79/0.79/0.79
Grade 5 0.78/0.82/0.84 0.76/0.80/0.83 0.80/0.80/0.80 0.79/0.79/0.80
Grade 6 0.79/0.80/0.84 0.77/0.78/0.84 0.78/0.79/0.80 0.80/0.80/0.80
Grade 7 0.85/0.86/0.87 0.76/0.79/0.84 0.82/0.83/0.84 0.82/0.82/0.83
Grade 8 0.81/0.84/0.80 0.77/0.78/0.83 0.80/0.79/0.82 0.81/0.79/0.78
GA SCHOOL LEVEL CORRELATIONS
PRIOR SCALE SCORE TO CURRENT SCALE SCORE 2014/2015/2016
Mathematics ELA Social Studies Science
Grade 4 0.89/0.90/0.93 0.94/0.93/0.95 0.85/0.87/0.88 0.92/0.92/0.93
Grade 5 0.87/0.90/0.92 0.95/0.95/0.96 0.88/0.87/0.88 0.91/0.90/0.91
Grade 6 0.89/0.90/0.91 0.95/0.95/0.96 0.84/0.83/0.86 0.89/0.89/0.89
Grade 7 0.96/0.96/0.96 0.96/0.96/0.97 0.89/0.89/0.91 0.91/0.90/0.92
Grade 8 0.93/0.93/0.90 0.96/0.95/0.97 0.89/0.87/0.90 0.93/0.88/0.89
ANALYSIS: CONCLUSIONS
• Results suggest that increasing correlations are not associated with
model anomalies but reflect actual changes occurring with Georgia
schools.
• The results suggest that in the second year of Georgia Milestones
anchored to new achievement standards, schools serving higher
achieving students are transitioning better than schools serving lower
achieving students.
• We welcome any advice on how we might continue to investigate
this phenomenon to reach definitive conclusions.
SCATTER PLOTS:
MEAN PRIOR ACHIEVEMENT BY MEAN SIMEX SGP
SCATTER PLOTS:
MEAN PRIOR ACHIEVEMENT BY YEAR-TO-YEAR INCREASE
IN GROWTH
SCATTER PLOTS:
MEAN PRIOR ACHIEVEMENT BY YEAR-TO YEAR
IMPROVEMENT
SCATTER PLOTS (SUGGESTED BY DEREK):
MEAN PRIOR ACHIEVEMENT BY YEAR-TO YEAR CHANGE IN
SGP
Georgia: Mean Prior Achievement by Growth Difference: ELA 2015 & 2016 (N >= 15)
Correlation = 0.15 Intercept: −0.11
Slope: 1.37
Mean Different SIMEX SGP: 2015−2016
Mean Prior SS: 2014 and 2015
Georgia: Mean Prior Achievement by Growth Difference: Mathematics 2015 & 2016 (N >= 15)
Correlation = 0.07 Intercept: −0.1
Slope: 0.22
Mean Different SIMEX SGP: 2015−2016
Mean Prior SS: 2014 and 2015
Georgia: Mean Prior Achievement by Growth Difference: Science 2015 & 2016 (N >= 15)
Correlation = 0.13 Intercept: 0.03
Slope: 1.28
Mean Different SIMEX SGP: 2015−2016
Mean Prior SS: 2014 and 2015
Georgia: Mean Prior Achievement by Growth Difference: Social Studies 2015 & 2016 (N >= 15)
Correlation = 0.11 Intercept: −0.1
Slope: 0.86
Mean Different SIMEX SGP: 2015−2016
Mean Prior SS: 2014 and 2015
About this copy
Georgia Summary SGP Results — Georgia Department of Education, December 2016.
This is an image-based copy of the original Keynote slides; builds and animations are flattened.