Georgia Summary SGP Results

Georgia Department of Education · December 2016

Georgia Summary SGP Results — Georgia Department of Education, December 2016.
Published

December 16, 2016

Event
State agency
Slides
58
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. GEORGIA SUMMARY SGP RESULTS CORRELATIONS BETWEEN GROWTH AND PRIOR ACHIEVEMENT Damian W Betebenner & Adam VanIwaarden National Center for the Improvement of Educational Assessment Dover, New Hampshire December 15th, 2016
  2. ISSUE: BACKGROUND • Initial SGP analyses with preliminary 2016 ELA and Mathematics EOG data were completed in late July. Preliminary EOC data was also available • As part of validation of results, correlations between mean prior achievement (mean of prior scale score standardized within grade) and mean SGP (unadjusted and SIMEX adjusted) were examined at the school level. • In all but one content area, the correlations increased from 2015 to 2016. • We want to explore whether these increases reflect a change in the system or are a manifestation of the model (e.g., measurement error).
  3. ANALYSIS: RESULTS • From 2015 to 2016 there is a definite trend upward in correlations between prior achievement and growth. • Examination of data (see supplementary scatterplots) rule out outliers/ changes in marginal distributions as the reason. • For example, non-random assignment of students to school by level of achievement is one of the factors for measurement error bias. Yet no marked changes in stratification of schools based upon achievement was observed. • The increasing correlations observed are synonymous with larger increases in mean SGP for schools serving higher achieving students (higher mean achievement) than their counterparts serving lower achieving students.
  4. ANALYSIS: RESULTS • Examination of data (see supplementary scatterplots) rule out outliers/changes in marginal distributions as the reason. • For example, non-random assignment of students to school by level of achievement is one of the factors for measurement error bias. Yet no marked changes in stratification of schools based upon achievement was observed. • The increasing correlations observed are consistent with changes in percent proficient for schools between 2015 and 2016. Schools serving higher achieving students had, on average, higher rates of improvement than schools serving lower achieving students (see supplementary scatterplots)
  5. ANALYSIS: RESULTS • To investigate more broadly, we examined results associated with assessment transitions in 7 states: • 2 non-consortium affiliated states (including Georgia) • 2 PARCC affiliated states • 3 SBAC affiliated states • All summary statistics calculated using a 15 student minimum.
  6. CORRELATIONS: MEAN PRIOR ACHIEVEMENT BY MEAN SGP
  7. GA: SCHOOL LEVEL CORRELATIONS SIMEX ADJUSTED/MEAN PRIOR ACHIEVEMENT 2014/2015/2016 2014 (CRCT/EOCT) 2015 (Milestones) 2016 (Milestones) American Lit 0.28 (427) 0.18 (427) 0.5 (436) Analytic Geometry 0.34 (426) 0.37 (436) 0.45 (346) Biology 0.11(438) 0.21(449) 0.23 (447) Coordinate Algebra 0.23 (649) 0.23 (725) 0.21(246) Economics 0.09 (426) 0.05 (427) 0.29 (428) ELA 0.38 (1,745) 0.3 (1,760) 0.44 (1.742) Grade 9 Lit 0.22 (465) 0.25 (540) 0.36 (584) Mathematics 0.35 (1,744) 0.38 (1,760) 0.42 (1,752) Physical Science 0.22 (465) 0.25 (540) 0.27 (584) Science 0.27 (1,747) 0.3 (1,760) 0.39 (1,752) Social Studies 0.24 (1,747) 0.21(1,760) 0.34 (1,752) US History 0.03 (423) 0.06 (426) 0.18 (431)
  8. GA: SCHOOL LEVEL CORRELATIONS MEAN SIMEX ADJUSTED SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Social Studies Science Grade 4 0.18/0.23/0.26 0.23/0.09/0.29 0.09/0.10/0.18 0.18/0.20/0.32 Grade 5 0.11/0.13/0.20 0.11/0.11/0.24 -0.05/-0.10/0.10 0.06/0.06/0.13 Grade 6 0.27/0.18/0.24 0.28/0.22/0.25 0.08/0.06/0.15 0.16/0.18/0.20 Grade 7 0.35/0.22/0.29 0.23/0.36/0.32 0.10/0.07/0.08 0.04/-0.01/0.06 Grade 8 0.23/0.23/0.30 0.38/0.22/0.34 0.00/-0.07/0.07 0.08/0.06/0.21 Entire School 0.35/0.38/0.42 0.38/0.30/0.44 0.24/0.21/0.34 0.27/0.30/0.39
  9. NON-ALIGNED 1 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.14/0.12/0.02/0.03 0.31/0.23/0.09/0.10 Grade 5 0.13/0.07/0.12/0.13 0.31/0.22/0.19/0.19 Grade 6 0.13/0.03/0.03/0.09 0.13/0.15/0.18/0.06 Grade 7 0.26/0.18/0.14/0.13 0.24/0.22/0.17/0.09 Grade 8 —/0.26/0.15/0.18/ 0.17/0.19/0.17/0.12 Entire School 0.26/0.26/0.23/0.26 0.34/0.40/0.33/0.21
  10. NON-ALIGNED 2 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.06/0.10/0.10 0.08/0.19/0.20 Grade 5 0.09/0.12/0.18 0.12/0.19/0.18 Grade 6 0.13/0.18/0.15 0.15/0.11/0.23 Grade 7 0.24/0.17/0.27 0.19/0.28/0.31 Grade 8 0.13/0.35/0.32 0.17/0.26/0.33 Entire School 0.27/0.33/0.34 0.24/0.35/0.39
  11. PARCC 1 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.32/0.25/0.18 0.50/0.40/0.28 Grade 5 0.34/0.27/0.30 0.42/0.25/0.28 Grade 6 0.34/0.20/0.25 0.35/0.24/0.21 Grade 7 0.20/0.24/0.17 0.33/0.26/0.22 Grade 8 0.31/0.31/0.21 0.30/0.15/0.12 Grade 9 —/0.14/0.13 Entire School 0.43/0.33/0.31 0.58/0.29/0.29
  12. PARCC 2 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.11/0.19/0.12 0.23/0.11/0.16 Grade 5 0.08/0.11/0.21 0.10/0.03/0.18 Grade 6 0.20/0.16/0.26 0.15/0.06/0.22 Grade 7 0.28/0.18/0.19 0.17/0.17/0.19 Grade 8 0.15/0.33/0.37 0.26/0.17/0.17 Grade 9 0.21/0.20/0.25 Grade 10 0.25/0.26/— Entire School 0.34/0.25/0.39 0.33/0.24/0.29
  13. PARCC 3 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.14/0.15/0.31 0.32/0.26/0.27 Grade 5 0.07/0.14/0.20 0.24/0.32/0.32 Grade 6 0.17/0.29/0.31 0.32/0.14/0.31 Grade 7 0.45/0.15/0.36 0.17/0.33/0.49 Grade 8 0.43/0.30/0.58 0.36/0.18/0.51 Grade 9 —/0.07/0.33 Entire School 0.43/0.37/0.53 0.32/0.25/0.45
  14. SBAC 1 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.12/0.20/0.05 0.44/0.15/0.15 Grade 5 0.05/0.07/0.06 -0.05/-0.04/0.17 Grade 6 -0.05/0.15/0.14 0.04/0.23/0.11 Grade 7 -0.19/0.04/0.07 0.08/0.29/0.27 Grade 8 0.07/0.25/0.28 -0.10/0.22/0.12 Entire School 0.07/0.28/0.18 0.13/0.29/0.29
  15. SBAC 2 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 0.14/0.02/0.15 0.17/0.15/0.19 Grade 5 0.24/0.08/0.04 0.34/0.14/0.17 Grade 6 0.25/0.05/0.08 0.41/0.14/0.11 Grade 7 0.15/0.03/-0.04 0.26/0.16/0.09 Grade 8 0.16/0.13/0.15 0.21/0.06/0.21 Entire School 0.40/0.16/0.18 0.32/0.22/0.26
  16. SBAC 3 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PRIOR MEAN SCALE SCORE BY GRADE Mathematics ELA Grade 4 -0.05/0.14/0.00 0.10/-0.04/0.13 Grade 5 0.14/0.00/0.06 0.24/0.01/0.09 Grade 6 -0.10/0.21/0.21 0.32/0.11/0.11 Grade 7 0.22/0.23/0.25 0.30/0.25/0.35 Grade 8 0.10/0.28/0.11 0.09/0.26/0.30 Grade 9 0.36/0.21/0.20 0.30/0.07/0.20 Grade 10 0.25/0.08/0.26 0.20/0.18/-0.03 Grade 11 0.31/0.16/0.23 0.30/0.01/-0.02 Entire School 0.24/0.27/0.27 0.30/0.16/0.30
  17. CORRELATIONS: CURRENT ACHIEVEMENT BY PERCENT FREE/REDUCED LUNCH
  18. GA SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Social Studies Science Grade 4 -0.50/-0.77/-0.75 -0.56/-0.80/-0.78 -0.50/-0.72/-0.71 -0.53/-0.74/-0.73 Grade 5 -0.50/-0.72/-0.70 -0.58/-0.81/-0.79 -0.56/-0.73/-0.70 -0.56/-0.74/-0.72 Grade 6 -0.52/-0.70/-0.68 -0.53/-0.72/-0.71 -0.50/-0.61/-0.64 -0.49/-0.65/-0.64 Grade 7 -0.48/-0.72/-0.71 -0.51/-0.72/-0.74 -0.45/-0.64/-0.67 -0.47/-0.67/-0.69 Grade 8 -0.49/-0.67/-0.68 -0.55/-0.71/-0.69 -0.50/-0.62/-0.64 -0.50/-0.64/-0.51 Entire School -0.48/-0.69/-0.68 -0.52/-0.70/-0.68 -0.51/-0.65/-0.65 -0.52/-0.67/-0.67
  19. PARCC I SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.68/-0.67/-0.70 -0.78/-0.71/-0.71 Grade 5 -0.57/-0.58/-0.66 -0.74/-0.63/-0.65 Grade 6 -0.48/-0.45/-0.49 -0.62/-0.44/-0.43 Grade 7 -0.52/-0.46/-0.48 -0.61/-0.43/-0.43 Grade 8 -0.57/-0.25/-0.34 -0.55/-0.46/-0.45 Entire School -0.73/-0.49/-0.51 -0.75/-0.57/-0.57
  20. PARCC 2 SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.69/-0.58/-0.65 -0.72/-0.63/-0.63 Grade 5 -0.70/-0.64/-0.56 -0.70/-0.68/-0.57 Grade 6 -0.64/-0.64/-0.65 -0.64/-0.62/-0.56 Grade 7 -0.65/-0.55/-0.53 -0.63/-0.56/-0.51 Grade 8 -0.60/-0.37/-0.45 -0.68/-0.50/-0.53 Entire School -0.71/-0.64/-0.64 -0.73/-0.62/-0.62
  21. PARCC 3 SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.73/-0.68/-0.40 -0.74/-0.69/-0.43 Grade 5 -0.76/-0.57/-0.52 -0.66/-0.67/-0.45 Grade 6 -0.73/-0.43/-0.46 -0.66/-0.48/-0.43 Grade 7 -0.53/-0.72/-0.23 -0.51/-0.52/-0.35 Grade 8 -0.43/-0.40/-0.43 -0.54/-0.54/-0.56 Entire School -0.66/-0.68/-0.39 -0.74/-0.62/-0.49
  22. SBAC 1 SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.55/-0.57/-0.61 -0.66/-0.69/-0.69 Grade 5 -0.47/-0.60/-0.61 -0.63/-0.63/-0.59 Grade 6 -0.46/-0.60/-0.65 -0.48/-0.64/-0.63 Grade 7 -0.49/-0.40/-0.60 -0.52/-0.42/-0.58 Grade 8 -0.30/-0.57/-0.43 -0.39/-0.63/-0.44 Entire School -0.59/-0.65/-0.66 -0.71/-0.68/-0.68
  23. SBAC 2 SCHOOL LEVEL CORRELATIONS CURRENT SCALE SCORE TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.42/-0.47/-0.54 -0.34/-0.41/-0.53 Grade 5 -0.45/-0.46/-0.55 -0.52/-0.54/-0.60 Grade 6 -0.36/-0.39/-0.43 -0.38/-0.43/-0.50 Grade 7 -0.41/-0.53/-0.48 -0.36/-0.44/-0.57 Grade 8 -0.48/-0.49/-0.53 -0.40/-0.49/-0.49 Entire School -0.50/-0.52/-0.56 -0.50/-0.54/-0.60
  24. CORRELATIONS: MEAN SGP AND PERCENT FREE/REDUCED LUNCH
  25. GA SCHOOL LEVEL CORRELATIONS MEAN SIMEX ADJUSTED SGP TO CURRENT PERCENT ED 2014/2015/2016 Mathematics ELA Social Studies Science Grade 4 -0.17/-0.41/-0.36 -0.18/-0.19/-0.33 -0.14/-0.21/-0.34 -0.16/-0.20/-0.35 Grade 5 -0.07/-0.20/-0.17 -0.12/-0.15/-0.22 -0.11/-0.04/-0.15 -0.12/-0.10/-0.14 Grade 6 -0.28/-0.28/-0.25 -0.19/-0.22/-0.22 -0.14/-0.10/-0.21 -0.11/-0.18/-0.24 Grade 7 -0.13/-0.21/-0.30 -0.12/-0.30/-0.25 -0.05/-0.08/-0.17 -0.01/-0.06/-0.15 Grade 8 -0.09/-0.09/-0.16 -0.30/-0.19/-0.23 -0.11/-0.01/-0.05 -0.07/-0.05/-0.12 Entire School -0.14/-0.31/-0.29 -0.18/-0.18/-0.27 -0.17/-0.14/-0.26 -0.13/-0.16/-0.26
  26. PARCC I SCHOOL LEVEL CORRELATIONS MEAN SGP TO PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.31/-0.29/-0.24 -0.53/-0.33/-0.26 Grade 5 -0.22/-0.09/-0.21 -0.40/-0.19/-0.10 Grade 6 -0.12/-0.07/-0.12 -0.39/-0.19/-0.10 Grade 7 -0.25/-0.08/0.00 -0.28/-0.04/0.02 Grade 8 -0.24/-0.01/-0.04 -0.11/0.00/0.00 Entire School -0.34/-0.13/-0.18 -0.49/-0.14/-0.07
  27. SBAC 2 SCHOOL LEVEL CORRELATIONS MEAN SGP TO PERCENT ED 2014/2015/2016 Mathematics ELA Grade 4 -0.19/-0.18/-0.14 -0.09/-0.31/-0.18 Grade 5 -0.16/-0.20/-0.17 -0.31/-0.31/-0.16 Grade 6 -0.20/-0.07/-0.08 -0.16/-0.14/-0.05 Grade 7 -0.11/-0.22/-0.06 -0.07/-0.22/-0.09 Grade 8 -0.05/-0.16/-0.12 -0.03/-0.19/-0.20 Entire School -0.21/-0.24/-0.17 -0.20/-0.31/-0.20
  28. CORRELATIONS: MISCELLANEOUS CORRELATIONS
  29. GA STUDENT LEVEL CORRELATIONS PRIOR SCALE SCORE TO CURRENT SCALE SCORE 2014/2015/2016 Mathematics ELA Social Studies Science Grade 4 0.79/0.81/0.84 0.78/0.78/0.82 0.76/0.77/0.77 0.79/0.79/0.79 Grade 5 0.78/0.82/0.84 0.76/0.80/0.83 0.80/0.80/0.80 0.79/0.79/0.80 Grade 6 0.79/0.80/0.84 0.77/0.78/0.84 0.78/0.79/0.80 0.80/0.80/0.80 Grade 7 0.85/0.86/0.87 0.76/0.79/0.84 0.82/0.83/0.84 0.82/0.82/0.83 Grade 8 0.81/0.84/0.80 0.77/0.78/0.83 0.80/0.79/0.82 0.81/0.79/0.78
  30. GA SCHOOL LEVEL CORRELATIONS PRIOR SCALE SCORE TO CURRENT SCALE SCORE 2014/2015/2016 Mathematics ELA Social Studies Science Grade 4 0.89/0.90/0.93 0.94/0.93/0.95 0.85/0.87/0.88 0.92/0.92/0.93 Grade 5 0.87/0.90/0.92 0.95/0.95/0.96 0.88/0.87/0.88 0.91/0.90/0.91 Grade 6 0.89/0.90/0.91 0.95/0.95/0.96 0.84/0.83/0.86 0.89/0.89/0.89 Grade 7 0.96/0.96/0.96 0.96/0.96/0.97 0.89/0.89/0.91 0.91/0.90/0.92 Grade 8 0.93/0.93/0.90 0.96/0.95/0.97 0.89/0.87/0.90 0.93/0.88/0.89
  31. ANALYSIS: CONCLUSIONS • Results suggest that increasing correlations are not associated with model anomalies but reflect actual changes occurring with Georgia schools. • The results suggest that in the second year of Georgia Milestones anchored to new achievement standards, schools serving higher achieving students are transitioning better than schools serving lower achieving students. • We welcome any advice on how we might continue to investigate this phenomenon to reach definitive conclusions.
  32. SCATTER PLOTS: MEAN PRIOR ACHIEVEMENT BY MEAN SIMEX SGP
  43. SCATTER PLOTS: MEAN PRIOR ACHIEVEMENT BY YEAR-TO-YEAR INCREASE IN GROWTH
  50. SCATTER PLOTS: MEAN PRIOR ACHIEVEMENT BY YEAR-TO YEAR IMPROVEMENT
  54. SCATTER PLOTS (SUGGESTED BY DEREK): MEAN PRIOR ACHIEVEMENT BY YEAR-TO YEAR CHANGE IN SGP
  55. Georgia: Mean Prior Achievement by Growth Difference: ELA 2015 & 2016 (N >= 15) Correlation = 0.15 Intercept: −0.11 Slope: 1.37 Mean Different SIMEX SGP: 2015−2016 Mean Prior SS: 2014 and 2015
  56. Georgia: Mean Prior Achievement by Growth Difference: Mathematics 2015 & 2016 (N >= 15) Correlation = 0.07 Intercept: −0.1 Slope: 0.22 Mean Different SIMEX SGP: 2015−2016 Mean Prior SS: 2014 and 2015
  57. Georgia: Mean Prior Achievement by Growth Difference: Science 2015 & 2016 (N >= 15) Correlation = 0.13 Intercept: 0.03 Slope: 1.28 Mean Different SIMEX SGP: 2015−2016 Mean Prior SS: 2014 and 2015
  58. Georgia: Mean Prior Achievement by Growth Difference: Social Studies 2015 & 2016 (N >= 15) Correlation = 0.11 Intercept: −0.1 Slope: 0.86 Mean Different SIMEX SGP: 2015−2016 Mean Prior SS: 2014 and 2015