New Jersey PARCC 2016 SGP Analyses

New Jersey Department of Education · March 2016

New Jersey PARCC 2016 SGP Analyses — New Jersey Department of Education, March 2016.
Published

March 4, 2016

Event
State agency
Slides
23
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. NEW JERSEY PARCC 2016 SGP ANALYSES Damian W Betebenner National Center for the Improvement of Educational Assessment Dover, NH March 4th, 2016
  2. BACKGROUND • 2016 SGP analyses of New Jersey PARCC data were conducted in December 2015/January 2016 • Analyses were conducted using PARCC theta values (instead of scaled scores) and scale scores for NJASK as priors to avoid any potential issues with ceiling/ oor effects. • Of the 4 PARCC states we work with on SGP analyses, 3 are using theta values. • Results from SGP analyses (only ELA and Mathematics) are used, in part, for educator evaluation. fl
  3. RESULTS: DATA PREPARATION • 2016 NJ PARCC SGPs were calculated for: • ELA and Mathematics EOGTs • Algebra I, Algebra II and Geometry EOCTs. • Valid cases by Grade and Content area: Content 3 4 5 6 7 8 9 10 Content Area Area Algebra I 91,716 ELA 95,288 93,830 94,651 92,668 90,301 88,479 82,683 73,165 Algebra II 58,022 Math 95,997 94,555 95,360 93,239 87,375 58,097 Geometry 71,130
  4. RESULTS: DATA ANALYSIS • EOGTs are analyzed based upon same content area and consecutive grades. • EOCTs are analyzed based upon same/similar content area (e.g., Mathematics to Algebra I) but not necessarily consecutive grades: • 2013 Math, Grade 8 to 2015 Algebra I; 2014 Math, Grade 7 to 2015 Algebra 1; 2014 Math, Grade 8 to 2015 Algebra 1 • 2013 Math, Grade 8 to 2015 Algebra II; • 2013 Math, Grade 8 to 2015 Geometry: 2014 Math, Grade 8 to 2015 Geometry
  5. RESULTS: SGP PROPERTIES • SGPs are uncorrelated with prior achievement at the student level (i.e., a conditional probability) • SGPs are correlated with current achievement. I.e., higher growth is associated with higher achievement and vice-versa. • SGPs for students are uncorrelated from year to year.
  6. RESULTS: DATA ANALYSIS • 2016 NJ PARCC median (mean) SGPs by content area and grade: Content 4 5 6 7 8 9 10 Area ELA 50 (50) 50 (49.9) 50 (49.8) 50 (49.8) 50 (49.8) 50 (50.1) 50 (50.2) Math 50 (50) 50 (50.1) 50 (50.1) 50 (50.2) 50 (50.2) 50 (50.2) Content Area Algebra I 50 (50.2) Algebra II 50 (50.0) Geometry 50 (50.1)
  7. GOODNESS OF FIT (MATHEMATICS) Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 Mathematics SGP, Grade 4 (N = 90,216) 2015 Mathematics SGP, Grade 5 (N = 91,196) 2015 Mathematics SGP, Grade 6 (N = 88,800) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) Prior Achievement Level Prior Achievement Level Prior Achievement Level Advanced (35.8%) Advanced (39%) Advanced (41.5%) Proficient (23.3%) Proficient (24%) Proficient (19.2%) Part Proficient (40.9%) Part Proficient (36.9%) Part Proficient (39.3%) Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−10,−3.213) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−3.533) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−3.841) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 27 Extremes (Range) Extremes (Range) Extremes (Range) Highest OSS: [3.754,4.065) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25 Highest OSS: [3.883,4.176) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25 Highest OSS: [3.854,4.393) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[100,170] 9.64 10.10 9.85 10.29 9.80 10.15 9.71 10.01 10.15 10.31 (8.6%) 100 1st/[100,172) 9.48 10.10 10.03 10.16 10.02 10.04 9.61 9.90 9.99 10.68 (9.7%) 100 1st/[100,181) 9.46 10.12 10.25 10.01 9.98 10.05 9.83 9.71 9.64 10.95 (8.8%) 100 2nd/(170,193] 9.19 9.62 9.90 9.91 10.09 10.25 10.50 9.89 9.83 10.81 (10.4%) 2nd/[172,191) 9.63 10.05 9.43 9.69 9.81 9.56 10.43 10.12 10.36 10.92 (9.8%) 2nd/[181,200) 9.66 10.06 9.78 10.41 10.13 9.55 9.88 10.27 9.85 10.39 (10.3%) Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * 3rd/(193,207] 9.83 10.40 10.19 9.99 9.88 9.45 9.63 10.01 9.76 10.84 (9.3%) 3rd/[191,206) 9.80 10.05 10.40 10.04 10.06 9.90 9.60 10.04 9.71 10.40 (9.9%) 3rd/[200,212) 9.64 9.88 10.17 9.59 9.91 9.99 10.14 9.96 9.82 10.90 (9.2%) Empirical SGP Distribution Empirical SGP Distribution Empirical SGP Distribution 4th/(207,221] 9.37 10.30 10.29 9.97 9.97 10.04 9.90 10.07 10.31 9.75 (11.1%) 4th/[206,216) 9.63 10.15 10.04 9.86 10.17 10.45 9.64 9.46 10.20 10.40 (8.4%) 4th/[212,225) 9.42 10.28 9.77 10.36 10.14 10.06 9.83 10.02 9.76 10.36 (10.4%) 5th/(221,233] 9.85 9.60 9.72 10.12 10.32 10.27 10.05 9.75 9.67 10.65 (9.5%) 5th/[216,232) 9.48 9.84 9.69 10.49 9.94 9.59 10.34 9.99 9.78 10.87 (12.2%) 5th/[225,235) 9.66 9.78 10.46 9.61 9.50 9.85 9.87 10.18 10.22 10.87 (8.9%) 6th/(233,250] 8.78 9.94 9.60 9.71 9.72 10.16 10.26 10.38 10.25 11.20 (10.3%) 6th/[232,244) 9.23 10.11 10.18 9.46 9.52 10.28 9.73 10.35 10.32 10.83 (9.7%) 6th/[235,246) 9.18 10.10 9.38 9.81 10.36 9.79 10.43 9.86 10.29 10.80 (9.6%) 7th/(250,256] 10.64 10.09 10.81 10.22 10.23 9.35 9.62 9.79 9.65 9.59 (7.3%) 7th/[244,259) 9.67 9.85 9.93 10.03 10.28 9.91 10.28 9.73 9.93 10.39 (10%) 7th/[246,258) 10.15 9.86 10.47 9.90 9.61 9.83 10.08 10.18 9.61 10.32 (10.5%) 8th/(256,274] 9.36 10.06 9.84 10.10 9.95 10.46 9.90 9.89 9.98 10.44 (11.1%) 8th/[259,270) 9.57 10.29 9.95 10.10 10.04 10.01 9.84 9.96 9.74 10.50 (7%) 8th/[258,273) 9.50 10.12 9.81 10.10 9.99 10.30 9.89 9.82 10.15 10.33 (11.1%) 9th/(274,300) 9.47 10.01 10.14 9.92 10.05 9.78 10.06 10.00 10.09 10.46 (11.1%) 9th/[270,291) 9.10 9.78 9.95 9.84 9.69 9.85 10.14 10.16 10.33 11.15 (10.3%) 9th/[273,296) 9.06 9.60 9.76 9.81 9.88 9.97 10.07 10.34 10.32 11.18 (10.8%) 10th/300 9.40 9.97 9.88 9.98 10.03 9.91 10.14 10.00 10.13 10.56 (11.4%) 10th/[291,300] 9.64 10.02 10.03 9.98 10.06 10.00 9.85 10.01 10.07 10.34 (13.1%) 10th/[296,300] 9.71 10.18 10.02 9.90 10.03 10.01 10.05 9.79 9.99 10.34 (10.2%) *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 Mathematics SGP, Grade 7 (N = 83,277) 2015 Mathematics SGP, Grade 8 (N = 55,093) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) Prior Achievement Level Prior Achievement Level Advanced (45.5%) Advanced (46.7%) Proficient (20.3%) Proficient (46.5%) Part Proficient (34.2%) Part Proficient (6.8%) Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−10,−5.452) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 30 Lowest OSS: [−10,−4.4) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 20 Extremes (Range) Extremes (Range) Highest OSS: [3.552,3.792) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 24 Highest OSS: [2.646,2.911) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 24.0 8.0 64.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[100,178) 9.67 9.91 9.93 9.40 9.83 10.20 10.02 10.18 9.91 10.96 (9.7%) 100 1st/[100,151) 9.76 9.90 10.02 9.79 9.65 9.92 9.69 10.18 9.90 11.19 (7.9%) 100 2nd/[178,194) 9.51 9.85 9.87 10.27 9.98 9.60 10.28 10.02 9.84 10.77 (8.7%) 2nd/[151,169) 9.64 10.11 9.72 9.85 9.90 9.87 10.11 10.01 9.80 10.97 (11.2%) ore Decile/Range * ore Decile/Range * 3rd/[194,209) 9.71 10.20 9.98 9.78 9.95 10.17 9.94 9.89 10.01 10.38 (11.5%) 3rd/[169,180) 9.87 9.23 10.22 9.99 9.99 10.18 9.93 10.14 9.86 10.60 (9.6%) GP Distribution GP Distribution 4th/[209,218) 9.44 9.69 9.74 10.02 9.81 9.97 9.99 9.95 10.01 11.38 (8.6%) 4th/[180,191) 9.67 10.65 9.82 10.13 9.63 9.27 9.96 9.56 10.29 11.01 (10.5%) 5th/[218,228) 9.47 9.84 9.78 9.98 9.74 10.08 10.09 10.68 10.04 10.30 (9.8%) 5th/[191,200) 9.60 9.77 9.90 9.15 10.24 10.04 10.17 10.07 9.92 11.14 (7.3%)
  8. GOODNESS OF FIT (ELA) Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 ELA SGP, Grade 4 (N = 89,960) 2015 ELA SGP, Grade 5 (N = 90,955) 2015 ELA SGP, Grade 6 (N = 88,681) SGP Deciles by Prior Achievement Level (ELA) SGP Deciles by Prior Achievement Level (ELA) SGP Deciles by Prior Achievement Level (ELA) Prior Achievement Level Prior Achievement Level Prior Achievement Level Advanced (57%) Advanced (54.7%) Advanced (62.1%) Proficient (36.8%) Proficient (33.4%) Proficient (39.3%) Part Proficient (4.5%) Part Proficient (3.7%) Part Proficient (8.5%) Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−5.805,−3.508) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 26 Lowest OSS: [−5.503,−3.457) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−5.926,−3.459) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Extremes (Range) Extremes (Range) Extremes (Range) Highest OSS: [3.868,4.174) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 23 Highest OSS: [3.655,3.861) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.0 88.0 N = 25 Highest OSS: [3.884,4.094) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 16.0 84.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[105,176) 9.81 10.07 9.92 10.14 9.68 9.82 9.95 9.88 10.08 10.66 (9.6%) 100 1st/[100,173) 10.23 10.09 10.07 10.13 9.92 9.79 9.77 9.79 9.76 10.46 (9.8%) 100 1st/[100,169) 9.79 10.61 10.27 9.93 9.97 9.89 9.85 9.46 9.89 10.35 (8.3%) 100 2nd/[176,185) 9.56 10.30 9.84 9.51 9.92 9.99 9.99 10.02 10.19 10.67 (8.2%) 2nd/[173,183) 9.61 10.40 10.16 9.94 9.54 10.27 10.20 10.03 9.69 10.17 (8.5%) 2nd/[169,181) 10.63 10.19 9.47 9.63 9.66 10.23 9.76 10.40 9.50 10.53 (9%) Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * 3rd/[185,194) 9.05 9.60 10.14 10.05 10.41 10.37 10.29 10.24 9.76 10.10 (11.3%) 3rd/[183,189) 9.86 9.83 10.23 9.82 10.08 9.94 9.79 9.92 10.20 10.33 (7.5%) 3rd/[181,192) 9.91 10.24 9.99 10.12 9.62 9.95 10.05 9.83 9.88 10.40 (12.4%) Empirical SGP Distribution Empirical SGP Distribution Empirical SGP Distribution 4th/[194,201) 9.95 10.28 10.68 10.99 10.30 9.64 9.62 9.48 9.38 9.68 (9.1%) 4th/[189,200) 9.39 10.02 9.36 9.98 10.21 9.90 10.08 10.19 10.20 10.68 (13.4%) 4th/[192,200) 9.65 9.91 10.62 9.51 10.46 9.52 9.91 9.97 9.96 10.49 (7%) 5th/[201,207) 9.40 10.40 9.72 9.54 9.64 9.95 10.17 9.98 10.16 11.04 (10.1%) 5th/[200,205) 10.18 10.03 10.32 10.04 10.03 10.02 10.24 9.66 9.63 9.86 (10.4%) 5th/[200,206) 9.94 10.11 9.99 10.09 10.01 9.81 10.13 9.84 10.03 10.05 (11.4%) 6th/[207,214) 9.24 9.55 9.82 9.79 10.14 9.99 9.84 10.35 10.41 10.87 (10.9%) 6th/[205,208) 9.62 10.17 9.68 9.76 10.17 9.53 9.90 9.90 10.01 11.26 (5.4%) 6th/[206,212) 9.75 10.28 9.78 9.57 9.87 9.91 9.87 10.47 9.81 10.71 (7.9%) 7th/[214,218) 9.62 9.76 9.20 9.76 9.86 10.35 10.20 10.18 10.53 10.53 (5.5%) 7th/[208,215) 9.62 9.92 10.36 9.96 10.12 10.00 9.50 10.15 9.88 10.48 (10.7%) 7th/[212,221) 9.58 9.99 9.81 10.21 10.10 10.50 9.76 9.80 9.89 10.37 (12.1%) 8th/[218,225) 9.83 10.38 10.41 10.19 9.67 10.14 9.76 9.84 9.63 10.15 (11%) 8th/[215,222) 9.49 10.13 9.92 9.95 9.40 10.11 10.08 10.14 10.26 10.51 (10.3%) 8th/[221,231) 9.41 9.88 10.22 9.90 9.89 9.68 10.41 9.93 9.92 10.77 (11.4%) 9th/[225,239) 9.47 9.79 9.97 10.01 10.21 9.83 9.90 9.99 10.10 10.73 (14%) 9th/[222,233) 9.53 10.02 9.80 9.94 10.44 9.94 10.03 9.74 9.98 10.59 (12.7%) 9th/[231,242) 9.63 10.04 9.79 10.02 9.92 9.92 9.77 10.36 9.96 10.59 (9.3%) 10th/[239,300] 9.36 9.97 9.99 9.94 10.03 9.98 10.36 9.98 9.97 10.42 (10.3%) 10th/[233,300] 9.42 9.89 10.03 10.02 9.91 10.02 10.03 10.22 9.99 10.49 (11.2%) 10th/[242,300] 9.51 10.08 9.67 9.96 9.78 9.97 10.34 9.89 10.14 10.65 (11.1%) *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 ELA SGP, Grade 7 (N = 86,673) 2015 ELA SGP, Grade 8 (N = 85,473) 2015 ELA SGP, Grade 9 (N = 74,503) 2015 ELA SGP, Grade 10 (N = 64,693) SGP Deciles by Prior Achievement Level (ELA) SGP Deciles by Prior Achievement Level (ELA) SGP Deciles by Prior Achievement Level (ELA) SGP Deciles by Prior Achievement Level (ELA) Prior Achievement Level Prior Achievement Level Prior Achievement Level Prior Achievement Level Advanced (49.5%) Advanced (58.8%) Advanced (67.6%) Advanced (72.1%) Proficient (34.5%) Proficient (31.9%) Proficient (19.3%) Proficient (16%) Part Proficient (9.3%) Part Proficient (16%) Part Proficient (13.1%) Part Proficient (11.9%) Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−4.547,−3.412) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−6,−3.592) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−6,−3.645) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−6,−3.589) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Extremes (Range) Extremes (Range) Extremes (Range) Extremes (Range) Highest OSS: [3.766,3.972) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.0 88.0 N = 25 Highest OSS: [3.647,3.83) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.0 96.0 N = 25 Highest OSS: [3.651,3.902) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 16.0 80.0 N = 25 Highest OSS: [3.714,3.999) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 20.0 76.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[100,176) 10.10 10.14 10.11 10.11 10.17 9.26 10.18 9.51 9.67 10.75 (10%) 100 1st/[100,169) 10.40 10.41 10.10 10.09 10.10 9.85 9.68 9.72 9.23 10.41 (9.4%) 100 1st/[112,188) 10.25 10.15 10.54 9.68 9.64 10.00 9.89 9.60 9.81 10.44 (9.9%) 100 1st/[113,193) 9.77 10.13 10.32 9.88 9.77 9.56 9.74 10.21 10.29 10.33 (9.8%) 100 2nd/[176,188) 10.31 10.23 10.32 10.05 9.89 10.05 9.63 9.62 9.85 10.04 (9.7%) 2nd/[169,182) 10.62 9.86 10.12 9.75 9.01 10.07 9.99 9.70 10.16 10.72 (9.4%) 2nd/[188,200) 9.77 10.62 9.61 9.68 10.09 9.99 9.88 9.57 9.92 10.86 (9.4%) 2nd/[193,203) 9.85 10.55 9.82 10.07 9.28 10.01 10.23 9.35 9.91 10.94 (9.1%) Score Decile/Range * Score Decile/Range * Score Decile/Range * Score Decile/Range * 3rd/[188,197) 9.94 9.33 9.80 9.78 10.09 9.81 10.22 10.42 10.04 10.58 (9.6%) 3rd/[182,193) 10.48 10.24 9.95 9.32 10.31 9.82 9.70 10.07 9.49 10.61 (10.1%) 3rd/[200,209) 9.49 9.71 10.39 10.07 9.66 10.04 9.94 9.80 10.22 10.69 (10.5%) 3rd/[203,210) 9.75 9.72 9.82 9.49 10.74 10.08 9.67 10.09 10.08 10.57 (10.3%) al SGP Distribution al SGP Distribution al SGP Distribution al SGP Distribution 4th/[197,205) 10.03 10.52 9.95 10.24 9.78 9.70 9.97 9.86 9.64 10.31 (9.8%) 4th/[193,203) 9.85 9.95 9.74 10.72 10.20 10.04 9.88 9.53 9.57 10.51 (11%) 4th/[209,215) 10.08 9.79 9.99 10.14 10.60 9.65 9.61 10.27 9.79 10.08 (9.1%) 4th/[210,215) 9.06 10.21 9.74 10.43 9.71 9.81 9.94 9.94 9.78 11.38 (8.6%) 5th/[205,213) 9.93 10.33 9.98 9.79 10.05 10.05 9.51 9.97 9.95 10.43 (10.3%) 5th/[203,211) 9.79 10.17 9.88 9.60 9.82 9.59 9.81 9.95 10.03 11.35 (8%) 5th/[215,222) 9.35 10.22 9.61 9.90 9.80 10.08 9.89 9.57 9.96 11.62 (11%) 5th/[215,221) 9.65 10.48 10.07 10.10 9.70 9.35 9.83 9.63 10.30 10.89 (11%) 6th/[213,219) 9.98 9.85 9.73 10.01 9.94 9.98 9.93 9.92 10.38 10.28 (10.1%) 6th/[211,220) 9.73 10.10 10.53 9.34 9.47 9.83 10.20 10.24 10.38 10.18 (10.4%) 6th/[222,228) 9.01 9.50 9.84 10.02 9.89 9.69 10.34 10.20 10.15 11.37 (8.2%) 6th/[221,227) 9.06 9.18 10.15 9.82 9.90 10.21 10.68 9.91 9.96 11.12 (10.3%)
  9. GOODNESS OF FIT (ALGEBRA I) Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 Algebra I SGP, Grade EOCT (N = 3,279) 2015 Algebra I SGP, Grade EOCT (N = 23,047) 2015 Algebra I SGP, Grade EOCT (N = 36,250) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) Advanced (26.8%) Prior Achievement Level Prior Achievement Level Prior Achievement Level Advanced (35.4%) Advanced (46.6%) Proficient (4.3%) Proficient (70.8%) Proficient (31.6%) Part Proficient (60.3%) Part Proficient (21.7%) Part Proficient (2.5%) Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−10,−3.891) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−2.301) 96.0 0.0 0.0 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−5.127) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Extremes (Range) Extremes (Range) Extremes (Range) Highest OSS: [0.9944,1.269) 0.0 0.0 0.0 4.0 0.0 4.0 4.0 8.0 4.0 16.0 20.0 40.0 N = 25 Highest OSS: [3.831,4.264) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25 Highest OSS: [2.703,2.906) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 20.0 36.0 44.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[100,145) 9.21 11.18 10.53 10.53 9.21 10.86 9.21 8.55 10.20 10.53 (9.3%) 100 1st/[100,214) 9.86 10.00 10.13 9.95 10.04 9.78 9.86 10.17 10.13 10.08 (9.9%) 100 1st/[100,165) 9.79 9.67 10.62 9.73 9.85 10.42 10.06 9.64 9.88 10.33 (9.3%) 100 2nd/[145,154) 11.52 10.78 10.41 9.67 8.18 8.92 11.15 10.78 10.41 8.18 (8.2%) 2nd/[214,228) 9.74 10.27 10.03 10.03 9.99 10.76 9.99 9.41 9.55 10.23 (9%) 2nd/[165,181) 9.68 10.51 9.49 10.51 10.16 9.36 9.36 10.35 9.90 10.70 (8.7%) Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * 3rd/[154,163) 9.82 8.73 6.55 10.91 11.27 11.27 10.91 10.55 9.09 10.91 (8.4%) 3rd/[228,240) 9.28 10.00 9.32 9.44 10.31 9.56 10.27 10.87 10.39 10.55 (10.9%) 3rd/[181,195) 9.00 10.12 9.71 9.46 10.09 10.25 10.73 10.02 10.27 10.35 (10.8%) Empirical SGP Distribution Empirical SGP Distribution Empirical SGP Distribution 4th/[163,174) 8.81 9.69 12.78 9.69 11.23 9.69 7.71 10.57 9.25 10.57 (13.8%) 4th/[240,250) 9.29 9.33 11.21 10.25 9.33 9.51 9.99 9.77 10.47 10.86 (9.9%) 4th/[195,206) 9.74 9.28 10.01 10.16 10.24 10.01 9.83 10.27 10.10 10.36 (9.4%) 5th/[174,181) 11.26 7.28 9.93 11.59 8.94 7.95 12.58 9.60 10.60 10.26 (9.2%) 5th/[250,252) 9.70 10.16 9.24 10.39 10.32 10.01 10.85 9.85 9.78 9.70 (5.6%) 5th/[206,216) 10.17 10.53 10.17 10.45 9.45 10.04 9.58 9.68 9.58 10.35 (10.7%) 6th/[181,188) 9.83 11.53 10.17 10.51 9.15 12.20 9.15 10.17 9.49 7.80 (9%) 6th/[252,262) 10.39 10.32 10.18 10.39 10.14 10.32 9.40 9.72 9.15 10.00 (12.3%) 6th/[216,223) 8.41 10.48 10.19 9.21 10.70 9.83 10.34 10.12 9.94 10.77 (7.6%) 7th/[188,195) 10.42 11.11 5.90 8.68 9.03 8.68 13.89 10.76 11.46 10.07 (8.8%) 7th/[262,267) 9.24 10.52 9.64 9.51 10.45 10.72 9.78 8.90 11.13 10.11 (6.4%) 7th/[223,234) 9.93 9.68 9.66 10.05 9.76 9.98 9.95 10.12 10.25 10.61 (11.3%) 8th/[195,208) 9.05 10.44 11.37 10.44 8.58 11.14 8.82 9.28 9.51 11.37 (13.1%) 8th/[267,279) 8.64 10.12 9.83 9.91 9.58 9.54 10.09 10.85 9.76 11.68 (12%) 8th/[234,250) 9.31 9.83 10.51 10.09 9.73 10.35 10.12 10.04 10.04 9.99 (10.6%) 9th/[208,219) 10.76 10.07 10.07 9.72 12.50 9.03 7.64 11.11 9.03 10.07 (8.8%) 9th/[279,295) 9.78 9.01 9.86 9.94 10.18 10.02 10.59 9.90 10.38 10.34 (10.7%) 9th/[250,259) 9.56 10.04 9.89 10.43 10.58 9.56 9.77 9.65 9.77 10.73 (9.2%) 10th/[219,300] 10.99 10.72 9.38 9.92 9.38 10.46 10.99 10.19 8.85 9.12 (11.4%) 10th/[295,300] 9.72 10.41 10.28 10.05 10.01 10.08 9.72 9.82 9.75 10.18 (13.2%) 10th/[259,300] 9.67 10.03 9.83 9.78 9.83 10.14 10.03 10.20 10.11 10.38 (12.5%) *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
  10. GOODNESS OF FIT (GEOMETRY & ALGEBRA II) Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives 2015 Geometry SGP, Grade EOCT (N = 28,294) 2015 Geometry SGP, Grade EOCT (N = 15,704) 2015 Algebra II SGP, Grade EOCT (N = 15,501) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) SGP Deciles by Prior Achievement Level (Mathematics) Advanced (22.6%) Advanced (28.8%) Prior Achievement Level Prior Achievement Level Prior Achievement Level Advanced (50.8%) Proficient (2.2%) Proficient (5.1%) Proficient (31.8%) Part Proficient (75.2%) Part Proficient (66.1%) Part Proficient (17.3%) Median Student Growth Percentile Median Student Growth Percentile Median Student Growth Percentile Ceiling / Floor Effects Test Ceiling / Floor Effects Test Ceiling / Floor Effects Test Student Growth Percentile Range Student Growth Percentile Range Student Growth Percentile Range Current Observed Score Current Observed Score Current Observed Score 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to 5 9 to to to to to to to to to to to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95 Lowest OSS: [−10,−4.787) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−2.517) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [−10,−3.344) 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Extremes (Range) Extremes (Range) Extremes (Range) Highest OSS: [2.305,2.493) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 28.0 20.0 48.0 N = 25 Highest OSS: [3.018,3.224) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 N = 25 Highest OSS: [3.047,3.176) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.0 88.0 N = 25 Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 9 to to to to to to to to to 9 to to to to to to to to to 9 to to to to to to to to to to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1 to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1st/[100,167) 9.83 10.28 9.96 10.12 9.75 9.83 9.83 9.92 9.96 10.52 (8.7%) 100 1st/[104,223] 10.37 10.01 9.79 10.23 9.72 10.08 10.08 10.01 10.01 9.72 (8.7%) 100 1st/[104,212] 10.27 10.20 10.06 9.99 9.78 10.06 10.34 9.78 10.06 9.44 (9.3%) 100 2nd/[167,185) 9.63 9.92 9.92 9.85 10.30 9.72 10.24 9.85 10.04 10.52 (11%) 2nd/(223,238] 9.59 10.21 9.66 9.66 10.21 9.32 10.21 10.21 10.56 10.35 (9.2%) 2nd/(212,229] 9.19 9.57 9.89 10.20 10.08 10.01 10.01 10.58 9.38 11.09 (10.2%) Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * Prior Scale Score Decile/Range * 3rd/[185,195) 9.57 9.69 9.89 10.02 9.41 10.42 10.34 10.58 10.26 9.81 (8.8%) 3rd/(238,251] 9.54 9.88 10.81 10.64 9.94 10.58 9.54 9.31 9.13 10.64 (11%) 3rd/(229,241] 10.07 10.01 10.44 9.70 9.95 9.39 10.57 9.02 10.38 10.44 (10.4%) Empirical SGP Distribution Empirical SGP Distribution Empirical SGP Distribution 4th/[195,205) 9.11 9.74 10.47 10.33 10.30 10.27 9.41 9.57 10.33 10.47 (10.7%) 4th/(251,259] 10.04 9.89 10.04 9.53 9.82 9.60 9.89 10.97 9.82 10.40 (8.8%) 4th/(241,251] 9.08 10.29 10.59 9.53 9.98 10.59 8.93 10.89 10.51 9.61 (8.6%) 5th/[205,212) 9.28 10.09 10.26 9.88 10.22 10.05 9.75 10.05 9.50 10.90 (8.3%) 5th/(259,270] 8.91 10.00 9.26 10.00 10.17 10.86 10.23 9.77 10.46 10.34 (11.1%) 5th/(251,260] 10.35 10.89 9.10 10.65 9.93 9.81 10.05 9.69 9.87 9.63 (10.8%) 6th/[212,222) 9.98 10.15 9.18 9.78 10.21 9.67 10.41 10.09 10.27 10.27 (12.4%) 6th/(270,276] 10.69 8.99 10.48 8.99 9.95 9.21 10.37 9.84 10.58 10.90 (6%) 6th/(260,266] 9.29 9.08 10.15 9.51 10.26 10.47 10.68 9.08 10.36 11.11 (6%) 7th/[222,229) 9.61 9.70 10.52 10.09 9.18 10.05 10.96 9.87 8.88 11.13 (8.2%) 7th/(276,290] 9.53 9.82 10.25 10.68 10.20 9.82 9.77 10.11 9.77 10.06 (13.3%) 7th/(266,279] 9.08 9.64 10.15 10.46 9.85 10.05 9.18 10.51 9.64 11.43 (12.6%) 8th/[229,241) 9.03 10.23 10.45 9.95 10.42 9.85 9.25 10.54 10.48 9.79 (11.2%) 8th/(290,300) 9.25 11.52 9.25 9.54 9.54 10.10 10.20 9.82 10.01 10.76 (6.7%) 8th/(279,287] 9.08 9.08 9.59 8.88 10.00 9.90 11.12 11.12 10.61 10.61 (6.3%) 9th/[241,255) 10.41 10.04 9.31 10.41 10.25 10.37 9.72 9.15 9.88 10.45 (8.7%) 9th/300 9.54 9.98 10.03 10.03 10.00 10.03 9.98 10.03 9.98 10.41 (25%) 9th/(287,300) 10.88 10.78 10.41 10.67 10.15 9.88 9.57 9.04 9.78 8.83 (12.3%) 10th/[255,300] 9.49 9.94 10.14 9.82 9.67 9.91 10.23 10.14 10.11 10.55 (12%) 10th/300 8.83 9.70 9.75 9.65 9.89 10.18 10.18 10.42 10.08 11.33 (13.4%) *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
  11. RESULTS: DATA ANALYSIS • 2016 NJ PARCC median (mean) SGPs by content area and grade: Content 4 5 6 7 8 9 10 Area ELA 50 (50) 50 (49.9) 50 (49.8) 50 (49.8) 50 (49.8) 50 (50.1) 50 (50.2) Math 50 (50) 50 (50.1) 50 (50.1) 50 (50.2) 50 (50.2) 50 (50.2) Content Area Algebra I 50 (50.2) Algebra II 50 (50.0) Geometry 50 (50.1)
  12. RESULTS: DATA ANALYSIS New Jersey School Performance PARCC Growth & Prior Achievement 2015 PARCC ELA Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 2014 Prior Percent at/above Proficient 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2015 Median Student Growth Percentile
  13. RESULTS: DATA ANALYSIS New Jersey School Performance PARCC Growth & Achievement 2015 PARCC ELA Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2015 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2015 Median Student Growth Percentile
  14. RESULTS: DATA ANALYSIS New Jersey School Performance PARCC Growth & Achievement 2015 PARCC Mathematics Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2015 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2015 Median Student Growth Percentile
  15. RESULTS: DATA ANALYSIS New Jersey School Performance PARCC Growth & Prior Achievement 2015 PARCC Mathematics Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 2014 Prior Percent at/above Proficient 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2015 Median Student Growth Percentile
  16. APPENDIX: THE IRRELEVANCE OF GAIN SCORES
  17. “It is a common misconception that to measure student growth in education, the subject matter and grades over which growth is examined must be on the same scale—referred to as a vertical scale. Not only is a vertical scale not necessary, but its existence obscures fundamental concepts necessary to understand growth.” Me Norm- and Criterion-Referenced Student Growth
  18. “If you can’t solve a problem, then there is an easier problem you can solve: nd it.” George Pólya Mathematical Discovery on Understanding, Learning, and Teaching Problem Solving, Volume I fi
  20. A Math Story Problem • Whose high jump changed more? Explain your answer. • Whose high jump change is more remarkable? Explain your answer. • Which high jumper improved more? Explain your answer.
  21. A Math Story Problem Properties that don’t hold with simple examples will not hold in complicated situations. This simple example illustrates how minimally helpful a scale is to understanding change. • A vertical scale allows one to answer a very narrow question: Who changed more? • Remarkable & improvement need not be synonymous with change. • This example (ratio scale, no measurement error) illustrates that a vertical scale is of little help in answering substantive questions.
  22. “One Year’s Growth” If “one year’s growth” is to be useful as a value that makes points on the score scale more meaningful, it would be useful to know if the empirical average value of “one year’s growth” is very different for students at different levels of the score scale or from different demographic backgrounds. Because values for “one year’s growth” are not currently available, the answers to these questions are unknown. David Thissen Validity Issues Involved in Cross-Grade Statements About NAEP Results