Utah State Office of Education
Technical Advisory Committee
Salt Lake City, Utah
January 13-14, 2016
“Sufficient Growth”
(2) (a) For the purpose of determining whether a student demonstrates sufficient growth in
the 2014-15 school year, or a succeeding school year, in language arts, mathematics, or
science as provided in Section 53A-1-1107, the board shall establish a year 2 growth target
for a student for each statewide assessment the student takes.
(b) A student demonstrates sufficient growth if the student's scale score on a
statewide assessment in year 2 is equal to or exceeds the year 2 growth target established
pursuant to Subsections (2)(c) and (2)(d).
(c) The board shall establish a year 2 growth target for each cohort of students with
the same scale score on a particular statewide assessment in year 1.
(d) (i) The board shall establish a year 2 growth target based on actual student growth
in the 2014-15 school year as measured by statewide assessments administered at the end of
the 2013-14 and 2014-15 school years.
(ii) Among a cohort of students with the same scale score on a particular statewide
assessment in the 2013-14 school year, the scale score of the student who scores on a
similar statewide assessment in the 2014-15 school year, at a percentile determined by the
board in rule, becomes the year 2 growth target for statewide assessments administered in
the 2014-15 school year and succeeding years for any student with a year 1 scale score that
is the same as the cohort's scale score in the 2013-14 school year.
UT Technical Advisory Committee_Jan. 13-14, 2016 2
Options Discussed by TAC, PAC, and State Board
1. Tie the target a student growth percentile (SGP) of 30
2. Set the next-year target scale score to be the cut score one level up from
the current-year performance level (or the cut score for the highest
performance level for students already achieving at this level)
3. Set next-year target scale scores separately for each current-year scale
score deciles by calculating the average gain score for that decile based
on prior years’ data, and adding that to the current-year scale score for
every student.
4. Set the next-year target scale score to the Adequate Growth Percentile
(AGP) necessary to move to the next performance level over a specified
number of years or remain in the top performance level for a specified
number of years
5. Set sufficient growth to an SGP of 40 for each student
UT Technical Advisory Committee_Jan. 13-14, 2016 3
Preferred Option: SGP = 40
• Set the target at the 40th growth percentile for each
student
• Determine the percentage of students in each school
meeting the growth target
• Follow the Grading Schools aggregation rules
UT Technical Advisory Committee_Jan. 13-14, 2016 4
But…
• Utah’s key legislative leaders insisted that the law
requires the use of a specific scale score instead of
framing the target in the SGP metric
• This is not unique to Utah. Previously, Arizona and
Indiana have requested conversion of an SGP target to a
scale score target.
• The software used to calculates SGPs (SGP package) was
extended several years ago to support that use case.
UT Technical Advisory Committee_Jan. 13-14, 2016 5
• Unfortunately, there are many misconceptions regarding
SGP/scale score targets
• On the following slides, we present some key properties
associated with such targets and then offer two options
for fulfilling the legislative request
UT Technical Advisory Committee_Jan. 13-14, 2016 6
Zero-sum game
• A 40th percentile SGP/scale score target does not result
in a zero-sum game where 40 percent of students “lose”
and 60 percent of the students “win”
• Targets are set a year in advance so (ideally) teachers
know each student’s scale score goal at the beginning of
the school year.
• If the system improves in the ensuing year, then a higher
percentage of students will reach their target than in the
base year.
– That is, more than 60% of students (even, in theory, 100% of
students) can meet or exceed their target.
UT Technical Advisory Committee_Jan. 13-14, 2016 7
Comparable Targets
• Basing targets on SGP=40 ensures that all student have
the same relative difficulty in reaching their target, even
though scale score increases/gains differ by student
• There is a misconception that scale score increases/gain
scores are uniform regardless of where a student starts
– This simply is not true!
• Establishing a single SGP target of 40 creates a uniform
level of difficulty for all students even though the scale
score changes for each student will differ
UT Technical Advisory Committee_Jan. 13-14, 2016 8
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: ELA Grade 4
UT Technical Advisory Committee_Jan. 13-14, 2016 9
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: ELA Grade 5
UT Technical Advisory Committee_Jan. 13-14, 2016 10
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: ELA Grade 6
UT Technical Advisory Committee_Jan. 13-14, 2016 11
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: ELA Grade 7
UT Technical Advisory Committee_Jan. 13-14, 2016 12
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: ELA Grade 8
UT Technical Advisory Committee_Jan. 13-14, 2016 13
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Math Grade 4
UT Technical Advisory Committee_Jan. 13-14, 2016 14
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Math Grade 5
UT Technical Advisory Committee_Jan. 13-14, 2016 15
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Math Grade 6
UT Technical Advisory Committee_Jan. 13-14, 2016 16
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Math Grade 7
UT Technical Advisory Committee_Jan. 13-14, 2016 17
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Math Grade 8
UT Technical Advisory Committee_Jan. 13-14, 2016 18
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Science Grade 5
UT Technical Advisory Committee_Jan. 13-14, 2016 19
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Science Grade 6
UT Technical Advisory Committee_Jan. 13-14, 2016 20
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Science Grade 7
UT Technical Advisory Committee_Jan. 13-14, 2016 21
Comparable Targets
Mean Scale Score Gains by Prior Score Decile: Science Grade 8
UT Technical Advisory Committee_Jan. 13-14, 2016 22
Two Approaches
• Single Prior
• Multiple Priors
• Student growth projections, which are the basis of the
SGP/scale score targets utilized by Utah and other states
can be calculated using either a single or multiple scale
score priors (if available).
• The use of single or multiple scale score priors is usually
done to be consistent with the way in which SGPs are
calculated.
UT Technical Advisory Committee_Jan. 13-14, 2016 23
Single Prior
• Ensures that every student with the same prior (Year 1)
score will have the same scale score target (Year 2)
• An advantage for using a single prior stems from the
belief that it is more straightforward to communicate a
target based upon a single rather than multiple priors
• Disadvantage is that the single prior is the least accurate
(and perhaps least reliable) case of SGPs
UT Technical Advisory Committee_Jan. 13-14, 2016 24
Multiple Priors
• SGPs are almost always calculated with multiple scale
score priors (when available). Thus, projections for
students also utilize the same multiple scale score priors.
• Multiple priors lead to more accurate individual targets.
The disadvantage to using multiple priors often stems
from the belief that it is more difficult to communicate a
target based upon multiple priors than a single prior
– For example, two students with the same immediately
prior score may (likely will) have different targets
based on the full score history
UT Technical Advisory Committee_Jan. 13-14, 2016 25
Single versus multiple prior based targets
• Low ELA achieving student
in 2014 (below prof/360)
who had high growth in
2015 (SGP=92) raising their
status (proficient/450).
• Top Figure: Using only 2015
Score to set target for 2016
gives a scale score target of
465 (SGP 40 with 1 prior).
• Bottom Figure: Using both
2014 and 2015 Scores to set
student target for 2016
gives a scale score target of
445 (SGP 40 with 2 priors) .
• Ignoring the 2014 scale
score sets too high a target
for the student.
UT Technical Advisory Committee_Jan. 13-14, 2016 26
Single versus multiple prior based targets
• High ELA achieving student
in 2014 (advanced/508)
who had low growth in 2015
(SGP=3) dropping their
status (proficient/450).
• Top Figure: Using only 2015
Score to set target for 2016
gives a scale score target of
465 (SGP 40 with 1 prior).
• Bottom Figure: Using both
2014 and 2015 Scores to set
student target for 2016
gives a scale score target of
492 (SGP 40 with 2 priors) .
• Ignoring the 2014 scale
score sets too low a target
for the student.
UT Technical Advisory Committee_Jan. 13-14, 2016 27
Discussion and Recommendations
• Which option seems most promising and why?
• Does the TAC think the increased accuracy associated
with using multiple priors is worth the potential
explanatory challenges?
• If so, can the TAC suggest some good strategies for
helping USOE construct compelling explanations?
UT Technical Advisory Committee_Jan. 13-14, 2016 28
About this copy
Sufficient Growth Discussion — Utah Department of Education, January 10, 2016.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.