Intro
Student Growth Percentiles and Accountability
References
New Directions for Student Growth Models
The Colorado Growth Model
Damian W. Betebenner
National Center for the Improvement of Educational Assessment
Dover, NH
Georgia RTTT Accountability Symposium
Athens, Georgia
January 24th, 2011
Damian W. Betebenner New Directions for Student Growth Models
Intro Describing Student Growth
Student Growth Percentiles and Accountability Student Growth Percentiles
References Criterion Referenced Growth
About Me
Dr. Damian Betebenner, PhD
- Senior Associate at the Center for Assessment (NCIEA).
- Developed student growth percentiles and percentile growth
trajectories to help states and educational associations employ
student growth in decision
making [Betebenner, 2008, Betebenner, 2009].
- Currently refining and sharing these techniques with other
states including Colorado, Massachusetts, Indiana, Virginia and
15 other states in various stages of investigation/adoption.
Damian W. Betebenner New Directions for Student Growth Models
Intro Describing Student Growth
Student Growth Percentiles and Accountability Student Growth Percentiles
References Criterion Referenced Growth
Describing Student Growth
Measuring student growth, even with a vertical scale, is not a simple
task.
Some believe a vertical scale simplifies the task of measuring
student growth.
Even with an interval (or ratio) scale, growth is not easy to interpret.
Consider, for example, height.
A child might grow 4 inches between ages 3 and 4.
4 inches is a well understood quantity.
The 4 inch increase becomes really meaningful only when understood
alongside the growth of other 3 to 4 year olds.
Student growth percentiles were developed to provide a normative
context for describing student growth.
Damian W. Betebenner New Directions for Student Growth Models
600 co
09 600 re
le
ca le ca
S re 10
co
600 co
09 600 re
le
ca le ca
S re 10
co
600 co
09 600 re
le
ca le ca
S re 10
co
co
th
re
ro
i le
le
ce
nt ca
er
th
re
600 co
le
ca
Intro Describing Student Growth
Student Growth Percentiles and Accountability Student Growth Percentiles
References Criterion Referenced Growth
Student Growth Percentiles
Should we be surprised with a child’s current achievement given their prior
achievement?
Given a student’s prior scale scores and the associated conditional
density, their current scale score corresponds to a percentile of that
conditional distribution.
This percentile is the student’s growth percentile.
Growth percentiles are closely related to estimating the probability of
observing a student’s current achievement taking account of their
past achievement:
Pr(Current Achievement |Past Achievement ).
As such, growth percentiles describe the rarity of a student’s current
achievement conditional upon their prior achievement.
Damian W. Betebenner New Directions for Student Growth Models
Intro Describing Student Growth
Student Growth Percentiles and Accountability Student Growth Percentiles
References Criterion Referenced Growth
Student Growth Percentiles
Should we be surprised with a child’s current achievement given their
prior achievement?
Student growth percentiles answer this question.
Consider a low achieving student with 90th percentile growth and a
high achieving student with 10th percentile growth.
The low achieving student grew at a rate exceeding 90 percent of
similar students.
The high achieving student grew at a rate exceeding just 10 percent of
similar students.
The low achiever’s growth is more exemplary (probabilistically) than
the high achiever’s.
Judgments about the adequacy of student growth require external
criteria.
Damian W. Betebenner New Directions for Student Growth Models
Intro Describing Student Growth
Student Growth Percentiles and Accountability Student Growth Percentiles
References Criterion Referenced Growth
Establishing growth-standards based upon growth-norms
The most common adequacy criterion are judging growth toward an
achievement goal (i.e., growth-to-standard).
Results from student growth percentile analyses can be used to
calculate percentile growth trajectories for each student.
These trajectories indicate what future rates of growth will lead to
and are used to make adequacy judgments.
This growth-to-standard approach was approved as part of
Colorado’s successful application to the Growth Model Pilot
Program.
Damian W. Betebenner New Directions for Student Growth Models
Advanced Reading
High
Achievement
CSAP Reading
● Typical
Scale Score
Proficient Growth
Low
● Level Percentiles
High 66th − 99th
Typical 35th − 65th
Part Proficient
● Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 462 539 563 609
Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement
Growth Percentile 66 66 90
Growth Level High High High Growth
Example Student 12 (1234567812) Sample Middle School
Advanced
Math
Achievement
High
CSAP Math
● Scale Score
Typical
● Growth
Proficient
● Level Percentiles
Low
High 66th − 99th
● Typical 35th − 65th
Part Proficient
Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 379 388 444 462
Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement
Growth Percentile 16 69 51
Growth Level Low High Typical Growth
Advanced Reading
High
Achievement
CSAP Reading
● Typical
Scale Score
Proficient Growth
Low
● Level Percentiles
High 66th − 99th
Typical 35th − 65th
Part Proficient
● Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 462 539 563 609
Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement
Growth Percentile 66 66 90
Growth Level High High High Growth
Advanced
Writing
Achievement
High
CSAP Writing
● Scale Score
Typical
● Growth
Proficient
Level Percentiles
● Low
High 66th − 99th
Part Proficient ● Typical 35th − 65th
Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 425 449 469 504
Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement
Growth Percentile 55 53 79
Growth Level Typical Typical High Growth
Copyright 2009, Colorado Department of Education/Center for Assessment, Inc.
2008 CSAP Mathematics
Growth and Achievement Chart
2008 Achievement Percentile Percentile Growth Trajectory
Scale Score
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10
Intro School Accountability
Student Growth Percentiles and Accountability Accountability in Colorado
References Concluding Remarks
Going from Students to Schools/Classrooms
It’s of interest to examine schools or classrooms/teachers where
students demonstrate, on average, extraordinarily high and low
student growth.
To summarize the student growth percentiles associated with a
school (or other grouping) calculate the median of the student
growth percentiles.
If students were randomly assigned to schools, expect to see a
median of 50.
Values greatly above or below 50 are of interest in identifying best
practices or providing extra support.
Examining growth with achievement sheds new light on school
performance.
Damian W. Betebenner New Directions for Student Growth Models
District C: 2008 CSAP Math School Results
Student Growth versus Student Achievement by Percent Free/Reduced Lunch
School Percent
Higher Achievement Higher Achievement Free/Reduced Lunch
Lower Growth Higher Growth
Percent at/above Proficient in School
Less than 20 percent
● 20 to 40 percent
● 40 to 60 percent
● 60 to 80 percent
● More than 80 percent
School Size
50 Students
100 Students
200 Students
30 500 Students
1,000 Students
10 Lower Growth Higher Growth
Lower Achievement Lower Achievement
Median of Student Growth Percentiles in School
Intro School Accountability
Student Growth Percentiles and Accountability Accountability in Colorado
References Concluding Remarks
Growth & Accountability in Colorado
The growth data can be used in different ways by states to make
judgments about educator/education quality.
Colorado blends the norm- and criterion-referenced growth
percentile results to accredit districts and rate schools.
The state is investigating using the norm-referenced growth
percentile results as part of evaluations of educator quality.
To promote wide ranging use of data, the state has set up a website
http://www.schoolview.org that provides data visualization tools
and resources to help all stakeholders use the student growth data.
Damian W. Betebenner New Directions for Student Growth Models
Intro School Accountability
Student Growth Percentiles and Accountability Accountability in Colorado
References Concluding Remarks
Lessons Learned
On Questions John Tukey
It is better to have an approximate answer to the right question than a precise answer to the
wrong question.
On Statistical Models George E. P. Box
All models are wrong but some are useful.
On Understanding Aristotle
We understand best those things we see emerge from their very beginnings.
On Accountability Sherlock Holmes
It is a capital mistake to theorize before you have all the evidence. It biases the judgment.
On Theories of Action Confucius
Tell me and I will forget, show me and I will remember, involve me and I will understand.
Damian W. Betebenner New Directions for Student Growth Models
Intro School Accountability
Student Growth Percentiles and Accountability Accountability in Colorado
References Concluding Remarks
What’s Next?
Colorado, Massachusetts, and Indiana have publicly released
growth results.
The response has been overwhelmingly positive.
Calculating of the quantities is the easy part. Getting people to
understand them and use them correctly is the hard part.
Colorado, Massachusetts, Indiana, Virginia, and other states are
investigating a multi-state platform for data visualization and
reporting built from a common growth analysis.
Damian W. Betebenner New Directions for Student Growth Models
Intro
Student Growth Percentiles and Accountability
References
References
Betebenner, D. W. (2008).
Toward a normative understanding of student growth.
In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based
Educational Accountability, pages 155–170. Taylor & Francis, New
York.
Betebenner, D. W. (2009).
Norm- and criterion-referenced student growth.
Educational Measurement: Issues and Practice, 28(4):42–51.
Damian W. Betebenner New Directions for Student Growth Models
About this copy
New Directions for Student Growth Models: The Colorado Growth Model — Georgia Department of Education, January 24, 2011.
This is an image-based copy of the original beamer slides; builds and animations are flattened.