Growth & Accountability: The Colorado Growth Model
Future Directions
References
New Directions in Student Growth
The Colorado Growth Model
Damian W. Betebenner
National Center for the Improvement of Educational Assessment
Dover, NH
CCSSO National Conference on Student Assessment
Orlando, Florida June 19th, 2011
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
Growth & Accountability: The Colorado Growth Model
How much growth & is it enough?
Colorado begins conversations about education/educator
quality by answering two questions:
How much growth did a student make?
Is it enough growth to reach or maintain proficiency?
Answering these two questions in a credible and clear manner
sets the stage for subsequent accountability discussions.
Colorado has purposely separated the description (what is) from
the determination of responsibility (who/what is responsible).
This separation promotes greater ownership of results and
participation in finding root causes.
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
Describing Student Growth
Measuring student growth, even with a vertical scale, is not a simple
task.
Some believe a vertical scale simplifies the task of measuring
student growth.
Even with an interval (or ratio) scale, growth is not easy to interpret.
Consider, for example, height.
A child might grow 4 inches between ages 3 and 4.
4 inches is a well understood quantity.
The 4 inch increase becomes really meaningful only when understood
alongside the growth of other 3 to 4 year olds.
Student growth percentiles were developed to provide a normative
context for describing student growth.
Damian W. Betebenner New Directions in Student Growth
Advanced Reading
High
Achievement
CSAP Reading
● Typical
Scale Score
Proficient Growth
Low
● Level Percentiles
High 66th − 99th
Typical 35th − 65th
Part Proficient
● Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 462 539 563 609
Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement
Growth Percentile 66 66 90
Growth Level High High High Growth
Example Student 12 (1234567812) Sample Middle School
Advanced
Math
Achievement
High
CSAP Math
● Scale Score
Typical
● Growth
Proficient
● Level Percentiles
Low
High 66th − 99th
● Typical 35th − 65th
Part Proficient
Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 379 388 444 462
Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement
Growth Percentile 16 69 51
Growth Level Low High Typical Growth
Advanced Reading
High
Achievement
CSAP Reading
● Typical
Scale Score
Proficient Growth
Low
● Level Percentiles
High 66th − 99th
Typical 35th − 65th
Part Proficient
● Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 462 539 563 609
Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement
Growth Percentile 66 66 90
Growth Level High High High Growth
Advanced
Writing
Achievement
High
CSAP Writing
● Scale Score
Typical
● Growth
Proficient
Level Percentiles
● Low
High 66th − 99th
Part Proficient ● Typical 35th − 65th
Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 425 449 469 504
Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement
Growth Percentile 55 53 79
Growth Level Typical Typical High Growth
Copyright 2009, Colorado Department of Education/Center for Assessment, Inc.
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
Student Growth Percentiles
Should we be surprised with a child’s current achievement given their
prior achievement?
Student growth percentiles answer this question.
Consider a low achieving student with 75th percentile growth and a
high achieving student with 25th percentile growth.
The low achieving student grew at a rate exceeding 75 percent of
similar students.
The high achieving student grew at a rate exceeding just 25 percent of
similar students.
The low achiever’s growth is more exemplary (probabilistically) than
the high achiever’s.
Value judgments (e.g., good, bad, enough, adequate, . . . ) about
student growth require external criteria.
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
The Colorado Growth Model
The Colorado Growth Model is based upon the open source Student
Growth Percentile (SGP)
methodology [Betebenner, 2008, Betebenner, 2009]
The model is a norm and criterion-referenced growth model that
answers both questions: How much? and Is it enough?.
The software and visualizations used with the SGPs are free and
open source.
Currently over two dozen states are investigating or utilizing the
model.
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
Going from Students to Schools
It’s of interest to examine schools/classrooms where students
demonstrate, on average, extraordinarily high and low student
growth.
To summarize the student growth percentiles associated with a
school/classroom (or other grouping) Colorado calculates the
median of the student growth percentiles.
If students were randomly assigned to schools, expect to see a
median of 50.
Values greatly above or below 50 are of interest in identifying best
practices or providing extra support.
Examining growth with achievement sheds new light on
school/classroom performance.
Damian W. Betebenner New Directions in Student Growth
District C: 2008 CSAP Math School Results
Student Growth versus Student Achievement by Percent Free/Reduced Lunch
School Percent
Higher Achievement Higher Achievement Free/Reduced Lunch
Lower Growth Higher Growth
Percent at/above Proficient in School
Less than 20 percent
● 20 to 40 percent
● 40 to 60 percent
● 60 to 80 percent
● More than 80 percent
School Size
50 Students
100 Students
200 Students
30 500 Students
1,000 Students
10 Lower Growth Higher Growth
Lower Achievement Lower Achievement
Median of Student Growth Percentiles in School
Growth & Accountability: The Colorado Growth Model
Describing Student Growth
Future Directions
Teacher & School Accountability
References
Integrating Norm- and Criterion Referenced Growth
The major challenge facing ESEA reauthorization is, ultimately, how
to balance norm and criterion referenced based definitions of quality.
NCLB is fundamentally a criterion-referenced accountability system
stipulating universal proficiency.
Criterion referenced growth models adopt this ethical mandate to
distinguish enough/not enough.
Value-added models (and the norm-referenced part of the Colorado
Growth Model) are norm-referenced and use statistical expectation
to distinguish enough/not enough.
The former is “fairer” to students the latter to personnel/institutions.
Colorado, in its recently released school and district performance
frameworks, has tried to reconcile these two competing demands.
http://www.schoolview.org/PerformanceFrameworks.asp
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Descriptive Accountability
Future Directions
Web 2.0 Technologies
References
Descriptive Accountability
“This is the difference between a retrospective question of identifying
fault as opposed to a prospective strategy to engineer some corrective
measure, almost independent of considering whether there was
blame-worthiness. And to move away from the blame-worthiness
paradigm toward something that is more regulatory in nature. Where one
might seize upon disparities or circumstances that are for some reason
deemed unacceptable and engineer the interventions needed to bring
about the necessary change. . . . It’s the no-fault gap closing strategy in
which the effort is to build a consensus about a vision of an improved
society rather than figure out where’s the person we want to pillory.”
Christopher Edley (2006)
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Descriptive Accountability
Future Directions
Web 2.0 Technologies
References
Web 2.0: Data Visualization and Social Networking
The Colorado Department of Education has embraced advanced
data visualization as a means of communicating complex data in a
manner that is accessible to non-technically inclined audience (e.g.,
parents).
The goal: Transform conversations about education through active
engagement with data (i.e., evidence).
Colorado’s efforts have received tremendous interest and
recognition:
Recognized by Adobe for innovative uses of their technology as an
Adobe Max Award finalist in October, 2009.
2010 NCME Award for Outstanding Dissemination of Educational
Measurement Concepts to the Public.
15 states signing MOUs to co-develop a cloud-based analysis and
data visualization platform in a non-proprietary fashion.
Work has just begun in to extend Colorado’s efforts in multiple states
including Indiana, Nevada, New Hampshire, Nevada, Rhode Island,
Virginia, and West Virginia with many more to come.
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Descriptive Accountability
Future Directions
Web 2.0 Technologies
References
Web 2.0: Data Visualization and Social Networking
With a collaborative spirit, with a collaborative platform where people can
upload data, explore data, compare solutions, discuss the results, build
consensus, we can engage passionate people, local communities, media
and this will raise–incredibly–the amount of people who can understand
what is going on.
And this would have fantastic outcomes: the engagement of people,
especially new generations; it would increase knowledge, unlock
statistics, improve transparency and accountability of public policies,
change culture, increase numeracy, and in the end, improve democracy
and welfare.
E. Giovannini, Chief Statistician, OECD. June 2007
Damian W. Betebenner New Directions in Student Growth
Growth & Accountability: The Colorado Growth Model
Future Directions
References
References
Betebenner, D. W. (2008).
Toward a normative understanding of student growth.
In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based
Educational Accountability, pages 155–170. Taylor & Francis, New
York.
Betebenner, D. W. (2009).
Norm- and criterion-referenced student growth.
Educational Measurement: Issues and Practice, 28(4):42–51.
Edley, C. (2006).
Educational “Opportunity" is the highest civil rights priority. So what
should researchers and lawyers do about it?
Retrieved June 22, 2006 from the World Wide Web:
http://www.softconference.com/MEDIA/WMP/260407/#43.010.
Damian W. Betebenner New Directions in Student Growth
About this copy
New Directions in Student Growth The Colorado Growth Model — CCSSO, June 19, 2011.
This is an image-based copy of the original beamer slides; builds and animations are flattened.