Talks archive, 2005–2009 – The Colorado Growth Model: A Technical Overview

Overview Student Growth Percentiles and Accountability References The Colorado Growth Model: A Technical Overview Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH CCSSO Conference on Student Assessment Los Angeles, CA — June 23rd, 2009 Damian W. Betebenner The Colorado Growth Model

Overview The Colorado Growth Model Student Growth Percentiles and Accountability Student Growth Percentiles: What Is References Student Growth Percentiles: What Should Be The Colorado Growth Model Student Growth for Multiple Purposes Growth as the cornerstone of accountability: In Colorado student progress to judge state, district, school and student performance. The Colorado Growth Model addresses three related questions using the same metric: What is the level of growth for a student (i.e. Actual Growth)? What should the level of growth for a student be (i.e. Aspirational Growth)? What could the level of growth for a student be (i.e., Realistic Growth)? Student growth percentiles—a normative description of growth—forms the basis of the Colorado Growth Model [Betebenner, 2008] Damian W. Betebenner The Colorado Growth Model

Overview The Colorado Growth Model Student Growth Percentiles and Accountability Student Growth Percentiles: What Is References Student Growth Percentiles: What Should Be Student Growth Percentiles Normative Growth Question Should we be surprised with a student’s current achievement given their prior achievement? Given a student’s prior scale scores and the associated conditional density, their current scale score corresponds to a percentile of that conditional distribution. This percentile is the student’s growth percentile. Growth percentiles are closely related to estimating the probability of observing a student’s current achievement taking account of their past achievement: Pr(Current Achievement |Past Achievement ). Growth percentiles describe the rarity of a student’s current achievement conditional upon their prior achievement. Damian W. Betebenner The Colorado Growth Model

600 co 05 600 re S le S ca le ca S re 06 co

Overview The Colorado Growth Model Student Growth Percentiles and Accountability Student Growth Percentiles: What Is References Student Growth Percentiles: What Should Be Student Growth Percentiles Normative Growth Question Should we be surprised with a student’s current achievement given their prior achievement? Student growth percentiles address this question. Consider a low achieving student with 90th percentile growth and a high achieving student with 10th percentile growth. The low achieving student grew at a rate exceeding 90 percent of similar students. The high achieving student grew at a rate exceeding just 10 percent of similar students. The low achiever’s growth is more exemplary (probabilistically) than the high achiever’s. Judgments about the adequacy of student growth require external criteria. Damian W. Betebenner The Colorado Growth Model

Overview The Colorado Growth Model Student Growth Percentiles and Accountability Student Growth Percentiles: What Is References Student Growth Percentiles: What Should Be Combining Actual and Aspirational Growth What Is Each student receives a growth percentile quantifying their growth in each of three subject for the academic year. What Should Be Each student receives percentile growth projections/trajectories esti- mating: What level of growth is required to reach each of the 3 performance levels in 1, 2, 3, and 4 years. What Could Be The percentile metric establishes a normative foundation allowing stakeholders to set challenging yet realistic goals. Damian W. Betebenner The Colorado Growth Model

Not On Track to Reach Proficient − Not Catching Up Colorado's Growth Model uses each student's growth percentile in two ways: First, the growth percentile is used to describe how 73rd much a student has grown 95th● during the last year. Second, the growth percentile is used to determine whether the student is on track to reach/maintain proficiency. The following slides demonstrate, for an individual student, how Colorado's Growth Model is used to determine whether the student is On Track to Reach Proficient, that is "Catching Up". 7th ● Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

Is the student's growth, from 2007 to 2008, sufficient to put them on track to reach proficient within 3 years? 73rd ● 95th ● 7th ● Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

Is the student's growth, from 2007 to 2008, sufficient to put them on track to reach proficient within 3 years? 73rd ● 95th ● After 1 year the student 7th ● remains partially proficient, so their 1 year growth was not enough to get them to proficient. Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

In 2007 CDE estimated that it would take Is the student's growth, 92nd percentile growth, consecutively for two years, to reach proficient. from 2007 to 2008, sufficient Their 73rd percentile growth puts them behind that 2 year target. to put them on track to reach proficient within 3 years? 92nd 73rd ● 95th ● After 1 year the student 7th ● remains partially proficient, so their 1 year growth was not enough to get them to proficient. Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

In 2007 CDE estimated that it would take Is the student's growth, 92nd percentile growth, consecutively for two years, to reach proficient. from 2007 to 2008, sufficient Their 73rd percentile growth puts them behind that 2 year target. to put them on track to reach proficient within 3 years? 88th 92nd 73rd ● 95th ● In 2007 CDE estimated that it would take 88th percentile growth, consecutively for three years, to reach After 1 year the student proficient. Their 73rd percentile growth 7th ● remains partially proficient, puts them behind that 3 year target. so their 1 year growth was not enough to get them to proficient. Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

In 2007 CDE estimated that it would take Is the student's growth, 92nd percentile growth, consecutively for two years, to reach proficient. from 2007 to 2008, sufficient Their 73rd percentile growth puts them behind that 2 year target. to put them on track to reach proficient within 3 years? Conclusion: Because the student was not proficient in 2008 and their 2007−08 growth percentile of 73 was less than both the two and three year targets, the student's growth is considered to be insufficient to reach proficient within three years 88th In short, the student is not on track to be proficient and is not "catching 92nd up". 73rd ● 95th ● In 2007 CDE estimated that it would take 88th percentile growth, consecutively for three years, to reach After 1 year the student proficient. Their 73rd percentile growth 7th ● remains partially proficient, puts them behind that 3 year target. so their 1 year growth was not enough to get them to proficient. Grade 3/2005 Grade 4/2006 Grade 5/2007 Grade 6/2008 Grade 7/2009 Grade 8/2010

On Track to Remain Proficient − Keeping Up Colorado's Growth Model uses each student's growth percentile in two ways: First, the growth percentile is used to describe how 63rd ● much a student has grown during the ●last year. Second, the● growth percentile is used to determine whether the 66th 26th student is on track to reach/maintain proficiency. The following slides demonstrate, for an individual student, how Colorado's Growth Model is used to determine whether the student is On Track to Remain Proficient, that is "Keeping Up". Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Is the student's growth, from 2007 to 2008, sufficient to remain at or above proficient for the next 3 years? 63rd ● 66th ● 26th ● Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Is the student's growth, After 1 year the student from 2007 to 2008, sufficient remains proficient, to remain at or above so their 1 year growth was proficient for the next 3 years? enough to remain at proficient. 63rd ● 66th ● 26th ● Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Is the student's growth, After 1 year the student from 2007 to 2008, sufficient remains proficient, to remain at or above so their 1 year growth was proficient for the next 3 years? enough to remain at proficient. 63rd ● 66th ● 18th 26th ● In 2007 CDE estimated that it would take, at a minimum, 18th percentile growth, consecutively for two years, to maintain at or above proficient. Their 63rd percentile growth puts them above that 2 year minimal target. Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Is the student's growth, After 1 year the student from 2007 to 2008, sufficient remains proficient, to remain at or above so their 1 year growth was proficient for the next 3 years? enough to remain at proficient. 63rd ● 22nd 66th ● 18th 26th ● In 2007 CDE estimated that it would take, at a minimum, 22nd percentile growth, consecutively for three years, to In 2007 CDE estimated that it maintain at or above proficient. Their 63rd would take, at a minimum, 18th percentile growth, percentile growth puts them above that 3 year consecutively for two years, to maintain minimal target. at or above proficient. Their 63rd percentile growth puts them above that 2 year minimal target. Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Is the student's growth, After 1 year the student from 2007 to 2008, sufficient remains proficient, to remain at or above so their 1 year growth was proficient for the next 3 years? enough to remain at proficient. Conclusion: Because the student was proficient in 2008 and their 2007−08 growth percentile of 63 was greater than both the two and three year minimum targets, the student's growth is considered to be sufficient to remain ●proficient during the next three years. 22nd 63rd In short, 66th the student is on track to remain proficient and is "keeping 18th up". 26th ● In 2007 CDE estimated that it would take, at a minimum, 22nd percentile growth, consecutively for three years, to In 2007 CDE estimated that it maintain at or above proficient. Their 63rd would take, at a minimum, 18th percentile growth, percentile growth puts them above that 3 year consecutively for two years, to maintain minimal target. at or above proficient. Their 63rd percentile growth puts them above that 2 year minimal target. Grade 5/2005 Grade 6/2006 Grade 7/2007 Grade 8/2008 Grade 9/2009 Grade 10/2010

Overview Student Growth Percentiles and Accountability School Accountability References Going from Students to Schools It’s of interest to examine schools where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles associated with a school (or other grouping) calculate the median of the student growth percentiles. If students were randomly assigned to schools, expect to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Examining growth with achievement sheds new light on school performance. Damian W. Betebenner The Colorado Growth Model

District C: 2008 CSAP Math School Results Student Growth versus Student Achievement by Percent Free/Reduced Lunch School Percent Higher Achievement Higher Achievement Free/Reduced Lunch Lower Growth Higher Growth Percent at/above Proficient in School Less than 20 percent ● 20 to 40 percent ● 40 to 60 percent ● 60 to 80 percent ● More than 80 percent School Size 50 Students 100 Students 200 Students 30 500 Students 1,000 Students 10 Lower Growth Higher Growth Lower Achievement Lower Achievement Median of Student Growth Percentiles in School

Overview Student Growth Percentiles and Accountability References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Damian W. Betebenner The Colorado Growth Model

About this copy

The Colorado Growth Model: A Technical Overview — CCSSO, June 2009.

This is an image-based copy of the original beamer slides; builds and animations are flattened.