New Directions in Student Growth The Colorado Growth Model

CCSSO · June 19, 2011

New Directions in Student Growth The Colorado Growth Model — CCSSO, June 19, 2011.
Published

June 19, 2011

Event
Conference
Slides
14
Format
Converted from Beamer

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Growth & Accountability: The Colorado Growth Model Future Directions References New Directions in Student Growth The Colorado Growth Model Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH CCSSO National Conference on Student Assessment Orlando, Florida June 19th, 2011 Damian W. Betebenner New Directions in Student Growth
  2. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References Growth & Accountability: The Colorado Growth Model How much growth & is it enough? Colorado begins conversations about education/educator quality by answering two questions: How much growth did a student make? Is it enough growth to reach or maintain proficiency? Answering these two questions in a credible and clear manner sets the stage for subsequent accountability discussions. Colorado has purposely separated the description (what is) from the determination of responsibility (who/what is responsible). This separation promotes greater ownership of results and participation in finding root causes. Damian W. Betebenner New Directions in Student Growth
  3. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner New Directions in Student Growth
  4. Advanced Reading High Achievement CSAP Reading ● Typical Scale Score Proficient Growth Low ● Level Percentiles High 66th − 99th Typical 35th − 65th Part Proficient ● Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 462 539 563 609 Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement Growth Percentile 66 66 90 Growth Level High High High Growth
  5. Example Student 12 (1234567812) Sample Middle School Advanced Math Achievement High CSAP Math ● Scale Score Typical ● Growth Proficient ● Level Percentiles Low High 66th − 99th ● Typical 35th − 65th Part Proficient Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 379 388 444 462 Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement Growth Percentile 16 69 51 Growth Level Low High Typical Growth Advanced Reading High Achievement CSAP Reading ● Typical Scale Score Proficient Growth Low ● Level Percentiles High 66th − 99th Typical 35th − 65th Part Proficient ● Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 462 539 563 609 Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement Growth Percentile 66 66 90 Growth Level High High High Growth Advanced Writing Achievement High CSAP Writing ● Scale Score Typical ● Growth Proficient Level Percentiles ● Low High 66th − 99th Part Proficient ● Typical 35th − 65th Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 425 449 469 504 Achievement Level Part Proficient Part Proficient Part Proficient Part Proficient Achievement Growth Percentile 55 53 79 Growth Level Typical Typical High Growth Copyright 2009, Colorado Department of Education/Center for Assessment, Inc.
  6. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References Student Growth Percentiles Should we be surprised with a child’s current achievement given their prior achievement? Student growth percentiles answer this question. Consider a low achieving student with 75th percentile growth and a high achieving student with 25th percentile growth. The low achieving student grew at a rate exceeding 75 percent of similar students. The high achieving student grew at a rate exceeding just 25 percent of similar students. The low achiever’s growth is more exemplary (probabilistically) than the high achiever’s. Value judgments (e.g., good, bad, enough, adequate, . . . ) about student growth require external criteria. Damian W. Betebenner New Directions in Student Growth
  7. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References The Colorado Growth Model The Colorado Growth Model is based upon the open source Student Growth Percentile (SGP) methodology [Betebenner, 2008, Betebenner, 2009] The model is a norm and criterion-referenced growth model that answers both questions: How much? and Is it enough?. The software and visualizations used with the SGPs are free and open source. Currently over two dozen states are investigating or utilizing the model. Damian W. Betebenner New Directions in Student Growth
  8. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References Going from Students to Schools It’s of interest to examine schools/classrooms where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles associated with a school/classroom (or other grouping) Colorado calculates the median of the student growth percentiles. If students were randomly assigned to schools, expect to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Examining growth with achievement sheds new light on school/classroom performance. Damian W. Betebenner New Directions in Student Growth
  9. District C: 2008 CSAP Math School Results Student Growth versus Student Achievement by Percent Free/Reduced Lunch School Percent Higher Achievement Higher Achievement Free/Reduced Lunch Lower Growth Higher Growth Percent at/above Proficient in School Less than 20 percent ● 20 to 40 percent ● 40 to 60 percent ● 60 to 80 percent ● More than 80 percent School Size 50 Students 100 Students 200 Students 30 500 Students 1,000 Students 10 Lower Growth Higher Growth Lower Achievement Lower Achievement Median of Student Growth Percentiles in School
  10. Growth & Accountability: The Colorado Growth Model Describing Student Growth Future Directions Teacher & School Accountability References Integrating Norm- and Criterion Referenced Growth The major challenge facing ESEA reauthorization is, ultimately, how to balance norm and criterion referenced based definitions of quality. NCLB is fundamentally a criterion-referenced accountability system stipulating universal proficiency. Criterion referenced growth models adopt this ethical mandate to distinguish enough/not enough. Value-added models (and the norm-referenced part of the Colorado Growth Model) are norm-referenced and use statistical expectation to distinguish enough/not enough. The former is “fairer” to students the latter to personnel/institutions. Colorado, in its recently released school and district performance frameworks, has tried to reconcile these two competing demands. http://www.schoolview.org/PerformanceFrameworks.asp Damian W. Betebenner New Directions in Student Growth
  11. Growth & Accountability: The Colorado Growth Model Descriptive Accountability Future Directions Web 2.0 Technologies References Descriptive Accountability “This is the difference between a retrospective question of identifying fault as opposed to a prospective strategy to engineer some corrective measure, almost independent of considering whether there was blame-worthiness. And to move away from the blame-worthiness paradigm toward something that is more regulatory in nature. Where one might seize upon disparities or circumstances that are for some reason deemed unacceptable and engineer the interventions needed to bring about the necessary change. . . . It’s the no-fault gap closing strategy in which the effort is to build a consensus about a vision of an improved society rather than figure out where’s the person we want to pillory.” Christopher Edley (2006) Damian W. Betebenner New Directions in Student Growth
  12. Growth & Accountability: The Colorado Growth Model Descriptive Accountability Future Directions Web 2.0 Technologies References Web 2.0: Data Visualization and Social Networking The Colorado Department of Education has embraced advanced data visualization as a means of communicating complex data in a manner that is accessible to non-technically inclined audience (e.g., parents). The goal: Transform conversations about education through active engagement with data (i.e., evidence). Colorado’s efforts have received tremendous interest and recognition: Recognized by Adobe for innovative uses of their technology as an Adobe Max Award finalist in October, 2009. 2010 NCME Award for Outstanding Dissemination of Educational Measurement Concepts to the Public. 15 states signing MOUs to co-develop a cloud-based analysis and data visualization platform in a non-proprietary fashion. Work has just begun in to extend Colorado’s efforts in multiple states including Indiana, Nevada, New Hampshire, Nevada, Rhode Island, Virginia, and West Virginia with many more to come. Damian W. Betebenner New Directions in Student Growth
  13. Growth & Accountability: The Colorado Growth Model Descriptive Accountability Future Directions Web 2.0 Technologies References Web 2.0: Data Visualization and Social Networking With a collaborative spirit, with a collaborative platform where people can upload data, explore data, compare solutions, discuss the results, build consensus, we can engage passionate people, local communities, media and this will raise–incredibly–the amount of people who can understand what is going on. And this would have fantastic outcomes: the engagement of people, especially new generations; it would increase knowledge, unlock statistics, improve transparency and accountability of public policies, change culture, increase numeracy, and in the end, improve democracy and welfare. E. Giovannini, Chief Statistician, OECD. June 2007 Damian W. Betebenner New Directions in Student Growth
  14. Growth & Accountability: The Colorado Growth Model Future Directions References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Edley, C. (2006). Educational “Opportunity" is the highest civil rights priority. So what should researchers and lawyers do about it? Retrieved June 22, 2006 from the World Wide Web: http://www.softconference.com/MEDIA/WMP/260407/#43.010. Damian W. Betebenner New Directions in Student Growth