Talks archive, 2010–2014 – Growth Models, SGPs, and Indiana

Concluding Thoughts Growth Models, SGPs, and Indiana Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH Indianapolis, IN May 23rd, 2013 Damian W. Betebenner Growth Models, SGPs, and Indiana

Concluding Thoughts About Me Damian Betebenner Senior Associate National Center for the Improvement of Educational Assessment. Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009, Betebenner, 2012]. All materials associated with this work is open source with the goal of promoting beneficial uses of the data. Currently refining and sharing these techniques with approximately two dozen other states. Damian W. Betebenner Growth Models, SGPs, and Indiana

Concluding Thoughts Purpose The SGP Model is intended to allow stakeholders to answer three questions simultaneously: What is? What is a student’s growth is answered in a norm-referenced fashion using student growth percentiles. What should be? What should a student’s growth be to reach or maintain desired levels of achievement is a criterion-referenced question and is answered using student growth projections. What is reasonable? Because SGPs are percentiles and are conditional probabilities, we implicitly understand what is a modest growth goal versus what is an ambitious growth goal. One needs both norms and criteria to fully understand all phenomena, including student growth. Thus, to answer the debate question, one needs both student growth percentiles (norms) and student growth projections (criteria) to make sense. Damian W. Betebenner Growth Models, SGPs, and Indiana

Concluding Thoughts Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner Growth Models, SGPs, and Indiana

Kristen Brown Sample School How to interpret this student growth & achievement report ELA Pass + Achievement ISTEP+ ● ISTEP+ ELA ● Scale Score Scale Score High Growth Level Percentiles ● Typical ISTEP+ ● Pass High 66th − 99th Achievement ● Levels ● Low Typical 35th − 65th Low 1st − 34th Did Not Pass Student Growth Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Percentile Scale Score 449 487 504 466 Achievement Level Pass Pass Pass Did Not Pass Achievement Growth Percentile 72 57 2 Growth Level −− −− Low Growth Suggested Uses Review past growth to assess student academic progress toward Pass + Math ISTEP+ achievement goals. Achievement ● Develop remediation or enrich− High ISTEP+ Math ment plans based on rate of ● Scale Score Typical growth needed to reach higher ● ISTEP+ achievement levels. Pass Growth ● Low Level Percentiles ● Identify the rate of progress ● needed in order to reach or High 66th − 99th maintain proficient status on the ISTEP+ next year. Typical 35th − 65th Did Not Pass ● Low 1st − 34th Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Scale Score 306 429 496 450 Achievement Level Did Not Pass Did Not Pass Pass Did Not Pass Achievement Growth Percentile 68 94 18 Growth Level −− −− Low Growth For more information please visit the () at or contact Cooperatively developed by the & the Center for Assessment, Inc. Distributed by the .

Concluding Thoughts Software for Student Growth Percentiles Student Growth Percentiles and Percentile Growth Trajectories are calculated using the open source R Software Environment [R Development Core Team, 2013] and the open source SGP package [Betebenner et al., 2013] The SGP package is hosted on GitHub and multiple state analysis sites are present there. See https://github.com/SchoolView. In working with state departments of education, the goal is to build state capacity so states can perform and validate their own analyses using software available and easily used by the state. Arizona, Colorado, Georgia, Massachusetts, Idaho, Indiana, Wisconsin are four states (that I know of) who have state staff capable of generating the student growth percentiles together with aggregations that follow from their calculation. State ownership/oversight of the SGP calculations has been a big plus in terms of buy-in from the state, enhancing their ability to explain and work with local stakeholders. Damian W. Betebenner Growth Models, SGPs, and Indiana

Student Growth Percentile Adoption Adopted Model Considering Adoption Investigating Model

Indiana: 2012 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory Scale Score 2012 Achievement Percentile Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8

Indiana: 2012 ELA Norm & Criterion Referenced Growth & Achievement Scale Score Percentile Growth Trajectory 2012 Achievement Percentile Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8

Indiana: 2012 ELA Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory Scale Score 2012 Achievement Percentile Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8

Concluding Thoughts Lessons Learned Lessons Learned On Questions John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. On Statistical Models George E. P. Box All models are wrong but some are useful. On Understanding Aristotle We understand best those things we see emerge from their very beginnings. On Accountability Sherlock Holmes It is a capital mistake to theorize before you have all the evidence. It biases the judg- ment. On Theories of Action Confucius Tell me and I will forget, show me and I will remember, involve me and I will understand. Damian W. Betebenner Growth Models, SGPs, and Indiana

Concluding Thoughts Lessons Learned References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Betebenner, D. W. (2012). Growth, standards, and accountability. In Cizek, G., editor, Setting Performance Standards: Foundations, Methods, and Innovations, pages 439–450. Routledge, New York. Betebenner, D. W., Van Iwaarden, A., and Domingue, B. (2013). SGP: An R Package for the Calculation and Visualization of Student Growth Percentiles & Percentile Growth Trajectories. R package version 1.0-4.0. R Development Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Damian W. Betebenner Growth Models, SGPs, and Indiana

About this copy

Growth Models, SGPs, and Indiana — Indiana Department of Education, May 23, 2013.

This is an image-based copy of the original beamer slides; builds and animations are flattened.