Talks archive, 2010–2014 – On Student Growth & Accountability

References On Student Growth & Accountability Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH ETS Praxis Client Conference, Princeton, New Jersey May 2, 2012 Damian W. Betebenner On Student Growth & Accountability

References About Me Dr. Damian Betebenner, PhD - Senior Associate at the Center for Assessment (NCIEA). - Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009, Betebenner, 2012]. - Currently refining and sharing these techniques with other states including Colorado, Massachusetts, Indiana, Nevada, Hawaii and 25 other states in various stages of investigation/adoption. Damian W. Betebenner On Student Growth & Accountability

References From a Paucity to a Plethora of Data Data Data Everywhere (The Economist, 2010) More and better organized data: mega, giga, tera, peta, exa, zetta, yotta! Historical records of student achievement. Linkage of student achievement to student demographics, teachers, schools, educational programs, . . . Stakeholder interest in examining student achievement over time (student growth) derives from data availability. Damian W. Betebenner On Student Growth & Accountability

References A Recurring Metaphor Data as a window and a mirror Damian W. Betebenner On Student Growth & Accountability

Advanced Reading High Achievement CSAP Reading ● Typical Scale Score Proficient Growth Low ● Level Percentiles High 66th − 99th Typical 35th − 65th Part Proficient ● Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 462 539 563 609 Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement Growth Percentile 66 66 90 Growth Level High High High Growth

References Student Growth Percentiles 2007 Began develop of the Colorado Growth Model (student growth percentiles). 2008 Began work with Indiana on development of growth model. 2008 Successfully application of Colorado Growth Model (criterion-referenced component) as part of growth model pilot program. 2009-2010 Began/continued work with approximately two-dozen other states on developing state growth models. 2010 Worked with states on incorporating growth into RttT applications and with successful states on their use (Massachusetts, Georgia, New York, Rhode Island). 2011 Worked with a number of states on incorporating student growth percentiles into the ESEA waiver applications and teacher evaluation systems. Damian W. Betebenner On Student Growth & Accountability

References About the growth model The Student Growth Percentiles Methodology is a norm- and criterion-referenced growth analysis method. The norm-referenced component of the model is the student growth percentile which indicates the relative growth a student makes. The criterion-referenced component of the model is the student growth percentile target which indicates the amount of growth required by the student to reach/maintain desirable achievement goals in a specified time frame. For example, catching-up to proficiency within 3 years. The norm- and criterion-referenced components together allow stakeholders to understand what is, what should be, and what is reasonable simultaneously. The goal of the metric was to assist states to move beyond debates over what thermometer is best to discussions of how best to use the thermometer. Damian W. Betebenner On Student Growth & Accountability

References Built to maximize sharing A number of analytic tools (for doing calculations) and visualization tools have been developed with the goal to promote maximum sharing. Data isn’t valuable! It’s what one does with data that is potentially valuable! The analytic tools to calculate all the quantities associated with student growth percentiles are open source and freely available to all states. The visualizations associated with student growth percentiles are open source for non-commercial use. Conduct training sessions at national conferences to help other calculate these quantities (like Indiana) on their own. Build enhanced visualization and reporting systems to make data come alive to stakeholders and lead to better stakeholder engagement with data. Goal is to re-invent the user experience with education data and enable a “growth mindset”. Damian W. Betebenner On Student Growth & Accountability

References Lessons Learned On Questions John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. On Statistical Models George E. P. Box All models are wrong but some are useful. On Understanding Aristotle We understand best those things we see emerge from their very beginnings. On Accountability Sherlock Holmes It is a capital mistake to theorize before you have all the evidence. It biases the judg- ment. On Theories of Action Confucius Tell me and I will forget, show me and I will remember, involve me and I will understand. Damian W. Betebenner On Student Growth & Accountability

References Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner On Student Growth & Accountability

600 co 09 600 re le ca le ca S re 10 co

co th re ro i le le ce nt ca er th re 600 co le ca

References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Betebenner, D. W. (2012). Growth, standards, and accountability. In Cizek, G., editor, Setting Performance Standards: Foundations, Methods, and Innovations, pages 439–450. Routledge, New York. The Economist and Cukier, K. (2010). Data data everywhere. Damian W. Betebenner On Student Growth & Accountability

About this copy

On Student Growth & Accountability — ETS, May 2, 2012.

This is an image-based copy of the original beamer slides; builds and animations are flattened.