Talks archive, 2010–2014 – Using SGP Data to Inform Practice

Introduction Student Growth Percentiles Student Growth Percentiles and Accountability References Using SGP Data to Inform Practice Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH February 27, 2012 Damian W. Betebenner Missouri SGP Webinar

Introduction About me Student Growth Percentiles From Data to Information to Knowledge Student Growth Percentiles and Accountability Bringing together diverse leadership References About Me Dr. Damian Betebenner, PhD - Senior Associate at the Center for Assessment (NCIEA). - Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009, Betebenner, 2012]. - Currently refining and sharing these techniques with other states including Colorado, Massachusetts, Indiana, Nevada, Hawaii and 25 other states in various stages of investigation/adoption. Damian W. Betebenner Missouri SGP Webinar

Introduction About me Student Growth Percentiles From Data to Information to Knowledge Student Growth Percentiles and Accountability Bringing together diverse leadership References My Role: Developing State Growth Models Education as a process (growth) instead of a destination (status). Growth/Value-Added Model Selection: Important but the easiest step (You have a growth/value-added model, now what). Identifying stakeholder groups who will make use (who we want to make use) of these data: Parents, teachers, principals, policy makers, . . . . The data can tell a thousand stories: Crafting data narratives that foster educational improvement. The data do not speak for themselves. My role: Educate state clients about growth and help them develop a new vocabulary that enables conversations about the progress of students through the education system. Damian W. Betebenner Missouri SGP Webinar

Introduction About me Student Growth Percentiles From Data to Information to Knowledge Student Growth Percentiles and Accountability Bringing together diverse leadership References My Role: Developing State Growth Models Design, first and foremost, with use in mind: How should the data be used. What are the conversations we want stakeholders (parents, teachers, principals, administrators, and policy makers) to have? How exactly will the data supplied enable those conversations. Turning data into information is primarily a social process. The leader’s role is to create the environment allowing the social process to flourish. Damian W. Betebenner Missouri SGP Webinar

Introduction About me Student Growth Percentiles From Data to Information to Knowledge Student Growth Percentiles and Accountability Bringing together diverse leadership References The Role of Leadership: Creating a Virtuous Data Cycle Assessment Data Quality Assessment Data Use Damian W. Betebenner Missouri SGP Webinar

Introduction About me Student Growth Percentiles From Data to Information to Knowledge Student Growth Percentiles and Accountability Bringing together diverse leadership References Developing a Successful Growth Model In the states demonstrating success in implementing student growth models, one commonality stands out: Commitment to the growth model by three key groups: Directors of Measurement/Assessment. Directors of Accountability. Directors of Information-Technology. Measurement/Assessment specialists understand the state assessments and will need to be able to understand and explain growth data derived from those assessments. Accountability directors will be involved in the use of these data, particularly as they pertain to accountability initiatives. CIOs/IT directors will often be tasked will helping to disseminate the results of the growth model. These directors become mavens for the work to others within their organization that reach out to the diverse range of stakeholders. Damian W. Betebenner Missouri SGP Webinar

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts Wisdom On Understanding Aristotle We understand best those things we see emerge from their very begin- nings. Damian W. Betebenner Missouri SGP Webinar

Billy Shin Sample School How to interpret this MAP growth & achievement report Advanced ● Comm Arts ● Achievement MAP Comm Arts ● MAP Scale Score ● Scale Score Proficient ● Growth ● Level Percentiles MAP High 66th − 99th Achievement Basic Levels Typical 35th − 65th Low 1st − 34th Below Basic Student Growth Grade 3 Grade 4 Grade 5 Grade 6 Next Year Percentile Scale Score 648 658 689 703 Achievement Level Proficient Basic Proficient Proficient Achievement Growth Percentile 31 76 93 Growth Level Low High High Growth Suggested Uses Review past growth to assess student progress toward MAP ● Math achievement goals Advanced Achievement ● Develop remediation or enrich− MAP Math ment plans based on rate of ● ● Scale Score growth needed to reach higher MAP achievement levels Growth Level Percentiles Proficient High 66th − 99th Basic Typical 35th − 65th Low 1st − 34th Below Basic Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 612 690 737 752 Achievement Level Basic Advanced Advanced Advanced Achievement Growth Percentile 99 94 87 Growth Level High High High Growth For more information please visit Missouri Department of Elementary and Secondary Education at http://dese.mo.gov/ or call 573−751−4212 Produced and distributed by the Missouri Department of Elementary and Secondary Education/Center for Assessment, Inc.by the Missouri Department of Elementary and Secondary Education & the Center for Assessment, Inc. Cooperatively developed

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner Missouri SGP Webinar

20 core ca ca le le 900 S re 2012 co

co th 800 re ro i le le nt ca er ce th 800 re co 700 le ca

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts Student Growth Percentiles Should we be surprised with a child’s current achievement given their prior achievement? Student growth percentiles answer this question. Consider a low achieving student with 75th percentile growth and a high achieving student with 25th percentile growth. The low achieving student grew at a rate exceeding 75 percent of their academic peers. The high achieving student grew at a rate exceeding just 25 percent of their academic peers. The low achiever’s growth is more exemplary (probabilistically) than the high achiever’s. Judgments about the adequacy of student growth require external criteria. Damian W. Betebenner Missouri SGP Webinar

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts Wisdom On Questions John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. On Statistical Models George E. P. Box All models are wrong but some are useful. Damian W. Betebenner Missouri SGP Webinar

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts What are the relevant questions for parents? Different stakeholder groups will have different questions [Yen, 2007]. Parent Questions Did my child make a year’s worth of progress in a year? Is my child making sufficient growth toward meeting state standards? Is my child growing at the same rate in Math and Reading or is she/he experiencing lower or higher growth in one content area compared to the other? Did my child grow as much this year as last year? Damian W. Betebenner Missouri SGP Webinar

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts What are the relevant questions for teachers? Different stakeholder groups will have different questions [Yen, 2007]. Teacher Questions Did my students make a year’s worth of progress in a year? Did my students make sufficient growth toward meeting state standards? Did my students make as much progress as students in other classrooms within my school or across schools within the complex area and state? Did my students learn as much in mathematics as they did in reading? Are there students with unusually low growth who need special attention? Do these growth results generally appear consistent with or differ from what I already know about how my students have progressed during the school year? Damian W. Betebenner Missouri SGP Webinar

Introduction Review Student Growth Percentiles Describing Student Growth Student Growth Percentiles and Accountability Student Growth Percentiles References Relevant Questions/Concepts What are the relevant questions for administrators? Different stakeholder groups will have different questions [Yen, 2007]. Administrator Questions Did the students in our district schools grow as much on average as other students in the state? Are our students making sufficient growth toward meeting Missouri state standards? Do students in a particular classroom/school/program show as much growth as other classrooms/schools/programs? Do students in certain demographic subgroups show as much growth as other demographic subgroups? Are our students growing consistently over multiple years? Damian W. Betebenner Missouri SGP Webinar

Introduction Student Growth Percentiles Teacher & School Accountability Student Growth Percentiles and Accountability Case Studies References Going from Students to Classrooms/Schools It’s of interest to examine schools, classrooms and programs where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles associated with a school (or other grouping) calculate the median of the student growth percentiles. If students were randomly assigned to the group, expect to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Examining growth with achievement sheds new light on school performance. Damian W. Betebenner Missouri SGP Webinar

Example District 42 School Performance Growth and Achievement 2011 Communication Arts Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2011 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

Example District 42 School Performance Growth and Achievement 2011 Mathematics Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2011 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

School Performance Growth and Achievement 2011 Communication Arts Higher Achievement Higher Achievement School Size Lower Growth X Higher Growth 50 Students X X X 100 Students 2011 Percent At/Above Proficient X X X 250 Students 500 Students XX X XXX X 70 XX X X X XX X XX XXX X X X X XX X XX X XX X XXXX X XX XX XX X X X XX X X X XX XXXX X XX X XXXXXXXX X XX X X X XX X X XX X XX XX X X XXX X X XX X X XXXX X X XXXX X X X XX X X X X XX X XX X XX X X X X X X XX XX X X X X X X X X X X X X XXX X XX X X X X Percent Free/Reduced X XX XX X X X X X XX X XXXX X X XX X X X X XXX X XX X X X XXX XX X Lunch Students XX X XX X X X X ● Less than 20 percent ● 20 to 40 percent X XX ● 40 to 60 percent ● 60 to 80 percent Achievement ● More than 80 percent Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

School Performance Growth and Achievement 2011 Communication Arts Higher Achievement Higher Achievement School Size Lower Growth X Higher Growth 50 Students 100 Students 2011 Percent At/Above Proficient 250 Students 500 Students X XX X X X XX X X X X XX X X X XX XX X XXX X X X X X 60 X XX X X X XX X X X XX X XX XX X XX X X X X X XX XX X XX X X X X X XXXXXXXX XXX X XX XX XXXX XXX X XXX X XXX X X X XX X XX X X XX X XXX XXX XX XXX X XX X X Percent Free/Reduced X X X X XX XXX XXX X X X X XXX X X X X X XX XXX X XX X X XXX X X XX X X XX X XX XXXXX Lunch Students X X X X X X XXX XXX XXXX XXX X X X XX XX XX X X XXX X X X XXX XXXXXXX X XX XX X X XXX XX X X X XX X X X X XXX ● Less than 20 percent X XX XXX X X X X X XX X X X XX X X XXX X X XXXXX X XX X XXXXX X XX XX XX XX XX XX X X X XX XX XX XX ● 20 to 40 percent X XX X X XX X X X XX X X XX X X XXX X XX XX X X X X X XX X X XX X ● 40 to 60 percent XX X X ● 60 to 80 percent Achievement ● More than 80 percent Lower Growth X Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

School Performance Growth and Achievement 2011 Communication Arts Higher Achievement Higher Achievement School Size Lower Growth X Higher Growth 50 Students 100 Students 2011 Percent At/Above Proficient 250 Students 500 Students X XX X X X X XXX X XX X XX Percent Free/Reduced X XXX X XX XX X XXXX X XX XXX X X X X XX X X XX X X X X XX X X X XX X X X X X XXX X X XX XX XXX X XXX X X X XX X X X XXX Lunch Students XX X X XXXX XX XX X XXX X X X X XXXXX X X XXX XX XX XX X X XXX X ● Less than 20 percent X X X X XXX X X X X XXX X XX X XX XXXXXXX X XXX XXX X XX XX XXXX XXX X X X XX X X XX XX X X X X X XX XX XX X X XX X XXX X X X X XX X X XX XXX XXXX X X X XX X XXX XXX X X XX XX X X X X XX X XX XX X X XX X X ● 20 to 40 percent X X XXXXX X X X XX X XX X XX X XXXX XXX X X X XX X XX XX ● 40 to 60 percent X XX X X X X ● 60 to 80 percent Achievement X XX X X ● More than 80 percent Lower Growth X Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

School Performance Growth and Achievement 2011 Communication Arts Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2011 Percent At/Above Proficient 250 Students 500 Students X XX X Percent Free/Reduced X Lunch Students X XX X XXX ● Less than 20 percent X X XX XX X ● 20 to 40 percent ● 40 to 60 percent X X X XX X XX X X X X X X X XX X X X 30 XX X XX X X ● 60 to 80 percent Achievement XX XX X X XX X X X X X XX X X X XX X X X X XX X X XX XX XX X XX X X X X XX ● More than 80 percent X X X XXX X Lower Growth X X X X X XX X X XXX XX X X Higher Growth Lower Achievement Lower Achievement Growth 2011 Median Student Growth Percentile

Introduction Student Growth Percentiles Teacher & School Accountability Student Growth Percentiles and Accountability Case Studies References Using SGPs to Inform Practice Massachusetts rollout of student growth percentile data to pilot districts: School Transitions Boston Public Schools investigation of summer reading program efficacy. Newton School District investigation of technology efficacy. Damian W. Betebenner Missouri SGP Webinar

Introduction Student Growth Percentiles Teacher & School Accountability Student Growth Percentiles and Accountability Case Studies References Wisdom On Theories of Action Confucius Tell me and I will forget, show me and I will remember, involve me and I will understand. Damian W. Betebenner Missouri SGP Webinar

Introduction Student Growth Percentiles Student Growth Percentiles and Accountability References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Betebenner, D. W. (2012). Growth, standards, and accountability. In Cizek, G., editor, Setting Performance Standards: Foundations, Methods, and Innovations, pages 439–450. Routledge, New York. Yen, W. M. (2007). Vertical scaling and No Child Left Behind. In Dorans, N. J., Pommerich, M., and Holland, P. W., editors, Linking and Aligning Scores and Scales, pages 273–283. Springer, New York. Damian W. Betebenner Missouri SGP Webinar

About this copy

Using SGP Data to Inform Practice — Missouri Department of Education, February 27, 2012.

This is an image-based copy of the original beamer slides; builds and animations are flattened.