Talks archive, 2010–2014 – The Rhode Island Growth Model: Using Student Growth to Evaluate Teacher Performance

The Big Picture Guiding Principles and Questions The Rhode Island Growth Model References The Rhode Island Growth Model Using Student Growth to Evaluate Teacher Performance Ana Karantonis Rhode Island Department of Education September 23rd, 2010 Ana Karantonis The Rhode Island Growth Model

The Big Picture The Data Revolution Guiding Principles and Questions Why Growth? The Rhode Island Growth Model Accountability and Growth References From a Paucity to a Plethora of Data The Economist: The Data Deluge Historical records of student achievement Historical records of student demographics, teachers, schools, educational programs, . . . Stakeholder interest in an examination of this longitudinal data Interest in examining student achievement over time (student growth) derives from data availability. Ana Karantonis The Rhode Island Growth Model

The Big Picture The Data Revolution Guiding Principles and Questions Why Growth? The Rhode Island Growth Model Accountability and Growth References The Attractiveness of Growth What is growth and why measure it? Student learning is a central goal of education. Assessments of student achievement provide evidence of the current status of student knowledge and understanding. Learning is demonstrated by growth in student achievement from one point in time to another point in time–not by status at either point time alone. Ana Karantonis The Rhode Island Growth Model

The Big Picture The Data Revolution Guiding Principles and Questions Why Growth? The Rhode Island Growth Model Accountability and Growth References Accountability & Growth Growth versus Status Enthusiasm for growth in accountability stems from the belief that growth and teacher/school quality are more closely related than status. Enthusiasm for growth also stems from its potential diagnostic uses. How do we judge the use of growth related measures within an accountability system? Ana Karantonis The Rhode Island Growth Model

The Big Picture Guiding Principles and Questions Guiding Principles The Rhode Island Growth Model Guiding Questions References Guiding Principles John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. George E. P. Box All models are wrong but some are useful. Start at the end and work backward The best model answers the questions of greatest relevance to stakeholders. Ana Karantonis The Rhode Island Growth Model

The Big Picture Guiding Principles and Questions Guiding Principles The Rhode Island Growth Model Guiding Questions References Guiding Questions Growth models address specific questions Different growth analysis techniques are good at answering different questions. It is critical to understand these different questions. Different questions lead to different conversations which lead to different uses and outcomes. Changing conversations about education Starting with the right questions simplifies development and motivating the proper use of the growth model results. The questions set the table for those conversations. See Yen (2007) for an excellent list of questions derived from a survey of parents, teachers and administrators Ana Karantonis The Rhode Island Growth Model

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References The Rhode Island Growth Model How much growth did a student make? Is it enough? The Rhode Island Growth Model uses Student Growth Percentiles [Betebenner, 2008, Betebenner, 2009] to quantify individual student growth. This approach was developed and used by Colorado and is currently being employed in Massachusetts, Indiana, Virginia, and more than a dozen other states. Rhode Island has been investigating the model for uses with school improvement for the last 2 years. An important aspect of the model is that it separates the description of growth from discussions of responsibility/accountability. Ana Karantonis The Rhode Island Growth Model

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References Describing Student Growth Discussing student growth, even with a vertical scale (which NECAP does not possess), is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth and change require context. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a norm-referenced basis for describing student growth. Ana Karantonis The Rhode Island Growth Model

20 core ca ca le le 400 S re 2006 co

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References Student Growth Percentiles Should we be surprised with a child’s current achievement given their prior achievement? Student growth percentiles (SGPs) answer this question. Consider a low achieving student with 90th percentile growth and a high achieving student with 10th percentile growth. The low achieving student grew at a rate exceeding 90 percent of similar students. The high achieving student grew at a rate exceeding just 10 percent of similar students. The low achiever’s growth is more exemplary (probabilistically) than the high achiever’s. Judgments about the adequacy of student growth require external criteria together with standard setting. Ana Karantonis The Rhode Island Growth Model

Advanced Reading High Achievement CSAP Reading ● Typical Scale Score Proficient Growth Low ● Level Percentiles High 66th − 99th Typical 35th − 65th Part Proficient ● Low 1st − 34th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Next Year Scale Score 462 539 563 609 Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement Growth Percentile 66 66 90 Growth Level High High High Growth

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References From Students to Groups of Students Aristotle We understand best those things we see emerge from their very beginnings. It’s of interest to examine teachers and schools where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles (SGPs) associated with a school (or other grouping) one calculates the median of the student growth percentiles. Median SGPs can be calculated for one or several years across single and multiple subjects. Ana Karantonis The Rhode Island Growth Model

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References From Students to Groups of Students Based upon random sampling of SGPs, one expects to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Rhode Island is currently working on details of how these results will be associated with teachers. Based upon our work with schools, examining growth with achievement sheds new light on school performance. Ana Karantonis The Rhode Island Growth Model

Rhode Island: 2009 NECAP Growth & Achievement Growth and Achievement Reading Performance by School Poverty 100 Higher Achievement X Higher Achievement School Size 90 Lower Growth X X XX XX Higher Growth 50 Students X X X XX X X XX 100 Students X XX X X XX X X X XX XXX XX XX XX XX XX X X X X X X 250 Students XXXX XX XX X X X X XX X X X XXXXXX XX X X XX X 500 Students X XX 2009 Percent at/above Proficient XX X XX X XXXXX X XX XXX XX X X 70 X X XX XX X X X X XX X X XX X XXX X X X XX X XX X X X X XX Percent Free/Reduced X XXX X X XXX X X X Lunch Students X XXX X X XX X ● Less than 20 percent 40 X XX XX X X X ● 20 to 40 percent ● 40 to 60 percent ● 60 to 80 percent 20 ● More than 80 percent Lower Growth X Higher Growth 0 Lower Achievement Lower Achievement 2009 Median Student Growth Percentile

Rhode Island: 2009 NECAP Growth & Achievement Growth and Achievement NECAP Math Performance by School Poverty 100 Higher Achievement Higher Achievement School Size 90 Lower Growth X X XX X Higher Growth 50 Students X 100 Students XX XX 250 Students 80 XX X X XX X XX XX X X XX X X 500 Students 2009 Percent at/above Proficient XX X X X X X X XXX X X X XX X 70 X X X XXX XX X X XX X X XXXX XXX X XXX X X XX X X X X X XX XXXXXX X 60 XX X X X X X XX X X XX XX XXX X X X Percent Free/Reduced X XX X X XXXX X X X X Lunch Students XX X X X X ● Less than 20 percent XX XX X 40 XX X X XXXX X ● 20 to 40 percent XXX X XX X X X X X X XX XX X ● 40 to 60 percent X X XX X X X ● 60 to 80 percent 20 X X ● More than 80 percent Lower Growth X Higher Growth 0 Lower Achievement Lower Achievement 2009 Median Student Growth Percentile

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References Growth, Effectiveness, and Value-Added Fundamental Premise “Good” schools or teachers bring about student growth in ex- cess of that found at “bad” schools or teachers. In Rhode Island, “good” teachers are called highly effective teachers. What’s the relationship between growth and effectiveness? Effectiveness indicates who/what is responsible for the growth (value-added models). Assuming the above premise is, at least, partly true. How do we judge the correctness of a value-added model? Ana Karantonis The Rhode Island Growth Model

The Big Picture Describing Student Growth Guiding Principles and Questions Growth & Effectiveness The Rhode Island Growth Model Next Steps References Next Steps for Growth Data in Rhode Island Use of student growth for teacher evaluation has expedited work on and use of the Rhode Island Growth Model. RIDE is in the process of linking student NECAP data to unique teacher identifiers. In the coming weeks, RIDE will use preliminary teacher identifiers in a few districts to look at student growth for teachers. These preliminary results will be used to understand a range of issues including: The integrity of the teacher student links in the data files. The number of student NECAP results associated with teachers in a given year (minimum n). Empirical results to help set cuts distinguishing teacher performance. Ana Karantonis The Rhode Island Growth Model

The Big Picture Guiding Principles and Questions The Rhode Island Growth Model References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Yen, W. M. (2007). Vertical scaling and No Child Left Behind. In Dorans, N. J., Pommerich, M., and Holland, P. W., editors, Linking and Aligning Scores and Scales, pages 273–283. Springer, New York. Ana Karantonis The Rhode Island Growth Model

About this copy

The Rhode Island Growth Model: Using Student Growth to Evaluate Teacher Performance — Rhode Island Department of Education, September 2010.

This is an image-based copy of the original beamer slides; builds and animations are flattened.