Student Growth Percentiles: An Overview

New Mexico Department of Education · September 11, 2024

Student Growth Percentiles: An Overview — New Mexico Department of Education, September 11, 2024.
Published

September 11, 2024

Event
State agency
Slides
53
Format
Converted from Beamer

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Background Student Growth Percentiles Student Growth Percentiles and Accountability References Student Growth Percentiles: An Overview Damian W. Betebenner & Adam VanIWaarden National Center for the Improvement of Educational Assessment Dover, NH Albuquerque, NM September 11th, 2024 Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  2. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions About Me Dr. Damian Betebenner, PhD - Senior Associate at the Center for Assessment (NCIEA). - Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009, Betebenner, 2012]. - Currently refining and sharing these techniques with more than 25 states in the United States, testing companies (Renaissance Learning), and other countries including Australia. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  3. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions About Me Dr. Damian Betebenner, PhD - Educational Background: PhD in Education Measurement from the University of Colorado, Boulder and a PhD in Mathematics from the University of Wyoming. - Began using R in 1999 to produce high quality graphics and it has been seminal in expanding my work. - I’m more of a "data scientist" than a statistician. SGPs are more of a "data science" approach to examining student academic growth than a "statistician" approach. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  4. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions History 2004-2007 Assistant Professor Boston College, ERME. 2012 Worked with 5 states on 2005-2007 Began development of the Colorado co-developing SchoolView interactive Growth Model (Student Growth visualizations to help better Percentiles). communicate student growth 2008 Successful application of Colorado percentile results. Growth Model (criterion-referenced 2014 Built SIMEX measurement error component) as part of growth model correction into SGP pilot program. model [Shang et al., 2015] 2009-2010 Began/continued work with 2014 Extended analyses to accommodate approximately two-dozen other states time-dependency allowing for SGP on developing state growth models. modelling of interim assessments. 2010 Worked with states on incorporating 2017 Extended SGP package to better growth into RttT applications and with accommodate assessment successful states on their use transitions. (Massachusetts, Georgia, New York, 2015-present Work with about 20 states in different Rhode Island). capacities to calculate SGPs. 2011 Worked with a number of states on 2020-2024 Utilized SGPs to examine academic incorporating student growth impact and recovery due to the percentiles into the ESEA waiver COVID-19 pandemic. applications and teacher evaluation systems. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  5. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions From a Paucity to a Plethora of Data Data Data Everywhere (The Economist, 2010) More and better organized data: mega, giga, tera, peta, exa, zetta, yotta! Historical records of student achievement. Linkage of student achievement to student demographics, teachers, schools, educational programs, . . . Stakeholder interest in examining student achievement over time (student growth) derives from data availability.. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  6. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Lessons Learned On Questions John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. On Statistical Models George E. P. Box All models are wrong but some are useful. On Understanding Aristotle We understand best those things we see emerge from their very beginnings. On Accountability Sherlock Holmes It is a capital mistake to theorize before you have all the evidence. It biases the judgment. On Theories of Action Confucius Tell me and I will forget, show me and I will remember, involve me and I will understand. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  7. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions The Attractiveness of Growth What is growth and why measure it? Student learning is a central goal of education. Assessments of student achievement provide evidence of the current status of student knowledge and understanding. Learning is demonstrated by growth in student achievement from one point in time to another point in time–not by status at either point time alone. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  8. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Guiding Questions Growth models address specific questions Different growth analysis techniques are good at answering different questions. It is critical to understand these different questions. Different questions lead to different conversations which lead to different uses and outcomes. Changing conversations about education Starting with the right questions simplifies development and motivating the proper use of the growth model results. The questions set the table for those conversations. Yen (2007) for an excellent list of questions derived from a survey of parent, teachers and administrators Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  9. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Guiding Questions Growth models address specific questions Different growth analysis techniques are good at answering different questions. It is critical to understand these different questions. Different questions lead to different conversations which lead to different uses and outcomes. Changing conversations about education Starting with the right questions simplifies development and motivating the proper use of the growth model results. The questions set the table for those conversations. Yen (2007) for an excellent list of questions derived from a survey of parent, teachers and administrators Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  10. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions What are the relevant questions for parents? Yen (2007), from a state survey of parents, teachers and administrators, compiled a list of frequently voiced questions/concerns by stakeholder group. Parent Questions Did my child make a year’s worth of progress in a year? Is my child growing appropriately toward meeting state standards? Is my child growing as much in Math as Reading? Did my child grow as much this year as last year? Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  11. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions What are the relevant questions for teachers? Yen (2007), from a state survey of parents, teachers and administrators, compiled a list of frequently voiced questions/concerns by stakeholder group. Teacher Questions Did my students make a year’s worth of progress in a year? Did my students grow appropriately toward meeting state standards? How close are my students to becoming Proficient? Are there students with unusually low growth who need special attention? Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  12. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions What are the relevant questions for administrators? Yen (2007), from a state survey of parents, teachers and administrators, compiled a list of frequently voiced questions/concerns by stakeholder group. Administrator Questions Did the students in our district/school make a year’s worth of progress in all content areas? Are our students growing appropriately toward meeting state standards? Does this school/program show as much growth as that one? Can I measure student growth even for students who do not change proficiency categories? Can I pool together results from different grades to draw summary conclusions? Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  13. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions What are the relevant questions for policy makers? Policy Maker Questions Can/how should student growth be used in judgments of educator quality? Can/how should student growth be used in judgments of education (e.g., school) quality? Are students on track for career and college readiness? What adjustments for student demographics need to be considered? Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  14. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions About the growth model The Student Growth Percentiles Methodology is a norm- and criterion-referenced growth analysis method [Betebenner, 2009]. The norm-referenced component of the model is the student growth percentile which indicates the relative growth a student makes. The criterion-referenced component of the model is the student growth percentile target which indicates the amount of growth required by the student to reach/maintain desirable achievement goals in a specified time frame. For example, catching-up to proficiency within 3 years. The norm- and criterion-referenced components together allow stakeholders to understand what is, what should be, and what is reasonable simultaneously. The goal of the metric was to assist states to move beyond debates over what thermometer is best to discussions of how best to use the thermometer. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  15. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Built to maximize sharing A number of analytic tools (for doing calculations) and visualization tools have been developed with the goal to promote maximum sharing. Data isn’t valuable! It’s what one does with data that is potentially valuable! The analytic tools to calculate all the quantities associated with student growth percentiles are open source and freely available to all states [Betebenner et al., 2024]. sgp.io Conduct training sessions at national conferences to help others calculate these quantities on their own. Build enhanced visualization and reporting systems to make data come alive to stakeholders, allow for stakeholder collaboration and sharing, and lead to better stakeholder engagement with data. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  16. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Growth & Accountability Growth versus Achievement Enthusiasm for growth in accountability stems from the belief that learning (i.e., growth) and teacher/school quality are more closely related than status (i.e., status, attainment). Enthusiasm for growth also stems from its potential diagnostic uses. Growth and Achievement Initial addition to include growth was done as a pivot away from achievement. As pointed out yesterday, growth is directly connected to achievement: Growth (proximal indicator) leads to achievement (distal indicator). Growth is an indicator that gives some insight into process whereas achievement is an indicator that gives some insight into outcome. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  17. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Growth & Accountability Fundamental Premise “Good” schools/teachers sustain learning (i.e., bring about growth in student achievement) in excess of that found at “bad” schools/teach- ers. The inherent believability of this premise is at the heart of current enthusiasm to incorporate growth models into accountability systems. Note that current accountability systems (from NCLB through ESSA) invoke the same premise (with achievement substituted for growth). “Good schools/teachers” are commonly referred to as highly effective schools/teachers. It is critical to understand the distinction between growth and effectiveness. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  18. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Growth & Accountability: The Colorado Growth Model How much growth & is it enough? When initially rolled out in Colorado 15 years ago the goal was to focus conversations around two questions: How much: How much growth did a student make? Is is enough: Is it enough growth to reach or maintain proficiency? Answering these two questions in a credible and clear manner sets the stage for subsequent accountability discussions. Colorado purposely separated the description (what is) from the determination of responsibility (who/what is responsible). This separation promoted greater ownership of results and participation in finding root causes. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  19. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Value-Added & Growth Who or what is responsible for student Often, analyses of student growth attempt to determine the amount of student progress/growth that can be attributed to the school or teacher. Called Value-added analyses, these techniques attempt to estimate the teacher/school contribution to student achievement. Value-added models quantify effectiveness—purporting to measure the impact on achievement that this school or teacher would have, on average, upon similar students assigned to them. These models and their effectiveness claims are controversial for a number of reasons (EPI (2010), NRC/NAE (2010), Braun (2005), Rubin and Stuart (2004)).[Economic Policy Institute, 2010, National Research Council and National Academy of Education, 2010, Braun, 2005, Rubin et al., 2004] Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  20. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Value-Added & Growth Who or what is responsible for student Value-added models are better described as effectiveness models than as growth models. Value-added is an inference – a causal conclusion drawn from the data. Value-added isn’t a type of statistical model or type of assessment. All growth models can be used for value-added purposes but not vice-versa. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  21. About me Background The Data Revolution Student Growth Percentiles Lessons Learned Student Growth Percentiles and Accountability Why Growth? References Different Models/Different Questions Different Models/Different Questions It’s all about conversations starting with questions Descriptive Accountability Accountability system results can have value without making causal inferences about school quality, solely from the results of student achievement measures and demographic characteristics. Treating the results as descriptive information and for identification of schools that require more intensive investigation of organizational and instructional process characteristics are potentially of considerable value. Rather than using the results of the accountability system as the sole determiner of sanctions for schools, they could be used to flag schools that need more intensive investigation to reach sound conclusions about needed improvements or judgments about quality [Linn, 2008, p. 21]. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  22. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Describing Student Growth The concept of student growth isn’t as difficult to understand and popular perception suggests. Calculations may be complicated but knowing how to calculate student growth is not synonymous with understanding student growth: Thenumber π (pi=3.14159...)is difficult to calculate but is simple to understand conceptually. Height and weight percentiles associated with children are difficult to calculate but are simple to understand conceptually. Computational details are important but often obscure a simple conceptual basis. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  23. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  24. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References The Illusion of Vertical Scales Personal Bests in 2023 and 2024 for Two High Jumpers World Record: 6 feet, 10 1/2 inches Whose high jump changed more? Explain your answer. 7 6 feet, 6 inches Judy Height (feet) Whose high jump change is 6 6 feet, 4 inches more remarkable? Explain your 5 answer. 4 4 feet, 0 inches Anna Which high jumper improved 3 3 feet, 6 inches more? Explain your answer. Spring 2023 Spring 2024 Track Season Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  25. 460 co 23 360 re le ca le ca S re 24 co
  26. 460 co 23 360 re le ca le ca S re 24 co
  27. 460 co 23 360 re le ca le ca S re 24 co
  28. co re th ro le i le ca nt er ce S th re 360 co le ca
  29. Peer 3 6 0 m i re ade Sco Ac cale 202 75% 75th er ce nt le G wth ro 460 co 23 360 re S le S ca le S ca S re 20 24 co
  30. Th is d 75th s: dat erive perc eer 3 6 0 acaa and from ntile c P = dem co a cu ic p vers ll the t m i re eer all ade Sco g ro Ac cale ups 202 75% 75th er ce nt le G wth ro 460 co 23 360 re S le S ca le S ca S re 20 24 co
  31. eer 3 3 0 cPe= Th m i r is d 75th ade Sco dat erive perc acaa and from ntile Ac cale dem co a cu ic p vers ll the t 3S eer all g ro ups 75 th er nt ce le G wth ro 460 co 23 360 re le ca le ca S re 24 co
  32. Th is e dade 75 a c t a r iv t h a d a ed p emnd f er ic coromce peve natil errs ll e c gralltheut ou ps % 7 5 th P 30 75 er ce 0 nt 0 il e Gr ow th Sc or al 36 0 Sc Sc 43 al or Sc
  33. Th is e dade 75 a c t a r iv t h a d a ed p emnd f er ic coro cmen peve atil errs ll e c gralltheut ou ps 75 th Pe 30 r ce 0 nt il e Gr ow th Sc or al 36 0 Sc Sc 43 al or Sc
  34. Th is e dade 75 a c t a r iv t h a d a ed p emnd f er ic coro cmen peve atil errs ll e c gralltheut ou ps % 7 5 th P 30 75 er ce 0 nt 0 il e Gr ow th Sc or al 36 0 Sc Sc 43 al or Sc
  35. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Student Growth Percentiles Should we be surprised with a child’s current achievement given their prior achievement? Given a student’s prior scale scores and the associated conditional density, their current scale score corresponds to a percentile of that conditional distribution. This percentile is the student’s growth percentile. Growth percentiles are closely related to estimating the probability of observing a student’s current achievement taking account of their past achievement: Pr(Current Achievement |Past Achievement ). As such, growth percentiles describe the rarity of a student’s current achievement conditional upon their prior achievement. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  36. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Student Growth Percentiles Should we be surprised with a child’s current achievement given their prior achievement? Student growth percentiles answer this question. Consider a low achieving student with 90th percentile growth and a high achieving student with 10th percentile growth. The low achieving student grew at a rate exceeding 90 percent of similar students. The high achieving student grew at a rate exceeding just 10 percent of similar students. The low achiever’s growth is more exemplary (probabilistically) than the high achiever’s. Judgments about the adequacy of student growth require external criteria. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  37. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Model for Student Growth Percentiles Student growth percentiles are calculated using quantile regression with B-spline smoothing, a generalized additive model. Quantile regression is used to model the complete distribution of the response variable (current achievement) B-spline smoothing is uses to accommodate non-linearity and heteroscedasticity of the data. The model/method is descriptive and all about data fit—it’s a data mining procedure. The model quantifies distance = rate · time probabilistically. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  38. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Model for Student Growth Percentiles Conditional Decile Regression Curves Conditional Decile Regression Lines 2006−07 Math: Grade 5 versus 6 2006−07 Math: Grade 5 versus 6 1000 Unsatisfactory Partially Prof. 1000 Unsatisfactory Partially Prof. Proficient Proficient Partially Prof. Proficient Advanced Partially Prof. Proficient Advanced Grade 6 Scale Score Grade 6 Scale Score Grade 5 Scale Score Grade 5 Scale Score Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  39. Background Describing Student Growth Student Growth Percentiles Student Growth Percentiles Student Growth Percentiles and Accountability Criterion Referenced Growth References Establishing growth-standards based upon growth-norms The most common adequacy criterion are judging growth toward an achievement goal like proficiency, English Language master (i.e., growth-to-standard). Results from student growth percentile analyses can be used to calculate percentile growth trajectories for each student. These trajectories indicate what future rates of growth will lead to and are used to make adequacy judgments. This growth-to-standard approach was approved as part of Colorado’s successful application to the Growth Model Pilot Program. Growth-to-standard is most often used for English Language Proficiency testing (e.g., WIDA-ACCESS in Massachusetts) Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  40. Eduardo Manriquez Sample School Advanced How to interpret this student English growth & achievement report Language Arts Proficient Achievement MSSA MSSA English Language Arts Scale Score Scale Score Growth Near Proficient > 1 Year Level Percentiles MSSA About 1 Year > 1 Year 66th − 99th Achievement Levels < 1 Year About 1 Year35th − 65th Novice < 1 Year 1st − 34th Student Growth −− Grade 4 Grade 5 Grade 6 Percentile Scale Score −− 400 502 Achievement Level −− Novice Novice Achievement Growth Percentile −− 13 Growth Level −− < 1 Year Growth Suggested Uses Advanced Review past growth to assess student academic progress toward Math MSSA achievement goals. Proficient Achievement Develop remediation or enrich− MSSA Math ment plans based on rate of Scale Score growth needed to reach higher MSSA achievement levels. Growth Near Proficient Level Percentiles Identify the rate of progress needed in order to reach or > 1 Year > 1 Year 66th − 99th maintain proficient status About 1 Year on the MSSA next year. About 1 Year35th − 65th Novice < 1 Year < 1 Year 1st − 34th −− Grade 4 Grade 5 Grade 6 Scale Score −− 422 532 Achievement Level −− Novice Novice Achievement Growth Percentile −− 39 Growth Level −− About 1 Year Growth For more information please visit the New Mexico Public Education Department (NMPED) at http://ped.state.nm.us or contact 505−827−5800 Cooperatively developed by NMPED & the Center for Assessment, Inc. Distributed by NMPED.
  41. Brian Bollig Sample School How to interpret this student growth & achievement report Mathematics Advanced Achievement DEMO Catch Up DEMO Mathematics Scale Score Scale Score Catch Up Target Target Growth Proficient Level Percentiles DEMO Very High 81st − 99th Achievement High 61st − 80th Part Proficient Levels Typical 40th − 60th Low 20th − 39th Unsatisfactory Very Low 1st − 19th Student Grade 3 Grade 4 Grade 5 Grade 6 Growth 2021−2022 2022−2023 2023−2024 2024−2025 Percentile Scale Score 460 418 490 Achievement Level Achievement Target Proficient Part Proficient Proficient (484) Part Proficient Proficient (513) Achievement Growth Percentile 2 79 Catch Up/Keep Up Growth Level Very Low High Growth Growth Target Catch Up (74) Catch Up (70) Move Up/Stay Up Targets Suggested Uses Advanced Reading Achievement DEMO Reading Scale Score Review past growth to assess Catch Up student academic progress toward Target Growth DEMO achievement goals. Proficient Catch Up Target Level Percentiles Develop remediation or enrich− Very High 81st − 99th ment plans based on rate of High 61st − 80th growth needed to reach higher DEMO achievement levels. Part Proficient Typical 40th − 60th Low 20th − 39th Identify the rate of progress Very Low 1st − 19th needed in order to reach or Unsatisfactory maintain proficient status Grade 3 Grade 4 Grade 5 Grade 6 on the DEMO next year. 2021−2022 2022−2023 2023−2024 2024−2025 Scale Score 549 546 563 Achievement Level Achievement Target Proficient Part Proficient Proficient (581) Part Proficient Proficient (598) Achievement Growth Percentile 8 38 Growth Level Growth Target Very Low Low Catch Up (60) Catch Up (73) Growth For more information please visit the Student Growth Percentiles Package (the SGP Package) at www.sgp.io or contact dbetebenner@nciea.org Cooperatively developed by the SGP Package & the Center for Assessment, Inc. Distributed by the SGP Package.
  42. New Mexico: 2023 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 1/Level 2 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  43. New Mexico: 2023 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 2/Level 3 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  44. New Mexico: 2023 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 3/Level 4 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  45. New Mexico: 2023 ELA Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 1/Level 2 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  46. New Mexico: 2023 ELA Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 2/Level 3 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  47. New Mexico: 2023 ELA Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2023 Achievement Percentile SGP trajectories for a 3rd grade student starting from the Level 3/Level 4 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
  48. Background Student Growth Percentiles School Accountability Student Growth Percentiles and Accountability References Going from Students to Schools It’s of interest to examine schools where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles associated with a school (or other grouping) calculate the median of the student growth percentiles. If students were randomly assigned to schools, expect to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Examining growth with achievement sheds new light on school performance. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview
  49. DEMONSTRATION School Performance DEMO Growth & Prior Achievement 2023−2024 DEMO Mathematics 2022−2023 Prior Percent at/above Proficient Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Summary Statistics Correlation: 0.43 Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.43 Growth 2023−2024 Median Student Growth Percentile
  50. DEMONSTRATION School Performance DEMO Growth & Prior Achievement 2023−2024 DEMO Reading 2022−2023 Prior Percent at/above Proficient Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Summary Statistics Correlation: 0.39 Lower Achievement Lower Achievement Correlation (N ≥ 22): 0.39 Growth 2023−2024 Median Student Growth Percentile
  51. DEMONSTRATION School Performance DEMO Growth & Achievement 2023−2024 DEMO Mathematics Higher Achievement Higher Achievement School Size 2023−2024 Percent at/above Proficient Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Summary Statistics Correlation: 0.55 Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.55 Growth 2023−2024 Median Student Growth Percentile
  52. DEMONSTRATION School Performance DEMO Growth & Achievement 2023−2024 DEMO Reading Higher Achievement Higher Achievement School Size 2023−2024 Percent at/above Proficient Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Summary Statistics Correlation: 0.52 Lower Achievement Lower Achievement Correlation (N ≥ 22): 0.52 Growth 2023−2024 Median Student Growth Percentile
  53. Background Student Growth Percentiles Student Growth Percentiles and Accountability References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Betebenner, D. W. (2012). Growth, standards, and accountability. In Cizek, G., editor, Setting Performance Standards: Foundations, Methods, and Innovations, pages 439–450. Routledge, New York. Betebenner, D. W., Iwaarden, A. V., Domingue, B., and Shang, Y. SGP: Student growth percentiles & percentile growth trajectories. Available online complete with source code from https://sgp.io. Braun, H. I. (2005). Using student progress to evaluate teachers: A primer on value-added models. Technical report, Educational Testing Service, Princeton, New Jersey. Economic Policy Institute (2010). Problems with the use of student test scores to evaluate teachers. Technical report, Washington, D.C. Downloaded August 30th, 2010 from http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf. Linn, R. L. (2008). Educational accountability systems. In The Future of Test-Based Educational Accountability, pages 3–24. Taylor & Francis, New York. National Research Council and National Academy of Education Getting value of out value-added: Report of a workshop. Technical report, Washington, D.C. Commitee on Value-Added Methodology for Instructional Improvement. Rubin, D. B., Stuart, E. A., and Zanutto, E. L. (2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29(1):103–116. Shang, Y., VanIwaarden, A., and Betebenner, D. W. (2015). Covariate measurement error correction for student growth percentiles using the simex method. Educational Measurement: Issues and Practice, 34(1):4–14. The Economist and Cukier, K. (2010). Data data everywhere. Yen, W. M. (2007). Vertical scaling and No Child Left Behind. In Dorans, N. J., Pommerich, M., and Holland, P. W., editors, Linking and Aligning Scores and Scales, pages 273–283. Springer, New York. Damian W. Betebenner & Adam VanIwaarden Student Growth Percentiles: An Overview