How Student Growth Percentiles Transform ERB Data into Actionable Insights

NJASA · April 4, 2019

How Student Growth Percentiles Transform ERB Data into Actionable Insights — NJASA, April 4, 2019.
Published

April 4, 2019

Event
Conference
Slides
21
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. How Student Growth Percentiles Transform ERB Data into Actionable Insights Damian W Betebenner National Center for the Improvement of Educational Assessment Austin, TX April 4th, 2025
  2. About Me • Recovering Academic • At the Center for Assessment since 2007 • Developed student growth percentile (SGP) methodology that is currently used by approximately with 25 state assessments as well as ERB and with several other educational assessments. • Currently live just outside of Boston, Massachusetts (Needham, MA) with wife, daughter, and dog.
  3. Topics • What is student growth and why is it critical to pay attention to? • What is a student growth percentile (SGP)? • How to turn SGPs into actionable insights?
  4. What is student growth?
  5. Terminology • Achievement: A point in time measure/indication of what a student knows and can do. Also referred to as status, attainment. Reported as a scaled score, percentile • Growth: The “change” in the level of achievement associated with an individual student measure at least two points in time. • Improvement: The “change” in the level of achievement associated with two groups of students formed by school or demographic groups (e.g., 2024 versus 2025 outcomes for Grade 4 students in a given school). This is the most common metric of change. • Not all change is “growth”. There are different types of changes and
  6. Why Student Growth? If you care about student achievement, then you care about student growth. • Where you are is a function of where you started and how quickly you’ve travelled since. • Student growth and student attainment/achievement are related in this way. • Distance=Rate x Time • Those who start behind tend to stay behind and those that start ahead tend to stay ahead.
  7. Student Growth Introductory Concepts A Math Story Problem • Whose high jump changed more? Explain your answer. • Whose high jump change is more remarkable? Explain your answer. • Which high jumper improved more? Explain your answer.
  8. Student Growth: Introductory Concepts • There are 3 general approaches of calculating student growth: • Growth as an amount of change (i.e., gain). • Growth as a norm/peer referenced quantity. • Growth as a criterion referenced quantity.
  9. Student Growth: Introductory Concepts • Growth as an amount of change: • What could be more simple? • Like height or weight just calculate the amount of change. • Requires a vertical scale (which ERB has) • Growth = Score Time 2 - Score Time 1 • Not as simple as it seems: • Comparisons between students often leads to wrong conclusions • Interpreting change requires context.
  10. Student Growth: Introductory Concepts Back to the story problem!
  11. Student Growth: Introductory Concepts • Which high jumper improved more: ▪ Judy improved 2 inches. ▪ Anna improved 6 inches. ▪ An inch is an inch!!! ▪ But is an inch really an inch??? ▪ 2 inches for an elite jumper is far more remarkable than 6 inches for a novice.
  12. Student Growth: Introductory Concepts •This is analogous to height/weight measures for infants. •Growth amounts are tricky to interpret and require context. •Peer based comparisons used by growth analyses are a way to acquire that context. An infant male toddler is measured at 2 and 3 years of age and is shown to have grown 4 inches. The magnitude of increase—4 inches—is a well understood quantity that any parent can grasp and calculate at home using a simple yardstick. However, parents leaving their pediatrician’s office knowing only how much their child has grown would likely be wanting for more information: Parents are not interested in an absolute magnitude of growth, but instead in a norm-referenced quantity locating that 4 inch increase alongside the height increases of similar children. 12
  13. Student Growth: Introductory Concepts • Peer-based analyses are also called norm-referenced analyses. • Like with infant height/weight charts, peers are children/student with similar characteristics. • For growth analyses using large scale assessments, peers are students with (at least) the same scale score history on the same tests. • Peer based norms are invaluable in providing context for stakeholders to understand student growth. • The ERB SGP Model creates growth norms for all students using regression (i.e., peer-based) analyses. • These norms allow stakeholders to understand what an ambitious yet reasonable expectation is for each student.
  14. What is a student growth percentile (SGP)?
  15. Student Growth Percentiles • Research on the SGP model began in 2007. • Based upon work in Colorado, the norm-referenced model was extended to include a criterion-referenced component. • Work in other states incorporating the model into state reporting and accountability systems occurred in subsequent years and student growth became more common across state assessment systems. • Goal or the model is to create an individual level growth metric that is informative to different stakeholders ranging from parents to policy makers. • Currently, more than two dozen states and several assessment organizations use SGPs for diagnostic and accountability purposes.
  16. Student Growth Percentiles • Different growth models answer different questions. • Different stakeholder groups ask different questions. • Parents are most concerned about how their child is doing. • To effectively engage parents and communities in a productive conversation about student learning (i.e. growth), a growth model should describe: • What is? How much are students learning/growing. • What should be? How much would we like students to grow/learn. • What is reasonable? How much is it reasonable to expect students to grow/ learn. • Most other growth models do not provide a student level metric. They provide a school summary used for accountability purposes.
  17. Growth Model: Other Options • Almost all current growth model use large amounts of data to create “norms” against which student growth is judged. • Growth models utilize various forms of regression to adjust for initial student achievement (and, in some instances) other demographic characteristics. • Other growth models include value-added models, value- table models
  18. History of student growth with ERB • Began work with ERB to develop SGPs on their suite of assessments in 2022. • Worked with Dr. Adrienne Hu to refine the methodology to ERB assessment. • ERB began including these data in 2024.
  19. Properties of SGP growth • All students regardless of starting place can have high or low growth — growth is uncorrelated with prior achievement. • Growth is uncorrelated from year-to-year. High growth one year doesn’t indicate growth in the following year. • Growth analyses do not include demographic adjustments. Growth differs for different demographic subgroups. • Growth by ethnicity ranges from a median SGP of low 40s to high 50s (depending upon grade and content area) • Economically disadvantaged students have median SGP in mid to low 40s (depending upon grade and content area) • Special needs students have median SGP in low 40s (depending upon grade and content area) • Included the calculation of student growth percentiles and student growth projections (which provide SGTs). • When New Jersey transitioned to PARCC, NJ received both state and consortium normed . • Used (and not used) in different ways over the last decade.
  20. Differential Impact versus Bias • Differences in growth results (differential impact) do not necessarily constitute bias. • The measure can show differences without necessarily being “biased” • However, differential impact, particularly when used for accountability, can be VERY thorny issue. • I advocate for distinguishing between the MEASURE (e.g., status, growth, …) and the EVALUATION done with the measure: • The evaluation may involve adjustments for circumstances. • Those adjustments don’t need to be baked into the measure.