RELPA Student Growth Models Slides to Damian

Hawaii REL · September 29, 2016

RELPA Student Growth Models Slides to Damian — Hawaii REL, September 29, 2016.
Published

September 29, 2016

Event
Invited talk
Slides
24
Format
Converted from PowerPoint

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Dr. Damian Betebenner The National Center for the Improvement of Educational Assessment
  2. CHANGING CONVERSATIONS ABOUT STUDENT PERFORMANCE
  3. The State of Student Growth • The existence of annual student testing together with unique student identifiers forms the foundation for the calculation of student growth. • Over the last decade, most states have begun calculation of longitudinal student growth using large scale state assessments. • Analysis of student growth is controversial because of how it is most often used: For teacher and school evaluation. • For example, the development of the current Hawaii Growth Model involved the creation of a diagnostically useful quantity that could be used (or not used) at a later time for many purposes. 3
  4. Why Student Growth? If you care about student achievement, then you care about student growth. • Where you are is a function of where you started and how quickly you ve travelled since. • Student growth and student attainment/achievement are related in this way. • Distance = Rate x Time • Those who start behind tend to stay behind and those that start ahead tend to stay ahead.
  5. Student Growth: Introductory Concepts • The concept of student growth isn’t as difficult to understand and popular perception suggests. • Calculations may be complicated but knowing how to calculate student growth is not synonymous with understanding student growth: • The number π (pi=3.14159…) is difficult to calculate but is simple to understand conceptually. • Height and weight percentiles associated with children are difficult to calculate but are simple to understand conceptually. • Computational details are important but often obscure a simple conceptual basis.
  6. Pause and Reflect – Poll Please see the questions in the poll response box to your right.
  7. Student Growth: Introductory Concepts • There are 3 general approaches of calculating student growth: • Growth as an amount of change (i.e., gain). • Growth as a norm/peer referenced quantity. • Growth as a criterion referenced/growth- to-standard quantity.
  8. Student Growth: Growth as an Amount of Change • Growth as an amount of change: • What could be more simple? • Like height or weight just calculate the amount of change. • Requires a common, across grade reporting scale (which Hawaii has since 2015/SBAC). • Growth = Score Time 2 - Score Time 1 • Not as simple as it seems: • Comparisons between students often lead to wrong conclusions. • Interpreting change requires context.
  9. Student Growth: Growth as a Norm/Peer Reference Quantity Back to our question: Which high jumper improved more? • Judy improved 2 inches. • Anna improved 6 inches. • An inch is an inch! • But is an inch really an inch? • 2 inches for an elite jumper is far more remarkable than 6 inches for a novice.
  10. Student Growth: Growth as a Norm/Peer Reference Quantity Like high jumping, growth is not uniform for the vertically scaled tests (e.g., SBAC): Average Scale Score Change 2015 to 2016 Top 10% Reading Grade 4 2015 +26.5 Bottom 10% Reading Grade 4 2015 +65.5 Top 10% Mathematic Grade 4 2015 +28.9 Bottom 10% Mathematics Grade 4 2015 +69.0 Just as an inch isn t necessarily an inch; a scale score point isn t necessarily a scale score point.
  11. Student Growth: Growth as a Norm/Peer Reference Quantity • This is analogous to height/weight measures for infants. • Growth amounts are tricky to interpret and require context. • Looking at the progress of individuals with similar starting points (i.e., peer based comparison) is a way to acquire that context.
  12. Student Growth: Growth as a Norm/Peer Reference Quantity • Like with infant height/weight charts, peers are children/students with similar characteristics. • For growth analyses using large scale assessments, peers are students with (at least) the same scale score history on the same tests. • Currently, the Hawaii Student Growth Percentile Model (i.e., the SGP/ Colorado Growth Model) creates growth norms for all students using regression (i.e., peer-based) analyses. • Value added models often include demographic variables (e.g., ethnicity) are part of peer comparison. • Value-tables consider students starting in the same achievement level for comparison.
  13. Student Growth: Criterion-Referenced / Growth-to-Standard • The standards used to examine growth are often the state’s performance standards. • Peer based analyses (e.g., SGP) are used in many states as part of growth-to-standard analyses. • Growth-to-standard analyses produce growth targets determining whether a student’s growth is sufficient to reach/maintain state standards. • Absent norms, criterion-referenced growth produces results similar to achievement indicating: • Students that start behind tend to stay behind. • Students that start ahead tend to stay ahead.
  14. The Student Growth Percentile (SGP) Model • Research on the SGP model began in 2007. • Based upon work in Colorado, the norm-referenced model was extended to include a criterion-referenced component. • Work in other states incorporating the model into state reporting and accountability systems occurred in subsequent years and student growth became more common across state assessment systems. • The goal of the model is to create an individual level growth metric that is informative to different stakeholders ranging from parents to policy makers.
  15. Growth is achievement over time: An example Hawaii State Department of Education/NCIEA 15
  16. Growth is achievement over time: An example Hawaii State Department of Education/NCIEA 16
  17. Student Growth: Options for Small Sample Sizes • Almost all current growth models use large amounts of data to create “norms” against which student growth is judged. • Absent large amounts of data (>1,000 grade/content area), professional judgment replaces large-scale data-based norms.
  18. Student Growth: Options for Small Sample Sizes • Value-tables can be used to monitor progress of students between achievement bands established for year-to-year assessments. • Some states (e.g., Alaska) use value-tables, particularly with smaller scale assessments for evaluating student growth for accountability purposes.
  19. Value Table
  20. Student Growth Models: Engaging Families and Communities • Different growth models answer different questions. • Different stakeholder groups ask different questions. • Parents are often simply concerned with how their child is doing. • To effectively engage parents and communities in a productive conversation about student learning (i.e. growth), what should be done?
  21. Pause and Reflect - Poll Please share your thoughts in the poll response box to your right.
  22. Student Growth Models: Engaging Families and Communities • A useful growth model should describe: – What is? How much are students learning/ growing? – What should be? How much would we like students to grow/learn? – What is reasonable? How much is it reasonable to expect students to grow/learn? • Most growth models do not provide a student level metric. They provide a school summary used for accountability purposes.
  23. Conclusions & Recommendations • Subtraction isn’t a growth model. A growth model requires one consider where a student begins. • Norms require large numbers of students. Smaller numbers of students requires professional judgment to decide what is ambitious yet reasonable progress for a student. • Precision and accuracy of results must be balanced with utility: Precise and accurate enough for what purpose. • A student level metric is necessary to make student growth models meaningful for families and communities.
  24. QUESTIONS AND FEEDBACK