Dr. Damian Betebenner The National Center for the Improvement of Educational Assessment

CHANGING CONVERSATIONS ABOUT STUDENT PERFORMANCE

The State of Student Growth • The existence of annual student testing together with unique student identifiers forms the foundation for the calculation of student growth. • Over the last decade, most states have begun calculation of longitudinal student growth using large scale state assessments. • Analysis of student growth is controversial because of how it is most often used: For teacher and school evaluation. • For example, the development of the current Hawaii Growth Model involved the creation of a diagnostically useful quantity that could be used (or not used) at a later time for many purposes. 3

Why Student Growth? If you care about student achievement, then you care about student growth. • Where you are is a function of where you started and how quickly you ve travelled since. • Student growth and student attainment/achievement are related in this way. • Distance = Rate x Time • Those who start behind tend to stay behind and those that start ahead tend to stay ahead.

Student Growth: Introductory Concepts • The concept of student growth isn’t as difficult to understand and popular perception suggests. • Calculations may be complicated but knowing how to calculate student growth is not synonymous with understanding student growth: • The number π (pi=3.14159…) is difficult to calculate but is simple to understand conceptually. • Height and weight percentiles associated with children are difficult to calculate but are simple to understand conceptually. • Computational details are important but often obscure a simple conceptual basis.

Pause and Reflect – Poll Please see the questions in the poll response box to your right.

Student Growth: Introductory Concepts • There are 3 general approaches of calculating student growth: • Growth as an amount of change (i.e., gain). • Growth as a norm/peer referenced quantity. • Growth as a criterion referenced/growth- to-standard quantity.

Student Growth: Growth as an Amount of Change • Growth as an amount of change: • What could be more simple? • Like height or weight just calculate the amount of change. • Requires a common, across grade reporting scale (which Hawaii has since 2015/SBAC). • Growth = Score Time 2 - Score Time 1 • Not as simple as it seems: • Comparisons between students often lead to wrong conclusions. • Interpreting change requires context.

Student Growth: Growth as a Norm/Peer Reference Quantity Back to our question: Which high jumper improved more? • Judy improved 2 inches. • Anna improved 6 inches. • An inch is an inch! • But is an inch really an inch? • 2 inches for an elite jumper is far more remarkable than 6 inches for a novice.

Student Growth: Growth as a Norm/Peer Reference Quantity Like high jumping, growth is not uniform for the vertically scaled tests (e.g., SBAC): Average Scale Score Change 2015 to 2016 Top 10% Reading Grade 4 2015 +26.5 Bottom 10% Reading Grade 4 2015 +65.5 Top 10% Mathematic Grade 4 2015 +28.9 Bottom 10% Mathematics Grade 4 2015 +69.0 Just as an inch isn t necessarily an inch; a scale score point isn t necessarily a scale score point.

Student Growth: Growth as a Norm/Peer Reference Quantity • This is analogous to height/weight measures for infants. • Growth amounts are tricky to interpret and require context. • Looking at the progress of individuals with similar starting points (i.e., peer based comparison) is a way to acquire that context.

Student Growth: Growth as a Norm/Peer Reference Quantity • Like with infant height/weight charts, peers are children/students with similar characteristics. • For growth analyses using large scale assessments, peers are students with (at least) the same scale score history on the same tests. • Currently, the Hawaii Student Growth Percentile Model (i.e., the SGP/ Colorado Growth Model) creates growth norms for all students using regression (i.e., peer-based) analyses. • Value added models often include demographic variables (e.g., ethnicity) are part of peer comparison. • Value-tables consider students starting in the same achievement level for comparison.

Student Growth: Criterion-Referenced / Growth-to-Standard • The standards used to examine growth are often the state’s performance standards. • Peer based analyses (e.g., SGP) are used in many states as part of growth-to-standard analyses. • Growth-to-standard analyses produce growth targets determining whether a student’s growth is sufficient to reach/maintain state standards. • Absent norms, criterion-referenced growth produces results similar to achievement indicating: • Students that start behind tend to stay behind. • Students that start ahead tend to stay ahead.

The Student Growth Percentile (SGP) Model • Research on the SGP model began in 2007. • Based upon work in Colorado, the norm-referenced model was extended to include a criterion-referenced component. • Work in other states incorporating the model into state reporting and accountability systems occurred in subsequent years and student growth became more common across state assessment systems. • The goal of the model is to create an individual level growth metric that is informative to different stakeholders ranging from parents to policy makers.

Growth is achievement over time: An example Hawaii State Department of Education/NCIEA 15

Growth is achievement over time: An example Hawaii State Department of Education/NCIEA 16

Student Growth: Options for Small Sample Sizes • Almost all current growth models use large amounts of data to create “norms” against which student growth is judged. • Absent large amounts of data (>1,000 grade/content area), professional judgment replaces large-scale data-based norms.

Student Growth: Options for Small Sample Sizes • Value-tables can be used to monitor progress of students between achievement bands established for year-to-year assessments. • Some states (e.g., Alaska) use value-tables, particularly with smaller scale assessments for evaluating student growth for accountability purposes.

Value Table

Student Growth Models: Engaging Families and Communities • Different growth models answer different questions. • Different stakeholder groups ask different questions. • Parents are often simply concerned with how their child is doing. • To effectively engage parents and communities in a productive conversation about student learning (i.e. growth), what should be done?

Pause and Reflect - Poll Please share your thoughts in the poll response box to your right.

Student Growth Models: Engaging Families and Communities • A useful growth model should describe: – What is? How much are students learning/ growing? – What should be? How much would we like students to grow/learn? – What is reasonable? How much is it reasonable to expect students to grow/learn? • Most growth models do not provide a student level metric. They provide a school summary used for accountability purposes.

Conclusions & Recommendations • Subtraction isn’t a growth model. A growth model requires one consider where a student begins. • Norms require large numbers of students. Smaller numbers of students requires professional judgment to decide what is ambitious yet reasonable progress for a student. • Precision and accuracy of results must be balanced with utility: Precise and accurate enough for what purpose. • A student level metric is necessary to make student growth models meaningful for families and communities.

QUESTIONS AND FEEDBACK

About this copy

RELPA Student Growth Models Slides to Damian — Hawaii REL, September 29, 2016.

This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.