INDIANA STUDENT GROWTH ANALYSES 2016-2017 & BEYOND Damian W Betebenner National Center for the Improvement of Educational Assessment Indianapolis, IN May 30th, 2017

TOPICS FOR DISCUSSION • What is student growth? • History of student growth in Indiana. • Analyses associated with the 2015-2016 academic year. • Analyses associated with the 2016-2017 academic year and beyond.

Why Student Growth? If you care about student achievement, then you care about student growth. • Where you are is a function of where you started and how quickly you’ve travelled since. • Student growth and student attainment/achievement are related in this way. • Distance=Rate x Time • Those who start behind tend to stay behind and those that start ahead tend to stay ahead.

Student Growth: Introductory Concepts • The concept of student growth isn’t as difficult to understand and popular perception suggests. • Calculations may be complicated but knowing how to calculate student growth is not synonymous with understanding student growth: – The number π (pi=3.14159…) is difficult to calculate but is simple to understand conceptually. – Height and weight percentiles associated with children are difficult to calculate but are simple to understand conceptually. • Computational details are important but often obscure a simple conceptual basis.

Student Growth Introductory Concepts A Math Story Problem • Whose high jump changed more? Explain your answer. • Whose high jump change is more remarkable? Explain your answer. • Which high jumper improved more? Explain your answer.

Student Growth: Introductory Concepts • There are 3 general approaches of calculating student growth: – Growth as an amount of change (i.e., gain). – Growth as a norm/peer referenced quantity. – Growth as a criterion referenced quantity.

Student Growth: Introductory Concepts • Growth as an amount of change: – What could be more simple? – Like height or weight just calculate the amount of change. – Requires a vertical scale (which Indiana had up to 2014/ISTEP+) – Growth = Score Time 2 - Score Time 1 • Not as simple as it seems: – Comparisons between students often leads to wrong conclusions – Interpreting change requires context.

Student Growth: Introductory Concepts Back to the story problem!

Student Growth: Introductory Concepts • Which high jumper improved more: – Judy improved 2 inches. – Anna improved 6 inches. – An inch is an inch!!! – But is an inch really an inch??? – 2 inches for an elite jumper is far more remarkable than 6 inches for a novice.

Student Growth: Introductory Concepts • The same holds true for the vertically scaled tests (e.g., ISTEP+). Average Scale Score Change 2014 to 2015 Top 10% Reading Grade 4 2014 +26.5 Bottom 10% Reading Grade 4 2014 +65.5 Top 10% Mathematic Grade 4 2015 +28.9 Bottom 10% Mathematics Grade 4 2015 +69.0 • Just as an inch isn’t necessarily an inch; a scale score point isn’t necessarily a scale score point.

Student Growth: Introductory Concepts •This is analogous to height/weight measures for infants. •Growth amounts are tricky to interpret and require context. •Peer based comparisons used by growth analyses are a way to acquire that context. An infant male toddler is measured at 2 and 3 years of age and is shown to have grown 4 inches. The magnitude of increase—4 inches—is a well understood quantity that any parent can grasp and calculate at home using a simple yardstick. However, parents leaving their pediatrician’s office knowing only how much their child has grown would likely be wanting for more information: Parents are not interested in an absolute magnitude of growth, but instead in a norm- referenced quantity locating that 4 inch increase alongside the height increases of similar children.

HISTORY OF STUDENT GROWTH IN INDIANA • Started under Wes Bruce in 2008 with the Rooney Foundation. • Included the calculation of student growth percentiles and student growth projections. • Due to feedback from eld that they wanted to know in advance what their growth goal was, Indiana began providing scale score targets for typical and higher growth a year in advance. • This “rolling baseline” approach used up to 2014-2015. fi

2014-2015 STUDENT GROWTH IN INDIANA • Based upon the work of the student growth accountability work group, a new model of student growth was adopted for the 2014-2015 academic year. • Transitions of standards and assessments disrupts both status and growth calculations. • The “monkey wrench” with the new assessment is a different scale. • The new scale doesn’t impact calculation of historical growth. • The new scale does impact the calculation of growth-to-standard: Future adequate growth. • The following picture illustrates student achievement across an assessment transition.

Amer Farley Sample School How to interpret this student Pass + growth & achievement report Pass + High ISTEP+ ● Typical ● Pass Scale Score ● Low Pass ● ISTEP+ Achievement Levels Did Not Pass Did Not Pass ISTEP+ ISTEP+ ISTEP+ ISTEP+ ISTEP+ Student ELA ELA ELA ELA ELA Growth Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Percentile Scale Score 504 555 504 579 Achievement Level Pass Pass + Pass Pass + Achievement Growth Percentile 80 4 86 Growth Level High Low High Growth Suggested Uses ● Review past growth to assess Pass + student academic progress toward Pass + ISTEP+ achievement goals. Pass ● Develop remediation or enrich− ● ● ment plans based on rate of growth needed to reach higher Pass ISTEP+ achievement levels. ● Identify the rate of progress needed in order to reach or Did Not Pass maintain proficient status on the ISTEP+ next year. Did Not Pass ISTEP+ ISTEP+ ISTEP+ ISTEP+ ISTEP+ Math Math Math Math Math Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Scale Score 521 531 601 547 Achievement Level Pass + Pass Pass + Pass Achievement Growth Percentile 32 89 25 Growth Level Low High Low Growth For more information please visit the Indiana Department of Education (IDOE) at www.doe.in.gov or contact webmaster@doe.in.gov Cooperatively developed by the IDOE & the Center for Assessment, Inc. Distributed by the IDOE.

2015-2016 STUDENT GROWTH IN INDIANA • .Based upon the work of the accountability work group, a criterion-referenced growth model was developed that blends value-tables and observed growth. • The model sub-divides Indiana’s 3 achievement levels into 8 sub- achievement levels to determine starting ranges for students. • Based upon observed student growth toward reaching/ maintaining passing status, points are given to students based upon the progress made. • School summaries are derived from individual student scores.

2015-2016 STUDENT GROWTH TABLE

CONSIDERATIONS • The current model is robust to changes in standards and assessments. However, starting achievement levels and target ranges and points need to be re-considered when transitions occur. • Important to examine relationship between growth and relevant school characteristics including: • Average prior achievement. • School poverty. • Test mode effects have existed in the past and are a constant concern when different test modes (e.g., online and paper-pencil).

About this copy

Indiana Student Growth Analyses — Indiana Department of Education, May 30, 2017.

This is an image-based copy of the original Keynote slides; builds and animations are flattened.