On Student Growth Percentiles & Accountability in Hawaii

Hawaii Department of Education · February 12, 2014

On Student Growth Percentiles & Accountability in Hawaii — Hawaii Department of Education, February 12, 2014.
Published

February 12, 2014

Event
State agency
Slides
22
Format
Converted from Beamer

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. References On Student Growth Percentiles & Accountability in Hawaii Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH Educator Evaluation Advisory Group, Honolulu, Hawaii February 12, 2014 Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  2. References About Me Dr. Damian Betebenner, PhD - Senior Associate at the Center for Assessment (NCIEA). - Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009, Betebenner, 2012, Betebenner et al., 2014]. - Currently refining and sharing these techniques with other states including Colorado, Massachusetts, Indiana, Nevada, Hawaii and 25 other states in various stages of investigation/adoption. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  3. References Student Growth Percentiles 2007 Began develop of the Colorado Growth Model (student growth percentiles). 2008 Began work with Indiana on development of growth model. 2008 Successfully application of Colorado Growth Model (criterion-referenced component) as part of growth model pilot program. 2009-2010 Began/continued work with approximately two-dozen other states on developing state growth models. 2010-2014 Worked with states on incorporating growth into RttT applications and with successful states on their use (Hawaii, Massachusetts, Georgia, Rhode Island). 2011-2014 Worked with a number of states on incorporating student growth percentiles into the ESEA waiver applications and teacher evaluation systems. 2014- Working with states transitioning to PARCC, SBAC assessment systems. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  4. Student Growth Percentile Adoption Adopted Model Considering Adoption Investigating Model
  5. References Lessons Learned On Questions John Tukey It is better to have an approximate answer to the right question than a precise answer to the wrong question. On Statistical Models George E. P. Box All models are wrong but some are useful. On Understanding Aristotle We understand best those things we see emerge from their very beginnings. On Accountability Sherlock Holmes It is a capital mistake to theorize before you have all the evidence. It biases the judg- ment. On Theories of Action Confucius Tell me and I will forget, show me and I will remember, involve me and I will understand. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  6. References About the growth model The Student Growth Percentiles Methodology is a norm- and criterion-referenced growth analysis method. The norm-referenced component of the model is the student growth percentile which indicates the relative growth a student makes. The criterion-referenced component of the model is the student growth percentile target which indicates the amount of growth required by the student to reach/maintain desirable achievement goals in a specified time frame. For example, catching-up to proficiency within 3 years. The norm- and criterion-referenced components together allow stakeholders to understand what is, what should be, and what is reasonable simultaneously. The goal is to create a durable metric to assist in moving beyond debates over what thermometer is best to discussions of how best to use the thermometer. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  7. References Built to maximize sharing A number of analytic tools (for doing calculations) and visualization tools have been developed with the goal to promote maximum sharing. Data isn’t valuable! It’s what one does with data that is potentially valuable! The analytic and visualization tools to calculate all the quantities associated with student growth percentiles are open source and freely available to all states. Conduct training sessions at national conferences to help other calculate these quantities on their own. Build enhanced visualization and reporting systems to make data come alive to stakeholders and lead to better stakeholder engagement with data. Goal is to re-invent the user experience with education data and enable a “growth mindset”. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  8. References Describing Student Growth Measuring student growth, even with a vertical scale, is not a simple task. Some believe a vertical scale simplifies the task of measuring student growth. Even with an interval (or ratio) scale, growth is not easy to interpret. Consider, for example, height. A child might grow 4 inches between ages 3 and 4. 4 inches is a well understood quantity. The 4 inch increase becomes really meaningful only when understood alongside the growth of other 3 to 4 year olds. Student growth percentiles were developed to provide a normative context for describing student growth. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  9. Advanced Reading Achievement Proficient ● DEMO Reading Scale Score ● Catch Up Target Growth Part Proficient Level Percentiles Very High 81st − 99th High 61st − 80th Typical 40th − 60th Unsatisfactory Low 20th − 39th Very Low 1st − 19th Grade 3 Grade 4 Scale Score 449 Achievement Level Achievement Target Unsatisfactory Proficient (529) Achievement Growth Percentile Growth Level Growth Target Catch Up (64) Growth
  10. Advanced Reading Achievement DEMO Reading ● Scale Score Proficient ● Catch Up Growth Target Level Percentiles Very High 81st − 99th Part Proficient High 61st − 80th ● Typical 40th − 60th Low 20th − 39th Unsatisfactory Very Low 1st − 19th Grade 3 Grade 4 Grade 5 Scale Score 449 533 Achievement Level Achievement Target Unsatisfactory Part Proficient Proficient (562) Achievement Growth Percentile 69 Growth Level Growth Target High Catch Up (76) Growth
  11. Advanced Reading Achievement Proficient ● ● ● Catch Up Target DEMO Reading Scale Score Growth Part Proficient Level Percentiles Very High 81st − 99th High 61st − 80th Typical 40th − 60th Unsatisfactory Low 20th − 39th Very Low 1st − 19th Grade 3 Grade 4 Grade 5 Grade 6 Scale Score 449 533 579 Achievement Level Achievement Target Unsatisfactory Part Proficient Part Proficient Proficient (586) Achievement Growth Percentile 69 85 Growth Level Growth Target High Very High Catch Up (71) Growth
  12. Advanced Reading ● Move Up Achievement Target DEMO Reading ● Scale Score Keep Up ● Target Proficient Growth Level Percentiles ● Very High 81st − 99th High 61st − 80th Part Proficient Typical 40th − 60th Low 20th − 39th Very Low 1st − 19th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Scale Score 449 533 579 602 Achievement Level Achievement Target Unsatisfactory Part Proficient Part Proficient Proficient Proficient (620)/Advanced (661) Achievement Growth Percentile 69 85 74 Growth Level Growth Target High Very High High Keep Up (76)/Move Up (99) Growth
  13. References Going from Students to Classrooms/Schools It’s of interest to examine schools/classrooms where students demonstrate, on average, extraordinarily high and low student growth. To summarize the student growth percentiles associated with a school/classroom (or other grouping) Colorado calculates the median of the student growth percentiles. If students were randomly assigned to classrooms/schools, expect to see a median of 50. Values greatly above or below 50 are of interest in identifying best practices or providing extra support. Examining growth with achievement sheds new light on school/classroom performance. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  14. Hawaii School Performance HSA Growth & Achievement 2013 HSA Mathematics Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2013 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2013 Median Student Growth Percentile
  15. Hawaii School Performance HSA Growth & Prior Achievement 2013 HSA Mathematics Higher Achievement Higher Achievement School Size 2012 Prior Percent at/above Proficient Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2013 Median Student Growth Percentile
  16. Hawaii School Performance HSA Growth & Achievement 2013 HSA Reading Higher Achievement Higher Achievement School Size Lower Growth Higher Growth 50 Students 100 Students 2013 Percent at/above Proficient 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2013 Median Student Growth Percentile
  17. Hawaii School Performance HSA Growth & Prior Achievement 2013 HSA Reading Higher Achievement Higher Achievement School Size 2012 Prior Percent at/above Proficient Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students Achievement Lower Growth Higher Growth Lower Achievement Lower Achievement Growth 2013 Median Student Growth Percentile
  18. Hawaii Teacher Median SGPs Frequency 400 Median SGP
  19. Hawaii: 2013 Mathematics Norm & Criterion Referenced Growth & Achievement No Grade 9 Assessment 2013 Achievement Percentile Percentile Growth Trajectory Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 10
  20. Hawaii: 2013 Reading Norm & Criterion Referenced Growth & Achievement No Grade 9 Assessment 2013 Achievement Percentile Percentile Growth Trajectory Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 10
  21. References Web 2.0: Data Visualization and Social Networking The Hawaii Department of Education and the Center for Assessment have been working for the last two years on developing next generation data visualization to accompany growth model data. The goal: Transform conversations about education through active engagement with data (i.e., evidence). Our efforts have received tremendous interest and recognition: Recognized by Adobe for innovative uses of their technology as an Adobe Max Award finalist in October, 2009. 2010 NCME Award for Outstanding Dissemination of Educational Measurement Concepts to the Public. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii
  22. References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Betebenner, D. W. (2012). Growth, standards, and accountability. In Cizek, G., editor, Setting Performance Standards: Foundations, Methods, and Innovations, pages 439–450. Routledge, New York. Betebenner, D. W., Van Iwaarden, A., Domingue, B., and Shang, Y. SGP: An R Package for the Calculation and Visualization of Student Growth Percentiles & Percentile Growth Trajectories. R package version 1.2-2.0. Damian W. Betebenner On Student Growth Percentiles & Accountability in Hawaii