COVID-19 Impact & Recovery
Exemplar Schools
December 2022 Update
Damian Betebenner & Adam VanIwaarden — Center for Assessment
Hawaii DOE
December 14th, 2022
Exemplar Schools: Impact & Recovery
• The goal of the project is to utilize results from Hawaii SBA assessment to find
outlier schools that we think have a high probability of being able to teach us
something about beneficial and not-to-beneficial activities associated with impact
and recovery.
• We focus on identifying three types of schools:
• Large Impact/Large Recovery
• Large Impact/Poor Recovery
• Minimal Impact/Large Recovery
• We show an original figure and an updated figure after we revised the
methodology on what SGP to use in 2022 to calculate academic recovery.
• The relationship between impact and recovery at the school level is negligible.
Previous Impact/
Recovery: Elementary
Higher correlation (-0.4) between academic
impact (x-axis) and academic recovery (y-axis)
Suggests schools impacted more “bounced back”
more.
NOTE: There are no magic “bounce backs” in
education.
Revised Impact/
Recovery: Elementary
Revisions to recovery calculation utilized a 2nd
order SGP instead of a 1st order SGP.
Incorporation of 2nd order SGP attenuated the
value somewhat leading to lower SGPs.
Exemplar Schools: Impact & Recovery
• The modifications showed a substantial decrease in the relationship between
impact and recovery. Previously, high impact schools showed bigger recovery
(suggesting a regression effect at play)
• The pattern in middle schools is not the same
Revised Impact/
Recovery: Middle
Middle School pattern does not follow that of
elementary schools
The pattern suggests that those middle schools
most impacted continue to struggle
The difference between elementary and middle
schools may have something to do with the
curriculum: Is it easier to “catch up” with
elementary school curriculum than middle school
curriculum.
Revised Impact/
Recovery: Middle ELA
Middle School pattern does not follow that of
elementary schools
The pattern suggests that those middle schools
most impacted continue to struggle
The difference between elementary and middle
schools may have something to do with the
curriculum: Is it easier to “catch up” with
elementary school curriculum than middle school
curriculum.
Revised Impact/
Recovery: Middle Math
Middle School pattern does not follow that of
elementary schools
The pattern suggests that those middle schools
most impacted continue to struggle
The difference between elementary and middle
schools may have something to do with the
curriculum: Is it easier to “catch up” with
elementary school curriculum than middle school
curriculum.
Candidate Exemplar School for Case Study Analysis
Exemplar Schools: Impact & Recovery
• As can be seen, Hawaii and its schools experienced a largely positive 2021-2022
• We’ve created Excel spreadsheets with the identified schools to help to sort
through the many potential criteria to use to select exemplar schools
• As we discussed in Portsmouth, there was some concern that of low motivation
leading to poor results. To examine we filtered on students with very low to very
high and very high to very low growth.
• Filtering out these students led to non-appreciable changes to the impact-
recovery examinations of schools.
About this copy
COVID-19 Impact & Recovery Exemplar Schools — Hawaii Department of Education, January 11, 2023.
This is an image-based copy of the original Keynote slides; builds and animations are flattened.