Pandemic Related Academic Impact in 2021: What do state summative assessments tell us

Arizona Department of Education · August 25, 2021

Pandemic Related Academic Impact in 2021: What do state summative assessments tell us — Arizona Department of Education, August 25, 2021.
Published

August 25, 2021

Event
State agency
Slides
67
Format
Converted from PowerPoint
Also given
2021/Rhode_Island_TAC_082521

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Pandemic Related Academic Impact in 2021 What do state summative assessments tell us Damian Betebenner & Adam Van Iwaarden Center for Assessment CCSSO Assessment Directors September 21st, 2021
  2. Pandemic Related Academic Impact RI SGP Calculations 2021
  3. Pandemic Related Academic Impact • The pandemic and all the ensuing disruptions functions as a kind of “academic headwind”, impeding (in general) the academic progress of students. • Headwinds impede progress in two ways: • They slow one’s rate of progress • The slower rate of progress leads to less distance being travelled. • In education these two outcomes represent: • Decrease in student growth. • Decrease in student attainment. RI SGP Calculations 2021
  4. RICAS SGP Operational Considerations in 2021 • Due to no RICAS testing in Spring 2020, RICAS growth analyses will be skip-year/gap year growth from 2019 to 2021. • Two types of SGP analyses were conducted for RICAS data: • Cohort referenced analyses: These analyses create growth norms from the 2019 to 2021 students. These growth norms yield cohort means/median SGPs of 50 and obscure COVID-19 academic impact. • Baseline referenced analyses: Historical growth norms were created using 2019 and prior data. These growth growth analyses don’t have mean/median SGPs of 50 and can be used to investigate overall COVID-19 academic impact • Projections (lagged and straight) were calculated for 2021 data using baseline growth norms in case they were needed. RI SGP Calculations 2021
  5. WIDA-ACCESS SGP Operational Considerations in 2021 • Two types of SGP analyses were conducted for WIDA-ACCESS data: • Cohort referenced analyses: These analyses create growth norms from the 2020 to 2021 students. These growth norms yield cohort mean/median SGPs of 50 and obscure COVID-19 academic impact. • Baseline referenced analyses: Historical growth norms were created using 2020 and prior data. These growth growth analyses don’t have mean/median SGPs of 50 and can be used to investigate overall COVID-19 academic impact. • Use composite score from WIDA-ACCESS data. No analyses performed using sub-scores. • Projections (lagged and straight) were calculated for 2021 data using both cohort and baseline growth norms in case they were needed. RI SGP Calculations 2021
  6. SGP Summary of Cohort versus Baseline Referenced • Cohort referenced SGPs • Growth norms created from the current cohort of students. • Mean/median will by 50 for all grade x content area analyses • Cannot investigate overall impact of the pandemic on students. • Can investigate differential impact of the pandemic on students. • Does not require scale stability across years. • Baseline referenced SGPs. • Growth norms created from historical (i.e., pre-COVID) cohorts of students. • Mean/median 50 for historical cohort, not necessarily 50 for current cohort. • Extent of deviation from (i.e. below) 50 is indicative of less learning/pandemic related academic impact. • Mean SGP deviation of 5 corresponds to an effect size of 0.18. • Can investigate BOTH overall impact and differential impact • Requires scale stability across years.RI SGP Calculations 2021
  7. SGP Assumptions • SGP analyses presume valid scale scores • Students forming the growth norm group are presumed to have taken the same (or equivalent) test at the same time. Scores have to mean the same thing. • SGPs, however, are often used as a post-hoc check of scale score validity/ comparability via Reductio ad absurdum • In 2021, check form comparability and mode-of-administration comparability. • RIDE has extensive experience (e.g., PARCC paper/pencil versus online) investigating and accommodating mode-effects. • If scores are not comparable then we either attempt to make them comparable or run separate analyses. • We assume something close to population invariance with respect to IRT scaling. • The population of students tested in 2021 and differences to 2019 is a concern so we begin with data on the population of students tested. RI SGP Calculations 2021
  8. RICAS Participation ELA Mathematics Grade RI SGP Calculations 2021
  9. WIDA-ACCESS Participation Overall Overall Grade Grade RI SGP Calculations 2021
  10. SGP Calculations • SGP calculations were performed using the SGP package and R software environment based upon data supplied by the Rhode Island Department of Education. • To facilitate the calculation of baseline SGPs across multiple states, we created an additional R package, SGPmatrices, that contains Rhode Island matrices, amongst many others. • https://centerforassessment.github.io/SGPmatrices/ • Source code associated with calculations are available on GitHub: • RICAS: https://github.com/CenterForAssessment/Rhode_Island • WIDA-ACCESS: https://github.com/CenterForAssessment/WIDA_RI RI SGP Calculations 2021
  11. SGP Results • The following slides show SGP results for 2021. • We report only baseline referenced SGPs in these slides • The TAC recommended baseline referenced SGPs • Differential impact shown in baseline SGPs will mirror that found in cohort referenced SGPs. • Due to the highly skewed nature of the SGP results, we recommend median SGP over mean SGP in this instance. • We also provide 2019 results to understand pandemic related academic impact. RI SGP Calculations 2021
  12. SGP Analyses and Academic Impact • Understanding pandemic related impact necessarily requires looking back to pre-pandemic results. • The impulse of stakeholders to want to compare 2021 results with 2019 results is the correct impulse at both the individual, school, district, and state level. • Wanting to look at 2019 to 2021 change is an attempt to investigate impact. • The challenge is making 2019 to 2021 comparisons apples-to-apples comparisons. • Supporting these comparisons requires having a complete understanding is who constitutes the missing data. More on that after some initial comparisons. RI SGP Calculations 2021
  13. SGP Analyses for 2021 • With comparable results we can quantify pandemic related academic impact: • At the state level • Demographic subgroups including ethnicity, poverty • Academic subgroups (special education, low/high achievers) • By region/district/school • By mode of education (remote/hybrid/in-person). • Attendance • English Language Learner impact can be evaluated using WIDA-ACCESS results. RI SGP Calculations 2021
  14. RICAS Percent Proficient by Grade ELA Mathematics Grade RI SGP Calculations 2021
  15. RICAS Median/Mean Skip-Year SGPs by Grade ELA Mathematics Grade 3 NA NA NA NA 4 NA NA NA NA RI SGP Calculations 2021
  16. Discussion • Median baseline SGPs in ELA were in the mid 30s and median baseline SGPs in mathematics were lower (upper teens to lower • Deviation from normal growth (i.e., 50) represents the extent to which academic growth slowed down from 2019 to 2021, that is, academic impact. • Larger academic impact in ELA than mathematics. • Academic impact is fairly uniform across grades. • Academic impact is very large likely requiring multiple years and additional educational supports/interventions to recover. RI SGP Calculations 2021
  17. WIDA-ACCESS Median/Mean SGPs by Grade Overall Overall Grade Grade K NA NA 6 50.0/50.2 32.0/38.4 (887) RI SGP Calculations 2021
  18. Discussion of Grade Level Results • Median baseline SGPs are low in elementary school grades and are close to 50 in the high school grades. • Deviation from normal growth (i.e., 50) represents the extent to which academic growth slowed down from 2020 to 2021, that is, academic impact. • Academic impact is very large in elementary and middle school and close to non-existent in high school. • Lack of impact is consistent with: • Great academic support for the group during the pandemic. • Impoverished learning conditions prior to the pandemic such that the pandemic didn’t alter the status quo. RI SGP Calculations 2021
  19. RICAS Median/Mean Skip-Year SGPs by Ethnicity ELA Mathematics Ethnicity Am. In./AL Native 44.0/44.3 (259) 29.5/35.2 (218) 43.0/45.3 (263) 18.0/26.2 (221) Asian 54.0/53.5 (1,175) 41.0/44.6 (1,081) 56.0/55.0 (1,204) 33.0/39.1 (1,095) African American 41.0/43.6 (3,145) 31.0/35.8 (2,890) 42.0/45.1 (3,202) 20.0/27.0 (2,901) Hispanic or Lat. 46.0/46.8 (9,875) 31.0/36.2 (9,001) 44.0/46.1 (10,136) 20.0/27.5 (9,116) Multiple Ethnicities 47.0/47.8 (1,755) 34.0/37.3 (1,675) 47.0/47.9 (1,763) 23.0/30.6 (1,651) Native HI/PI 46.0/46.5 (51) 33.0/35.3 (49) 53.0/50.4 (53) 24.0/30.7 (47) No Race reported 47.0/45.9 (8) 28.5/33.4 (132) 46.5/41.7 (6) 14.0/21.3 (128) White 53.0/52.4 (22,316) 39.0/42.3 (18,733) 54.0/52.6 (22,358) 29.0/35.2 (18,598) RI SGP Calculations 2021
  20. Discussion of Ethnicity Results • Broken out by ethnicity, 2021 baseline SGP results show similar rank ordering (i.e., growth gaps) as seen in normal years. • Student academic growth is lower in mathematics than in ELA for all ethnic subgroups. • Even though growth is lower for ethnic minorities, academic impacts (growth difference with 2019) are uniform across ethnicity subgroups. RI SGP Calculations 2021
  21. RICAS Median/Mean Skip-Year SGPs by Free/Reduced Lunch Status ELA Mathematics Free/Reduced Lunch Status FRL Yes 44.0/45.9 (18,477) 30.0/35.0 (14,479) 45.0/46.4 (18,820) 20.0/27.2 (14,541) FRL No 55.0/53.7 (20,107) 41.0/43.4 (19,300) 55.0/53.6 (20,165) 30.0/36.0 (19,216) RI SGP Calculations 2021
  22. Discussion of Free/Reduced Lunch Results • Baseline SGP results show similar growth gaps in 2021 (students indicated as receiving free/reduced lunch have lower SGPs than students indicated as not receiving free/reduced lunch. • Academic impacts are uniform across the subgroups. RI SGP Calculations 2021
  23. RICAS Median/Mean Skip-Year SGPs by IEP Status ELA Mathematics IEP Status Non-IEP 51.0/50.9 (31,158) 37.0/40.8 (26,913) 51.0/50.8 (31,552) 25.0/32.3 (26,939) IEP 42.0/44.3 (5,398) 29.0/35.0 (4,802) 43.0/45.0 (5,408) 25.0/31.3 (4,780) Students with 504 52.0/50.8 (2,028) 35.0/39.2 (2,064) 53.0/52.0 (2,025) 25.0/32.4 (2,038) RI SGP Calculations 2021
  24. Discussion of IEP Results • Baseline SGP results show similar growth gaps in 2021 (students indicated as receiving free/reduced lunch have lower SGPs than students indicated as not receiving free/reduced lunch. • Academic impacts are, in general, uniform across the subgroups: • Academic impact for IEP group in mathematics is smaller than the other two groups. RI SGP Calculations 2021
  25. RICAS Median/Mean Skip-Year SGPs by ELL Status ELA Mathematics ELL Status ELL No 51.0/50.3 (36,261) 36.0/40.3 (30,942) 51.0/50.4 (36,249) 25.0/32.5 (30,607) ELL Yes 42.0/44.7 (2,323) 30.0/34.9 (2,837) 43.0/45.5 (2,736) 23.0/28.8 (3,150) RI SGP Calculations 2021
  26. Discussion of ELL Results • Baseline SGP results show somewhat similar growth gaps for ELA subgroups between 2019 and 2021. • The results for both ELA and mathematics show slightly less of an impact for ELL students than for their non-ELL counterparts, though the difference is practically insignificant. RI SGP Calculations 2021
  27. RICAS Median/Mean Skip-Year SGPs by Gender ELA Mathematics Gender Female 54.0/53.0 (18,917) 38.0/41.4 (16,438) 51.0/50.6 (19,137) 25.0/32.5 (16,375) Male 46.0/47.1 (19,659) 34.0/38.4 (17,167) 49.0/49.5 (19,842) 25.0/32.0 (17,205) Other --- 32.5/38.1 (42) --- 16.0/18.1 (49) RI SGP Calculations 2021
  28. Discussion of Gender Results • Growth gender gaps historically exist (favoring females) in both ELA and mathematics. • Growth gender gaps are historically larger in ELA than mathematics. • Gaps are smaller in 2021 between males and females suggesting the females might have been slightly more academically impacted than males RI SGP Calculations 2021
  29. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 5 ELA RI SGP Calculations 2021
  30. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 6 ELA RI SGP Calculations 2021
  31. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 7 ELA RI SGP Calculations 2021
  32. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 8 ELA RI SGP Calculations 2021
  33. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 5 Mathematics RI SGP Calculations 2021
  34. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 6 Mathematics RI SGP Calculations 2021
  35. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 7 Mathematics RI SGP Calculations 2021
  36. RICAS Median/Mean Skip-Year SGPs by Prior Achievement: Grade 8 Mathematics RI SGP Calculations 2021
  37. Discussion of Growth by Prior Achievement Results • In some grades and content areas median SGP baselines vary by the prior achievement level of students, sometimes substantially. • For example, in Grade 6 ELA lower achieving students had much lower baseline SGPs than higher achieving students. This implies that those students had much greater academic impact. • In Grade 7 mathematics higher achieving students had lower baseline SGPs than lower achieving students implying that higher achieving students were more impacted. • In general, there is a fairly uniform impact by prior achievement level. RI SGP Calculations 2021
  38. Participation Rates and Interpretation • An issue impacting the interpretation of growth (and status) results in 2021 is participation in the state assessment. • Low participation doesn’t impact calculation of cohort referenced growth norms nor use of baseline referenced growth norms. • Low participation does impact comparisons of group level results with historical (e.g., 2019) results based upon higher participation rates. • For example, how should one compare growth and status results for school in 2019 and 2021 with very different rates of participation. RI SGP Calculations 2021
  39. Multiple Imputation • Given the rich historical data available for students missing in 2021, and the observed performance of comparable students from 2019 to 2021, it is possible to simulate what we think their performance might have been. • Combining multiple simulations can inform us about potential bias in what we observed at state/district/school levels in 2021, as well as inform us about levels of uncertainty around them. • Multiple imputation (MI) of missing data is widely adopted and well-developed statistical analysis technique used in many fields to deal with missing data problems. RI SGP Calculations 2021
  40. Multiple Imputation • MI uses information from the observed data to generate parameter estimates in three steps: • Imputation: A prediction model generates a set of plausible values for the missing observations, resulting in M imputed data sets. • Analysis: The analysis (e.g., baseline student growth percentiles) is conducted on each of the M data sets. • Pooling: Parameter and (more importantly) uncertainty estimates are constructed by pooling across the M analyses. RI SGP Calculations 2021
  41. Multiple Imputation • In the context of COVID impact analyses, we use the multiple (plausible) parameter estimates to inform us about the uncertainty around what was observed. That is, we do not use MI to produce “corrected” parameter estimates, but rather to help determine whether we trust the statistics calculated from the observed (with missing) data. • Statistics of particular interest: • Mean/median scale scores • Mean/median baseline SGPs • Effect size estimates RI SGP Calculations 2021
  42. Multiple Imputation • Before performing a MI analysis with RICAS data, missing data patterns were investigated. • Patterns suggest some relationship to factors such as prior achievement and economic status. • These relationships suggest that missingness in RICAS is non- ignorable, but does not preclude the use of MI. • Data can be considered “Missing-at-random” – relationships exist between missingness and the observed data (and so can not be considered “missing completely-at-random”) but the relationships are not systematic (e.g. all students below proficiency in 2019 were missing in 2021). RI SGP Calculations 2021
  43. Multiple Imputation RI SGP Calculations 2021
  44. Multiple Imputation RI SGP Calculations 2021
  45. Multiple Imputation RI SGP Calculations 2021
  46. Multiple Imputation • Given the 2021 missingness patterns and the good coverage of observed scores for students from all demographic groups and prior achievement levels, MI is a reasonable approach to dealing with RICAS missing data. • Imputation models included: • Student prior achievement • Student demographics • FRL Status • Disability status • English language proficiency status • Gender • Ethnicity dummy variables for White, African American, Hispanic and Asian students • School level clustering • 30 imputations (M = 30 with a maximum of 10 iterations per imputation) were conducted. RI SGP Calculations 2021
  47. Multiple Imputation • Imputation models were restricted to include only students who were enrolled, but not tested. • Knowing school/district affiliation is critical for assessing the impact of missing data for subsequent analyses at these levels. • Excluding students who “disappeared” from the system may not give us a full account of the impact of missing data at the state level. However, a comparison of the enrolled vs. non-enrolled missing students does not lead us to believe their inclusion would meaningfully change inferences at the state level. RI SGP Calculations 2021
  48. Multiple Imputation RI SGP Calculations 2021
  49. School Level Results • Due to larger fluctuations in participation rates a the school level, growth comparisons between 2019 and 2021 must be approached more cautiously. • In general we have found via multiple imputation that comparisons of status and growth results from 2019 to 2021 are defensible. • Whether a school’s results support cross-year comparison depends more on the size of the school than on the participation rate/ enrollment change. • The following are preliminary academic impact results at the school level represented as bubble plots that can be used to help identify students/schools in the greatest need of assistance. RI SGP Calculations 2021
  50. School Level COVID Academic Impact RICAS: ELA Improvement 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 2.6% Higher Growth School Size 50 Students 100 Students 0% 2% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  51. School Level COVID Academic Impact RICAS: ELA Modest to None 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 20.5% Higher Growth School Size 50 Students 100 Students 45.5% 25.5% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  52. School Level COVID Academic Impact RICAS: ELA Moderate 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 35.4% Higher Growth School Size 50 Students 100 Students 18.2% 27.5% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  53. School Level COVID Academic Impact RICAS: ELA Large 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 23.1% Higher Growth School Size 50 Students 100 Students 27.3% 21.6% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  54. School Level COVID Academic Impact RICAS: ELA Severe 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 14.4% Higher Growth School Size 50 Students 100 Students 9.1% 23.5% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  55. School Level COVID Academic Impact RICAS: Mathematics Improvement 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 5.1% Higher Growth School Size 50 Students 100 Students 16.7% 0% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  56. School Level COVID Academic Impact RICAS: Mathematics Modest to None 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 13.8% Higher Growth School Size 50 Students 100 Students 16.7% 38.9% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  57. School Level COVID Academic Impact RICAS: Mathematics Moderate 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 13.8% Higher Growth School Size 50 Students 100 Students 33.3% 5.6% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  58. School Level COVID Academic Impact RICAS: Mathematics Large 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 32.8% Higher Growth School Size 50 Students 100 Students 16.7% 11.1% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  59. School Level COVID Academic Impact RICAS: Mathematics Severe 2019 Growth by Achievement Higher Achievement Higher Achievement Lower Growth 30.3% Higher Growth School Size 50 Students 100 Students 16.7% 44.4% 250 Students 500 Students 2019 Percent Proficient COVID Impact ES 2021 SGP − 2019 SGP Improvement (> 0.2) 40 Modest to None (−0.2 to 0.2) Moderate (−0.2 to −0.5) Large (−0.5 to −0.8) Achievement Severe (< −0.8) Lower Growth Higher Growth Lower Achievement Lower Achievement RI SGP Calculations 2021 Growth 2019 Median Student Growth Percentile
  60. SGP Analysis Plans post 2021 (2022 and beyond) • Spring 2021 represents an ideal time to get a snapshot of student attainment as we are presumably at or near the point of maximal impact on student attainment. • That is, recovery has not really begun for most students in earnest. • It’s impossible to talk about recovery sensibly without understanding how big the impact is. • Monitoring recovery will be at least as critical as monitoring the impact of the pandemic on student achievement. • Beware the K-shaped recovery! RI SGP Calculations 2021
  61. Communication of Results in 2021 • Technical challenges to the calculation of growth in 2021 are likely minor compared to communication challenges. • State reporting is, in general, highly aligned with accountability. • Even though analyses outlined herein deal with status/growth, they don’t necessarily align with accountability narratives. • It is critical for leadership (at the highest level) to map out a communication plan and then build analytics and reporting to support that plan. • Without a well considered, disciplined and coordinated communication plan, accountability narratives are likely to take over. RI SGP Calculations 2021
  62. Summary of Findings • SGP analyses were successfully conducted for both RICAS and WIDA- ACCESS. • Results of the analyses suggest large academic impacts on students due to the pandemic. • Results are larger in math and in ELA • Results are uniformly large across most demographic subgroups. • Results show variation across academic subgroups (e.g., high versus low achievers in some content areas). • Academic impact varied greatly by schools. • Recovery from impacts this large will likely take multiple years and require additional supports beyond that provided by normal schooling. RI SGP Calculations 2021
  63. Odds & Ends for Discussion • Massachusetts and Rhode Island use the same test (though it’s a little complicated this year/2021) • MA and RI both have their own baseline growth norms/coefficient matrices in SGPmatrices package. • Would it be of interest to run RI data with MA growth norms and compare? Would take < 1 hour to run RI SGP Calculations 2021
  64. SGP Goodness of Fit Plots: RICAS ELA RI SGP Calculations 2021
  65. SGP Goodness of Fit Plots: RICAS ELA RI SGP Calculations 2021
  66. SGP Goodness of Fit Plots: RICAS Mathematics RI SGP Calculations 2021
  67. SGP Goodness of Fit Plots: RICAS Mathematics RI SGP Calculations 2021