Illinois SGP Calculation 2022

Illinois Technical Advisory Committee · September 15, 2022

Illinois SGP Calculation 2022 — Illinois Technical Advisory Committee, September 15, 2022.
Published

September 15, 2022

Event
Technical advisory committee
Slides
30
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Illinois SGP Calculation 2022 Growth & Pandemic Academic Impact/Recovery Damian Betebenner & Adam Van Iwaarden Center for Assessment Illinois TAC Meeting September 15th, 2022
  2. Where we are in 2022 • Spring 2020 Testing • USED granted testing waivers in 2020 due to the onset of the pandemic in March 2020. • IL tested approximately 80,000 students in 2020 prior to the onset of the pandemic. • Besides those students, all other testing was cancelled in 2020. • No growth analyses were conducted due to the cancellation of testing in 2020 Illinois TAC 12/2/2021
  3. Where we are in 2022 • Spring 2021 Testing • Accountability waivers were granted for states but testing waivers were not universally granted. • The New Meridian consortium was broadly impacted by testing changes: • DC was given a waiver to not test in 2021. • NJ tested using an abbreviated form (Start Strong) in Fall 2021 • IL tested in spring 2021 and offered Fall 2021 testing • BIE and DoDEA testing in spring 2021. • States that tested did not see test participation at levels normally observed (i.e., 95%). USED granted testing waivers in 2020 due to the onset of the pandemic in March 2020. • Illinois (and New Meridian) conducted skip-year/gap year growth analyses from 2019 to • Academic impact results from these analyses were presented to the TAC in December, Illinois TAC 12/2/2021
  4. Where we are in 2022 • Illinois conducted consecutive year growth analyses from 2021 to 2022. • In 2022, four types of SGP analyses were conducted for Illinois data: • Cohort referenced analyses: These analyses create growth norms from the 2021 to 2022 students. These growth norms yield cohort means/median SGPs of 50 and obscure COVID-19 academic impact. Both SIMEX (measurement error corrected) and non-SIMEX SGPs are calculated. • Baseline referenced analyses: Historical, consecutive-year growth norms were created using 2018 to 2019 data. These growth growth analyses don’t have mean/median SGPs of 50 and can be used to investigate overall COVID-19 academic impact. Both SIMEX (measurement error corrected) and non-SIMEX SGPs are calculated. • All analyses were single prior analyses. • Projections (lagged and straight) were calculated for 2022 data using baseline growth norms in case they were needed. Illinois TAC 12/2/2021
  5. SGP Summary of Cohort versus Baseline Referenced • Cohort referenced SGPs • Growth norms created from the current cohort of students. • Typical way that SGPs are calculated. • Both consortium and state referenced norms are created. • Mean/median will by 50 for all grade x content area analyses. • Cannot investigate overall impact of the pandemic on students. • Can investigate differential impact of the pandemic on students. • Does not require scale stability across years. • Baseline referenced SGPs. • Growth norms created from historical (i.e., pre-COVID) cohorts of students. • Mean/median 50 for historical cohort, not necessarily 50 for current cohort. • Both consortium and state referenced baseline norms are created (using 2019 membership) • Extent of deviation from (i.e. below) 50 is indicative of less learning/pandemic related academic impact. • Mean SGP deviation of 5 corresponds to an effect size of 0.18. • Can investigate BOTH overall impact and differential impact • Requires scale stability across years. Illinois TAC 12/2/2021
  6. SGP Calculations • SGP calculations were performed using the SGP package and R software environment based upon data supplied by Pearson. • To facilitate the calculation of baseline SGPs across multiple states, we created an additional R package, SGPmatrices, that contains all baseline matrices for states/organizations. • SGPmatrices • Source code associated with calculations are available on GitHub (private account): • Illinois GitHub Repo • Analyses were conducted first by state/organization based upon availability of data provided by Pearson and then at the consortium level once all data was available. Illinois TAC 12/2/2021
  7. SGP Calculations 2022 • SGP analyses were successfully conducted in July with data provided by Pearson. • No analytic issues were encountered as part of the analyses and data was returned. • Cohort referenced analyses all displayed good model fit. Illinois TAC 12/2/2021
  8. SGP Analyses and Academic Impact & Recovery • Impact is synonymous with deceleration. • Recovery is synonymous with acceleration. • The pandemic, functioning as an academic “headwind” through all the edu-disruptions slows the rate of learning of students (i.e., deceleration). • Decreased rates of learning lead to lower attainment for students. • Growth and status changes are opposite sides of the same coin. However, status changes are much more sensitive to population changes (e.g., non-participation) than growth. Illinois TAC 12/2/2021
  9. Student Attainment and Growth (SGP) Results • The following slides show Illinois student attainment and growth results for 2022. • Results presented do not include any exclusion rules IDOE may apply for accountability purposes (e.g., full-academic year enrollment) • We report both cohort & baseline referenced SGPs in these slides • Cohort referenced results fail to show overall impact. • Due to the highly skewed nature of the SGP results, we recommend median SGP over mean SGP in this instance. • We also provide 2019 and 2021 results to understand pandemic related academic impact/recovery Illinois TAC 12/2/2021
  10. Testing Frequency Data for Spring & Fall 2019 to 2022 Content Spring 2019 Spring 2021 Fall 2021 Spring 2022 Area ELA Math ELA Math ELA Math ELA Math Overall 861,290 859,489 566,600 559,874 31,100 30,799 796,998 794,926 Grade 3 138,105 137,911 92,250 91,493 5,475 5,419 125,725 125,478 Grade 4 141,509 141,336 94,491 93,639 5,481 5,430 127,693 127,384 Grade 5 145,638 145,485 95,801 94,817 5,497 5,432 132,010 131,748 Grade 6 147,791 147,557 95,466 94,306 5,875 5,825 132,844 132,529 Grade 7 144,741 144,413 94,474 93,134 6,181 6,133 137,419 137,003 Grade 8 143,506 142,787 94,118 92,485 2,591 2,560 141,307 140,784 Illinois TAC 12/2/2021
  11. IAR Attainment: 2019, 2021 & 2022 Percent Proficient ELA Mathematics Grade 19 to 21/ 19 to 21/ 2019 2021 2022 19 to 22 2019 2021 2022 19 to 22 Illinois TAC 12/2/2021
  12. IAR Attainment: 2019 to 2021 Percent Proficient with Imputed Scores for Missing ELA Mathematics Grade Percent Percent Difference Imputed Percent Percent Difference Imputed Proficient Proficient 2019 to 2021 Percent Proficient Proficient 2019 to 2021 Percent 2019 2021 Proficient 2019 2021 Proficient Illinois TAC 12/2/2021 (142,787) (92,615) (136,175)
  13. Points of Clarification for Status Results • Decrease in attainment (as a decrease in percent proficient) between 2019 and 2021 is the most often reported statistic related to state testing in 2021. • This analysis & statistic (percent proficient) are problematic for several reasons: • Using 2019 as a counterfactual for where students would have been had the pandemic not occurred is problematic for several reasons including: • Changes in student cohort composition is not accounted for. • Improvement between 2019 and 2021 that would have occurred is not accounted for. • Missing data in 2021 not accounted for. • Multiple imputation results imply results would have been lower in 2021 had all students tested. Illinois TAC 12/2/2021
  14. Points of Clarification for Status Results • Effect sizes associated with decreases range from approximately 0.15 to 0.4. • Participation rates (derived from decreases in observed test scores between 2019 and 2021) vary by grade and state/organization but suggest participation between 60% and 80% (mostly toward the lower end) • In general, the achievement decreases observed are large and are consistent with large academic impact. • Other states, with and without missing data, show similar decreases. • In general, current results based upon incomplete data likely represent a best- case scenario compared to what would appear given fuller participation. Illinois TAC 12/2/2021
  15. IAR Attainment: 2019, 2021 & 2022, Median Baseline SGP ELA Mathematics Grade 19 to 21/ 19 to 21/ 2019 2021 2022 21 to 22 2019 2021 2022 21 to 22 3 NA NA NA NA/NA NA NA NA NA/NA 4 50 (135,033) NA 58.5 (93,266) NA/NA 50 (135,167) NA 58.0 (92,340) NA/NA Illinois TAC 12/2/2021
  16. IAR Growth: 2019 to 2021 Median SGP by Grade Recall: Imputed scores in 2021 suggested lower growth for the population than was observed. ELA Mathematics Grade Median Median Difference Imputed Median SGP Median Difference Imputed SGP 2019 SGP 2021 2019 to Median 2019 SGP 2021 2019 to Median 2021 SGP 2021 2021 SGP 2021 3 NA NA NA NA NA NA NA NA 4 NA NA NA NA NA NA NA NA Illinois TAC 12/2/2021 (130,869)
  17. Discussion • Status and growth analyses from 2019 to 2021 indicated large academic impact associated with the pandemic on students across all grade levels and content areas. • Status and growth analyses from 2021 to 2022 indicate (on average) a return to normal rates of learning. • NOTE: It is important to distinguish between the system returning to normal and the students in the system returning to normal. • The system itself can return to normal rates of learning without the students in the system catching-up to where they would have been. • Eventually, students impacted will leave the system, possibly without ever having caught up to where they would have been. • To help interpret and communicate results across various grades, content areas subgroups and states, we built custom graphics classes in R for visualization purposes. Illinois TAC 12/2/2021
  18. IAR Mathematics Grade 7 Illinois 2019 to 2021 COVID⌧19 Academic Impact All Students Growth: Grade 5 2019 to Grade 7 2021 Status: Grade 7 2019 to Grade 7 2021 Summary 2019 Grade 5 Scaled Score Decile Cuts 2019 Grade 7 Scaled Score Academic Impact: Overall and by Quintile Grade 7 Scaled/Standardized Score and Achievement Level Cuts 2017 to 2019 Mean Conditional Status 2019 Percentiles (Observed) Overall 2019 to 2021 Mean Conditional Status (Observed) 3.5 2021 Percentiles (Observed) 2019 to 2021 Mean Conditional Status (+ Imputed) 2021 Percentiles (+ Imputed) Moderate 2019 to 2021 Growth Impact 3 2019 to 2021 Status Impact Observed + Imputed 2019 and 2021 Mean Scaled Scores Moderate Moderate Moderate Large Large 2021 Grade 7 Scaled Score Decile Cuts 2021 Grade 7 Scaled Score Observed 90 1 Participation 80 750 0.5 Total Status Growth −1.5 2019: 29.9% Proficient 2021: 25.2% Proficient Growth/Status Change Legend 2019 Grade 5 Scaled Score with Achievement Level Cuts Grade 7 Overall Percentile Ranks Modest Severe Large Moderate to Improvement 2019 & 2021 Grade 7 Median SGPs by Quintile 2019 & 2021 Grade 7 Mean Scaled (Standardized) Scores by Quintile None Q1 Q2 Q3 Q4 Q5 Overall Q1 Q2 Q3 Q4 Q5 Overall 2019 Grade 7 Students 50.0 50.0 51.0 50.0 50.0 50.0 2019 Grade 7 Students 695 (−1.34) 718 (−0.56) 733 (−0.05) 750 ( 0.52) 778 ( 1.44) 735 (0.00) 2021 Grade 7 Students 43.0 37.0 35.0 33.0 27.0 34.0 2021 Grade 7 Students 694 (−1.36) 716 (−0.62) 731 (−0.14) 746 ( 0.38) 772 ( 1.26) 732 (−0.11) Calculations suppressed Academic Impact (Observed) −7.0 −13.0 −16.0 −17.0 −23.0 −16 Change (Observed) −1 (−0.03) −2 (−0.06) −3 (−0.10) −4 (−0.14) −6 (−0.19) −3 (−0.11) Academic Impact (+ Imputed) −11.0 −11.0 −14.0 −16.6 −22.0 −14.0 Change (+ Imputed) −0 (0.30) −3 (0.26) −3 (0.28) −4 (0.26) −6 (0.24) −6 (−0.20) due to small group N (< 50) Count Observed 2019 25,252 27,214 27,145 28,075 26,585 134,271 Count Observed 2019 29,700 26,173 31,210 29,121 28,209 144,413 Count Observed 2021 14,916 16,161 17,713 18,740 19,755 87,285 Count Observed 2021 19,651 18,894 17,189 18,918 18,480 93,132 2021 to 2019 Scaled/Standardized Score Difference 2021 to 2019 SGP Difference (Observed) 1.10 2021 to 2019 Grade 7 Status Change (Observed) 30 2021 to 2019 SGP Difference (+ Imputed) 30 1.00 2021 to 2019 Grade 7 Status Change (+ Imputed) 0.95 0.68: Correlation SGP and Status Percentile Change Conditional Status/Growth Change 2021 to 2019 Conditional Status Difference (Observed) 0.85 0.99: Correlation SGP and Conditional Status Difference 0.80 2019 to 2021 2019 Grade 5 Scaled Score with Decile & Achievement Level Cuts Grade 7 Percentile Ranks and Scaled Scores (2019 Norm)
  19. IAR Mathematics Grade 7 Illinois 2021 to 2022 COVID⌧19 Academic Recovery All Students Growth: Grade 6 2019 to Grade 7 2022 Status: Grade 7 2021 to Grade 7 2022 Summary 2021 Grade 6 Scaled Score Decile Cuts 2021 Grade 7 Scaled Score Academic Impact: Overall and by Quintile Grade 7 Scaled/Standardized Score and Achievement Level Cuts 2018 to 2019 Mean Conditional Status 4 2021 Percentiles (Observed) Overall 2021 to 2022 Mean Conditional Status (Observed) 2022 Percentiles (Observed) 2021 to 2022 Growth Impact 3.5 2021 to 2022 Status Impact Recovery 2021 and 2022 Mean Scaled Scores Recovery Recovery Recovery Recovery Stabilization 2022 Grade 7 Scaled Score Decile Cuts Observed 2022 Grade 7 Scaled Score Participation Total Status Growth −1.5 2021: 25.2% Proficient 2022: 25.7% Proficient Growth/Status Change Legend 2021 Grade 6 Scaled Score with Achievement Level Cuts Grade 7 Overall Percentile Ranks Deterioration Stabilization Recovery 2019 & 2022 Grade 7 Median SGPs by Quintile 2021 & 2022 Grade 7 Mean Scaled (Standardized) Scores by Quintile Q1 Q2 Q3 Q4 Q5 Overall Q1 Q2 Q3 Q4 Q5 Overall 2019 Grade 7 Students 50.0 51.0 50.0 51.0 50.0 50.0 2021 Grade 7 Students 694 (−1.32) 716 (−0.53) 731 (−0.03) 746 ( 0.52) 772 ( 1.44) 732 (0.00) 2022 Grade 7 Students 61.0 63.0 64.0 64.0 55.0 61.0 2022 Grade 7 Students 692 (−1.39) 715 (−0.60) 730 (−0.06) 745 ( 0.48) 773 ( 1.45) 731 (−0.02) Calculations suppressed Academic Impact (Observed) 11.0 12.0 14.0 13.0 5.0 11.0 Change (Observed) −2 (−0.08) −2 (−0.07) −1 (−0.03) −1 (−0.04) 0 ( 0.01) 0 (−0.02) Count Observed 2019 27,693 27,850 25,977 27,735 28,419 137,674 Count Observed 2021 19,653 18,894 17,189 18,918 18,480 93,134 due to small group N (< 50) Count Observed 2022 18,270 17,485 17,364 18,164 18,213 89,496 Count Observed 2022 26,268 29,474 24,936 28,962 27,363 137,003 2022 to 2021 Scaled/Standardized Score Difference 2022 to 2021 SGP Difference (Observed) 1.15 2022 to 2021 Grade 7 Status Change (Observed) 2022 to 2021 Conditional Status Difference (Observed) 1.00 −0.04: Correlation SGP and Status Percentile Change Conditional Status/Growth Change 25 0.90: Correlation SGP and Conditional Status Difference 25 0.90 2021 to 2022 2021 Grade 6 Scaled Score with Decile & Achievement Level Cuts Grade 7 Percentile Ranks and Scaled Scores (2021 Norm)
  20. IAR ELA and Mathematics Illinois 2019 to 2021 COVID⌧19 Academic Impact Grade & Content Area Impact Recovery −40 −25 −15 −5 5 20 Growth Change −0.55 −0.4 −0.25 −0.1 0.05 0.2 Status Change Modest Calculations suppressed Severe Large Moderate to Improvement None due to small group N (< 50) ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 All Students Large Large Large Moderate Large Moderate Large Severe Large Large Moderate Moderate Economically Disadvantaged Students Severe Large Large Moderate Large Moderate Severe Severe Large Large Moderate Moderate Non−Economically Disadvantaged Students Large Large Large Large Large Large Large Large Large Large Large Moderate African American Students Severe Large Moderate Moderate Moderate Moderate Severe Severe Large Moderate Moderate Moderate Asian Students Large Severe Large Large Large Large Large Large Large Large Large Large White Students Large Large Large Large Large Large Moderate Large Large Large Large Moderate Hispanic Students Severe Large Large Moderate Large Moderate Severe Severe Large Large Moderate Moderate ELL Students Large Large Moderate Moderate Moderate Moderate Severe Severe Large Large Moderate Moderate Special Education Students Large Moderate Moderate Moderate Moderate Moderate Large Moderate Large Moderate Moderate Moderate Female Students Large Large Large Large Large Large Severe Severe Large Large Large Large Male Students Large Large Large Moderate Large Moderate Large Large Large Large Moderate Moderate
  21. IAR ELA and Mathematics Illinois 2021 to 2022 COVID⌧19 Academic Recovery Grade & Content Area Deterioration Recovery −30 −7 10 30 Growth Change −0.55 −0.07 0.1 0.2 Status Change Calculations suppressed Deterioration Stabilization Recovery due to small group N (< 50) ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 All Students Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Economically Disadvantaged Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization Non−Economically Disadvantaged Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization African American Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization Asian Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization White Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Hispanic Stabilization Recovery Recovery Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization ELL Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Recovery Stabilization Special Education Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Stabilization Female Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Male Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization
  22. ELA And Mathematics Illinois 2019 to 2021 COVID⌧19 Academic Impact Grade, Content Area & Achievement Quintile Impact Recovery −40 −25 −15 −5 5 20 Growth Change NOTE: Quintiles for each student −0.55 −0.4 −0.25 −0.1 0.05 0.2 Status Change subgroup are based upon quintile Modest Calculations suppressed cuts defined for All Students. Severe Large Moderate to Improvement None due to small group N (< 50) Box width indicates group proportion. ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile All Students Economically Disadvantaged Students Non−Economically Disadvantaged Students African American Students Asian Students White Students Hispanic Students ELL Students Special Education Students Female Students Male Students
  23. ELA And Mathematics Illinois 2021 to 2022 COVID⌧19 Academic Recovery Grade, Content Area & Achievement Quintile Deterioration Recovery −30 −7 10 30 Growth Change NOTE: Quintiles for each student −0.55 −0.07 0.1 0.2 Status Change subgroup are based upon quintile Calculations suppressed cuts defined for All Students. Deterioration Stabilization Recovery due to small group N (< 50) Box width indicates group proportion. ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile All Students Economically Disadvantaged Non−Economically Disadvantaged African American Asian White Hispanic ELL Special Education Female Male
  24. IAR Mathematics Grade 5 Illinois 2021 to 2022 COVID⌧19 Academic Recovery African American Students Growth: Grade 4 2019 to Grade 5 2022 Status: Grade 5 2021 to Grade 5 2022 Summary 2021 Grade 4 Scaled Score Decile Cuts All Students 10 20 30 40 50 60 70 80 90 2021 Grade 5 Scaled Score Academic Impact: Overall and by Quintile Grade 5 Scaled/Standardized Score and Achievement Level Cuts African American Students 2018 to 2019 Mean Conditional Status 3.5 2021 Percentiles (Observed) Overall 2021 to 2022 Mean Conditional Status (Observed) 2022 Percentiles (Observed) 2021 to 2022 Growth Impact 2021 to 2022 Status Impact Stabilization 2021 and 2022 Mean Scaled Scores Deterioration Stabilization Stabilization Recovery Stabilization Observed 2022 Grade 5 Scaled Score 2022 Grade 5 Scaled Score Decile Cuts Participation 750 750 Total Status Growth −1 2021: 4.6% Proficient 2022: 4.8% Proficient −2 Growth/Status Change Legend 2021 Grade 4 Scaled Score with Achievement Level Cuts Grade 5 Overall and Subgroup Percentile Ranks Deterioration Stabilization Recovery 2019 & 2022 Grade 5 Median SGPs by Quintile 2021 & 2022 Grade 5 Mean Scaled (Standardized) Scores by Quintile Q1 Q2 Q3 Q4 Q5 Overall Q1 Q2 Q3 Q4 Q5 Overall 2019 Grade 5 Students 46.0 43.0 41.0 40.0 39.0 43.0 2021 Grade 5 Students 681 (−0.69) 705 (−0.69) 721 (−0.69) 740 (−0.69) 770 (−0.69) 702 (−0.69) 2022 Grade 5 Students 31.0 46.0 51.0 52.0 45.0 41.0 2022 Grade 5 Students 676 (−0.69) 704 (−0.69) 722 (−0.69) 741 (−0.69) 768 (−0.69) 702 (−0.69) Calculations suppressed Academic Impact (Observed) −15.0 3.0 10.0 12.0 6.0 −2.0 Change (Observed) −5 (−0.15) −1 (−0.04) 1 ( 0.04) 1 ( 0.03) −2 (−0.06) 0 (0.00) Count Observed 2019 8,674 5,910 4,324 2,508 1,353 22,769 Count Observed 2021 5,432 3,405 1,793 961 441 12,032 due to small group N (< 50) Count Observed 2022 5,353 2,938 1,486 978 410 11,165 Count Observed 2022 8,789 6,271 4,001 1,945 866 21,872 2022 to 2021 Scaled/Standardized Score Difference 2022 to 2021 SGP Difference (Observed) 0.95 2022 to 2021 Grade 5 Status Change (Observed) 2022 to 2021 Conditional Status Difference (Observed) 0.85 0.17: Correlation SGP and Status Percentile Change Conditional Status/Growth Change 25 1.00: Correlation SGP and Conditional Status Difference 25 0.75 2021 to 2022 2021 Grade 4 Scaled Score with Decile & Achievement Level Cuts Grade 5 Percentile Ranks and Scaled Scores (2021 Norm)
  25. All Students Multi⌧State 2019 to 2021 COVID⌧19 Academic Impact ELA And Mathematics by Grade Growth/Status Change Legend Modest Calculations suppressed Severe Large Moderate to Improvement None due to small group N (< 50) ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Illinois Large Large Large Moderate Large Moderate Large Severe Large Large Moderate Moderate State A Moderate Moderate Large Moderate Moderate Moderate Large Large Large Severe Large Severe State B Moderate Moderate Moderate Moderate Large Moderate State C Moderate Moderate Modest to None Moderate Modest to None Moderate Large Large Large Large Large Large State D Moderate Moderate Large Large Large Large Large Moderate Severe Severe Large Severe State E Large Moderate Large Moderate Large Moderate Severe Large Large Severe Large Large State F Moderate Moderate Large Large Large Large Severe Severe Severe Severe Large Severe State G Moderate Moderate Large Moderate Moderate Moderate Severe Large Severe Large Large Large State H Moderate Moderate Moderate Moderate Moderate Moderate Moderate Large Large Large Large Large State I Moderate Large Moderate Moderate Modest to None Moderate Moderate Moderate Moderate Moderate Moderate Moderate
  26. District Level Results • State averages mask important impact and recovery variation. • We investigated impact and recovery for numerous larger districts in Illinois to try and surface important variation with regard to impact and recovery. Illinois TAC 12/2/2021
  27. IAR ELA and Mathematics ELA And Mathematics ELA And Mathematics Illinois 2021 to 2022 COVID⌧19 Academic Recovery Grade & Content Area Illinois 2021 to 2022 COVID⌧19 Academic Recovery Grade, Content Area & Achievement Quintile Illinois 2021 to 2022 COVID⌧19 Academic Recovery Grade, Content Area & Achievement Quintile Deterioration Recovery Deterioration Recovery Deterioration Recovery −30 −7 10 30 Growth Change −30 −7 10 30 Growth Change NOTE: Quintiles for each student −30 −7 10 30 Growth Change NOTE: Quintiles for each student −0.55 −0.07 0.1 0.2 Status Change −0.55 −0.07 0.1 0.2 Status Change subgroup are based upon quintile −0.55 −0.07 0.1 0.2 Status Change subgroup are based upon quintile Calculations suppressed Calculations suppressed cuts defined for All Students. Calculations suppressed cuts defined for All Students. Deterioration Stabilization Recovery Deterioration Stabilization Recovery Deterioration Stabilization Recovery due to small group N (< 50) due to small group N (< 50) Box width indicates group proportion. due to small group N (< 50) Box width indicates group proportion. ELA Mathematics ELA Mathematics ELA Mathematics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile Quintile District Academic All Students Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization All Students All Students 010011720220000 Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization 040041000260000 Deterioration Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Recovery Recovery Stabilization Recovery Stabilization 041011220220000 Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration 041012050250000 Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization 050160150040000 Stabilization Stabilization Stabilization Deterioration Deterioration Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Results 050160210040000 Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Recovery Deterioration 050160250020000 Stabilization Deterioration Deterioration Deterioration Deterioration Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization 050160340040000 Stabilization Stabilization Stabilization Deterioration Stabilization Deterioration Recovery Recovery Recovery Stabilization Recovery Stabilization 050160390020000 Recovery Recovery Recovery Recovery Recovery Deterioration Recovery Recovery Recovery Recovery Deterioration Deterioration 050160540040000 Recovery Recovery Recovery Recovery Recovery Stabilization Recovery Recovery Recovery Recovery Recovery Stabilization 050160590040000 Recovery Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Stabilization Recovery Stabilization Stabilization Stabilization 050160620040000 Recovery Stabilization Stabilization Deterioration Stabilization Stabilization Recovery Stabilization Recovery Stabilization Recovery Stabilization (Attached as a 050160640040000 Recovery Recovery Stabilization Recovery Stabilization Stabilization Recovery Recovery Recovery Recovery Recovery Deterioration 050160650040000 Recovery Recovery Recovery Recovery Stabilization Deterioration Recovery Stabilization Stabilization Recovery Stabilization Stabilization 060160970020000 Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Recovery Recovery Recovery 060160990020000 Recovery Stabilization Stabilization Stabilization Stabilization Deterioration Recovery Recovery Stabilization Stabilization Stabilization Stabilization 060161000020000 Recovery Stabilization Stabilization Deterioration Deterioration Deterioration Recovery Recovery Recovery Stabilization Recovery Stabilization 070161170020000 Recovery Stabilization Stabilization Deterioration Recovery Stabilization Recovery Recovery Recovery Stabilization Recovery Stabilization separate PDF) 070161230020000 Stabilization Stabilization Recovery Stabilization Deterioration Stabilization Recovery Recovery Recovery Stabilization Stabilization Stabilization 070161350020000 Recovery Stabilization Stabilization Deterioration Deterioration Deterioration Recovery Recovery Recovery Stabilization Stabilization Deterioration 070161400020000 Stabilization Deterioration Recovery Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization 070161700020000 Recovery Stabilization Recovery Recovery Stabilization Stabilization Recovery Recovery Stabilization Recovery Recovery Stabilization 090100040260000 Deterioration Stabilization Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Stabilization 150162990250000 Stabilization Recovery Recovery Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization 160194280260000 Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Deterioration 170640050260000 Deterioration Stabilization Deterioration Stabilization Deterioration Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization 190220330020000 Recovery Stabilization Stabilization Deterioration Stabilization Stabilization Recovery Stabilization Stabilization Recovery Recovery Stabilization 190220410020000 Stabilization Stabilization Stabilization Recovery Recovery Recovery Recovery Recovery Recovery Recovery Recovery Recovery 190220450020000 Recovery Recovery Stabilization Stabilization Stabilization Deterioration Recovery Recovery Stabilization Stabilization Recovery Stabilization 190220930040000 Stabilization Stabilization Stabilization Stabilization Deterioration Deterioration Stabilization Stabilization Recovery Stabilization Stabilization Stabilization 190221810040000 Recovery Stabilization Stabilization Deterioration Stabilization Deterioration Stabilization Deterioration Stabilization Stabilization Recovery Deterioration 190222000260000 Stabilization Recovery Recovery Recovery Recovery Stabilization Recovery Stabilization Stabilization Stabilization Recovery Recovery 190222030260000 Recovery Stabilization Stabilization Stabilization Stabilization Deterioration Recovery Recovery Recovery Recovery Recovery Stabilization 190222040260000 Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization 190222050260000 Recovery Recovery Recovery Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Stabilization Stabilization 240322010040000 Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Deterioration Stabilization Recovery Stabilization Deterioration 240471150260000 Deterioration Deterioration Deterioration Deterioration Deterioration Deterioration Stabilization Deterioration Deterioration Stabilization Stabilization Deterioration 240473080260000 Stabilization Recovery Recovery Stabilization Deterioration Deterioration Recovery Recovery Recovery Stabilization Recovery Stabilization 310450460220000 Recovery Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Stabilization Stabilization Stabilization Stabilization Stabilization 310451010220000 Stabilization Stabilization Recovery Stabilization Stabilization Deterioration Recovery Stabilization Stabilization Stabilization Recovery Deterioration 310451290220000 Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Recovery Stabilization Stabilization Stabilization Stabilization 310451310220000 Deterioration Deterioration Deterioration Deterioration Stabilization Stabilization Recovery Recovery Stabilization Recovery Recovery Stabilization 310453000260000 Recovery Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Recovery Stabilization Recovery Stabilization Stabilization 310453010260000 Recovery Stabilization Recovery Stabilization Stabilization Recovery Recovery Deterioration Recovery Recovery Recovery Stabilization 310453030260000 Stabilization Stabilization Recovery Deterioration Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration 310453040260000 Recovery Recovery Stabilization Deterioration Stabilization Recovery Recovery Recovery Recovery Stabilization Recovery Stabilization 320461110250000 Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Recovery Recovery Stabilization Recovery Stabilization 340490460040000 Recovery Recovery Stabilization Stabilization Deterioration Deterioration Recovery Stabilization Stabilization Stabilization Stabilization Deterioration 340490600260000 Recovery Stabilization Stabilization Stabilization Deterioration Deterioration Recovery Recovery Stabilization Stabilization Stabilization Stabilization 340490730040000 Recovery Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Recovery Recovery Stabilization Stabilization Stabilization 340490950260000 Recovery Recovery Recovery Recovery Recovery Recovery Stabilization Stabilization Stabilization Recovery 340490960040000 Recovery Stabilization Stabilization Deterioration Stabilization Deterioration Recovery Stabilization Stabilization Stabilization Recovery Stabilization 340491090020000 Stabilization Stabilization Stabilization Stabilization Deterioration Deterioration Stabilization Stabilization Stabilization Stabilization Recovery Deterioration 340491120020000 Recovery Stabilization Deterioration Stabilization Deterioration Stabilization Stabilization Deterioration Stabilization Deterioration Stabilization Deterioration 340492200260000 Stabilization Recovery Recovery Recovery Deterioration Stabilization Stabilization Recovery Recovery Stabilization Stabilization Stabilization 390550610250000 Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization 410570020260000 Recovery Stabilization Stabilization Stabilization Stabilization Deterioration Deterioration Deterioration Stabilization Stabilization Deterioration Stabilization 410570070260000 Deterioration Stabilization Stabilization Recovery Deterioration Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization 410570100260000 Recovery Stabilization Recovery Stabilization Stabilization Deterioration Recovery Stabilization Stabilization Recovery Recovery Stabilization 410570110260000 Stabilization Recovery Stabilization Stabilization Deterioration Stabilization Recovery Recovery Stabilization Stabilization Deterioration Deterioration 440630150040000 Recovery Stabilization Stabilization Stabilization Recovery Stabilization Recovery Recovery Deterioration Stabilization Recovery Stabilization 440630470040000 Stabilization Stabilization Stabilization Deterioration Deterioration Deterioration Stabilization Stabilization Stabilization Stabilization Recovery Stabilization 440631580220000 Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization 440632000260000 Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Recovery Recovery Recovery 480721500250000 Stabilization Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Stabilization 480723230260000 Stabilization Stabilization Stabilization Recovery Deterioration Deterioration Stabilization Stabilization Deterioration Stabilization Stabilization Deterioration 490810400220000 Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Recovery Stabilization Recovery Recovery Stabilization 490810410250000 Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization 500820190260000 Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Deterioration Stabilization Recovery Stabilization Stabilization 500820900040000 Recovery Stabilization Stabilization Stabilization Stabilization Deterioration Stabilization Stabilization Recovery Stabilization Recovery Deterioration 500821180020000 Recovery Stabilization Stabilization Stabilization Deterioration Stabilization Recovery Stabilization Stabilization Recovery Recovery Stabilization 500821890220000 Recovery Stabilization Stabilization Deterioration Deterioration Stabilization Recovery Recovery Deterioration Stabilization Recovery Stabilization 510840050260000 Recovery Recovery Deterioration Stabilization Deterioration Stabilization Recovery Recovery Stabilization Stabilization Deterioration Stabilization 510841860250000 Recovery Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Recovery Stabilization Recovery Recovery Stabilization 530901080020000 Recovery Recovery Recovery Stabilization Deterioration Stabilization Stabilization Recovery Stabilization Stabilization Stabilization Stabilization 56099030c040000 Stabilization Stabilization Stabilization Stabilization Deterioration Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Deterioration 56099033c040000 Recovery Recovery Stabilization Stabilization Stabilization Stabilization Recovery Recovery Deterioration Stabilization Recovery Stabilization 560990860050000 Recovery Recovery Recovery Stabilization Stabilization Recovery Recovery Recovery Stabilization Stabilization Stabilization Stabilization 560991220020000 Deterioration Deterioration Deterioration Deterioration Deterioration Deterioration Stabilization Deterioration Stabilization Stabilization Stabilization Deterioration 56099157c040000 Recovery Stabilization Stabilization Deterioration Deterioration Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization Stabilization 560991610020000 Recovery Deterioration Recovery Stabilization Stabilization Stabilization Recovery Stabilization Deterioration Recovery Stabilization Recovery 560992020220000 Recovery Stabilization Recovery Stabilization Stabilization Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization 56099365u260000 Recovery Recovery Stabilization Stabilization Deterioration Stabilization Recovery Recovery Stabilization Stabilization Recovery Stabilization
  28. SGP Analysis Plans post 2022 • Continued use of the current assessment allows Illinois to compare student growth and attainment to pre-COVID times. • Important to distinguish between system level recovery and student level recovery. • Growth and status analyses will continue to be valuable assets in understanding the extent to which students are recovering from pandemic related academic impact. Illinois TAC 12/2/2021
  29. Summary of Findings • SGP analyses were successfully conducted for Illinois in Spring 2022 • Cohort and baseline referenced SGPs were calculated. • Baseline referenced analyses suggest, on average, that students in the Illinois education system have returned to typical rates of learning. • However, typical rates of learning are not sufficient to catch students up to where they would have been. • Average, of course, mask important variation with some students demonstrating robust recovery and other demonstrating further declines. Illinois TAC 12/2/2021
  30. Questions