Multiple Imputation Approaches to Missing

Rhode Island Department of Education · August 11, 2021

Multiple Imputation Approaches to Missing — Rhode Island Department of Education, August 11, 2021.
Published

August 11, 2021

Event
State agency
Slides
12
Format
Converted from PowerPoint

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Multiple Imputation Approaches to Missing Data with 2021 RICAS Data Adam Van Iwaarden & Damian Betebenner Center for Assessment Rhode Island Department of Education August 11th, 2021
  2. Missing Data and Inferences about School Summaries • Suboptimal participation rates on the RICAS have led to concerns about the reporting of summary statistics (e.g., percent proficient, median SGP, …) for schools. • How would summary statistics at the school look if all (or at least 95%) of the students participated. • Related to this: Is there a participation rate below which the summary statistics cannot be trusted to be representative of all students? • An answer to either question clearly rests on an understanding of the missing data. • Clearly, if the data are missing completely at random, then even a small percentage of student testing yields an unbiased estimate of the entire school. • Thus, an answer to the 2nd question is not likely to yield a single number without a uniform mechanism causing missingness across all schools. • However, number that works in most cases might be possible. Multiple Imputation Approaches to Missing RICAS Data 2
  3. Missing Data and Inferences about School Summaries • The simplest brute force approach to understanding what impact missing data could have is presented by in the paper where: • Missing data are replaced with the LOSS/LOT (lowest obtainable scale score/ lowest obtainable theta) • Missing data re replaced with the HOSS/HOT (highest obtainable scale score/ highest obtainable theta) • Summary statistics for each school can be calculated based upon the worst-case and best-case scenario, respectively. • Not surprisingly, this approach leads to substantial changes in observed values with even modest missing data. Multiple Imputation Approaches to Missing RICAS Data 3
  4. Multiple Imputation • A more sophisticated approach is to utilize modern approaches to characterizing and imputing missing data. • Multiple imputation is a widely used technique to infer missing data, especially in longitudinal studies. • NOTE: Our goal for these techniques is NOT to replace the observed data with a “better” estimate. • The goal is to calculate the better estimate and its standard error/ confidence band. If the observed score lies outside of this confidence band (at a designated significance level), then the missing data is believed to be a biased estimate. Multiple Imputation Approaches to Missing RICAS Data 4
  5. Summary of Results: Mean SGP for ELA and Mathematics Multiple Imputation Approaches to Missing RICAS Data 5
  6. Summary of Results: Mean Scale Score for ELA and Mathematics Multiple Imputation Approaches to Missing RICAS Data 6
  7. Significance by Percent Missing Multiple Imputation Approaches to Missing RICAS Data 7
  8. Investigation of Who’s Missing Multiple Imputation Approaches to Missing RICAS Data 8
  9. Investigation of Who’s Missing Multiple Imputation Approaches to Missing RICAS Data 9
  10. Investigation of Who’s Missing Multiple Imputation Approaches to Missing RICAS Data 10
  11. Investigation of Who’s Missing Multiple Imputation Approaches to Missing RICAS Data 11
  12. Investigation of Who’s Missing Multiple Imputation Approaches to Missing RICAS Data 12