Multiple Imputation Approaches to Missing
Data with 2021 RICAS Data
Adam Van Iwaarden & Damian Betebenner
Center for Assessment
Rhode Island Department of Education
August 11th, 2021
Missing Data and Inferences about School Summaries
• Suboptimal participation rates on the RICAS have led to concerns about the reporting
of summary statistics (e.g., percent proficient, median SGP, …) for schools.
• How would summary statistics at the school look if all (or at least 95%) of the
students participated.
• Related to this: Is there a participation rate below which the summary statistics
cannot be trusted to be representative of all students?
• An answer to either question clearly rests on an understanding of the missing data.
• Clearly, if the data are missing completely at random, then even a small percentage of
student testing yields an unbiased estimate of the entire school.
• Thus, an answer to the 2nd question is not likely to yield a single number without a
uniform mechanism causing missingness across all schools.
• However, number that works in most cases might be possible.
Multiple Imputation Approaches to Missing RICAS Data 2
Missing Data and Inferences about School Summaries
• The simplest brute force approach to understanding what impact
missing data could have is presented by in the paper where:
• Missing data are replaced with the LOSS/LOT (lowest obtainable scale score/
lowest obtainable theta)
• Missing data re replaced with the HOSS/HOT (highest obtainable scale score/
highest obtainable theta)
• Summary statistics for each school can be calculated based upon the
worst-case and best-case scenario, respectively.
• Not surprisingly, this approach leads to substantial changes in
observed values with even modest missing data.
Multiple Imputation Approaches to Missing RICAS Data 3
Multiple Imputation
• A more sophisticated approach is to utilize modern approaches to
characterizing and imputing missing data.
• Multiple imputation is a widely used technique to infer missing data,
especially in longitudinal studies.
• NOTE: Our goal for these techniques is NOT to replace the observed
data with a “better” estimate.
• The goal is to calculate the better estimate and its standard error/
confidence band. If the observed score lies outside of this confidence
band (at a designated significance level), then the missing data is
believed to be a biased estimate.
Multiple Imputation Approaches to Missing RICAS Data 4
Summary of Results: Mean SGP for ELA and Mathematics
Multiple Imputation Approaches to Missing RICAS Data 5
Summary of Results: Mean Scale Score for ELA and
Mathematics
Multiple Imputation Approaches to Missing RICAS Data 6
Significance by Percent Missing
Multiple Imputation Approaches to Missing RICAS Data 7
Investigation of Who’s Missing
Multiple Imputation Approaches to Missing RICAS Data 8
Investigation of Who’s Missing
Multiple Imputation Approaches to Missing RICAS Data 9
Investigation of Who’s Missing
Multiple Imputation Approaches to Missing RICAS Data 10
Investigation of Who’s Missing
Multiple Imputation Approaches to Missing RICAS Data 11
Investigation of Who’s Missing
Multiple Imputation Approaches to Missing RICAS Data 12
About this copy
Multiple Imputation Approaches to Missing — Rhode Island Department of Education, August 11, 2021.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.