Idaho SMART Project
An investigation of program efficacy using IRI data
Damian Betebenner
Center for Assessment
March 5th, 2025
SMART Background
• Launched in May 2021, Idaho Striving to Meet Achievement in
Reading Together (SMART) is a multi-year cohort project training
K-3 educators in science-based reading instruction and
intervention.
• Participants engage in facilitated learning with Speech to Print by
Dr. Louisa Moats, guided implementation of evidence-based
practices, and coaching for sustained instructional improvement.
• The goal is to equip K-3 educators with science-aligned reading
strategies to improve student outcomes on the Idaho Reading
Inventory (IRI).
SMART: IRI Impact
• To investigate whether SMART benefits IRI outcomes, we
examine whether students learn more in SMART classrooms
than non-SMART classrooms.
• Using 2022-2023 and 2023-2024 IRI assessment results we
calculate the growth (i.e., learning) of students and compare
rates of learning between groups to examine SMART impact.
• NOTE: Simply comparing IRI outcomes between SMART and
non-SMART students can potentially provide misleading results
as students are not randomly-assigned to the SMART program.
SMART: IRI Impact
Positive SMART Impact: No SMART Impact:
SMART students learn more SMART students learn as much
than non-SMART students as non-SMART students over
over the course of the year the course of the year
SMART
SMART
IRI non-SMART IRI
non-SMART
Fall Spring Fall Spring
IRI: Processes to Impacts to Outcomes
Assessment Indicators
Proximal-Leading Distal-Lagging
Growth/Learning Achievement/Status
lead to lead to
Processes Impacts Outcomes
The diverse set of factors and
activities including
instructional practices,
professional climate, and
student experiences, that are
implemented or cultivated to
support student learning.
Research Questions:
• Do students instructed by SMART participating teachers
demonstrate more learning (i.e., greater growth) than other
students.
• Does student growth/learning differ by grade of student/
teacher?
• Does student growth/learning differ by number of years of
teacher participation in SMART?
• Does student growth/learning differ by number of years of
student receiving SMART instruction?
Background on Student Growth
• Calculation of student growth requires longitudinal, individual results from at least
two assessments (usually in the same content area administered at the same time).
• When thinking about student growth/learning, most users immediately think of
gain (i.e., change) scores.
• Calculation of change across two tests requires the test be scaled vertically placing
the results of both tests on the same scale.
• The IRI is vertically scaled.
• However, gain scores are not a viable growth model because student change is
dependent upon where a student starts.
• All defensible growth models utilize regression to accommodate dependency upon
where a student starts.
Student Growth Percentiles (SGPs)
• SGPs were derived to obviate issues related to the interpretation of change on score scales.
• The SGP properties I discuss are associated with those derived via the methodology
implemented in the SGP package for the R software environment.
• SGPs are integers ranging from 1 to 99 where 1st percentile growth is the lowest and 99th
percentile growth is the highest.
• SGPs are independent of prior achievement meaning that low and high achieving students
have the same chances of demonstrating low or high growth.
• SGPs utilize non-parametric (B-spline) based quintile regression to accommodate both non-
linearity of the multi-variate score distribution as well as heteroskedasticity.
• SGPs are invariant to mono-tone transformations of scale.
• In general, assessment changes resulting in the change of score scale do not impact the
calculation of SGPs (cohort referenced).
IRI Data
• For SGP calculations, IRI data was utilized from the 2022-2023
and 2023-2024 academic years.
• Fall and Spring IRI data was utilized to calculate Fall-to-
Spring SGPs in both 2022-2023 and 2023-2024
• Students with appropriate Fall and Spring scaled scores in
grade K, 1, 2, and 3 received SGPs.
Potential Issues related to SGPs and inferences
regarding SMART
• Because SGP calculations utilize scaled scores, properties of the scaled score
distribution can impact the results.
• Ceiling/Floor effects: SGPs model the multi-variate distribution and
attempt to model 99 conditional quantile curves based upon the scaled
score supplied. When the score distribution is capped and
disproportionate numbers of students are assigned HOSS/LOSS, SGPs will
be impacted.
• Inferences regarding SMART impact (using SGPs or any metric) are only as
good as the data on which they are based.
• If the IRI does not assess what SMART instruction teaches children, then
results from the IRI will not be sensitive to changes in those outcomes.
SMART-IRI Data & SGP Calculation
SMART Participation
• IRI data for the 2022-2023 and 2023-2024 academic years was examined.
• At the start of 2022-2023, between 3.2% (Grade 3) and 5.3% (Grade 1)
students were associated with teachers participating in the SMART
program.
• At the start of 2023-2024, between 5.5% (Grade 3) and 11.7% (Grade
1) students were associated with teachers participating in the SMART
program.
• The achievement level of students participating in the program in
2022-2023 was slightly lower in 3 out of 4 grades and in 2023-2024 was
slightly lower in 1 out of 4 grades.
Starting Achievement Level*: SMART-IRI
Participant
All Students SMART Students Non-Participating
Grade Mean Scale Score Mean Scale Score Students Mean Scale
Percentile (Count) Percentile (Count) Score Percentile (Count)
Fall 2023 2 50.2 (22,966) 50.4 (819) 50.2 (22,147)
Fall 2024 2 50.3 (23,339) 50.9 (1,818) 50.2 (21,521)
* Idaho only normed achievement
SGP Calculation
• Though initial achievement level of students associated with SMART instruction was, in general,
comparable to those not associated with the program, we examine growth/learning by conditioning
upon prior (Fall) achievement.
• SGPs were calculated using IRI reading data supplied by IDOE for 2022-2023 and 2023-2024 in
grades K, 1, 2, and 3.
• SGPs were calculated for Fall-to-Spring in 2022-2023 and 2023-2024.
• SGPs for Spring 2023 utilized a Fall 2022 prior.
• SGPs for Spring 2024 utilized a Fall 2023 prior and Spring 2023 prior (if available)
• Due to the multi-week testing window associated with IRI administration, time between the spring
and fall administration was incorporated into the analyses to account for variability in time-lag
between test administration.
• Analyses were conducted using the R software environment (4.4.3) and the SGP package (2.2-1.93).
• All analyses ran without error and no technical issues emerged in the calculation of the SGPs.
SGP Calculation: IRI Data
SMART SGP SGP SMART
Grade Student Student Student Student Instructor
Count Count Count Count (SMART) Count
Fall 2022 2 22,966 819 41
Spring 2023 2 23,240 830 22,434 801 41
Fall 2023 2 23,339 1,818 90
Spring 2024 2 23,411 1,859 22,681 1,813 93
SGP Calculation: Goodness of Fit
• SGP results show excellent fit in both years and in all grades.
• No ceiling or floor effects were observed on any of the assessments.
• SGPs were uniformly distributed across all prior achievement levels.
• By definition, the mean/median of the SGPs (for the entire grade) is
• However, subgroups (e.g., students associated with SMART
instruction) can deviate from 50.
• Results above 50 indicate increased rates of learning/growth.
• Results below 50 indicate decreased rates of learning/growth
SGP Results
• Did students associated with SMART instruction learn more
than their counterparts not associated with SMART
instruction?
• The following two tables display:
• Mean SGPs by Year, Grade, and SMART subgroup.
• Mean SGPs by Number of Years teacher in SMART program
by grade.
SMART-IRI SGP Results
All Students SMART Students Non-Participating
Grade Mean SGP Mean SGP Students Mean SGP
(Count) (Count) (Count)
Spring 2023 2 50.0 (22,434) 52.3 (801) * 49.9 (21,633)
Spring 2024 2 50.4 (22,681) 48.7 (1,813) * 50.4 (20,868)
* Indicates statistically significant at the .05 level
SMART-IRI SGP Results by Number of Years
Teaching in SMART
0 Years in SMART 1 Year in SMART 2 Years in SMART
Grade Mean SGP Mean SGP Mean SGP
(Count) (Count) (Count)
Spring 2024 2 50.4 (20,868) 47.7 (1,247) 50.8 (566)
SGP Results
• In 2022-2023, Grades K and 2 showed statistically significant
increases in growth/learning.
• In 2023-2024, Grades K, 1, and 3 showed statistically significant
increases in growth/learning while Grade 2 showed a statistically
significant decrease.
• Though the results are statistically significant due to large sample
size, the results are not practically significant.
• Results based upon number of years participating in SMART mirror
the increases but do not show overall increases in learning with two
years participation over one year.
Instructor Level Performance
• De-identified IDs were provided for instructors participating
in the program.
• We examined performance (student growth/learning by
entering level of achievement).
• The results are shown for 2023-2024 for K, 1, 2, and 3 in the
following figures.
SMART Instructor Performance: Grade K
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.08
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.11
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade K
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.08
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.11
2 Year Instructors Highlighted
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.14
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.14
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0
2 Year Instructors Highlighted
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 2
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.03
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.03
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 2
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.03
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.03
2 Year Instructors Highlighted
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.01
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.08
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.01
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.08
2 Year Instructors Highlighted
Growth Mean Student Growth Percentile
Instructor Level Performance
• In general, the results suggest broad variability in rates of
student learning associated with factors that extend beyond
the curriculum/pedagogy (SMART) being utilized.
• The slightly negative correlations suggest SMART instruction
is potentially of greater benefit to lower achieving students.
SMART Impact by Prior Achievement
• The summary results suggest that student associated with SMART participants
demonstrated greater rates of learning if they were lower achieving.
• Discussions with IDE (Meghan) indicated that SMART emphasizes learning to
read over reading to learn which suggest greater efficacy for lower achieving
readers.
• To investigate this hypothesis, we examine student growth/learning of
students associated with SMART by prior achievement quintile.
• In some grades, the results, in general, indicate slightly better rates of learning
for lower achieving students.
• The following figures illustrate results for Grade 1, 2023-2024
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.02
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.16
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.03
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.11
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.1
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.07
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.05
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.04
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.18
Growth Mean Student Growth Percentile
IRI School Performance
2022⌧2023 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.18
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.17
Growth Mean Student Growth Percentile
IRI School Performance
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.17
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.18
Growth Mean Student Growth Percentile
IRI School Performance
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.17
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.18
SMART > 20% highlighted
Growth Mean Student Growth Percentile
IRI School Performance: Grade KG.3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.05
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.09
Growth Mean Student Growth Percentile
IRI School Performance: Grade 1.3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.01
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.04
Growth Mean Student Growth Percentile
IRI School Performance: Grade 2.3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.08
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.07
Growth Mean Student Growth Percentile
IRI School Performance: Grade 3.3
2023⌧2024 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement School Size
Lower Growth Higher Growth 25 Students
100 Students
Mean Scale Score Percentile
250 Students
500 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.05
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.01
Growth Mean Student Growth Percentile
Conclusions
• I commend IDE for investigating the efficacy of the SMART program using IRI
data in “real time”.
• Overall, the results suggest, at best, modest improvement in student learning
for students/teachers in the SMART program.
• These results confirm, I think, some things that we all know:
• “Everything works somewhere. Nothing works everywhere.” Dylan Wiliam
• There are no silver bullets.
• There are likely numerous pedagogical factors beyond SMART participation at
play that also need to be addressed to substantially improve student rates of
learning.
Goodness of Fit (Kindergarten)
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2022−2023.3 Reading SGP, Grade KG.3 (N = 20,764) 2023−2024.3 Reading SGP, Grade KG.3 (N = 19,404)
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score Current Observed Score
5 9 to to to to to to to to to to 5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [100,134) 80.0 16.0 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [100,137) 76.0 8.0 16.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range) Extremes (Range)
Highest OSS: [504,522) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 8.0 16.0 20.0 52.0 N = 25 Highest OSS: [504,521) 0.0 0.0 0.0 0.0 0.0 4.0 4.0 0.0 0.0 20.0 12.0 60.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to 9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[100,198) 9.88 10.17 10.36 10.32 10.07 9.83 9.98 10.22 9.39 9.78 (9.9%) 100 1st/[100,201) 10.24 10.18 10.40 10.29 9.76 9.71 9.97 9.82 9.50 10.13 (9.8%) 100
2nd/[198,218) 9.26 10.14 9.41 9.12 9.70 10.19 10.09 10.09 10.24 11.75 (9.9%) 2nd/[201,220) 8.62 9.83 9.83 9.67 9.83 10.15 10.41 10.04 10.94 10.68 (9.8%)
Prior Scale Score Decile/Range Prior Scale Score Decile/Range
3rd/[218,231) 8.76 9.34 10.01 10.79 10.30 10.01 9.82 9.34 11.32 10.30 (10%) 3rd/[220,233) 9.62 9.57 9.98 9.98 10.96 10.49 9.26 9.93 9.52 10.70 (10%)
Empirical SGP Distribution Empirical SGP Distribution
4th/[231,242) 9.81 10.45 10.06 10.06 9.72 9.86 10.55 9.81 9.81 9.86 (9.9%) 4th/[233,244) 9.50 10.38 9.60 9.08 9.24 9.86 10.22 10.64 10.74 10.74 (9.9%)
5th/[242,252) 10.12 10.37 10.37 9.68 10.72 10.22 9.18 10.67 8.78 9.88 (9.7%) 5th/[244,255) 9.47 9.91 10.40 11.24 9.76 10.06 10.01 9.71 9.57 9.86 (10.5%)
6th/[252,263) 9.28 9.48 9.62 9.57 9.23 10.06 10.54 10.40 10.16 11.66 (9.9%) 6th/[255,265) 10.49 10.27 9.44 9.55 10.82 9.06 9.99 9.77 9.33 11.26 (9.3%)
7th/[263,275) 9.18 10.03 9.93 10.22 10.55 9.89 10.08 9.18 10.03 10.93 (10.2%) 7th/[265,277) 8.96 10.03 10.18 10.08 9.98 10.18 10.08 9.69 10.86 9.98 (10.6%)
8th/[275,289) 10.04 10.27 10.42 10.23 9.42 9.56 9.75 10.27 10.56 9.47 (10.2%) 8th/[277,291) 9.63 9.89 9.78 9.53 9.63 11.22 9.73 10.71 9.48 10.40 (10.1%)
9th/[289,309) 9.13 9.55 9.87 9.97 10.52 10.34 10.11 9.92 9.87 10.71 (10.4%) 9th/[291,310) 9.11 9.91 10.76 10.49 10.17 8.74 10.28 9.54 9.80 11.18 (9.7%)
10th/[309,440] 9.78 10.21 9.97 10.01 9.78 10.01 9.97 10.06 9.73 10.49 (10.1%) 10th/[310,464] 9.73 10.03 9.58 10.03 9.93 10.38 9.98 10.18 10.18 9.98 (10.4%)
*Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
Goodness of Fit (1st grade)
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2022−2023.3 Reading SGP, Grade 1.3 (N = 22,040) 2023−2024.3 Reading SGP, Grade 1.3 (N = 21,732)
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score Current Observed Score
5 9 to to to to to to to to to to 5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [100,184) 83.3 4.2 12.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 24 Lowest OSS: [100,170) 76.0 8.0 12.0 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range) Extremes (Range)
Highest OSS: [580,595) 0.0 0.0 0.0 0.0 0.0 0.0 4.0 4.0 4.0 16.0 12.0 60.0 N = 25 Highest OSS: [578,593) 0.0 0.0 0.0 0.0 3.8 0.0 0.0 3.8 3.8 15.4 26.9 46.2 N = 26
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to 9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[100,269) 9.36 9.92 9.68 10.34 10.10 10.47 10.34 9.78 9.64 10.38 (9.7%) 100 1st/[100,271) 10.00 9.86 10.74 9.76 9.81 9.25 10.97 9.53 10.04 10.04 (9.9%) 100
2nd/[269,290) 9.88 10.20 10.16 9.79 9.74 9.65 9.32 10.48 9.74 11.04 (9.8%) 2nd/[271,292) 8.87 10.71 9.25 10.15 10.38 11.00 8.64 10.29 9.86 10.85 (9.8%)
Prior Scale Score Decile/Range Prior Scale Score Decile/Range
3rd/[290,304) 9.10 9.96 10.32 9.28 10.32 9.64 10.46 9.73 11.19 10.00 (10%) 3rd/[292,306) 9.27 9.54 9.63 10.13 9.81 10.41 9.45 10.50 10.31 10.95 (10.1%)
Empirical SGP Distribution Empirical SGP Distribution
4th/[304,316) 9.96 9.24 10.00 10.90 10.09 10.00 10.31 9.69 9.42 10.40 (10.1%) 4th/[306,318) 9.46 9.84 9.94 9.84 10.32 9.89 10.89 9.42 10.03 10.37 (9.7%)
5th/[316,328) 9.04 10.81 10.10 9.66 10.46 10.19 9.35 10.50 9.17 10.72 (10.2%) 5th/[318,330) 9.43 10.38 10.83 9.66 9.56 8.98 10.83 9.88 10.20 10.24 (10.2%)
6th/[328,340) 9.52 10.26 9.43 9.52 9.47 10.07 10.17 10.12 11.18 10.26 (9.8%) 6th/[330,342) 10.46 10.18 9.68 10.87 9.82 10.50 9.09 10.00 9.36 10.05 (10.1%)
7th/[340,353) 10.03 9.43 10.49 10.44 9.29 10.17 10.44 9.84 9.47 10.40 (9.8%) 7th/[342,356) 9.09 9.32 10.07 10.30 9.74 10.16 10.39 9.88 10.11 10.95 (9.9%)
8th/[353,369) 8.93 9.82 10.13 9.91 10.62 9.60 9.60 10.04 10.31 11.03 (10.2%) 8th/[356,372) 9.31 10.63 9.90 9.22 10.36 9.63 9.72 10.22 10.49 10.54 (10.1%)
9th/[369,391) 9.95 10.76 9.50 10.04 9.90 10.31 9.95 9.72 9.95 9.95 (10.1%) 9th/[372,393) 10.08 9.48 10.04 9.48 10.77 10.31 9.76 10.13 10.04 9.90 (10%)
10th/[391,549] 9.42 9.69 10.22 10.09 9.96 9.91 10.09 10.13 9.91 10.58 (10.3%) 10th/[393,605] 9.19 10.21 9.95 10.52 9.41 9.99 10.39 9.86 9.72 10.75 (10.4%)
*Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
Goodness of Fit (2nd Grade)
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2022−2023.3 Reading SGP, Grade 2.3 (N = 22,434) 2023−2024.3 Reading SGP, Grade 2.3 (N = 22,681)
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score Current Observed Score
5 9 to to to to to to to to to to 5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [105,196) 88.0 0.0 12.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [100,186) 60.0 20.0 8.0 8.0 4.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25
Extremes (Range) Extremes (Range)
Highest OSS: [716,762) 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.0 0.0 4.0 12.0 80.0 N = 25 Highest OSS: [743,794) 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.0 4.0 16.0 8.0 68.0 N = 25
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to 9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[130,322) 9.42 10.00 9.96 10.18 10.18 10.18 10.18 9.28 10.18 10.45 (9.9%) 100 1st/[100,320) 9.81 10.26 9.77 10.30 10.08 9.99 9.42 9.06 10.17 11.14 (10%) 100
2nd/[322,353) 9.38 10.02 10.65 9.07 9.79 9.43 9.29 11.06 10.20 11.11 (9.8%) 2nd/[320,351) 9.78 8.69 9.42 9.37 9.65 9.51 9.96 11.51 10.69 11.42 (9.7%)
Prior Scale Score Decile/Range Prior Scale Score Decile/Range
3rd/[353,373) 10.41 9.45 9.18 11.38 10.02 10.54 10.33 10.11 9.18 9.40 (10.1%) 3rd/[351,372) 9.53 9.30 9.26 9.85 9.85 10.52 10.16 9.94 10.57 11.02 (9.8%)
Empirical SGP Distribution Empirical SGP Distribution
4th/[373,389) 8.70 11.35 10.18 9.29 10.54 9.42 10.81 8.79 10.00 10.90 (9.9%) 4th/[372,388) 9.82 10.26 10.21 10.12 9.90 10.48 9.86 9.86 9.15 10.34 (10%)
5th/[389,403) 9.63 8.79 9.72 10.12 9.77 10.53 9.23 10.48 10.88 10.84 (10%) 5th/[388,403) 9.14 10.63 10.12 10.20 9.61 9.48 9.74 9.78 10.29 11.01 (10.4%)
6th/[403,416) 9.43 10.61 11.50 9.34 9.10 9.67 9.19 10.84 9.90 10.42 (9.5%) 6th/[403,416) 9.45 9.99 9.27 10.39 10.12 9.36 10.48 10.03 10.30 10.62 (9.8%)
7th/[416,431) 9.68 9.84 8.46 11.14 10.43 10.31 10.85 9.93 9.80 9.55 (10.6%) 7th/[416,430) 9.43 8.99 10.39 9.21 10.61 10.13 9.96 10.70 9.74 10.83 (10.1%)
8th/[431,447) 9.13 9.86 10.64 8.81 10.32 9.73 10.14 9.86 9.68 11.83 (9.8%) 8th/[430,446) 10.81 9.03 9.81 10.89 9.07 10.20 9.98 9.42 9.94 10.85 (10.2%)
9th/[447,472) 10.01 10.50 9.62 10.10 9.75 10.28 10.15 9.31 10.23 10.06 (10.1%) 9th/[446,472) 8.98 11.22 10.12 9.15 9.60 10.12 9.90 10.43 9.86 10.61 (10%)
10th/[472,709] 9.36 9.67 10.32 10.28 10.10 9.89 9.67 10.37 9.97 10.37 (10.2%) 10th/[472,676] 9.55 9.50 9.98 9.98 10.37 9.85 9.81 9.94 10.20 10.81 (10.1%)
*Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
Goodness of Fit (3rd Grade)
Student Growth Percentile Goodness−of−Fit Descriptives Student Growth Percentile Goodness−of−Fit Descriptives
2022−2023.3 Reading SGP, Grade 3.3 (N = 22,505) 2023−2024.3 Reading SGP, Grade 3.3 (N = 22,850)
Ceiling / Floor Effects Test Ceiling / Floor Effects Test
Student Growth Percentile Range Student Growth Percentile Range
Current Observed Score Current Observed Score
5 9 to to to to to to to to to to 5 9 to to to to to to to to to to
to to 10 20 30 40 50 60 70 80 90 95 to to 10 20 30 40 50 60 70 80 90 95
Lowest OSS: [100,224) 88.0 12.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 25 Lowest OSS: [110,212) 96.2 3.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 N = 26
Extremes (Range) Extremes (Range)
Highest OSS: [817,847) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.5 20.8 66.7 N = 24 Highest OSS: [845,882) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 3.8 11.5 80.8 N = 26
Student Growth Percentile Range QQ−Plot: Student Growth Percentiles Student Growth Percentile Range QQ−Plot: Student Growth Percentiles
19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution 19 29 39 49 59 69 79 89 99 Theoretical SGP Distribution
9 to to to to to to to to to 9 to to to to to to to to to
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100 1
to 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 100
1st/[181,374) 9.39 10.37 9.52 9.43 9.79 10.06 10.10 10.19 10.15 11.00 (9.9%) 100 1st/[141,375) 9.67 9.75 10.37 9.75 9.97 9.62 9.80 10.19 10.37 10.50 (10%) 100
2nd/[374,407) 10.02 9.76 10.20 10.91 10.47 9.67 10.02 9.62 9.76 9.58 (10%) 2nd/[375,409) 9.47 10.42 9.51 10.64 10.01 10.55 10.73 9.87 9.10 9.69 (9.7%)
Prior Scale Score Decile/Range Prior Scale Score Decile/Range
3rd/[407,426) 8.86 9.74 10.76 9.51 9.88 10.39 9.46 10.76 9.60 11.04 (9.6%) 3rd/[409,430) 9.24 10.30 10.13 10.05 9.96 9.79 8.91 10.05 10.47 11.10 (10.4%)
Empirical SGP Distribution Empirical SGP Distribution
4th/[426,441) 9.34 10.28 9.70 9.74 9.92 10.28 10.41 10.54 10.01 9.79 (10%) 4th/[430,444) 9.29 9.38 9.38 9.24 9.29 10.28 11.09 10.46 10.14 11.45 (9.7%)
5th/[441,456) 10.12 9.90 10.12 10.20 9.77 9.29 9.90 8.98 10.51 11.21 (10.2%) 5th/[444,460) 10.51 10.02 10.95 10.37 10.64 9.44 9.62 9.49 9.71 9.26 (9.9%)
6th/[456,471) 9.06 10.05 9.96 10.05 10.41 9.96 9.91 9.46 10.50 10.64 (9.9%) 6th/[460,475) 8.97 10.49 10.36 9.98 10.58 10.58 9.48 9.69 9.81 10.07 (10.3%)
7th/[471,488) 9.85 9.98 9.98 10.62 9.94 10.45 11.00 10.58 8.32 9.30 (10.4%) 7th/[475,492) 10.30 10.08 9.33 11.04 9.73 9.55 9.86 9.77 9.29 11.04 (10%)
8th/[488,508) 9.58 10.31 9.95 9.04 10.04 9.63 8.68 9.86 11.30 11.62 (9.8%) 8th/[492,511) 8.52 9.59 9.64 8.65 9.23 10.62 10.62 10.89 11.21 11.03 (9.8%)
9th/[508,538) 9.10 9.40 9.66 10.19 9.71 10.58 10.32 10.27 10.23 10.54 (10.2%) 9th/[511,541) 8.96 9.52 9.95 10.04 10.39 9.65 10.26 10.51 10.47 10.26 (10.1%)
10th/[538,765] 9.81 10.30 10.16 10.25 10.12 9.63 10.03 9.90 9.68 10.12 (10%) 10th/[541,787] 10.32 10.45 10.36 10.28 9.98 10.06 9.59 9.16 9.34 10.45 (10.2%)
*Prior score deciles can be uneven depending upon the prior score distribution *Prior score deciles can be uneven depending upon the prior score distribution
SMART Instructor Performance: Grade K
2022⌧2023 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.11
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.13
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 1
2022⌧2023 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: −0.2
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.22
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 2
2022⌧2023 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.14
Lower Achievement Lower Achievement Correlation (N ≥ 10): −0.1
Growth Mean Student Growth Percentile
SMART Instructor Performance: Grade 3
2022⌧2023 Growth & Achievement Growth & Prior Achievement
Higher Achievement Higher Achievement Class Size
Lower Growth Higher Growth 10 Students
25 Students
Mean Scale Score Percentile
50 Students
100 Students
Prior Achievement
Lower Growth Higher Growth Summary Statistics
Correlation: 0.38
Lower Achievement Lower Achievement Correlation (N ≥ 10): 0.23
Growth Mean Student Growth Percentile
Know what you want to accomplish
and focus on the process rather
than the outcome.
Nick Saban
IRI: Processes to Impacts to Outcomes
Assessment Indicators
Proximal-Leading Distal-Lagging
Growth/Learning Achievement/Status
lead to lead to
Processes Impacts Outcomes
The diverse set of factors and
activities including
instructional practices,
professional climate, and
student experiences, that are
implemented or cultivated to
support student learning.
About this copy
Idaho SMART Project — Idaho Department of Education, March 5, 2025.
This is an image-based copy of the original Keynote slides; builds and animations are flattened.