Norm- and Criterion-Referenced Student Growth
New Directions
Exploring Student Growth in New Hampshire:
Considering Results and Next Steps
Damian W. Betebenner
National Center for the Improvement of Educational Assessment
Dover, New Hampshire
AYP Taskforce Meeting
Concorde, New Hampshire February 8th, 2010
Damian W. Betebenner New Hampshire Growth Model
Norm- and Criterion-Referenced Student Growth
New Directions
Agenda for today
Overview of student growth percentile model (norm and
criterion referenced student growth)
A review New Hampshire school performance (percent
proficient versus median SGPs) based upon Fall 2010
NECAP results
Issues regarding growth model deployment/implementation
Damian W. Betebenner New Hampshire Growth Model
Describing Student Growth
Norm- and Criterion-Referenced Student Growth
Criterion Referenced Growth
New Directions
Growth & Effectiveness
Describing student growth
Measuring student growth, even with a vertical scale, is not a
simple task.
Some believe a vertical scale simplifies the task of measuring
student growth.
Even with an interval (or ratio) scale, growth is not easy to
interpret. Consider, for example, height.
A child might grow 4 inches between ages 3 and 4.
4 inches is a well understood quantity.
The 4 inch increase becomes really meaningful only when
understood alongside the growth of other 3 to 4 year olds.
Student growth percentiles were developed to provide a
context for describing student
growth [Betebenner, 2009, Betebenner, 2008].
Damian W. Betebenner New Hampshire Growth Model
Describing Student Growth
Norm- and Criterion-Referenced Student Growth
Criterion Referenced Growth
New Directions
Growth & Effectiveness
Student Growth Percentiles
Should we be surprised with a child’s current achievement given
their prior achievement?
Student growth percentiles answer this question.
Consider a low achieving student with 90th percentile growth
and a high achieving student with 10th percentile growth.
The low achieving student grew at a rate exceeding 90 percent
of similar students.
The high achieving student grew at a rate exceeding just 10
percent of similar students.
The low achiever’s growth is more exemplary (probabilistically)
than the high achiever’s.
Judgments about the adequacy of student growth require
external criteria.
Damian W. Betebenner New Hampshire Growth Model
Describing Student Growth
Norm- and Criterion-Referenced Student Growth
Criterion Referenced Growth
New Directions
Growth & Effectiveness
Combining norms/actual with standards/aspirational
Growth adequacy is determined by whether a student’s
growth is sufficient to reach/maintain desired achievement
levels (e.g., proficiency).
With established performance standards, percentile growth
projections/trajectories are calculated for each student to
reach/maintain desired levels of achievement.
We have investigated Follow the Child targets previously
using student growth percentiles
Student growth percentiles can be seen as an
extension/refinement (not a replacement) of New Hampshire’s
Follow the Child initiative
Damian W. Betebenner New Hampshire Growth Model
Advanced Reading
High
Achievement
CSAP Reading
● Typical
Scale Score
Proficient Growth
Low
● Level Percentiles
High 66th − 99th
Typical 35th − 65th
Part Proficient
● Low 1st − 34th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Next Year
Scale Score 462 539 563 609
Achievement Level Unsatisfactory Part Proficient Part Proficient Proficient Achievement
Growth Percentile 66 66 90
Growth Level High High High Growth
Describing Student Growth
Norm- and Criterion-Referenced Student Growth
Criterion Referenced Growth
New Directions
Growth & Effectiveness
Going from students to groups of students
It’s of interest to examine schools where students
demonstrate, on average, extraordinarily high and low student
growth.
To summarize the student growth percentiles associated with
a school (or other grouping) one calculates the median of the
student growth percentiles.
If students were randomly assigned to schools, one expects to
see a median of 50.
Values greatly above or below 50 are of interest in identifying
best practices or providing extra support.
Examining growth with achievement sheds new light on
school performance.
Damian W. Betebenner New Hampshire Growth Model
New Hampshire: 2009−10 NECAP Growth & Achievement
Growth and Achievement Reading Performance by School Poverty
100 Higher Achievement Higher Achievement
School Size
X XX X X
90 Lower Growth X X
X XX
XX X
XXXXXXX
XXX
X XX Higher
X Growth X
X 50 Students
XX X X
XXX XX
XXX X X X XX X
X 100 Students
X XXX
X X XXX X
X X XX X X
X X XX XXX X X X
XXX X
XXX XX 250 Students
X XXXX XX
XXXX XX
XX X
X X X XXX X XX
X X X X XX
XX
XX
XX X X X
Fall 2010 Percent at/above Proficient
X XX
XXX X
XXX X X X X X X
XXXX
XXX X X X
XX XX X X X
X 500 Students
X XXXX X X X
XX
XX
XXX
X XX
X X XX
70 X X X X XXX X XXX XXX
X X X X X X X XX X X
X X X XX X X
X X X XX
XXX X XXX XX X XXX X
X X XX X X X
XX
XX XX
X X Percent Free/Reduced
XX
X X Lunch Students
X ● Less than 20 percent
X ● 20 to 40 percent
● 40 to 60 percent
X ● 60 to 80 percent
● More than 80 percent
Lower Growth
Higher Growth
0 Lower Achievement Lower Achievement
Fall 2010 Median Student Growth Percentile
New Hampshire: 2009−10 NECAP Growth & Achievement
Growth and Achievement Reading Performance by School Poverty
100 Higher Achievement Higher Achievement School Size
90 Lower Growth Higher Growth 50 Students
100 Students
X 250 Students
Fall 2010 Percent at/above Proficient
500 Students
X X Percent Free/Reduced
Lunch Students
X ● Less than 20 percent
X ● 20 to 40 percent
30 ● 40 to 60 percent
● 60 to 80 percent
20 ● More than 80 percent
Lower Growth Higher Growth
0 Lower Achievement Lower Achievement
Fall 2010 Median Student Growth Percentile
New Hampshire: 2009−10 NECAP Growth & Achievement
Growth and Achievement NECAP Math Performance by School Poverty
100 Higher Achievement X
Higher Achievement
School Size
90 Lower Growth X X
X Higher Growth
XX
50 Students
XX
X X X X X XX XX X
X X 100 Students
XX XXXX
XX
X X XXX X XX XX
XX XX X X
XXX XX XXXXX
X XX X
XX
X X X 250 Students
80 X X XX XX X X XX
X XXX
Fall 2010 Percent at/above Proficient
X XX X X X
XX
X X X X X X X XXXXX X X X
X XX X X X X XXX X XX 500 Students
X XX X
XX
XX X
XX X X XX X X X XX X X X
X X XX XXX X X
XXX XXXXXX X XXXX XX XX X
XX
X XX
X X XX XX XXX X
X XX
X X X X XXX X X XX X
X X X XX X X
XX
X X X X XX
X X XX X X X
XX
X XX X XX
XX XX X XXX
60 X X X XXX X
X X XXXX X XXX X
XX X
XX X X Percent Free/Reduced
XX X X
XX X X X
Lunch Students
X ● Less than 20 percent
X X ● 20 to 40 percent
30 ● 40 to 60 percent
● 60 to 80 percent
● More than 80 percent
Lower Growth X Higher Growth
0 Lower Achievement Lower Achievement
Fall 2010 Median Student Growth Percentile
New Hampshire: 2009−10 NECAP Growth & Achievement
Growth and Achievement NECAP Math Performance by School Poverty
100 Higher Achievement Higher Achievement School Size
90 Lower Growth Higher Growth 50 Students
100 Students
250 Students
Fall 2010 Percent at/above Proficient
X 500 Students
X X Percent Free/Reduced
50 X Lunch Students
● Less than 20 percent
● 20 to 40 percent
30 ● 40 to 60 percent
● 60 to 80 percent
20 ● More than 80 percent
Lower Growth Higher Growth
0 Lower Achievement Lower Achievement
Fall 2010 Median Student Growth Percentile
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
Descriptive Accountability
“Accountability system results can have value without making
causal inferences about school quality, solely from the results of
student achievement measures and demographic characteristics.
Treating the results as descriptive information and for identification
of schools that require more intensive investigation of
organizational and instructional process characteristics are
potentially of considerable value. Rather than using the results of
the accountability system as the sole determiner of sanctions for
schools, they could be used to flag schools that need more
intensive investigation to reach sound conclusions about needed
improvements or judgments about quality.”
R. L. Linn (2008)
Damian W. Betebenner New Hampshire Growth Model
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
Turning Data into Information
Growth Model
Data Calculations
Growth Model
Data Use
Damian W. Betebenner New Hampshire Growth Model
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
Turning Data into Information
Calculating growth quantities is the first and easiest step in
the development and deployment of a growth model.
Student growth percentiles and quantities derived from them
are just data. That data needs to be transformed into
information.
Turning data into information requires a statewide effort
focused on dissemination and use of the data.
Data visualization vis-à-vis reporting is an important means to
help users quickly find the important “stories” that the data
have to tell.
Good reporting/data visualization is an integral part of
outreach/professional development/training efforts.
These efforts are just a part of the conversation about using
data to inform practice and our larger understanding of
education quality.
Damian W. Betebenner New Hampshire Growth Model
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
Web 2.0: Data Visualization and Social Networking
The Colorado Department of Education and the Center for
Assessment have been working for the last two year on
developing next generation data visualization to accompany
growth model data.
The goal: Transform conversations about education through
active engagement with data (i.e., evidence).
Our efforts have received tremendous interest and
recognition:
Recognized by Adobe for innovative uses of their technology
as an Adobe Max Award finalist in October, 2009.
Multiple states signing MOUs to co-develop a cloud-based
reporting platform in a non-proprietary fashion.
Colorado recently devoted $2.5 million of stimulus funds to the
development efforts
Damian W. Betebenner New Hampshire Growth Model
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
Web 2.0: Data Visualization and Social Networking
With a collaborative spirit, with a collaborative platform where
people can upload data, explore data, compare solutions, discuss
the results, build consensus, we can engage passionate people,
local communities, media and this will raise–incredibly–the amount
of people who can understand what is going on.
And this would have fantastic outcomes: the engagement of
people, especially new generations; it would increase knowledge,
unlock statistics, improve transparency and accountability of public
policies, change culture, increase numerary, and in the end,
improve democracy and welfare.
E. Giovannini, Chief Statistician, OECD. June 2007
Damian W. Betebenner New Hampshire Growth Model
Growth & Accountability
Norm- and Criterion-Referenced Student Growth
Web 2.0 Technologies
New Directions
References
References
Betebenner, D. W. (2008).
Toward a normative understanding of student growth.
In Ryan, K. E. and Shepard, L. A., editors, The Future of
Test-Based Educational Accountability, pages 155–170. Taylor
& Francis, New York.
Betebenner, D. W. (2009).
Norm- and criterion-referenced student growth.
Educational Measurement: Issues and Practice, 28(4):42–51.
Linn, R. L. (2008).
Educational accountability systems.
In The Future of Test-Based Educational Accountability, pages
3–24. Taylor & Francis, New York.
Damian W. Betebenner New Hampshire Growth Model
About this copy
Exploring Student Growth in New Hampshire: Considering Results and Next Steps — New Hampshire Department of Education, February 8, 2010.
This is an image-based copy of the original PDF slides; builds and animations are flattened.