Understanding the West Virginia
Growth Model
February 20, 2015
Morgantown, West Virginia
Welcome and Overview
Justin Baer, Ph.D.
Director, REL Appalachia
What is a REL?
• A REL is a regional educational laboratory.
• There are 10 RELs across the country.
• The REL program is administered by the U.S. Department of
Education, Institute of Education Sciences (IES).
• A REL serves the education needs of a designated region.
• The REL works in partnership with the region’s school districts, state
departments of education, and others to use data and research to
improve academic outcomes for students.
REL Appalachia’s mission
• Meet the applied research and technical assistance needs of
Kentucky, Tennessee, Virginia, and West Virginia.
• Conduct empirical research and analysis.
• Bring evidence-based information to policymakers and practitioners:
– Inform policy and practice—for states, districts, schools, and other
stakeholders.
– Focus on high-priority, discrete issues and build a body of knowledge
over time.
http://www.RELAppalachia.org
Follow us! @REL_Appalachia
Workshop goals
• Provide participants with the research foundation and background of
the WV Growth Model
• Increase participants’ understanding of specific applications of the
growth model in WV schools.
• Increase participants’ understanding of how the WV Growth Model
and forthcoming WV A-F school grading system are linked.
West Virginia Growth Model:
Changing Conversations about Student Performance
Dr. Damian Betebenner
Senior Associate
The National Center for the Improvement of Educational Assessment
West Virginia Growth Model
• The existence of annual student testing together with unique student
identifiers forms the foundation for the calculation of student
growth.
• Over the last decade, most states have begun calculation of
longitudinal student growth using large scale state assessments.
• The analysis of student growth is controversial because of the uses to
which it is put: Teacher & School Evaluation.
• Development of the West Virginia Growth Model involved the creation
of a diagnostically useful quantity that could be used (or not used) at
a later time for many purposes.
West Virginia Growth Model
• Research on the SGP model began in 2007 (Betebenner, 2008)
• Based upon work in Colorado, the norm-referenced model was
extended to include a criterion-referenced component. (Betebenner,
• Work in other states incorporating the model into state reporting and
accountability systems occurred in subsequent years and student
growth became more common across state assessment systems.
(Betebenner, 2012)
West Virginia Growth Model
Why Student Growth:
If you care about student achievement,
then you care about student growth.
• Where you are is a function of
where you started and how
quickly you’ve travelled since.
• Student growth and student
attainment/achievement are
related in this way.
• Distance=Rate x Time
• Those who start behind tend to stay
behind and those that start ahead
tend to stay ahead.
West Virginia Growth Model
• The concept of student growth isn’t as difficult to understand and
popular perception suggests.
• Calculations may be complicated but knowing how to calculate
student growth is not synonymous with understanding student
growth:
– The number π (pi=3.14159…) is difficult to calculate but is simple to
understand conceptually.
– Height and weight percentiles associated with children are difficult
to calculate but are simple to understand conceptually.
• Computational details are important but often obscure a simple
conceptual basis.
West Virginia Growth Model
A Math Story Problem
• Whose high jump changed
more? Explain your answer.
• Whose high jump change is
more remarkable? Explain
your answer.
• Which high jumper improved
more? Explain your answer.
West Virginia Growth Model
• There are 3 general approaches of calculating student growth:
– Growth as an amount of change (i.e., gain).
– Growth as a norm/peer referenced quantity.
– Growth as a criterion referenced quantity.
West Virginia Growth Model
• Growth as an amount of change:
– What could be more simple?
– Like height or weight just calculate the amount of change.
– Requires a vertical scale (which West Virginia has up to this year and
will have going forward with SBAC).
– Growth = Score Time 2 - Score Time 1
• Not as simple as it seems:
– Comparisons between students often leads to wrong conclusions
– Interpreting change requires context.
West Virginia Growth Model
Back to the story problem!
West Virginia Growth Model
• Which high jumper improved
more:
– Judy improved 2 inches.
– Anna improved 6 inches.
– An inch is an inch!!!
– But is an inch really
an inch???
– 2 inches for an elite
jumper is far more
remarkable than 6
inches for a novice.
West Virginia Growth Model
• The same holds true for the vertically scaled tests (e.g., SBAC).
Average Scale Score Change 2013 to 2014
Top 10% Reading Grade 4 2013 -7.5
Bottom 10% Reading Grade 4 2013 +24.5
Top 10% Mathematic Grade 4 2013 -12.6
Bottom 10% Mathematics Grade 4 2013 +18.4
• Just as an inch isn’t necessarily an inch; a scale score point isn’t
necessarily a scale score point.
West Virginia Growth Model
•This is analogous to height/weight measures for infants.
•Growth amounts are tricky to interpret and require context.
•Peer based comparisons used by growth analyses are a way to acquire that context.
An infant male toddler is measured at 2 and 3 years of age and is
shown to have grown 4 inches. The magnitude of increase—4 inches
—is a well understood quantity that any parent can grasp and
calculate at home using a simple yardstick. However, parents
leaving their pediatrician’s office knowing only how much their
child has grown would likely be wanting for more information:
Parents are not interested in an absolute magnitude of growth, but
instead in a norm-referenced quantity locating that 4 inch increase
alongside the height increases of similar children.
West Virginia Growth Model
• Peer-based analyses are also called norm-referenced analyses.
• Like with infant height/weight charts, peers are children/student
with similar characteristics.
• For growth analyses using large scale assessments, peers are
students with (at least) the same scale score history on the same
tests.
• Peer based norms are invaluable in providing context for
stakeholders to understand student growth.
• Peer based norms are possible even across test transitions like West
Virginia’s transition from WESTEST 2 to SBAC.
West Virginia Growth model
600 0 0 co
14 2 5 S re
700 0 le
Sca ca
le 800
core 230 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth model
600 0 0 co
14 2 5 S re
700 0 le
Sca ca
le 800
core 230 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth model
20 core
14 700 0 le S
Sca ca
le 800
S re 3 0 0 15
co 2 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth model
th
Per
ce nt i
500 leG
th
row
20 core
S le
ca le Sca
core 2 3 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth model
th
Per
400 ce nt i
le
wth
ro
20 core
S le
ca le S ca
core 2 3 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth model
Pe 6
i re =
cad e Sco
A cal
75% 75th
Per
400 nt i
ce le G
wth
ro
20 core
Sca 4 0 0 ca le
le 2 S
core 2 3 20
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth Model
• Peer-based growth calculations are only as good as the assessments
on which they are based.
• With quality assessments, peer based analyses can allow for all
students to demonstrate growth.
• However, ceiling or floor effects on a test can impede the analysis of
growth for students at the extremes.
• The goal is to be able to demonstrate for stakeholders student
growth that is exemplary as well as that which is problematic.
West Virginia Growth Model
• Growth is achievement over time
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth Model
• Peer based comparisons (norms) are helpful but limited by their
very nature.
• One needs criteria as well as norms to have a complete
understanding.
• A criterion-referenced interpretation examines the quantity against
a set of standards.
• Student performance on the WESTEST 2 within the 5 performance
levels and the new SBAC within the 4 performance levels are
familiar examples of a criterion referenced interpretation of
attainment/achievement.
West Virginia Growth Model
• Growth is achievement over time
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth Model
• Criterion-referenced growth is also referred to as growth-to-
standard.
• The standards used to examine growth are often the state’s
performance standards.
• Peer based analyses (e.g., SGP) are used in many states as part of
growth-to-standard analyses.
• Growth-to-standard analyses produce growth targets determining
whether a student’s growth is sufficient to reach/maintain state
standards.
• Absent norms, criterion-referenced growth produces results similar
to achievement indicating:
– Students that start behind tend to stay behind.
– Students that start ahead tend to stay ahead.
West Virginia Growth Model
• Most of the controversy surrounding student growth analyses centers
on their (high stakes) uses:
– Teacher evaluation
– School evaluation/accreditation
– Principal evaluation
• There is broad disagreement on whether results derived from state
summative assessments are appropriate for personnel or
institutional evaluation.
• Measures and uses often get conflated leading to push back against
the measures (annual state assessments) themselves.
• States tailor their use of SGPs to their particular state policies.
West Virginia Growth Model
• Like most states, West Virginia will transition to a new state assessment in
• West Virginia is a part of the Smarter Balanced (SBAC) consortium and will
use their test.
• The transition from the WESTEST 2 to SBAC have some impact on
the growth analyses:
– It will have minimal impact upon the norm-referenced analyses.
– It will have a larger impact upon the criterion-referenced (growth-to-
standard) analyses.
• Even with two distinct tests (WESTEST 2 & SBAC), it’s possible to
monitor the relative progress of students.
• Monitoring growth-to-standard during the transition is more difficult
for two reasons:
– The standards have not been officially established
– Data is not yet available on how students have progressed relative to
the new test and standards.
West Virginia Growth Model
• Growth is achievement over time
Source: West Virginia Department of Education/
NCIEA
West Virginia Growth Model
• New tests, performance standards, and accountability systems have
made the use of growth results for high stakes purposes even more
controversial.
• Many states are diminishing/eliminating state assessment results
from accountability calculations during the assessment transition.
• New Jersey counts student growth as 10% of overall teacher
evaluation in 1st year (versus 30% in later years).
• Idaho postponing growth to standard results across assessment
transition for school evaluation.
• Stakeholders often conflate the test itself with the use of the test.
• The challenges associated with the use of growth during the
assessment transition are more about communication than about
technical issues.
The West Virginia Growth Model:
What it Means for Your School
Timothy Butcher
Coordinator, Office of Assessment and Accountability
West Virginia Department of Education
Results Results
• Student Growth
– Approximately 182,000 tested students per year
– Approximately 174,000 students are accountable under growth model
– Growth data observed for all students in grades 4-11
– Adequate growth data obtained for all students in grades 4-10
Results Results
• Student observed growth (grades 4-11)
– Most meaningful at the student level
– State Level Results:
▪ 35% of students in the state show low growth
▪ 30% of students show typical growth
▪ 35% of students show high growth
– District and school results are more meaningful because it varies
– Next slide shows available reports
Growth Model Reports: Summary
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Observed
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Adequate
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Observed Growth
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Grade
Source: West Virginia Department of Education Internal WVEIS WOW Application
Results Results
• Student Adequate Growth (grades 4-10)
– Are students demonstrating enough growth to meet grade level
expectations in the future?
– Not be proficient now, but considered to be on track
– If proficient, are you keeping up or on track to fall behind?
Results: Math Results: Math
Not Proficient Proficient
24% are 61% are
Catching Up Keeping Up
76% are Falling 39% are Falling
Behind Behind
Source: West Virginia Department of Education
Results: RLA
Results: Reading and Language Arts
Not Proficient Proficient
31% are Catching 66% are Keeping
Up Up
69% are Falling 34% are Falling
Behind Behind
Source: West Virginia Department of Education
Growth Model Reports: Adequate Growth
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Mathematics by Grade
Source: West Virginia Department of Education Internal WVEIS WOW Application
Growth Model Reports: Reading/LA by Grade
Source: West Virginia Department of Education Internal WVEIS WOW Application
Data Data
• Review your own data to determine how many students are:
– Keeping up with grade level expectations
– Catching up to grade level expectations
– Falling Behind grade level expectations
– Staying Behind grade level expectations
Available sources of data Available Sources of Data
• Private site for ZoomWV for District Personnel and Principals
• WVEIS on the Web (WOW) Principal and Teacher Access
Why Growth? Why Growth?
• Why growth? 4 Driving Questions
1. How much growth did my student exhibit?
2. What does this growth mean?
3. How much growth does my student need to be proficient in the
future?
4. How likely is this growth?
• Student Reports – the primary benefit of the WV Growth Model
Why Growth? Why Growth?
• Why growth?
– Growth tells us the what
▪ Contextualizes student changes in performance
▪ Identifies successful progress for an individual student, regardless of
performance
– Growth does NOT tell us why
▪ Concretizes data-driven decision making
▪ Builds interest into where we should look next
The Student Growth Report
The Student Growth Report
Source: West Virginia Department of Education Sample Report 55
Why Growth? Why Growth?
• Electronic reports are available (student level) bundled at the
grade, school, and district level.
• Reports are printed by your counties.
Student Growth ReportsStudent Growth Reports
• Keep conversations simple.
– Drive questions towards the why:
▪ Why is my student exhibiting high/typical/low growth?
▪ Why is my student catching up/keeping up?
▪ Why is my student falling behind/staying behind?
– Get people to target standards analysis
– Must be ongoing through the year to assess a student’s learning
trajectory
Break-Out Session:
Growth Model
Lydotta Taylor, Ed.D..
Alliance Lead, REL Appalachia
Debriefing of Break-Out Session
Lydotta Taylor, Ed.D..
Alliance Lead, REL Appalachia
WV’s A-F Grading System
for Schools & Districts
Carla Howe, Ph.D.
Interim Executive Director of Data Governance and
Accountability
West Virginia Department of Education
Purpose of A-F
• Provides clear and concise information on how well schools are
doing
• Puts student achievement as the central focus of our education
accountability and accreditation system
• Assists the Governor, the Legislature, the WVDE, and county boards
of education in ensuring:
– that high quality educational standards and annual performance
measures and progress are met by all schools and school systems
– and that a thorough and efficient system of schools is provided.
A-F Policy
Establishes an accountability system for West Virginia Public Schools
and an approval process for school systems based on:
(1) measures of student performance
(2) a system of reviews through which school and school system
quality is examined and publically reported
A-F Components
• Proficiency
• Growth
• Graduation Rates for schools with a Grade 12
A-F Point Values A-F Point Values
Component Points Possible Total Points Possible
Proficiency
Mathematics 200
Reading 200
Observed Growth
Mathematics 100
Reading 100
Adequate Growth
Mathematics 100
Reading 100
Accelerated Growth 1000 points possible
Mathematics 100
Reading 100
Graduation Rate 1200 points possible
Four-year rate 100 for schools with Grade
Five-year rate 100 12
Source: West Virginia Department of Education 66
Proficiency
Proficiency rates for mathematics and reading are calculated with the
following criteria for each subject independently.
• Students with full academic year (FAY) status
• Students enrolled in grades 3-11
• Students with a valid assessment result
• At least 10 FAY students for each subject at the school level
To calculate the proficiency components:
• Sum the number of students (that met the criteria above) that were
proficient, and divide by the total number of students (that met the
criteria above) for each subject.
• Multiply the proficiency rate for each subject by 200 to determine
the number of points earned out of the total 200.
Observed Growth
To calculate observed growth for each subject independently, the
following criteria must be met:
• Students Enrolled in grades 4-11
• Students who are FAY in the current year
• Students with valid scale scores in the current and prior year to get
growth results
• 10 FAY students with growth calculations at the school level.
This calculation takes the sum of students with typical and high growth
by subject; then divides by the total number of students with growth
measures at a school by subject. Each rate is then multiplied by 100 to
determine the number of points earned out of the total 100.
Adequate Growth
To calculate adequate growth for each subject independently, the
following criteria must be met:
• Students Enrolled in grades 4-11
• Students who are FAY in the current year
• Students with valid scale scores in the current and prior year to get
growth results
• 10 FAY students with growth calculations at the school level.
This calculation takes the sum of students with a Yes indicator in either
the Keeping Up or Catching Up category for each subject (reading and
mathematics), then divides by the total number of students with
growth measures at a school by subject. Each rate is then multiplied by
100 to determine the number of points earned out of the total 100.
Accelerated Growth
In order to calculate Accelerated Growth of the Lowest 25% of
students, first, the LOW25 subgroup must be identified.
Identifying the LOW25 subgroup
To identify the LOW25 subgroup in a school, the following conditions
are to be satisfied independently for mathematics and reading scores:
• Students enrolled in grades 4-11 in the current year
• Students with a full academic year (FAY) indicator in the current
year
• Students that take state assessments
Low25 subgroup (cont.)
To calculate the LOW25 subgroup, the following steps will be taken for
reading and mathematics independently.
1. Count the total number of students who are FAY by grade level in
the current year.
2. Determine the number of students in each grade that equals 25% of
the accountable student population with growth results.
– In cases where the result is a decimal, round up to the next whole
number.
3. Using the students’ prior year scale scores, rank the students’
scale scores from lowest to highest by grade.
4. Select the number of students as determined from Step 2
starting with the lowest scale score.
− If a student included in the LOW25 subgroup has the same scale
score as a student(s) who just missed the 25% cutoff, include the
student(s) in the LOW25 subgroup.
LOW25 subgroup (cont.)
5. Set a LOW25 indicator for the students identified as the lowest
performers in each grade.
6. Combine students indicated as being in the LOW25 subgroup across
all grades into one LOW25 subgroup for the school.
7. If there are fewer than 10 students in the LOW25 subgroup, add an
additional student participating in the state assessment per grade
across all grades until the minimum of at least 10 FAY students can
be met.
Calculating Accelerated Growth
To calculate accelerated growth for each subject independently, the
following criteria must be met:
• A minimum of 10 FAY students in the LOW25 subgroup
Using current year growth results, calculate the accelerated growth
rate:
1. Sum the LOW25 students across grades that have High growth.
2. Divide by the total number of LOW25 students.
3. Multiple by 100 to determine the number of points earned out of the
total 100.
Graduation Rates
• The certified graduation rates will be included as part of the A-F
report card by converting the graduation rate to points.
• Graduation rates are certified during the October 1 Certified
Collection for the prior year. These rates will then carry forward and
be part of the report card for the year in which the rates are
certified.
June 2014 graduates and Summer 2014 graduates are certified in
October 2014. The rates that are certified will be part of the score
card for the 2014-15 school year.
Adding it Up!
• A school’s scores for each component will be summed as shown in
prior referenced point value table.
• Schools will be ranked from highest points earned to the lowest
points earned (elementary/middle schools will be ranked separately
from high schools).
• Based on the points earned rankings, stanines will be calculated for
elementary/middle schools, and stanines will be calculated for high
schools in Year 1.
What is a stanine?
• Stanine (STAndard NINE) is a method of scaling test scores on a
nine-point standard scale where the distribution of scores is divided
into nine intervals and where the mean lies at the center of the
fifth interval.
Assign a letter grade
• The letter grade associated with each stanine will be applied to
each school within the stanine as shown in the example below.
• The percentages in the example below show the approximate
percent of schools that will fall into each stanine.
Grade & Point Spans for future years
• Once grades have been assigned to schools, the point spans will be
established for future years.
• The school with the lowest points and the school with the highest
points in a letter grade span will create the point span.
Break-Out Session:
Accountability
Lydotta Taylor, Ed.D..
Alliance Lead, REL Appalachia
Debriefing of Break-Out Session
Lydotta Taylor, Ed.D..
Alliance Lead, REL Appalachia
Expert Panel
Dr. Damian Betebenner, Panelist
Timothy Butcher, Panelist
Dr. Carla Howe, Panelist
Dr. Lydotta Taylor, Facilitator
Wrap-Up and Closing Remarks
Stakeholder Feedback Survey
Dr. Justin Baer
Director, REL Appalachia
Dr. Lydotta Taylor
Alliance Lead, REL Appalachia
About this copy
Understanding the West Virginia Growth Model — REL West Virginia, October 1, 2015.
This is an image-based copy of the original Keynote slides; builds and animations are flattened.