The Big Picture
Value-Added Analysis
Guiding Wisdom
Student Growth and Value-Added: Redux
Damian W. Betebenner
National Center for the Improvement of Educational Assessment
Dover, New Hampshire
Governor’s Council on Educator Effectiveness
Denver, Colorado October 29th, 2010
Damian W. Betebenner Student Growth and Value-added: Redux
The Big Picture
Value-Added Analysis
Guiding Wisdom
The Big Picture
No curriculum
framework
Curriculum Framework
but no Assessment
Voc Ed Phys Ed
Teachers
8th Grade History Assessment, but
no growth
(10%) K-4 Reading
using DIBELS &
AP and IB Growth Indirect
Grade **
Teachers** (17%) Health
9 & 10 ELA
Visual Growth Direct and
Arts (16%) Math
3rd Grade ELA and Math 4-8 Subject
Teachers Self Contained Classes and teachers
Pre-K – 2 Middle School Subject teachers
Special Business
& Mkting
Education 4-10
7th Grade
K-12 ELL
Reading Specialists (4-8) History
Teachers (MEPA)
Teachers*
Gr. 11 & 12
STE MS & HS School STE Gr 10 & 11
Teachers US History*
Drama
Foreign Language
Special
Education * HSS Tests
Specialists Music HS Electives have been
(non 4-10)
suspended
**These
MS & HS teachers have
Computers not been
April 29, 2010 Draft
linked yet
Damian W. Betebenner Student Growth and Value-added: Redux
No curriculum
framework
Curriculum Framework
but no Assessment
Voc Ed Phys Ed
Teachers
8th Grade History Assessment, but
no growth
(10%) K-4 Reading
using DIBELS &
AP and IB Growth Indirect
Grade **
Teachers** (17%) Health
9 & 10 ELA
Visual Growth Direct and
Arts (16%) Math
3rd Grade ELA and Math 4-8 Subject
Teachers Self Contained Classes and teachers
Pre-K – 2 Middle School Subject teachers
Special Business
& Mkting
Education 4-10
7th Grade
K-12 ELL
Reading Specialists (4-8) History
Teachers (MEPA)
Teachers*
Gr. 11 & 12
STE MS & HS School STE Gr 10 & 11
Teachers US History*
Drama
Foreign Language
Special
Education * HSS Tests
Specialists Music HS Electives have been
(non 4-10)
suspended
**These
MS & HS teachers have
Computers not been
April 29, 2010 Draft
linked yet
The Big Picture
Value-Added Analysis
Guiding Wisdom
How can we evaluate ALL teachers on student growth?
Create a LOT of standardized tests?
Consider other ways to evaluate student progress:
Quantitative growth versus qualitative growth.
Damian W. Betebenner Student Growth and Value-added: Redux
The Big Picture
Contextual Value-Added
Value-Added Analysis
Corrupting Influences
Guiding Wisdom
Value-Added Analysis
http://projects.latimes.com/value-added/
A teacher’s value-added rating is based on his or her students’
progress on the California Standards Tests for English and
math. The difference between a student’s expected growth and
actual performance is the "value" a teacher added or subtracted
during the year. A school’s value-added rating is based on the
performance of all students tested there during that period.
Better phrased as the difference between a student’s
expected status and their actual status.
Whose “expectation”?
Damian W. Betebenner Student Growth and Value-added: Redux
Expectation for 2010 Student Achievement: Many Possibilities
2010 Scale Score Mathematics
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: No Child Left Behind
700 Proficient ●
2010 Scale Score Mathematics
Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: No Child Left Behind, School 1
700 Proficient ●
2010 Scale Score Mathematics
Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: No Child Left Behind, School 2
700 Proficient ●
2010 Scale Score Mathematics
400 Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: No Child Left Behind, White Students
700 Proficient ●●
2010 Scale Score Mathematics
Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: No Child Left Behind, Minority Students
700 Proficient ●
2010 Scale Score Mathematics
Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: Gain More than 20 Points
700 Proficient
2010 Scale Score Mathematics
400 Not Proficient
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: Gain More than 20 Points
2010 − 2009 Gain Score Mathematics
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: Statistical (least squares) Expectation
2010 Scale Score Mathematics
2009 Scale Score Mathematics
Expectation for 2010 Student Achievement: Statistical (least squares) Expectation
2010 Scale Score Mathematics
2009 Scale Score Mathematics
The Big Picture
Contextual Value-Added
Value-Added Analysis
Corrupting Influences
Guiding Wisdom
Contextual Value Added
Contextual Value Added (CVA) – Great Britain
CVA takes into account nine factors that are known to affect the perfor-
mance of children, but outside of the schools control. (From Wikipedia)
Gender
Special Educational Needs
Eligibility for free school meals.
First language
Whether pupils move between schools
Ethnicity
The age (i.e. the month they were born in) of different pupils within the
year group
Whether a pupil has been taken into care (e.g. Foster Care) at any
stage
The level of deprivation in the area the pupil lives (using the Income
Deprivation Affecting Children Index)
Damian W. Betebenner Student Growth and Value-added: Redux
The Big Picture
Contextual Value-Added
Value-Added Analysis
Corrupting Influences
Guiding Wisdom
Value-added analyses can be (n.b., I didn’t say should be)
conducted on other assessment data including interim data,
end-of-course assessments, and combinations thereof.
Some districts in the state are doing this
Campbell’s Law Donald T. Campbell
The more any quantitative social indicator is used for social
decision-making, the more subject it will be to corruption pres-
sures and the more apt it will be to distort and corrupt the social
processes it is intended to monitor.
Damian W. Betebenner Student Growth and Value-added: Redux
The Big Picture
Value-Added Analysis
Guiding Wisdom
Guiding Wisdom
On Questions John Tukey
It is better to have an approximate answer to the right question than a precise answer
to the wrong question.
On Statistical Models George E. P. Box
All models are wrong but some are useful.
On Understanding Aristotle
We understand best those things we see emerge from their very beginnings.
On Accountability Sherlock Holmes
They say that genius is an infinite capacity for taking pains. It’s a very bad definition,
but it does apply to detective work.
On Theories of Action Confucius
Tell me and I will forget, show me and I will remember, involve me and I will understand.
Damian W. Betebenner Student Growth and Value-added: Redux
About this copy
Student Growth and Value-Added: Redux — Governors Council on Educator Effectiveness, October 29, 2010.
This is an image-based copy of the original PDF slides; builds and animations are flattened.