Validating NAEP
Preparedness Measures
Presentation to:
THE 2008 REIDY INTERACTIVE LECTURE SERIES:
Validating Assessment and Accountability Programs
Friday, September 26, 2008
Portsmouth, NH
Presenter:
Lauress L. Wise
September 26, 2008 Validating NAEP Preparedness Measures 1
NAEP 12th Grade Readiness Measures
• The National Assessment Governing Board (NAGB) has
conducted an extensive review of 12th grade NAEP
– Launched in response to Roy Truby’s farewell address
– A key focus has been on redefining content standards in terms of
student readiness for college, work, and military service
– NAGB hopes to report results from the 2009 assessment in terms
of readiness standards
• National Commission on 12th Grade NAEP (2004)
– Concern about student motivation for 12th grade assessment
– Recommended consideration of readiness reporting
• Ad Hoc (NAGB) Committee on Planning for NAEP 12th
Grade Assessments in 2009
– Recommended focus on academic preparation (Preparedness)
– Targets: Credit-bearing college courses and job training programs
– Led to adoption of revised frameworks for math and reading
September 26, 2008 Validating NAEP Preparedness Measures 2
Panel to Recommend Validity Evidence for
NAEP Preparedness Measures
• In June 2007, NAGB commissioned a Technical
Panel on 12th Grade Preparedness Research
– 7 experts in educational policy and measurement and
industrial psychology, chaired by Michael Kirst
– Recommendations are due at NAGB’s November
2008 meeting
• Preliminary recommendations discussed at
NAGB’s August 2008 meeting
– Panel believes that preparedness reporting is feasible
with appropriate validity studies
– Research studies to start in October 2008
– Completed by May 2010, followed by standard setting
– Results to be reported by October 2010
September 26, 2008 Validating NAEP Preparedness Measures 3
Types of Validity Studies Identified by the
NAEP Panel
• Content Alignment
– To SAT/ACT, ASVAB, AccuPlacer, Compass, Work Keys
• Statistical Relationships
– Score linkages to many of these same tests
• Judgmental Standard Setting with Expert Panels
– Identify KSA’s for samples of exemplar “reference” courses and
job training programs
– Relate to NAEP math and reading score levels
• Survey of Higher Education and Job Training Institutions
– Identify requirements and cut-scores used for admissions and
placement decisions
September 26, 2008 Validating NAEP Preparedness Measures 4
Group Discussion Questions
1. Do you think it is a good idea to define
high school content standards in terms of
readiness for what lies beyond? (Why or
why not)
2. What barriers do you see to the adoption
of such measures?
3. What evidence do you consider most
important for establishing the validity of
readiness measures
September 26, 2008 Validating NAEP Preparedness Measures 5
About this copy
Validating NAEP Preparedness Measures — Center for Assessment Reidy Interactive Lecture Series, September 2008.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.