Assessment Transition and
Implications for Accountability
Systems
CCSSO Growth Transition Meeting
Orlando, FL
July 11, 2014
Domaleski and Hall - Accountability Transition Planning 1
Background
∗ The purpose of the paper is to highlight the
accountability implications related to a shift to
Common Core aligned assessments.
∗ We identify 5 key questions to be considered in
defining an accountability transition plan, and
broadly describe approaches for addressing core
issues.
• Potentially something that state accountability
leaders can use to guide conversations with
chiefs, board members and/or other policy
makers.
Domaleski and Hall - Accountability Transition Planning 2
Five Central Questions to Address in
an Accountability Transition Plan
∗ Expected Impact: What impact are the new assessments
expected to have on school accountability outcomes?
∗ Design Decisions: What inputs should be included in the
accountability system and what outcomes result from their
inclusion?
∗ Performance standards: How should the state establish
criteria for ‘good enough’ performance given defined inputs
and outputs?
∗ Results: How should accountability outcomes be reported?
∗ Consequences: What supports and inventions should
correspond to system outcomes?
Domaleski and Hall - Accountability Transition Planning 3
Expected Impact
∗ Estimating expected impact of shifting to common
core assessments makes it easier to think about how
and/or why an existing accountability system may
need to be revised.
∗ For example, an expected impact to the
accountability system may be that the percentage of
schools receiving unfavorable ratings will
dramatically increase.
∗ Even in the absence of information about
performance standards for the new tests, existing
external indicators such as NAEP or ACT may be
useful to roughly predict impact.
Domaleski and Hall - Accountability Transition Planning 4
Design Decisions
∗ States should review defined inputs and outputs to make
sure they are still relevant. The accountability transition
process offers a good opportunity to make design decisions
that better reflect the state’s goals related to school
improvement .
∗ Input examples:
∗ Rich performance tasks
∗ Assessments in additional content areas (e.g. science)
∗ Measures of ‘trans-academic’ competencies (e.g.
communication, persistence)
∗ Output examples:
∗ Indicators weights
∗ Number and type of classification categories
Domaleski and Hall - Accountability Transition Planning 5
Performance Standards
∗ Just as cut-scores on an assessment change when a test is revised,
when the components underlying an accountability system
change, the standards which define adequate or expected
performance within the context of that system must be reviewed.
∗ Accountability standards must be established with an
understanding of the inferences and implications associated with
assignment to a given level (e.g. what does a letter grade of ‘B’
mean?)
∗ Accountability standards must be relevant, defensible and
reasonable given the expected impact of the new assessments on
accountability inputs and the state’s overall accountability goals .
Domaleski and Hall - Accountability Transition Planning 6
Results
∗ Is comparability of results desired? Three possible
approaches
∗ Maintain current outcomes with old standards (e.g.
‘equated’ outcomes). Desire is to produce same
distribution of results and report them in the same
way.
∗ Maintain current outcomes with new standards.
Desire is to produce new distribution of results but
report in the same way.
∗ Establish new outcomes and new standards. A new
‘look and feel’ may avoid confusion with the old
system and old set of expectations.
Domaleski and Hall - Accountability Transition Planning 7
Consequences
∗ A view that consequences and supports can be ‘carried-
over’ from the legacy system may fall short of promoting
the types of outcomes that the system is designed to
reinforce.
∗ States should consider if the existing set of consequences
should be revised and/or assigned in the same manner.
∗ For example, if the most substantial consequences are
provided to schools earning a ‘D’ or ‘F’ and the number of
schools receiving these classifications dramatically increase
in the new system, is appropriate or even feasible to apply
the same consequences?
Domaleski and Hall - Accountability Transition Planning 8
Evaluation
∗ Important to evaluate the system prior to and
following release
∗ Some important components include:
∗ Are the measures relatively stable across years?
∗ Are the results associated with variables not
generally related to effectiveness (e.g. school
size)?
∗ Are the classifications credible? (e.g. associated
with other known indicators of quality?)
∗ Are negative consequences mitigated?
Domaleski and Hall - Accountability Transition Planning 9
About this copy
Assessment Transition and Implications for Accountability Systems — CCSSO, July 2014.
This is an image-based copy of the original PowerPoint slides; builds and animations are flattened.