Accountability: An Unfulfilled Promise? Damian Betebenner, Chris Domaleski, and Susan Lyons Center for Assessment Reidy Interactive Lecture Series September 27, 2018

Background • Test based accountability is not a “new” idea. • Nearly two decades ago, Linn was discussing “balanced accountability” and the use of summative assessment for high stakes accountability (2000, 2002, 2003). • Since then, states have investigated a myriad of ways of creating accountability systems through NCLB, waivers, and now ESSA. • Has accountability fulfilled its promise?

Background • The Center has been extensively involved in accountability design from the beginning. • At the Center we’ve had the honor to work with some of the most intelligent and hard working state leads to develop and implement accountability systems. • We’ve tried (and invented) almost everything under the sun with regards to summative accountability. • Has accountability fulfilled its promise?

Background • There’s some evidence to suggest small systemic improvement attributable to high stakes accountability (Ahn & Vigdor, 2014) • I’d wager that most who were enthusiastic about the possibilities of large scale assessment and accountability are underwhelmed with the results in 2018. • Has accountability fulfilled its promise? • Probably not.

Insanity Insanity Is Doing the Same Thing Over and Over Again and Expecting Different Results

Stepping Back • Seriously, how much systemic improvement should we expect from an optimally designed accountability system? • In terms of John Hattie’s Visual Learning Effect Sizes, what’s the upper bound? 0.25, 0.5, 1.0? • Expectations for what accountability can, in theory, yield leads to different paths forward.

Stepping Back • If there is an expectation that an ideal system can yield significantly superior results to current systems: • Worthwhile to improve current systems. • Changes to improve current systems to reach optimal status likely to be significant. • Small tweaks (e.g., fifth indicator, demographic adjustments) unlikely to bend the needle yield big changes.

Stepping Back • If there is an expectation that an ideal system will do about as good as current systems: • From an efficacy perspective not clear whether • Changes to improve current systems to reach optimal status likely to be significant. • Small tweaks (e.g., fifth indicator, demographic adjustments) unlikely to bend the needle yield big changes.

Conclusions • We’re optimistic: We feel there is likely significant improvements possible from an optimally designed accountability system. • Will likely require most than just tweaks to existing systems. • My colleagues have some ideas in that regard :-)

For more information: Center for Assessment www.nciea.org

About this copy

Accountability: An Unfulfilled Promise? — Center for Assessment Reidy Interactive Lecture Series, September 2018.

This is an image-based copy of the original Keynote slides; builds and animations are flattened.