Accountability: An Unfulfilled Promise?

Center for Assessment Reidy Interactive Lecture Series · September 2018

Accountability: An Unfulfilled Promise? — Center for Assessment Reidy Interactive Lecture Series, September 2018.
Published

September 25, 2018

Event
Colloquium
Slides
10
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Accountability: An Unfulfilled Promise? Damian Betebenner, Chris Domaleski, and Susan Lyons Center for Assessment Reidy Interactive Lecture Series September 27, 2018
  2. Background • Test based accountability is not a “new” idea. • Nearly two decades ago, Linn was discussing “balanced accountability” and the use of summative assessment for high stakes accountability (2000, 2002, 2003). • Since then, states have investigated a myriad of ways of creating accountability systems through NCLB, waivers, and now ESSA. • Has accountability fulfilled its promise?
  3. Background • The Center has been extensively involved in accountability design from the beginning. • At the Center we’ve had the honor to work with some of the most intelligent and hard working state leads to develop and implement accountability systems. • We’ve tried (and invented) almost everything under the sun with regards to summative accountability. • Has accountability fulfilled its promise?
  4. Background • There’s some evidence to suggest small systemic improvement attributable to high stakes accountability (Ahn & Vigdor, 2014) • I’d wager that most who were enthusiastic about the possibilities of large scale assessment and accountability are underwhelmed with the results in 2018. • Has accountability fulfilled its promise? • Probably not.
  5. Insanity Insanity Is Doing the Same Thing Over and Over Again and Expecting Different Results
  6. Stepping Back • Seriously, how much systemic improvement should we expect from an optimally designed accountability system? • In terms of John Hattie’s Visual Learning Effect Sizes, what’s the upper bound? 0.25, 0.5, 1.0? • Expectations for what accountability can, in theory, yield leads to different paths forward.
  7. Stepping Back • If there is an expectation that an ideal system can yield significantly superior results to current systems: • Worthwhile to improve current systems. • Changes to improve current systems to reach optimal status likely to be significant. • Small tweaks (e.g., fifth indicator, demographic adjustments) unlikely to bend the needle yield big changes.
  8. Stepping Back • If there is an expectation that an ideal system will do about as good as current systems: • From an efficacy perspective not clear whether • Changes to improve current systems to reach optimal status likely to be significant. • Small tweaks (e.g., fifth indicator, demographic adjustments) unlikely to bend the needle yield big changes.
  9. Conclusions • We’re optimistic: We feel there is likely significant improvements possible from an optimally designed accountability system. • Will likely require most than just tweaks to existing systems. • My colleagues have some ideas in that regard :-)
  10. For more information: Center for Assessment www.nciea.org