A Primer on Design Thinking

Center for Assessment Reidy Interactive Lecture Series · September 22, 2023

A Primer on Design Thinking — Center for Assessment Reidy Interactive Lecture Series, September 22, 2023.
Published

September 22, 2023

Event
Colloquium
Slides
24
Format
Converted from Keynote

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. A Primer on Design Thinking Application to Accountability System Design Damian Betebenner Center for Assessment September 24th, 2023
  2. RILS 2021: Innovation Design
  3. Design Innovation in Assessment • In response to the pandemic, the Center pivoted to host the RILS conference virtually in 2021. • Spearheaded by Jeri Thompson (now retired) and myself, the conference centered on the pivotal concept of "Design Innovation" in educational assessment. • The aim was to capitalize on the extensive momentum in design innovation, exploring its applicability to modernizing and enhancing assessment development.
  4. First Exposure to Design Thinking • My introduction to Design Thinking originated in the early 2010s through my involvement in software development for the SchoolView data visualization platform. • Concepts such as "Agile Development," "Scrums," "Personas," and "Prototyping" became integral to our development dialogue. • At that time, I began to ponder the potential for these design methodologies to enhance the Center's work in educational assessment and accountability system design.
  5. DESIGN INNOVATION PROCESS DERSTANDIN R O TOTYPIN G SCALING UN T H E ALUATE YP S TE S T IZ EV ER CRE ST PLA OV JU PIL NE C AT E O T FI RE • Identify and engage a range • Design assessments for • Set up and run the of diverse stakeholders user needs assessment system with pilot group(s) • Identify problem • Administration with a statement(s) small sample • Collect data and evaluate applying evaluation criteria • Engage in root cause • Collect data and review analysis applying pre-established • Gather insights needed to criteria redefine or revise • Create a Theory of Action underlying the assessment • Prioritize user needs • Improve user satisfaction system design • Redesign, refine and revise • Repeat to make revisions • Clarify, analyze, and assessments based on for large-scale use synthesize learning formative evaluation and user feedback • Share the findings broadly • Repeat and review process to ensure understanding • Repeat with multiple rounds and issues of equity of iterations at multiple levels
  6. A Brief Introduction to Design Thinking
  7. What is Design Thinking? • Definition: Design Thinking is a human-centered approach to problem-solving that focuses on understanding user needs, ideation, prototyping, and iteration. • Core Phases: • Empathize • Define • Ideate • Prototype • Test • Importance: Allows for creative solutions to complex problems by integrating the needs of people, the possibilities of technology, and the requirements for program success.
  8. Core Phases: Empathize Objective: Understand your users' needs, motivations, and challenges. Methods: User interviews, observations, personas, and immersion in the user environment. Importance: Builds a foundation for designing solutions that are rooted in real-world needs and problems.
  9. Core Phases: Define Objective: Clearly articulate the user's needs and the problem to be solved. Methods: Synthesis of data, framing the problem statement, theory of action. Importance: Ensures that everyone is aligned and focused on solving the right problem.
  10. Core Phases: Ideate Objective: Generate a wide variety of potential solutions. Methods: Brainstorming sessions, mind mapping, and lateral thinking. Importance: Encourages creative problem-solving and lays the groundwork for innovative solutions.
  11. Core Phases: Prototype Objective: Create low-fidelity models of the best ideas from the ideation phase. Methods: Sketching, mock-ups, digital wireframes. Importance: Allows for rapid testing and iteration, helping to identify the most viable solutions.
  12. Core Phases: Test Objective: Validate the effectiveness of the prototype by collecting user feedback. Methods: Evaluation, user testing sessions, A/B testing, surveys. Importance: Provides valuable insights for refining the solution and ensures it meets user needs before full-scale implementation.
  13. Design Thinking in Educational Accountability
  14. Aligning Design Thinking with Educational Accountability Aligning Design Thinking with Educational Understanding Users: Parents, Teachers, Students, Administrators, and Policy Makers Defining Problems: What issues do existing accountability systems face? (E.g., Lack of Fairness, Inequality, Ineffective, Obscure and Complex) Ideation & Prototyping: Development and introduction of new indicators.
  15. Understanding Aligning DesignUsers Thinking with Educational Objective: Identify the needs, motivations, and challenges of all stakeholders involved in educational accountability. Who are the Users: Parents, Teachers, Students, Policy Makers, Educational Assessment Specialists Methods: Stakeholder interviews, surveys, and participatory design sessions, personas. Importance: Builds a robust understanding of stakeholder needs, shaping the direction of accountability system design.
  16. Defining Design Aligning Problems Thinking with Educational Objective: Clearly state the issues and challenges that existing accountability systems face. Examples of Problems: Lack of fairness, inequality, inadequate representation of student growth, etc. Methods: Data analysis, literature reviews, and expert consultations. Importance: Ensures focus on solving the most pressing problems in educational accountability, increasing the efficacy of the resulting solutions.
  17. Ideation Design Aligning & Prototyping Thinking with Educational Objective: Develop creative and effective indicators and dissemination strategies for educational accountability systems. Methods: Brainstorming sessions, prototyping workshops, and iterative testing. Importance: Facilitates the creation of new tools and metrics that better capture the nuances of educational outcomes and foster equity.
  18. Benefits of Applying Design Thinking to Accountability Systems
  19. Transformative Aligning Design Outcomes Thinking with Educational Enhanced Stakeholder Engagement: Active involvement of all parties in the design and implementation stages. Dynamic Indicators: Allows for the creation of more responsive and fairer methods of assessment. Continuous Improvement: Regular feedback loops enable ongoing refinement of accountability systems.
  20. Enhanced Stakeholder Aligning Design Engagement Thinking with Educational Objective: Involve all stakeholders in the process of design and implementation, ensuring their needs and perspectives are integrated. Methods: Workshops, focus groups, and continuous feedback loops. Importance: Creates a sense of ownership among stakeholders, leading to more successful and sustainable solutions.
  21. Dynamic Indicators Aligning Design Thinking with Educational Objective: Develop indicators that are adaptable, fair, and focused on defensible outcomes. Methods: Iterative prototyping and testing, analytics, and longitudinal studies. Importance: Allows for indicators that can evolve with educational paradigms, meeting the needs of a diverse student population.
  22. Continuous Improvement Aligning Design Thinking with Educational Objective: Establish mechanisms for the ongoing refinement and adaptation of educational accountability systems. Methods: Regular review cycles, analytics, and user feedback to iteratively improve the system. Importance: Encourages a culture of continuous improvement, ensuring the system remains effective and relevant over time.
  23. Caveats
  24. Getting AligningOur of Our Design Own Way Thinking with Educational Jargon/Terminology: Our field is a MESS with terminology • Words are used incorrectly • Words are used inconsistently We need to get our house in order if we want to MEANINGFULLY engage non-technical stakeholders in this design process.