ENLIGHTENING WITH DATA
Damian W Betebenner
National Center for the Improvement of Educational Assessment
Minneapolis, MN October 25th, 2016
MY BACKGROUND:
• Started at the Center for Assessment in 2007.
• Began work Summer 2007 with Colorado
Department of Education on Colorado Growth
Model.
• Worked on both analytics and reporting of
growth data.
• Have since worked with about 30 states on
issues related to student growth.
BIGGEST LESSON I’VE LEARNED
COMMUNICATING DATA MEANINGFULLY
TO STAKEHOLDERS IS THE MOST
DIFFICULT TASK ASSOCIATED WITH
TESTING/ACCOUNTABILITY
THE ANALYTICS ARE THE EASY PART
COMMUNICATING RESULTS IS THE HARD PART
IT’S EASY TO TAKE A COMPLICATED IDEA
AND MAKE IT LOOK HARD
(THAT’S BAD TEACHING)
IT’S HARD TO TAKE A COMPLICATED IDEA
AND MAKE IT LOOK SIMPLE
(THAT’S GREAT TEACHING)
IMHO (IN MY HUMBLE OPINION):
OVERALL, AS A FIELD, WE’VE DONE AN
ABYSMAL JOB COMMUNICATING
ASSESSMENT AND ACCOUNTABILITY
RESULTS, ESPECIALLY TO NON-TECHNICAL
STAKEHOLDERS.
Individual Student Reports
Advanced Reading
● Move Up
Achievement
Target
DEMO Reading
● Scale Score
Keep Up
● Target
Proficient Growth
Level Percentiles
● Very High 81st − 99th
High 61st − 80th
Part Proficient Typical 40th − 60th
Low 20th − 39th
Very Low 1st − 19th
Unsatisfactory
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7
Scale Score 449 533 579 602
Achievement Level
Achievement Target
Unsatisfactory Part Proficient Part Proficient Proficient
Proficient (620)/Advanced (661)
Achievement
Growth Percentile 69 85 74
Growth Level
Growth Target
High Very High High
Keep Up (76)/Move Up (99)
Growth
Bubble Plots
District C: 2008 CSAP Math School Results
Student Growth versus Student Achievement by Percent Free/Reduced Lunch
School Percent
Higher Achievement Higher Achievement Free/Reduced Lunch
Lower Growth Higher Growth
Percent at/above Proficient in School
Less than 20 percent
● 20 to 40 percent
● 40 to 60 percent
● 60 to 80 percent
● More than 80 percent
School Size
50 Students
100 Students
200 Students
30 500 Students
1,000 Students
10 Lower Growth Higher Growth
Lower Achievement Lower Achievement
Median of Student Growth Percentiles in School
Demonstration: 2015−2016 Mathematics
Growth & Achievement Plots
Norm & Criterion Referenced Growth & Achievement
Baseline Referenced Percentile Growth Trajectory
Scale Score
2015−2016 Achievement Percentile
SGP trajectories for student 1000372 starting from their 5th grade result.
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10
REPORTING/DATA
VISUALIZATION
IS THE KEY
Next big lesson I’ve learned
“Software will eat the world”
Marc Andreesson,
Inventor of Netscape
Web Browser
erns
nc
Co Data Visualization
ra te
pa
Se
Data Preparation
OPEN DATA:
• Many countries, states, cities, counties are
publishing data (by law) under open data
initiatives.
• Data on an unbelievable variety of governmental
activities are available.
• Increases transparency and accountability and
ultimately enhances democracy.
• Allows for improved services as deficiencies are
surfaced.
BOATING ACCESS SITES IN OREGON:
APIS POWER THE WEB:
• API: Application Programming Interface.
• “An API isn’t the same as the remote server —
rather it is the part of the server that receives
requests and sends responses.”
• APIs are the efficient way to separate concerns.
• Serve data to visualizations (both your own and
to 3rd party developers)
APIS POWER THE WEB:
• Often return data in a format called JSON.
• More flexible than csv/tabular data formats.
Dogfooding
Eat your own down food
A slang term used to reference a
scenario in which a company uses its
own product to test and promote the
product.
Dogfooding is about
creating a virtuous
cycle.
DASHBOARDS OR SUMMATIVE RATINGS
• Why not both?
• With a robust API serving your data, you (and
others) can build reports/visualizations around
that data.
LESSONS LEARNED
• Separate data preparation from data
visualization.
• Build modern web-services to serve data.
• Watch a thousand flowers bloom.
About this copy
Enlightening with Data — CCSSO SCASS, October 25, 2016.
This is an image-based copy of the original PDF slides; builds and animations are flattened.