Data Visualization using R

Northeastern Educational Research Association · October 2016

Data Visualization using R — Northeastern Educational Research Association, October 2016.
Published

October 27, 2016

Event
Conference
Slides
22
Format
Converted from Beamer

This deck is an image-based copy of the original slides. Builds and animations are flattened; the slide text is reproduced below for search and screen readers.

Slide text
  1. Overview Data & Data Visualization R Resources References Data Visualization using R Damian W. Betebenner National Center for the Improvement of Educational Assessment Dover, NH NERA Training Session Trumbull, Connecticut October 27th, 2016 Damian W. Betebenner Data Visualization using R
  2. Overview Data & Data Visualization About the R Software R Resources References Purpose of today’s session To learn about R and its capabilities for visualizing data. To provide participants with a comprehensive list of resources for producing their own data visualizations. To demonstrate the power of programatic drawing over WYSIWYG drawing. To inspire participants to develop their own customized visualizations and push the envelope for what is possible Damian W. Betebenner Data Visualization using R
  3. Overview Data & Data Visualization About the R Software R Resources References About Me Dr. Damian Betebenner, PhD Senior Associate at the Center for Assessment (NCIEA). Developed student growth percentiles and percentile growth trajectories to help states and educational associations employ student growth in decision making [Betebenner, 2008, Betebenner, 2009]. In the process of refining and sharing these techniques with other states including Colorado, Massachusetts, Arizona, Indiana, and 15 other states in various stages of investigation/adoption. Interested in the rise of the data sciences/scientist: Data analysis and data visualization and their use within education. Began using R in 1998. Damian W. Betebenner Data Visualization using R
  4. Overview Data & Data Visualization About the R Software R Resources References About the R Software R is an GNU open source, free, statistical software environment (package/language) that is available for source compilation or in pre-compiled binary form for numerous operating systems. R can be downloaded from CRAN (Comprehensive R Archive Network) http://cran.r-project.org/ The R language has become a de facto standard among statisticians for the development of statistical software. The philosophy behind R/S (John Chambers): “To Turn Ideas into Software Quickly and Faithfully” [Chambers, 2000, p. v] Damian W. Betebenner Data Visualization using R
  5. Overview Data & Data Visualization About the R Software R Resources References About the R Software Daryl Pregibon, Google R is really important to the point that it’s hard to overvalue it. R had a recent New York Times article written about it http://www.nytimes.com/2009/01/07/technology/ business-computing/07program.html. A particular strength of R is its data visualization capabilities The greatest strength of R (in my humble opinion) is the IMMENSE amount of code available online to learn from. Learn from the masters. This training session will introduce users to data visualization using R as well as the resources available to continue their explorations. Damian W. Betebenner Data Visualization using R
  6. Overview Data & Data Visualization Data R Resources Data Visualization References About Data Rutherford D. Roger We are drowning in information [data] and starving for knowledge [in- formation]. The Economist, February 27th, 2010, devoted a special issue to “The Data Deluge”. In 2008, 1,200 exabytes (1.2 ZB) of digital data was created (n.b., giga, tera, peta, exa, zetta, yotta). Compound annual increase of data at 60%. Petabyte level computing is reaching commodity levels with Amazon EC2/S3 offering specials for petabyte scale projects. Damian W. Betebenner Data Visualization using R
  7. Overview Data & Data Visualization Data R Resources Data Visualization References About Data Hal Varian, Chief Economist, Google A new kind of professional has emerged, the data scientist, who combines the skills of a software programmer, statistician, and story- teller/artist to extract nuggets of gold hidden under mountains of data. The job of the data scientist will become the sexiest around. Data are widely available; what is scarce is the ability to extract wisdom from them. The Data doesn’t speak for itself. The data can tell a thousand stories. Transforming data into information and ultimately into knowledge requires a broad range of expertise (i.e., the emerging data scientist). Goal: The right data to the right people at the right time in the right format! To reach this goal subject matter experts MUST work more closely with IT specialists, ensuring the right stories are told. Damian W. Betebenner Data Visualization using R
  8. Overview Data & Data Visualization Data R Resources Data Visualization References About Data Visualization Data visualization leverages the immense capacity of the human eye to extract patterns from visual stimuli. Communicating with data is a form of storytelling. Visualization allows the storyteller to communicate more effectively and engage the user in more complicated stories. Color, pattern, and symbols can be combined to communicate highly complicated stories that, without pictures, are difficult even for experts to follow. As such, data visualization, as communication, is a form of teaching. Damian W. Betebenner Data Visualization using R
  9. Overview Data & Data Visualization Data R Resources Data Visualization References About Programmatic Drawing Byron Ellis via Twitter [R] is for making new things. Point and click is for redoing old things. Almost all graphical programs (especially those incorporated in statistical analysis packages) rely upon WYSIWYG interface. WYSIWYG pros: Easy to learn. WYSIWYG cons: Limited ability to customize. Because of constrained options, WYSIWYG interfaces both dictate and limit the stories you can tell with your data. Programming your drawings (i.e., programatic drawing) allows for unlimited possibilities, constrained only by the creator’s imagination. Damian W. Betebenner Data Visualization using R
  10. DEMONSTRATION: 2015−2016 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2015−2016 Achievement Percentile Scale Score SGP trajectories for a 3rd grade student starting from the Level 1/Level 2 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10
  11. DEMONSTRATION: 2015−2016 Mathematics Norm & Criterion Referenced Growth & Achievement Percentile Growth Trajectory 2015−2016 Achievement Percentile Scale Score SGP trajectories for a 3rd grade student starting from the Level 1/Level 2 cut. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10
  12. Observed Percent Proficient = 10 % Observed Percent Proficient = 20 % Observed Percent Proficient = 30 % Observed Percent Proficient = 40 % Observed Percent Proficient = 50 % Observed Percent Proficient = 60 % Observed Percent Proficient = 70 % Observed Percent Proficient = 80 % Observed Percent Proficient = 90 % Bias in Observed Percent Proficient (Observed − True) Observed under−estimates true Observed over−estimates true
  13. Advanced Reading Move Up Achievement ● Target DEMO Reading ● Scale Score Keep Up Catch Up ● Target Proficient Target ● ● Growth Level Percentiles ● Very High 81st − 99th High 61st − 80th Part Proficient Typical 40th − 60th Low 20th − 39th Very Low 1st − 19th Unsatisfactory Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Scale Score 449 533 579 602 Achievement Level Achievement Target Unsatisfactory Part Proficient Part Proficient Proficient Proficient (592) Proficient (620)/Advanced (661) Achievement Growth Percentile 69 85 74 Growth Level Growth Target High Very High High Catch Up (60) Keep Up (76)/Move Up (99) Growth
  14. Billy Gonzales North End Middle School How to interpret this student growth & achievement report Mathematics Advanced Achievement ● Catch Target Up DEMO ● DEMO Mathematics ● Catch Up ●● Target ●● Scale Score Scale Score ● ● Growth Proficient ● Level Percentiles DEMO Very High 81st − 99th Achievement High 61st − 80th Part Proficient Levels Typical 40th − 60th Low 20th − 39th Unsatisfactory Very Low 1st − 19th Student Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Growth 2012−2013 2013−2014 2014−2015 2015−2016 2016−2017 Percentile Scale Score 438 447 489 495 Achievement Level Achievement Target Proficient Part Proficient Part Proficient Part Proficient Proficient (514) Proficient (547) Achievement Growth Percentile 18 64 44 Catch Up/Keep Up Growth Level Very Low High Typical Growth Move Up/Stay Up Growth Target Catch Up (70) Catch Up (82) Targets Suggested Uses Advanced Reading Move Up Achievement ● Target DEMO Reading ● Scale Score ● Review past growth to assess ● Keep Up student academic progress toward ● Catch Up ●● Target Proficient Target ●● Growth DEMO achievement goals. ● Level Percentiles ● Develop remediation or enrich− ● Very High 81st − 99th ment plans based on rate of High 61st − 80th growth needed to reach higher DEMO achievement levels. Part Proficient Typical 40th − 60th Low 20th − 39th ● Identify the rate of progress ● Very Low 1st − 19th needed in order to reach or Unsatisfactory maintain proficient status Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 on the DEMO next year. 2012−2013 2013−2014 2014−2015 2015−2016 2016−2017 Scale Score 449 533 579 602 Achievement Level Achievement Target Unsatisfactory Part Proficient Part Proficient Proficient Proficient (592) Proficient (620)/Advanced (661) Achievement Growth Percentile 69 85 74 Growth Level Growth Target High Very High High Catch Up (60) Keep Up (76)/Move Up (99) Growth For more information please visit the Student Growth Percentiles Package (SGP Package) at www.sgp.io or contact dbetebenner@nciea.org Cooperatively developed by the SGP Package & the Center for Assessment, Inc. Distributed by the SGP Package.
  15. District C: 2008 CSAP Math School Results Student Growth versus Student Achievement by Percent Free/Reduced Lunch School Percent Higher Achievement Higher Achievement Free/Reduced Lunch Lower Growth Higher Growth Percent at/above Proficient in School Less than 20 percent ● 20 to 40 percent ● 40 to 60 percent ● 60 to 80 percent ● More than 80 percent School Size 50 Students 100 Students 200 Students 30 500 Students 1,000 Students 10 Lower Growth Higher Growth Lower Achievement Lower Achievement Median of Student Growth Percentiles in School
  16. 2009 Statewide Middle School Performance Growth and Achievement MCAS Math by School Poverty 100 Higher Achievement Higher Achievement School Size 90 Lower Growth Higher Growth 50 Students 100 Students X XX X XX 250 Students 80 X XXX XX XX X XX X X X XXX X XX X X 500 Students X X XXXX 2009 Percent at/above Proficient X XX X X X XX XX XXX X X XX 70 X XX XX XXX XX X X X X XX X X 1000 Students XX X X XXX XX X X XXXX XXX X X X XX XXX XXX X XX X X X XX X X X 60 X X XX X X X XXXXX X X X X X XX X X XX X X XXX X XXX X XX X X X X X X XX X XX X X School Percent X X X X X XXX X X X X X XX XX XXXX X X Free/Reduced Lunch 50 XX X XXXX XXX XXXXX X XXX X X X X X X XX XXXXX X XX X XXX XXX X X XXX X X XX XX ● Less than 20 percent 40 XXXXXX XXXX X XX X X XX XX X XX X XX XX X XXX X X X ● 20 to 40 percent XXX X X 30 XXX XX X X X ● 40 to 60 percent X XX X X X X X XXXX XX X XX ● 60 to 80 percent X X X XX ● More than 80 percent X XX X X XX X Lower Growth X X X X XX Higher Growth 0 Lower Achievement Lower Achievement X 2009 Median Student Growth Percentile
  17. 2009 Statewide Middle School Performance Growth and Achievement MCAS Math by School Poverty 100 Higher Achievement Higher Achievement School Size 90 Lower Growth Higher Growth 50 Students 100 Students 250 Students 500 Students 2009 Percent at/above Proficient 70 1000 Students School Percent Free/Reduced Lunch X XX ● Less than 20 percent X ● 20 to 40 percent XX X X ● 40 to 60 percent X XXX X ● 60 to 80 percent X XX X X X X X ● More than 80 percent XX X XX Lower Growth X X X X XX Higher Growth 0 Lower Achievement Lower Achievement 2009 Median Student Growth Percentile
  18. Overview Data & Data Visualization Online/Free R Resources Books References So Many Amazing Resources: Free Numerous help manuals installed when R is installed (Under the Help dropdown menu in Windows) One can download R and numerous resources at the CRAN (Comprehensive R Archive Network) http://cran.r-project.org/. Click on the “Contributed” hyperlink at CRAN to access dozens of free resources in many languages. Most resources provide many nuggets of wisdom that proves to be useful. Good places to start include: John Maindonald’s Using R for Data Analysis and Graphics–Introduction, Examples and Commentary John Verzani’s Simple R R has a very active listserve for R (R-help) that is easy to search using Google and keywords. Damian W. Betebenner Data Visualization using R
  19. Overview Data & Data Visualization Online/Free R Resources Books References So Many Amazing Resources: Free The R Graph Gallery. Great examples with source code: http://addictedtor.free.fr/graphiques/. Spatial Data and R. Great examples with source code http://r-spatial.sourceforge.net/. A four part online video tutorial for using ggplot http://blog.revolution-computing.com/2010/03/ video-hadley-wickham-gives-a-short-course-on-graphics-with-r html Microsoft R Open (formerly Revolution R): https://mran.microsoft.com/open/ Damian W. Betebenner Data Visualization using R
  20. Overview Data & Data Visualization Online/Free R Resources Books References So Many Amazing Resources: Published Books The Grid Graphics Package (and traditional graphics overview) [Murrell, 2006] Lattice Graphics Package: Depyan Sarkar [Sarkar, 2008]. ggplot Graphics Package: Hadley Wickham [Wickham, 2009]. Spatial Data Analysis and Visualization [Bivand et al., 2008]. Damian W. Betebenner Data Visualization using R
  21. Overview Data & Data Visualization Online/Free R Resources Books References Inspiration “A Thing of Beauty is a Joy Forever” John Keats Damian W. Betebenner Data Visualization using R
  22. Overview Data & Data Visualization R Resources References References Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L. A., editors, The Future of Test-Based Educational Accountability, pages 155–170. Taylor & Francis, New York. Betebenner, D. W. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4):42–51. Bivand, R. S., Pebesma, E. J., and Gòmez-Rubio, V. (2008). Applied Spatial Data Analysis with R. Springer, New York. Chambers, J. M. (2000). Programming with Data. Springer, New York. Murrell, P. (2006). R Graphics. Springer, New York. Sarkar, D. (2008). Lattice: Multivariate Data Visualization with R. Springer, New York. Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis. Springer, New York. Damian W. Betebenner Data Visualization using R