Data Analysis and Visualisation Courses

Data Visualisation with Processing

Duration: 5 Days

Course Background

Processing is yet another language invented by those clever guys at MIT. One of its aims was to simplify the implementation of graphics applications by producing a language dialect and development environment that would make it much easier for creative individuals to develop powerful interactive graphics applications.

Processing brings “art” to data visualisation. Outstanding exponents of Processing are able to actually realise the reality behind the quotation “a picture is worth a thousand words”. Just as computer programmers are, typically, not the best people to design graphical user interfaces, so statisticians and programmers are not best suited to designing stunning and intuitive visualisations of data. This course aims to provide the basic skills needed by “the artists of the data visualisation world”, and to statisticians and scientist wanting to see what can be achieved using Processing, and to be able to communicate their visions to artists who, by use of Processing, can convert those visions into reality.

This course is aimed at those people who care passionately about data presentation and visualisation, and either individually, or (better) as members of a multi-disciplinary team want to master Processing - both the Java variant and also Processing.js

This course comes in three parts

The design aspects and the design process of data visualisation
The Processing and Processing.js programming aspects that convert the design into reality
More advanced topics

Communicating with databases such as MySQL, PostgreSQL and SQLite
Working with DOM in Processing.js

The objectives of the sesign aspects of the course are to provide attenedees with a comprehensive introduction to data visualisation design and ways of thinking about the visualisation design process so as to be able to efficiently rationalise the choices various choices and pathways that arise in any visualisation process,

By challenging the attendee's existing design approaches and perceptions of effective visualisation designs
By laying the foundation for adopting portable, versatile and flexible data visualisation design approaches
By providing an opportunity to practice various aspects of the visualisation creation process in a rich, interactive environment
By providing inspiration, new insights and introducing some novel approaches to data visualisation.

Course Prerequisites and Target Audience

Attendees should have some knowledge of programming such as might have been gained from e.g writing shell scripts, or basic Python programs, or JavaScript programming. Attendees should also have an appreciation for aesthetics and a feeling for how users might interact with visual presentations. This is not a “high powered programming course”, rather, its aim is to provide a common ground within which graphics artists and data analysts can communicate and experiment with various approaches to data visualisation.

Course Outline

Part 1:

What is of Visualisation

An art form
A valuable business tool
When good it is timeless
When good it has relevance and impact
When good it has clarity and compactness
When good it is memorable
When good it provides an effective opportunity to clarify and communicate

The Context of Data Visualization

Vital in the digital age
Visualization as a discovery tool
Visualization as a technique for conveying knowledge

Visualization design objectives

Striving for form and function
Justifying the selection of everything done
Creating accessibility through intuitive design
Never deceiving the receiver
Setting the Purpose and Identifying Key Factors
Clarifying the purpose of the project

The reason
The effect

Establishing intent – the purpose of the visualisation

To explain or explore or to exhibit data
Establishing intent – the visualization's tone
Pragmatic and analytical vs. Emotive and abstract

The roles involved in data visualization design

The initiator
The data scientist
The journalist
The computer scientist
The designer
The cognitive scientist
The communicator
The project manager

Editorial Focus and Learning About The Data

The importance of editorial focus
Gaining familiarity with the data
Visualisation tells a story - Using visual analysis techniques to find stories

Conceiving and Reasoning about Visualization Design Options

Data visualization design is all about choices
Visualization structure – data representation

Heuristics for choosing the moat appropriate visualization method
Consideration of the physical properties of the data
Determining the degree of accuracy in interpretation
Creating an appropriate design metaphor
Selecting a solution

Elements of Visualisation – data presentation elements

The use of color
Creating interactivity
Annotation
Arrangement

Taxonomy of Data Visualization Methods and survey of Chart types

Comparing categories

Dot plot
Bar chart (or column chart)
Floating bar (or Gantt chart)
Pixelated bar chart
Histogram
Slopegraph (or bumps chart or table chart)
Radial chart
Glyph chart
Sankey diagram
Area size chart
Small multiples (or trellis chart)
Word cloud

Assessing hierarchies and part-to-whole relationships

Pie chart
Stacked bar chart (or stacked column chart)
Square pie (or unit chart or waffle chart)
Tree map
Circle packing diagram
Bubble hierarchy
Tree hierarchy

Showing changes over time

Line chart
Sparklines
Area chart
Horizon chart
Stacked area chart
Stream graph
Candlestick chart (or box and whiskers plot, OHLC chart)
Barcode chart
Flow map

Plotting connections and relationships

Scatter plot

Bubble plot
Scatter plot matrix
Heatmap (or matrix chart)
Parallel sets (or parallel coordinates)
Radial network (or chord diagram)
Network diagram (or force-directed/node-link network)

Mapping geo-spatial data

Choropleth map
Dot plot map
Bubble plot map
Isarithmic map (or contour map or topological map)
Particle flow map
Cartogram
Dorling cartogram
Network connection map

Constructing and Evaluating Design Solutions

Selection of tools and packages
Delivery and deployment
Post-launch evaluation
Developing visualisation design skills

Part 2:

Introduction to Processing

Background to processing
Sketching with Processing
Exporting and Distributing work
Functions
Sketching and Scripting

Key visualisation and analysis activities

Acquire
Parse
Filter
Mine
Represent
Refine
Interact

Mapping - case study

Drawing a Map
Locations on a Map
Data on a Map

Time Series and visualisation - basic example case study

Acquire and Parse a data table
Cleaning the Table - Filter and Mine
Creating a Simple Plot - Represent and Refine

Labeling the Current Data Set - Refine and Interact
Drawing Axis Labels - Refine
Choose a Proper Representation - Represent and Refine
Using Rollovers to Highlight Points - Interact
Different Ways to Connect Points - Refine
Text Labels As Tabbed Panes - Interact
Interpolation Between Data Sets - Interact

Connections and Correlations

Examining Data Sources and Problem Statement
Preprocessing
Using the Preprocessed Data - Acquire, Parse, Filter, Mine
Displaying the Results - Represent
Returning to the Question - Refine
Effective Sorting e.g.Using some other attribute as a Tiebreaker - Mine
Moving to Multiple Days - Interact
Smoothing Out the Interaction - Refine
Deployment Issues - Acquire, Parse, Filter

Scatterplot Maps - Case Study

Loading the Data - Acquire and Parse
Drawing a Scatterplot of Zip Codes - Mine and Represent
Highlighting Points While Typing - Refine and Interact
Show the Currently Selected Point - Refine
Progressively Dimming and Brightening Points - Refine
Zooming In - Interact
Changing How Points Are Drawn When Zooming - Refine
Deployment Issues - Acquire and Refine

Trees, Hierarchies, and Recursion

Introduction to recursion and recursive thinking for problem solving
Using Recursion to Build a Directory Tree
Using a Queue to Load Data Asynchronously - Interact
Treemaps - the basics
Filesystem example case study

Which Files Are Using the Most Space?
Viewing Folder Contents - Interact
Improving the Treemap Display - Refine
Going through the Files - Interact

Networks and Graphs

An introduction to graphs and networks and their uses
Examples of simple and complex graphs for data visualisation
Strategies for working with Network Problems
Advanced Network Graph Examples
Mining Additional Information from Networks and Graphs

Part 3:

Acquiring Data

Finding the Data
Tools for Acquiring Data from the Internet
Locating Files for Use with Processing
Loading Text Data
Handling multiple Files and Folders
Asynchronous Image and graphics Downloads
Using openStream( ) As a Bridge to Java
Handling Byte Arrays
Advanced Web Techniques
Interacting with Databases - basics of relational databases and SQL

Parsing Data

Tools and techniques for parsing data
Knowing when to call in the experts
Overview of regular expressions and pattern matching
Text Markup Languages
Overview of Grammars and BNF Notation
Handling Compressed Data
Introduction to Vectors and Geometry
Introduction to Binary Data Formats

Integrating Processing with Java - approaches to more complex applications

Programming Modes
Additional Source Files - Tabs
Understanding the Preprocessor
API Structure
Embedding PApplets (Processing applets) into Java Applications
Using Java Code in a Processing Sketch
Using Libraries
Building applications with the Source for processing.core

Processing.js