Data Visualization

CS 159.35/CS 295.S65
Reina Reyes

Ateneo de Manila University

Tell (powerful) stories with (interesting) data.

Outline

Motivation

Examples

Process

Tools

Timeline

Demo

People are visual creatures.

Illustration: Opinion piece on the Pork Barrel Issue

http://opinion.inquirer.net/61051/a-story-of-greed-and-abuse-of-power#ixzz2erDdHfuC%C2%A0

Numbers can tell the most fascinating stories, and this one has to do with greed and abuse of power: the servants of the people conspiring with each
other to steal from their masters in even greater, mind-boggling magnitudes.
Start with P720 million. That is the total amount of the first pork barrel in the form of the Mindanao Development Fund (P480 million) and the Visayas
Development Fund (P240 million). Established in 1989 by President Cory Aquino, who finally gave in to the importuning (I have personal knowledge
of this) of politicians.
Continue with P2.3 billion. This is the increase in the pork barrel in 1990, one year later. It is now called the Countrywide Development Fund (CDF),
because the Luzon politicians also wanted to partake of the pork. Note, Reader, that hardly had President Cory offered a hand of help to the
legislators than they reached for her whole arm (and seemed to have gotten it). Between one year and the next, the pork barrel more than tripled.
Fast forward to 1996, when this newspaper came out with an expos detailing how members of both the legislative and executive branches were
dividing the pork among themselves, with the people, for whom the D in CDF was intended, allegedly getting, in the form of actual cost of the
project, as little as 7 percent and at most 40 percent of the funds intended for them. The restfrom 60 to 93 percentwas apparently being divided
among the legislators (12-40 percent), the implementing agencies, the pre-bids and awards committees (plus the resident auditor), and the local
government units. Everybody had their hand in that pork barrel. As it turns out, the Deep Throat, the supplier of the information, was the sitting
congressman from Marikina, Romeo Candazo, who apparently could not stomach the situation.
At this point, it is appropriate to compare the pork-sharing system then and in the recent past, assuming that so-called whistle-blower Benhur Luys
testimony is as unimpeachable as Candazos was 17 years ago: The numbers show that a) the legislators share (although Luy was referring only to
senators, because it was the Senate blue ribbon committee holding the hearing) of the pork has gone up from 40 percent at most to a definite 50
percent, not including the share of their chiefs of staff at 5 percent; b) the rest of the gang (implementing agencies, executive agencies) get 10
percent; and c) the NGOs and/or their mastermind get 35 percent. Which means that all the foregoing got the priority with respect to assistance,
while the objects of development got absolutely zero.
Theres more to come. There of course was public outrage and public outcry then (as there is now). And the government listened then, as it seems
to be listening now. Apparently, reforms were promised, but when that was not enough, the CDF was abolished in 2000. Does that sound familiar?
But, it was replaced. By the Priority Development Assistance Fund (PDAF). The same dogoops, pigwith a different collar, supposedly tighter, in
the form of much-ballyhooed safeguards with regard to type of projects, bidding, monitoring, vetting of NGOs, procedures, financial and liquidation
requirements, etc., etc. The same safeguards enumerated in the recent Commission on Audit report.
But it would seem that, not content with PDAF, the legislators also introduced other forms of pork: The Various Infrastructure including Local Projects
(VILP, also known as hard pork, to distinguish it from the PDAFs soft pork) came into being. The VILP, by the way, was more than double the
PDAF, which means the pork barrel tripled in amount.
Still not content with that, some legislators in the Arroyo administration realized that by reducing the assumed exchange rate for converting foreign
debt service into pesos, a lot of additional funds could be made available for congressional reallocation. Thus was born the
Congressional Insertions (CI), which was more than double the VILP, and five times the PDAF.
Now go to P83 billion. That was the amount of the pork barrel in 2009, which marks the 20th year of the pork barrel, and the almost-end of the
Arroyo administration. The pork was composed of the PDAF appropriations of P10 billion; VILP, P23 billion; and CI, P50 billion.
Now compare the 1989 and 2009 numbers: 1989, P720 million; 2009, P83 billion. The pork barrel had increased by a factor of One Hundred Fifteen
(115, i.e., 83 billion is 115 times 720 million). Now bring in another statistic: the increase in the general level of prices between 1989 and 2009. Prices
in 2009 were, on the average, four times those of prices in 1989.
In other words, while the general level of prices went up fourfold, the pork barrel ballooned to 115 times its original level. Hows that for greed? Note,
Reader, that the legislators were getting a much larger share of a humongously larger pork barrel.
Actually, the CI for 2010 amounted to an even larger P64 billion, but Gloria Arroyo gave a conditional vetoshe vetoed it unless funds other than
the supposed savings from foreign exchange rates could be found. In spite of that, by the time the Aquino administration came in, at least one-third
of the CI had been spent.
It is to the credit of the present administration that it put a stop to the CI. It also folded the VILP into the PDAF for greater control (besides which
Public Works Babes Singson didnt want the VILP in his budgetso he must have known the kind of hanky-panky that could take place).
Unfortunately, this action was deliberately or unintentionally misunderstood, and P-Noy was accused of more than doubling the PDAF compared to
Arroyo. See how statistics can be misinterpreted?
The good news is that from a high of P83 billion in 2009, the pork barrel plummeted to about P25 billion in 2012 (alas, the hanky-panky continues).
And it will be zero for 2014. But as we have seen, the PDAF is not the pork barrel. Until P-Noy says that he is abolishing the pork barrel, I will not be
satisfied, and neither should anybody else. Weve been hoodwinked enough. Abolish the pork barrel. Not just the PDAF. Now.

http://www.gmanetwork.com/news/story/326487/opinion/a-visual-evolution-of-the-pork-barrel

Putting the World in Perspective

Japan

GAPMINDER WORLD 2012

Israel

Mapping the Wealth and Health of Nations

Spain
South Korea

life expectancy in years

Rich
Sick

70

Micronesia

Bangladesh

Life Expectancy of the World

North Korea
Nepal

65

Ghana

Kyrgyzstan

Laos

Philippines

55

Niger

Malawi

50

Mozambique

Dem. Rep. Congo

Nigeria

Cameroon
Zambia

Sierra Leone

1 000

Afghanistan

Lesotho

2 000

Colour by region

Angola

income per person

100

1000
millions

2011 data for all 193 UN Members and for

Hong Kong, Kosovo, Palestine, Puerto
Rico and Taiwan.

Documentation and
pdf version for print at:
gapminder.org/downloads/world-pdf
Botswana

Equatorial Guinea

Free to copy, share and

remix, but attribute to
Gapminder Foundation.
If you want to see more data visit:

www.gapminder.org

Chad

Central African Rep.

Guinea-Bissau

500

South Africa

Mali

Kuwait

3 or 10
less

Uganda

Guinea

Somalia

Burundi

Burkina Benin
Faso
Cote d'Ivoire

Rwanda

Zimbabwe

Congo, Rep.

Kenya

Togo

United Arab Emirates

Size by population

Gabon

Namibia

Mauritania
Djibouti

Liberia

Qatar

Brunei

Kazakhstan

Kiribati

Timor-Leste

South Sudan

Senegal

Tanzania

Luxembourg

Turkmenistan

Sudan

Ethiopia
Gambia

Bhutan

India

Comoros

60

Russia

Ukraine

Liechtenstein

Nauru
Tuvalu

Papua New Guinea

Haiti

Grenada Serbia

Indonesia

Fiji
Mongolia

Bolivia

Sao Tome
and Principe

Cambodia
Eritrea

Uzbekistan

Yemen

Myanmar

USA

Taiwan Denmark

Bosnia and H.

Singapore

China

Iraq

Moldova

Pakistan

Madagascar

Canada

Brazil

Solomon
Islands

Tajikistan

Norway

Sweden

Income per Person of the World

Poor

Puerto Rico

Mexico

Belize

Australia

Ireland

Uruguay Barbados
Czech Rep.
Poland
DominicaCroatia
Vietnam
Slovak Rep.
Panama
Ecuador
Montenegro
ArgentinaAntigua
Bahamas
&Barbuda
St.Lucia
Macedonia
Thailand
Bahrain
Cape
St.
Kitts
Sri Lanka
Libya
Palestine
Estonia
Venezuela
Verde
& Nevis
Tunisia
Armenia
Seychelles
Malaysia
Nicaragua
Hungary
Peru
Romania Mauritius
Egypt Dom.R.
Georgia
Jordan
Saudi
Colombia Turkey
Latvia
Honduras
Oman
Iran
Paraguay Samoa
Jamaica
Arabia
Lebanon
Algeria
Lithuania
Palau
Bulgaria
Morocco
Tonga El Salvador St.Vincent
and G.
Vanuatu
Guatemala
Suriname
Belarus
Azerbaijan
Guyana
Trinidad and Tobago
Kosovo
Maldives Albania

Syria

Iceland
San Marino
Hong Kong
Andorra Switzerland

France

Netherlands
New Zealand
Austria
Greece Cyprus UK
Germany
Chile Portugal
Finland Belgium
Malta
Slovenia

Cuba Costa Rica

Healthy

75

Italy

map layout by Paolo Fausone

80

Monaco

Swaziland

5 000

Version 11 September 2012

10 000

20 000
in US Dollars (GDP/capita, PPP$ inflation adjusted, log scale)

www.gapminder.org; also look for TED talks by Hans Rosling (highly recommended!)

50 000

Putting the Philippines in Perspective

Putting the Philippines in Perspective

Just for fun!

Just for fun!

Each pixel represents a death (in the Iraq war):

U.S. soldiers blue, Iraqi troops green,
enemies grey, and civilians orange.
Charts made by Kamel Makhloufi

Start with questions

Who is your audience?

What questions do they have?

What answers do you find for them?

What other questions does it inspire?

What conversations will result?

From Visual Analysis Best Practices: Simple Tips for Making Every Data Visualization Useful and Beautiful.
Tableau Software White Paper. Available at: http://www.tableausoftware.com/asset/10-tips-to-create-usefulbeautiful-visualizations

Data Cake

http://epicgraphic.com/data-cake/

Data Cake
Get Data

Check Data

Clean Data

Data Cake
Analyze Data

Visualize Data

Data Cake
Design Chart

Design Graphic

Write text for the reader

Data Cake
Tell the story

Get reader feedback

Tools

Out-of-the-Box Software

Programming

Python, R
HMTL, Javascript and CSS, Flash and Actionscript

Illustration

Microsoft Excel, Google Spreadsheet

Tableau Software (Public, Desktop, Server, Online)

Adobe Illustrator, Inkscape

Pen and paper

Tools

Maps

Google Maps, OpenStreetMaps (API)

Python, R
ModestMaps, Kartograph, MapBox

Big Data

Python, R
Hadoop
Databases - SQL, noSQL

Tools - for this class

Python

Matplotlib - matplotlib.org
!

d3 Javascript library

d3js.org

Timeline
Dates

Python

Topics/Tasks

Nov 11-27 ! Data visualization overview!

(3 weeks/! Setting up Python and iPython notebook!
6 sessions) Manipulating datasets with Python numpy and pandas!
Plotting charts with Python matplotlib
Dec 2-18 ! Hands-on exercises: Python!
(3 weeks/! Critique of submitted work
6 sessions)
Christmas break (Dec 21-Jan 4)

d3

Jan 6-29 ! Setting up d3!

(4 weeks/! Plotting charts with d3!
8 sessions) Interactive visualizations with d3
Feb 3-26 ! Hands-on exercises: d3!
(4 weeks/! Critique of submitted work
7 sessions)
Mar 3-19 ! Group project (proposal, implementation, presentation, critique)!
(3 weeks/! *Final exam days for Seniors (Mar 4-6)
6 sessions)
Mar 23-27

Final exam week for college and graduate students

Demo!

Homework

Show and Tell - your favorite data visualization

Bring your laptop - setup work environment

(Python, etc.)

Setting up

Enthought Canopy (Python 2.7 distribution + core

packages): www.enthought.com

-> Products -> Get Canopy

-> Canopy Express - Free Download

Mac OS/Windows/Linux - 64-bit/32-bit

Your first chart

Import libraries:!

from numpy import *

from matplotlib import pyplot as plt

Define variables:!

xvar = array([1,2,3,4,5])

yvar = xvar*2

Make scatter plot of x vs. y!

plt.clf()

plt.plot(xvar,yvar,ko)

Your first chart

Customize your plot!

plt.xlim((0,5))

plt.ylim((0,5))

Label your plot!

plt.xlabel(variable X)

plt.ylabel(variable Y)

plt.title(My first Python plot)

Save to file!

plt.savefig(my_first_chart.png)