Introduction To Open Data For Journalists
Uploaded by
Open Data InstituteIntroduction To Open Data For Journalists
Uploaded by
Open Data InstituteIntroduction to open data for journalists
London, September 2013
Before we start
Wi-Fi = ODINET
Password: OpenData
Please download the latest Chrome at: http://tinyurl.com/install-google-chrome-now
Please set up a Google account if you don't have one already at: https://accounts.google.com/SignUp
Introductions
Ulrich Atz
Statistician, Open Data Institute
@statshero
Kathryn Corrick
Head of training, Open Data Institute
@kcorrick
Introductions
Your name
Where youve come from
Role
Your aims for the day
Agenda - Today
What is data?
Using data in journalism
Finding reliable data sources
Is there really a story in this data?
Using your data, including law and licensing
Cleaning and visualising your data
Common data analysis mistakes
Presenting your story
Special invited guest
A note on the exercises and additional links
Exercises - http://bit.ly/odidjexercises Other materials, further reading and links http://bit.ly/odidjlinks The slides will be made available online after the course
WHAT IS DATA?
Discussion
In your groups discuss what is data for you?
http://theodi.github.io/data-denitions/
Open data is data that can be freely used, reused and redistributed by anyone subject only, at most, to the requirement to attribute and share alike.
Opendenition.org
USING DATA IN JOURNALISM
http://www.theguardian.com/news/datablog/ 2012/may/24/data-journalism-punk
http://www.ft.com/cms/s/0/4b1a2f64-2048-11e3-9a9a-00144feab7de.html
http://numeroteca.org/2011/12/05/surface-newspapers-front-pagesvs-twitter-nov30th-occupy-ows/
http://road.cc/content/news/93687-bikes-faster-public-transport-most-londonjourneys-under-8-miles
http://wheredoesmymoneygo.org/
http://openspending.org/
http://behindthewire.theglobalmail.org/
Case Study
http://smtm.labs.theodi.org/
An approach to using data in journalism (Data Percolation)
Source Prepare Analyse
See also: the Data Journalism Handbook
An approach (also see: Data Journalism Handbook) The process of Data Percolation
Source
Prepare
Analyse
Eventually, write the report (=simplify)
We recommend you offer someone a coffee and nd a second pair of eyes for your work.
How should I budget my time?
How should I budget my time?
Time for a coffee
1.1 - FINDING RELIABLE DATA SOURCES
Discussion
What makes a trusted (data) source?
Plenty of data sources exist
Exercise 1.1: explore data portals see your handouts
Exercise 1.1b: A basic test with 3 questions see handouts
1. Compared to what? 2. Since when? 3. Says who?
From the Statistical literacy guide How to spot spin and inappropriate use of statistics http://www.getstats.org.uk/wp-content/uploads/2012/02/How-to-spot-error.pdf
How to stay up to date with government data releases
Ofce for National Statistics release calendar Parliamentary releases mailing list Planning alerts mailing list Press releases RSS feeds Twitter See your links hand out for further details
Now that Google Reader has gone? RSS feed readers
Most recommended Feedly http://cloud.feedly.com/#welcome The Old Reader - http://theoldreader.com/ Digg Reader - http://digg.com/login?next=%2Freader
1.2 - CONFIRM THAT THE DATA IS REVELANT IS THERE A STORY?
Questions you may want to ask yourself
Is the dataset complete? Does it include crucial variables? Time and geography If a statistic is the answer, what was the question?
1.3 USING YOUR DATA UNDERSTANDING LAW & LICENSING
Law and licensing
Please note, I am not a lawyer and this should not be treated as legal advice.
Key laws affecting data journalism
Intellectual Property - copyright and database rights Computer Misuse Data Protection Freedom of Information Act
What are intellectual property rights?
Rights which are given which allow ownership of creations Patents Trade marks Design rights Copyright Database rights Many creations are a bundle of rights protected by more than one or all of the above
Copyright Designs & Patents Act 1988
Original works - e.g. content, graphics, text, music Gives exclusive rights to the author of the work allowing the author to control the copying and exploitation of it Arises automatically Fair dealing - criticism or review, reporting current events, noncommercial research, educational use Beware public domain assumption and myth
Database denition
A collection of independent works, data or other materials which are arranged in a systematic or methodical way and are individually accessible by electronic or other means
Databases
Copyright
Creative effort and substantial investment in the selection and presentation
Individual components of the database
Database rights
Substantial investment in obtaining, verifying and presenting the database
Rule of thumb
Do you have rights or permission to publish?
Do you have rights to use the information/data?
Is the data derived from other sources?
(see licensing)
Computer Misuse Act
Offences Unauthorised access to computer material Unauthorised access with intent to commit or facilitate further offences Unauthorised modication of computer material Penalties 2 10 years imprisonment Fines
Rule of thumb
Leaks and whistleblowing get your editor and the legal team in
Data Protection
Personal Data
Data Protection Act 1998
Data relating to a living identiable person must be processed fairly and lawfully
Processing that is not immediately apparent to users e.g. cookies (new laws and guidance) damages available to data subjects
Rule of thumb
Does this data contain personal identiable data?
Could this data be used combined with another data set to create personal identiable data?
Anonymisation is hard.
See
http://www.scribd.com/doc/128356210/Business-considerationsfor-privacy-and-open-data-how-not-to-get-caught-out http://www.scribd.com/doc/125638490/Getting-to-grips-with-theNational-Pupil-Database-personal-data-in-an-open-data-world
Licences: what to look for
Licenses identify the scope and limited of how intellectual property can be used Commonly used in the UK: All rights reserved Royalty free license Paid-for license Open Government Licence Creative Commons Licence
http://www.nationalarchives.gov.uk/doc/open-government-licence/version/2/
Rule of thumb
If you are uncertain about what rights you may have over a piece of content or dataset or how you can use it
Contact the owner. Ask.
And what about crowd-sourced data?
http://paidcontent.org/2013/05/24/crowdsourcing-thenews-do-we-need-a-public-license-for-citizenjournalism/
Freedom of Information Act 2000
Provides public access to recorded information held by public authorities The Act does not necessarily cover every organisation that receives public money Recorded information includes printed documents, computer les, letters, emails, photographs, and sound or video recordings
FOIA tips
Sign up to 'What Do They Know?
https://www.whatdotheyknow.com/
Always check commercial condentiality. See Information Commissioner Ofce advice:
http://www.ico.org.uk/~/media/documents/library/Environmental_info_reg/ Practical_application/ eir_condentiality_of_commercial_or_industrial_information.ashx
2.1 CLEANING YOUR DATA
Introducing Open Rene
https://code.google.com/p/google-rene/
Exercise 2: Open Rene
We will use Land Registrys Price Paid Data (PPD). Use the latest one from August 2013. Link in the exercise pack.
Merging data sets is the DANGER ZONE
Most mistakes happen when combining data and the big problem is you might not notice! Be extra careful or seek help
3.1 VISUALISE AND EXPLORE YOUR DATA
Exercise 3.1: Google spreadsheets
See exercise pack for two suggested data sets and some tasks you can try.
3.2 ANALYSING YOUR DATA COMMON MISTAKES
Other common mistakes.
1. 2. 3. 4. 5. The difference between percentages & percentage points Average, Mean, Median Visualisations Correlation versus causation
Percentages
Know the difference between a percentage and a percentage point.
VAT increased from 17.5% to 20% on January 2011.
This is a rise of 2.5 percentage points not a rise of 2.5%.
How much would a rise in 2.5% actually be?
From: Stories and Statistics by Frank Swain
Averages
Where is the mode?
Problems with maps
Map projections
Mercator projection
Kavrayskiy VII
XKCD
Creative Commons Attribution Non-Commercial http://xkcd.com/552/
3.3 DO YOUR RESULTS PASS A SENSECHECK?
Exercise 3.3: Sense-checking
The best way to do this is to get a second pair of eyes to help you. List two ways how you would double-check an analysis you did previously.
PRESENTING YOUR STORY
Three tips for using statistical language
1. 2. Use substantial instead of signicant (except if you mean it) Do not abbreviate or alter statements if you are unsure E.g. change in percentage points with change in percentage. Be precise with your denitions. DO: 200,000 economic migrants came to the UK from Eastern Europe last year. DONT: 200,000 workers came to the UK from Eastern Europe last year.
3.
Tips for data visualisations
1. Show the data integrity density Emphasise the data data-ink ratio avoid chart-junk Inform and engage context audience
Ulrichs interpretation of Edward Tuftes principles
2.
3.
An example from the Scottish Government
http://www.scotland.gov.uk/Topics/Statistics/16002/DataTrendsInternet
Improved version
Inform and engage
Finally make sure you include
Data sources and links to your analysis How they can be used and reused by others (licences)
Final exercise: assess the following article
http://globalnews.ca/news/622513/opendata-alberta-oil-spills-1975-2013/
Special guest
Nick Scott
Import.io
CONCLUSIONS AND TIME FOR QUESTIONS
Our top tip: BBC Radio 4 / Podcast More or Less
http://www.bbc.co.uk/podcasts/series/moreorless
Thank you!
Further reading and links http://bit.ly/odidjlinks
The slides will be made available online after the course
