Reading in data locally and from the web

 Reading data is the gateway for any data analysis.

 Data can be read from local device or from web.
 In R, “Reading” or “loading” is the process of converting data (stored as plain text, a
database, HTML, etc.) into an object (e.g., a data frame)
 There are many ways to store data as well as many ways to read them.
 Different functions are available in R to import data from various file formats.
 While loading a data set into R, we need to tell R where those files live. The file could live
on your computer (local) or somewhere on the internet (remote).
 The place where the file lives on your computer is called the “path.”
 There are two kinds of paths: relative paths and absolute paths.
 A relative path is where the file is with respect to our current computer.
 An absolute path is where the file is in respect to the computer’s file system.
 As per the figure,
o We are working in a file named worksheet_02.ipynb .
o If we want to read the .csv file named happiness_report.csv into R, we could do this
using either a relative or an absolute path.

Reading happiness_report.csv using a relative path

happy_data <- read_csv("data/happiness_report.csv")

Reading happiness_report.csv using an absolute pat:

happy_data <- read_csv("/home/dsci-100/worksheet_02/data/happiness_report.csv")

 In case of remote files, a Uniform Resource Locator (URL) (web address) indicates the
location of a file/resource.
Reading tabular data from a plain text file into R
 read_csv() to read in comma-separated files (csv file)
data <- read_csv("data/xyz.csv")

Data filename is “xyz.csv” stored under “data” folder.

 read_tsv to read in tab-separated files

data <- read_tsv("data/xyz.tsv")

Reading tabular data directly from a URL

 read_csv( ), read_tsv( ), read_delim( ) functions are used to read in data directly from
a Uniform Resource Locator (URL) that contains tabular data.
url <- "https://xxx.com/data/xyz.csv"
data <- read_csv(url)

Reading tabular data from a Microsoft Excel file

data <- read_excel("data/xyz.xlsx")

Reading data from a database

 Relational database is a common form of data storage for large data sets or multiple
users working on a project.
 There are many relational database management systems, such as SQLite, MySQL,
PostgreSQL, Oracle and many more.
 Reading data from a SQLite database
o SQLite database is self-contained and usually stored and accessed locally.
o Data is usually stored in a file with a .db extension.
o To read data into R from a database we need to connect the database.
o dbConnect( ) function is used from the DBI (database interface) package to
connect the database.
data <- dbConnect(RSQLite::SQLite(), "data/xyz.db")
o Relational databases may have many tables. In order to retrieve data from a
database, we need to know the name of the table in which the data is stored.
o We can get the names of all the tables in the database using
the dbListTables function:
tables <- dbListTables(conn_lang_data)

Obtaining data from the web using API

 Accessing data stored in a plain text, spread sheets, comma or tab separated files from a
web URL using one of the read_* functions from the tidyverse.
 Now websites use Application Programming Interface (API), which provides a
programmatic way to read data set.
 This allows the website owner to control who has access to the data, what portion of the
data they have access to, and how much data they can access.
 We can collect data programmatically - in the form of Hypertext Markup Language
(HTML) and Cascading Style Sheet (CSS) code - and process it to extract useful
information.
 HTML provides the basic structure of a site and CSS helps style the content.
What is Tidy Data?
 In a Data Science project, tidying data is a necessary after importing data in order to
communicate results.

 Tidy datasets provide a standardized way to link the structure of a dataset (its physical
layout) with its semantics (its meaning).
o Structure is the form and shape of data. In statistics, most datasets are rectangular
data tables(data frames) and are made up of rows and columns.
o Semantics is the meaning for the dataset. Datasets are a collection of values,
either quantitative or qualitative. These values are organized in 2 ways —
variable & observation.
 Variables — all values that measure the same underlying attribute across units
 Observations — all values measured on the same unit across attributes
o The 3 rules of tidy data help simplify the concept and make it more intuitive.

 Each variable is a column

 Each observation is a row
 Each type of observational unit is a table

Messy Data
 Messy data is any kind of data that does not follow the above framework.
 To narrow it down, the paper gives 5 common problems of messy data:
o Column headers are values, not variable names.
o Multiple variables are stored in one column.
o Variables are stored in both rows and columns.
o Multiple types of observational units are stored in the same table.
o A single observational unit is stored in multiple tables.

Why is Tidy Data important?

 If the data set is in standardized framework then we spend less time on data cleaning and
wrangling and more time to focus on answering the problem.
 It is a good practice to have the data in a format which makes it reproducible and easy for
others to understand.
 Another more technical reason is that the concept of tidy data is complemented with the tools
in R to work with. Since R works with vectors of values (R functions are vectorized by nature),
we able to naturally apply our tidy data to the tools used.