DATA ANALYSIS INDUSTRY

TRAINING PROGRAM

Elnaz Gholipour
2022 - 2023
 Elnaz Gholipour
2022

Page No

▪ Introduction to Data Analysis, Data cleaning & Transformation, Types of data 3-6

▪ Different Statistics for Analysis; Descriptive, Inferential 7

▪ Practical part 1(SPSS) 8 - 12

▪ Practical part 2(SPSS) 13 - 15

▪ Normal Distribution 16 - 19

▪ Practical part 3(SPSS) 20

▪ When data is not normally distributed & Interpretation of the normality result & Z-Score 21 - 23

▪ Practical part 4 24

▪ Visual Representation of data, Conclusion of normality test 25 - 27

▪ Sampling 28

2

Data analytics is all about helping organizations make
decisions based on data. Page visits can inform
marketing strategies, housing costs can affect policy
changes, and patient outcomes can impact a hospital’s
operations. Data analytics helps us find patterns and tell
stories from a large number of data organizations. To do
that, data analysts take a business question and translate
it into a data question. Part of their job is collecting and
reformatting data, analyzing it with statistics and
probability, and sharing actionable insights in the form
of visuals and reports. “Every company is collecting
some data. And a lot of companies need to leverage
their data to make good data-driven decisions. There’s a
huge opportunity for data analysts to really put that data
to work.”
Elnaz Gholipour
2022
Data science is a broad field that includes data
analytics. It also covers making predictions with
machine learning, working with big data, and
developing artificial intelligence. Data scientists create
algorithms to automate data processes, recognize
patterns in new information, and make
recommendations based on past behavior. They work on
things like forecasting the financial future, creating
customer-facing chatbots, detecting tumors in X-ray
images, and making suggestions of things you might
like. “Data science tends to be more specialized than
data analytics because not every company needs to
make predictive data decisions, and not every company
needs to leverage big data”.
3
 Elnaz Gholipour
2022

Global job statistics and income of DA

OECD Data, WORLD BANK 4

 Elnaz Gholipour
2022

Data Cleaning refers to removing incorrect,
corrupted, incorrectly formatted, duplicate, or
incomplete data from a dataset. The possibility of
duplication or mislabeling data increases when
multiple data sources are combined. Incorrect data
can lead to unreliable outcomes and algorithms,
even if they seem to be correct.
Data Transformation process extracts data from a
source, converts it into a usable format, and delivers it to
a destination. Extraction involves gathering data from
different locations and sources and integrating them into
one place. The entire process is known as ETL (Extract,
Load, Transform).

Steps of data cleaning: (Excel file :teacher1)

Step 1: Remove irrelevant data.
Step 2: Deduplicate your data.
Step 3: Fix structural errors. For example, you may find
“N/A” and “Not Applicable”, but they should be in the
same category.
Step 4: Deal with missing data.
Step 5: Filter out data outliers.

5
 Elnaz Gholipour
2022

No meaningful order can be given to these categories. We can only do

frequency distribution, and it is just related to categorizing. For example, for
Nominal the nominal variable of a preferred mode of transportation, you may have the
categories of car, bus, train, or bicycle.
An ordinal data set is a type of statistical data in which the variables are
categorized by natural, ordered groups and the distance between the groups is
Ordinal
unknown. Examples: socio-economic status (“low income”,” middle income",
"high income”), an education level (“high school”, ” BS”,” MS”,” Ph.D.”)

Interval
Interval and ratio data can both be categorized, ranked, and have equal spacing
between adjacent values, however, only ratio scales have a true zero. The
temperature in Celsius or Fahrenheit has an interval scale because zero is not
the lowest possible value.
Ratio

6
 Elnaz Gholipour
2022

▪ Descriptive Statistics; Utilizes numerical and graphical methods to look for patterns in a data set. Descriptive
statistics provides tools to describe a sample. To summarize the information revealed in
a data set and to present that information in a convenient form. i.e. Average, Range,
Frequency, Histogram, Median, Scatter Plot, Mode, …( Excel file : teacher2)
Frequency Central Tendency
Distribution Mean = Average
How many
observations
in each Dispersion Mode =Most frequent number
group

Median = The middle

Standard Range value of the sorted data
Deviation Variance

Uses sample data to make estimates, decisions, predictions, or other generalizations

about a larger set of data. Starting from the sample, inferential statistics are used to
▪ Inferential Statistics;
make a statement about the population i.e. Hypothesis test, Z score, ANOVA,
Correlation, Regression, Confidence Interval, …

7
 Elnaz Gholipour
2022

1) In this part, I am going to introduce the environment and version of SPSS. How to enter data entering data : csv file (bank)
into SPSS, and how to define the variables.

Free trial of SPSS: 30 Days; use the link below;

https://www.ibm.com/analytics/spss-trials
Academic version for students: It may be 35$ for 6 months

2) Secondly, How to get a sample from a hidden part of SPSS :

For Macintosh: Mac./Applications/IBM/SPSS/Statistics/22/Samples/English

For Windows : C:\Program Files\IBM\SPSS\Statistics\22\Samples\English
.

8
 Elnaz Gholipour
2022

Three Steps of Cleaning Data:

1. Missing Value Analysis

2. Out-of-the-Range Values ( Find the maximum and minimum values of each variable, and check the out-of-range
values. If you have access to that observation that is out of the range, you should correct it, otherwise, it must be
deleted)

3. Detecting and Removing Outliers: An Outlier is an observation in a given dataset that lies far from the rest of the
observations. That means an outlier is vastly larger or smaller than the remaining values in the set. Natural
variations in the population, as outliers should be left as are in your data set. There are other types of outliers that
must be removed because they represent measurement errors, data entry errors, and poor sampling errors. (To do
so, we go Analyze Descriptive Statistics Explore (box plot shows)) (Excel file: teacher3)
(To compute Z-Score: Analyze Descriptive Statistics Descriptive Save standardized values as variables).

9
 Elnaz Gholipour
2022

1. Check if the outlier is the entry mistake.

2. Test the result with and without the outlier. Being transparent in the final report is a great way to make sure that
your final analysis is reliable.
3. Examine the normal distribution of data with the outlier(s) if it still follows the normality.

10

Handling Missing values: If you don't filter or identify
these data, your analysis may not provide accurate
results. Missing responses for variables per variable
may range from 4% to 10%. Such a range is often
considered normal (Hair & Anderson, 2010).
Patterns of missing data:(Sav file: teacher1)
Missing completely at random (MCAR); If the
missing data of one variable is totally unrelated to any
Certainty other variable in the data set.
is Missing at random(MAR); If the missing data of one
increasing
variable can be explained by any other variable in the
data set.
Missing non-random (MNR); This missing data can
not be missed. (Sav file: teacher1)
Elnaz Gholipour
2022
If there are no patterns detected for missing values
(MCAR) and the percentage of them exceeds 10%, then
pairwise or listwise deletion could be done. However, In
this case, imputation is required if the missing values
show a pattern. first, you need to run descriptive
statistics to check if the missing is related to a specific
variable, case, or setting (MAR), then you need to run a
skewness test to check if the distribution is normal or
skewed for the variables, if the distribution is
approximate to normal, then you can use the mean for
replacing the missing, otherwise, you need to use the
median.
By increasing the certainty of missing data (MNR); we
are certain about the reason for the missing values, we
should apply models like KNN, EM, and Regression,…
to replace the missing ones.
11
 Elnaz Gholipour
2022

5 ways of Dealing with missing data:
Listwise deletion; All missing data of the variable will be
deleted from the data set.
Pairwise deletion or delete cases analysis by analysis; In
any analysis, the missing data of the variables will be
omitted, for another analysis, the corresponding missing
data of the selected variables will be removed.
Mean Substitution; missing data will be replaced by the
mean of the variable (SPSS: Transform Replace missing
values).
Regression-based imputation; missing data is replaced with
the predicted value generated by using multiple regression
based on non-missing data on another variable (SPSS:
Analyze Missing Value Analysis).
Expectation – Maximization (EM)Algorithm; Maximum
likelihood estimation(SPSS: Analyze Missing Value
Analysis).
SPSS Syntax:
a) What is going behind SPSS
b) After a while if it is forgotten how was that
particular analysis, It is only needed to run the
saved Syntax.
c) Re-doing the same analysis over and over again
in a quick way.
To deselect missing value: Data Select cases
if-condition (Sav file: teacher1)
NOTE!
After replacing the missing value, we can use
Paired-sample-t-test to check whether the old
and new values are alike or not.

12

Types of Descriptive Analysis
Measures of Frequency
An essential part of descriptive analysis is knowing
how frequently a certain event or response occurs. This
is the prime purpose of measures of frequency to make
like a count or percent.
Measures of Central Tendency
An important part of any descriptive analysis is finding
out what the Central (or average) Tendency or
Response is. In order to determine central tendency,
three averages are used: the mean, the median, and the
mode.
Measures of Dispersion
Often, it is useful to know how data is spread across a
range.
Measures of Position
Identifying a value's position or its response in relation
to others is another aspect of descriptive analysis.
Elnaz Gholipour
2022
Think about a survey in which 500 participants are
asked which football team they support. A database
with 500 responses would be hard to handle, but
measuring how often one football team was selected
can make the data more accessible.
Take the case of a survey that measures the weight of
1,000 people as an example. A mean average would
be an excellent descriptive metric in this case.
A sample of two people's weight can be used to
illustrate this. The average weight will be 60 kg if
both people weigh 60 kilos. The average weight will
still be 60 kg if one individual weighs 50 kg and the
other 70 kg.
Measures like percentiles, quartiles, and cross-
tabulation become very useful in this area of
expertise.
13
 Elnaz Gholipour
2022

Your aim is to find out what activities are most popular # of values below x
among women and men. Participants are asked to rate
Percentile rank of x =
n * 100
how often they have done these things during the past
year: (Sav file: teacher6) What value exists at the percentile ranking of 25%?
Go to a library percentile
Watch a movie at a theater Value # = (n+1)
100
Visit a national park
A survey's responses make up your data set. Now you SPSS: Analyze Descriptive Statistics Frequencies
can use descriptive statistics to find out the overall Statistics ( percentile & quartile).
frequency of each activity (frequency), the averages for
each activity (central tendency), the spread of responses
for each activity (dispersion), and a measure can tell us
whether a value is about the average, or whether it's
unusually high or low (position). Percentiles are values
that separate the data into 100 equal parts. For example,
The 95th percentile separates the lowest 95% of the
values from the top 5%. 25th percentile = 1st quartile, 50th
percentile = 2nd quartile, and median.

14
 Elnaz Gholipour
2022

Crosstabs can be used to investigate the relationship
between two variables. Since a crosstab is a descriptive
statistic, it can only report on a sample. The chi-square
test must be used when describing the population.
In general, independent variables are plotted in columns,
and dependent variables in rows. (Excel file: teacher2)
Cross–Tabulation: It is very common for market
researchers to use crosstabs to compare customers or
products. A crosstabulation table is a simple way to
examine the relationship between two categorical
(nominal or ordinal) variables. i.e. which age group
prefers which insurance?

SPSS: Analyze Descriptive Statistics Crosstabs

15