Stating and Refining the Question
• For example, higher income may be one of the final
• Doing data analysis requires quite a bit of thinking
and when you’ve completed a good data analysis,
you’ve spent more time thinking than doing.
• The thinking begins before you even look at a
dataset, and it’s well worth devoting careful thought
to your question.
Descriptive question
• A descriptive question is one that seeks to
summarize a characteristic of a set of data.
• Examples include determining the proportion of
males, the mean number of servings of fresh fruits
and vegetables per day, or the frequency of viral
illnesses in a set of data collected from a group of
individuals.
• There is no interpretation of the result itself
as the result is a fact, an attribute of the set
of data that you are working with.
Exploratory question
• An exploratory question is one in which you analyze
the data to see if there are patterns, trends, or
relationships between variables.
• These types of analyses are also called “hypothesis-
generating” analyses because rather than testing a
hypothesis as would be done with an inferential,
causal, or mechanistic question, you are looking for
patterns that would support proposing a hypothesis.
Inferential question
• An inferential question would be a restatement of
this proposed hypothesis as a question and would be
answered by analyzing a different set of data, which in
this example, is a representative sample of adults in
the Indonesia.
• You will be able to infer what is true, on average, for
the adult population in the Indonesia from the
analysis you perform on the representative sample.
Predictive question
• A predictive question would be one where you ask
what types of people will eat a diet high in fresh fruits
and vegetables during the next year.
• In this type of question you are less interested
in what causes someone to eat a certain diet, just
what predicts whether someone will eat this certain
diet.
set of predictors, and you may not know (or even
care) why people with higher incomes are more likely
to eat a diet high in fresh fruits and vegetables, but
what is most important is that income is a factor that
predicts this behavior.
Causal question
• A causal question asks about whether changing one
factor will change another factor, on average, in a
population.
• Sometimes the underlying design of the data
collection, by default, allows for the question that you
ask to be causal.
• An example of this would be data collected
in the context of a randomized trial, in which people
were randomly assigned to eat a diet high in fresh
fruits and vegetables or one that
Mechanistic question
• Finally, none of the questions described so far will
lead to an answer that will tell us, if the diet does,
indeed, cause a reduction in the number of viral
illnesses, how the diet leads to a reduction in the
number of viral illnesses.
• A question that asks how a diet high in fresh fruits
and vegetables leads to a reduction in the number of
viral illnesses would be a mechanistic question.
Characteristics of a Good Question (Peng &
Matsui, 2015)
There are five key characteristics of a good question
for a data analysis:
• It should be of interest to your audience
• It has not already been answered.
• It should stem from a plausible framework.
• It should be answerable.
• It should be specific.
• Is eating a healthier diet better for you?
• Does eating at least 5 servings per day of fresh
fruits and vegetables lead to fewer upper respiratory
tract infections (colds)? >> More Spesific
Good Economic Research Question
• Especially for research in the field of economics,
there are additional criteria that need to be
considered.
• These criteria are as follows:
 1. Tampaknya seperti topik penelitian di bidang
educational psychology
2. Topik ini merupakan isu penelitian di
bidang labor economics dan teori human capital.
3. Merupakan aplikasi dari teori educational
production function

Feasible
Data mengenai kecerdasan kognitif dan Ibu
bekerja dan data mengenai characteristic ibu-anak
terdapat di IFLS.

Examples
Contoh:
• Wanita Karir (Ibu Bekerja) dan Perkembangan
Kecerdasan Anak

Problem oriented

Analytical

Untuk membahas mengenai isu (research question) ini

dibutuhkan sebuah studi mengenai proses
pengembangan kecerdasan anak dan peran ibu atau
pengasuh / childcare dalam hal tsb.

Interesting and Significant

 Studi sebelumnya menemukan bahwa

kurangnya perhatian ibu menyebabkan
kecerdasan anak berkurang.
 Namun belum ada kesepakatan yang permanen
diantara para ahli. Hal ini menarikuntuk diteliti
karena hasilnya dapat digunakan untuk
membuat kebijakan yang dapat berpengaruh
pada banyak keluarga di Indonesia.

Data Pre-processing
Amenable to Economic Analysis

• 80% of project time is typically getting data
ready.
• Data in the real world is often dirty; that is, it
needs being cleaned up before it can be used for a
desired purpose.
• This is often called data pre-processing.
• What makes data “dirty”? Here are some of the
factors that indicate that data is not clean or ready
to process:
• Incomplete. When some of the attribute values
are lacking, certain attributes of interest are
lacking, or attributes contain only aggregate data.
• Noisy. When data contains errors or outliers. For
example, some of the data points in a dataset may
contain extreme values that can severely affect the
dataset’s range.
• Inconsistent. Data contains discrepancies in
codes or names. For example, if the “Name”
column for registration records of employees
contains values other than alphabetical letters, or
if records do not start with a capital letter,
discrepancies are present.
Forms of data preprocessing
Data Cleaning
• Since there are several reasons why data could
be “dirty,” there are just as many ways to “clean”
it.
• Three key methods that describe ways in which
data may be “cleaned,” or better organized:
• Data Munging
• Handling Missing Data
• Smooth Noisy Data
Data Munging
• Often, the data is not in a format that is easy to
work with.
• Thus, we need to convert it to something more
suitable for a computer to understand.
• This can be done manually, automatically, or,
in many cases, semiautomatically.
EXAMPLE:
Consider the following text recipe.
“Add two diced tomatoes, three cloves of garlic,
and a pinch of salt in the mix.” This can be turned
into a table
Handling Missing Data
• Sometimes data may be in the right format, but
some of the values are missing.
• Missing data:
• problems with the process of collecting data
• equipment malfunction
• comprehensiveness may not have been
considered important at the time of collection.
• system or human error while storing or
transferring the data.
• Strategies to combat missing data include:
• ignoring that record,
• using a global constant to fill in all missing
values,
• imputation,
• inference-based solutions (Bayesian formula
or a decision tree), etc.
Smooth Noisy Data
• There are times when the data is not missing, but
it is corrupted for some reason.
• Data corruption may be a result of faulty data
collection instruments, data entry
problems, or technology limitations.
• Just as there is no single technique to take care
of missing data, there is no one way to
remove noise, or smooth out the noisiness in
the data.
Example:
A digital thermometer measures temperature
to one decimal point (e.g., 70.1°F), but the storage
system ignores the decimal points. Now we have
70.1°F and 70.9°F both stored as 70°F. This may
not seem like a big deal, but for humans a 99.4°F
temperature means you are fine, and 99.8°F
means you have a fever, and ifm our storage
system represents both as 99°F, then it fails to
differentiate between healthy and sick persons!
Data Integration
• To be as efficient and effective for various data
analyses as possible, data from various sources
commonly needs to be integrated.
• The following steps describe how to integrate
multiple databases or files:
1. Combine data from multiple sources into a
coherent storage place (e.g., a single file or a
database).
2. Engage in schema integration, or the
combining of metadata from different sources.
3. Detect and resolve data value conflicts.
4. Address redundant data in data integration.
Data Transformation
Data must be transformed so it is consistent and
readable (by a system).
The following five processes may be used for data
transformation:
• Smoothing: Remove noise from data.
• Aggregation: Summarization, data cube
construction.
• Generalization: Concept hierarchy climbing.
• Normalization: Scaled to fall within a small,
specified range and aggregation. Some of the
techniques that are used for accomplishing
normalization are:
• Min–max normalization.
• Z-score normalization.
• Normalization by decimal scaling.
• Attribute or feature construction:
• New attributes constructed from the given
ones.
Data Reduction
• Data reduction is a key process in which a
reduced representation of a dataset that produces
the same or similar analytical results is obtained.
• One example of a large dataset that could
warrant reduction is a data cube.
• Data cubes are multidimensional sets of data that
can be stored in a spreadsheet.
• Data Reduction Techniques:
Data Cube Aggregation. We reduce the data
to its more meaningful size and structure for the
task at hand.
Dimensionality Reduction. In contrast with
the data cube aggregation method, where the data
reduction was with the consideration of the task,
dimensionality reduction method works with
respect to the nature of the data.
Data Discretization
• We are often dealing with data that are collected
from processes that are continuous, such as
temperature, ambient light, and a company’s stock
price. But sometimes we need to convert these
continuous values into more manageable parts.
• This mapping is called discretization.
• There are three types of attributes involved in
discretization:
Nominal: Values from an unordered set
Ordinal: Values from an ordered set
Continuous: Real numbers
• To achieve discretization, divide the range of
continuous attributes into intervals.
• For instance, we could decide to split the range
of temperature values into cold, moderate, and
hot, or the price of company stock into above or
below its market valuation
Machine Learning: Introduction

Machine Learning
• Machine learning is part of data science.
• What happens most often is data gets generated
in massive volumes and it becomes totally tedious
for a data scientist to work on it. That is when
machine learning comes into action.
• Definition:
- Machine learning is the ability given to a
system to learn and process data sets
autonomously without human intervention. Or,
- Machine learning is the ability of
algorithms to learn from data
• Machine Learning (ML) uses various statistical
approaches for making predictions.
• It also has a major role in pattern finding in data,
that is, it can find various patterns in complex data
given to it.
• ML is a part of its larger domain, which is
Artificial Intelligence (AI).
• ML deals with algorithms that learn from given
data and make predictions.
Why Do We Need Machine Learning?
• Machine Learning has made the analysis of large
amounts of data very efficient. Normal algorithms
are not capable of doing complex tasks, which is
why ML is in use.
• They learn from previous computations to
supply reliable, repeatable decisions and results.
• ML is widely used in fields like banking, Machine Learning Approaches
healthcare, science and many more. • There are various approaches when it comes to
• Tasks like pattern recognition, prediction of implementing Machine Learning algorithms.
future data would not be possible without ML. • The approaches in ML are now classified into
two categories:
How Does Machine Learning Work? • Grouping of the algorithms by their
• Machine Learning has a basic working pattern. learning style.
The algorithm takes both the input and output of
the program.
• Then it trains a model using the given data. The
algorithm looks for various patterns in the given
data.
• These patterns help in future predictions. The
results obtained from training the model will help
in improving the model’s working.

• Grouping of the algorithms by their

similarity.

ML: Supervised Learning

• Supervised learning is the machine learning task
of learning a function that maps an input to an
output based on example input-output pairs.
• It infers a function from labeled training data
consisting of a set of training

Machine Learning Applications examples.

• In this type of learning, we have labeled input
data. This means that the data presented to the
model already contains the correct answer.
• Some might wonder why we are providing data
that already has the right answer. The answer is
simple.
• We are giving this pre-labeled data to make the
model learn from it. This means if data with
similar features is given to the machine in the
future, it will recognize it.
Advantages and Disadvantages of
• In addition, the labeled data (training data) help
Machine Learning
to train the model to improve its accuracy.
 2. Regression
• It is a way which helps find the correlation
between variables and enables us to predict the
continual output variable supported the one or
more predictor variables.
• The output in this problem is a real value like
‘mass’, ‘percent’, ‘rupees’, ‘dollars’ etc.

ML: Unsupervised Learning

• Unsupervised learning is a type of machine
learning algorithm used to draw inferences from
datasets consisting of input data without labeled
responses.
• The most common unsupervised learning
method is cluster analysis, which is used for
exploratory data analysis to find hidden patterns
or grouping in data.
• In this type of learning, there is no labeled input
data. This means that the machine won’t be given
any training data to learn from.
• Thus, the task of the machine is to group similar
types of data together. This is done based on
patterns and differences without any sort of
previous training.
• Thus, the machine will have to figure everything
from scratch without any help.

Supervised learning is of two types:

1. Classification
• It specifies the category to which data elements
belong to and is best used when the output has
finite and discrete values.
• It predicts a category for an input variable also.
• The output in this problem is a category.
Such as ‘blue’, ‘green’, ‘sunny’, ‘no sunny’,
‘disease’, ‘no disease’ etc.
• The output here is in the form of classes.
Unsupervised learning is of two types:
1. Clustering
• It which involves segregating data supported the
similarity between data instances.
• It is an iterative process to seek out cluster
centers called centroids and assigning data points
to at least one of the centroids.
• Here, we have groups of data according to
certain criteria.
• For example, people who drink coffee are
grouped separately. Whereas people who don’t
drink come in a different group.

2. Association
ML: Decision Tree Algorithms
• Association rules allow you to determine
• In trees, the data splits according to specific
associations amongst data objects inside large
parameters. It consists of nodes and leaves.
databases.
• Here, leaves are the result whereas the nodes
• This helps to find more diverse rules in the
represent the point where the data is split.
data.
• Splitting here means, if there are two options of
• Association rule provides us with deeper
yes and no, at a time only one result is there.
information about groups.
• There are two types of trees:
• For example, people who drink coffee may also
- classification trees and (a yes/no type)
like to drink tea.
- regression trees (the data is continuous)
• There are various algorithms using which the
ML: Semi-Supervised Learning
decision tree is constructed:
supervised learning
- ID3 algorithm (Iterative Dichotomiser 3
• we have to manually label the data
algorithm)
• This requires a lot of time and can be expensive.
- CART (Classification and Regression
• Also, it requires ML engineers or special data
Testing)
scientists to do such jobs
- Chi-square method
- Decision Stump
unsupervised learning
- M5 algorithm
• the results obtained are less accurate.
• This is because the data is not labeled and
ML: Bayesian Algorithm
unknown.
• For Bayesian methods, it is obvious that the
Bayes theorem is there in all methods.
semi-supervised learning
• Bayes theorem is all about probability,
• Eliminates supervised and unsupervised learning
especially conditional probability.
problems.
• It suggests that an event A will happen if an
• The algorithm trains using both labeled and
event B has already happened.
unlabeled data.
• The most famous algorithm used is the Naïve
• The labeled data is a small part as compared to
Bayes theorem.
the unlabeled data.
• It works on probability and it can calculate the
• The programmer uses unsupervised learning to
likelihood of events to happen.
first group the unlabeled data.
• There are various other algorithms like:
• Then the supervised learning labels all the
- Gaussian Naïve Bayes theorem
remaining unlabeled data.
- Multinomial Naïve Bayes theorem
 - Bayesian Belief Network

ML: Clustering Algorithms

• This is an unsupervised learning approach,
which is very useful when it comes to grouping of
data.
• Here, in clustering, similar types of data occur in
a single group or cluster.
• Whereas, if data is not similar, then it occurs in
some other group or cluster.
• The algorithms used are:
•K-means
• K-medians
• Hierarchical clustering

ML: Artificial Neural Network (ANN)

Algorithms
• ANNs work on the exact concept of biological
neurons (the nerve cells) in your brain. An ANN
consists of many artificial neurons, which perform
the exact task as the nerve cell in a brain.
• The neural networks are a concept of Deep
Learning. They can simulate the biological
nervous system.
• It is capable of both Machine Learning and
pattern recognition.
• ANN is mainly an information processing
technique. It is also a type of graph consisting of
nodes and connecting arcs.
• The neural network consists of three layers: The
input layer, the hidden layer, and the output layer.
• The input layer takes the input. The hidden layer
processes the input data.
• It performs various tasks on the given data. Then
the processed data passes through the output layer.
• Each neuron is represented by a circle. Whereas
the connections here are arc-shaped.
• There are various algorithms in ANN:
• Perceptron learning
• Multilayer perceptron
• Back-propagation
• Stochastic gradient descent