DECISION TREE ALGORITHM

Hello There!
 Algorithm
An algorithm is a step-by-step procedure that defines
a set of instructions that must be carried out
in a specific order to produce the desired result

Condition
Input for Desired Output
Result
 What is
Machine Learning?

Supervised

Unsupervised
Ordinary System Artificial Intelligence Machine Learning
An approach to achieve AI
Operates at normal Reinforcement
Human Intelligence through systems that can learn
condition
exhibit by machines. from its pas experience
 Decision Tree
Terminologies
1. Root Node
2. Splitting Represents entire
Dividing nodes into two population
or more sub-nodes

TR
LS
3.Decision Node

FA

UE
Dividing nodes into two or
4. Leaf / Terminal Node more sub-nodes
The node that doesn’t split

TR
LS

UE
FA
5. Pruning
Removing sub-nodes of a
decision node
 Criteria for choosing the attributes
to form a perfect tree:

Purest Tree
If we select a random tree, it won’t be optimum. We must follow a technique by
which we can get the best tree. The best tree is the smallest complete tree

Root Node
If we want to have the best tree we must select the first one. Because if we select
the outlook as the root attribute, we will get the most relevant tree. In another
word, we can say that it is the purest tree.

The most popular impurity criterion is ‘Information Gain’. The attribute which results
maximum information gain will be chosen as our desired nodes. And the whole process
is called ‘ID3 algorithm’. We will discuss ID3 algorithm for making the decision tree.
 SAMPLE DATA SET

DAY OUTLOOK TEMP HUMIDITY WIND PLAY

D1 Sunny Hot High Weak No
D2 Sunny Hot High Strong No
D3 Overcast Hot High Weak Yes
D4 Rain Mild High Weak Yes
D5 Rain Coo; Normal Weak Yes
D6 Rain Coo Normal Strong No
D7 Overcast Coo Normal Strong Yes
D8 Sunny Mild High Weak No
D9 Sunny Coo Normal Weak Yes
D10 Rain Mild Normal Weak Yes
D11 Rain Mild Normal Strong Yes
D12 Overcast Mild High Strong Yes
D13 Overcast Hot Normal Weak Yes
D14 Rain Mild High Strong No
ID3 Algorithm
Numerical Computation

Elaborate on what you want to discuss.

 COMPUTING ENTROPY

Gain = IG – E(S)

S = ([x , y]) = entropy (x , y) = - ( ) log ( ) - ( ) log ( )

IG = - log ( ) - ( ) log ( )

EA =entropy
v
S v
log 10 𝑥
log 2 𝑥=
log 10 2
 Decision Tree Diagram
[D1, D2, …,D14]
[9+ 5, …]

OUTLOOK

Sunny Rain
Overcast

HUMIDITY WIND
[D3,D7,D12,D13]
High [4+, 0-] Strong
Normal Weak

[D9,D11] [D6, D14] [D4, D5, D10]

[D1, D2,D8] Yes
NO YES NO YES
Group Name:
nan0-Blast
A team is more than collection of people
It is a process of gove and take

Jules Capuras Rojhanne Monsanto Jeric Cabilogan

Leader Lead Programmer Presentor
Thank you!
for Listening