Classification by

decision tree
induction
What is a decision tree?

❑ A decision tree is a structure that can be used to divide

up a large collection of records into successfully smaller
sets of records by applying a sequence of simple
decision rules.
❑ With each successive division, the members of the
resulting sets become more and more similar to each
other.
❑ A decision tree model consist of a set of rules for dividing
a large heterogenous population into smaller, more
homogeneous(mutually exclusive)groups with respect to
a particular target.
Decision tree

 Decision trees applies to:

✓ Classification
✓ prediction
They represent rules (generate rules)
Visually define the rules for simple interpretation and
understanding
Decision tree has 3 types of nodes

o Root nodes(main questions):top(or left most) node with

no incoming edges and zero or more outgoing edges.
o Child or internal node : descendent node which has
exactly one incoming edge and two or more outgoing
edges
o Leaf node (answer): terminal node which has exactly
one incoming edge and no outgoing edges.
o The rules or branches are the unique path(edges) with a
set of conditions(attribute) that divide the observations
into smaller subset.
2 important factors

 Information gain : measure of how much information the

answer to a specific question provide
 Entropy : measure of how much uncertainity there us in
the information
They have inverse relation
example