Fakulti Teknologi Maklumat & Komunikasi

Universiti Teknikal Malaysia Melaka (UTeM)

BITI 1113
Artificial Intelligence

Laboratory 10: Solution

Instruction: Answer all questions individually and submit a hardcopy at the end of the lab
session.

Given a data set below about deciding to play tennis:

Weather Temperature Wind Play Tennis?

Sunny 85 Weak No
Sunny 90 Strong No
Sunny 95 Weak No
Sunny 70 Weak Yes
Sunny 70 Strong Yes
Cloudy 78 Weak Yes
Cloudy 65 Strong Yes
Cloudy 90 Strong Yes
Cloudy 75 Weak Yes
Rainy 96 Weak Yes
Rainy 80 Weak Yes
Rainy 80 Weak Yes
Rainy 70 Strong No
Rainy 80 Strong No

1. Calculate the Entropy for PlayTennis result.

Entropy(PlayTennis)
= - (5/14) log2 (5/14) – (9/14) log2 (9/14)
= - (0.36 log2 0.36) – (0.64 log2 0.64)
= 0.94

1
2. Calculate the IG for Weather, Humidity and Wind attributes. Which attribute will be chosen
to be the root of a decision tree?

Weather

Entropy(Weather-Sunny)
= - (2/5) log2 (2/5) – (3/5) log2 (3/5)
= - (0.4 log2 0.4) – (0.6 log2 0.6)
= 0.97

Entropy(Weather-Cloudy)
= - (4/4) (log2 (4/4) – 0
= - (1 log2 1) – 0
=0

Entropy(Weather-Rainy)
= - (3/5) (log2 (3/5) – (2/5 (log2 (2/5))
= - (0.6 log2 0.6) – (0.4 log2 0.4)
= 0.97

IG(Weather) =
= 0.94 – ((5/14) *0.97 + (4/14)*0 + (5/14) *0.97)
= 0.94 – ((0.36)* 0.97 + (0.29)*0 + (0.36)*0.97)
= 0.94 – (0.35 + 0 + 0.35)
= 0.24

Humidity

Entropy(Temperature<=80) =
= - (7/9) log2 (7/9) – (2/9) log2 (2/9)
= - 0.7 ((log10 (0.7) / log10 (0.301)) - 0.2 ((log10 (0.2) / log10 (0.301))
= 0.76

Entropy(Temperature>80) =
= - (2/5) log2 (2/5) – (3/5) log2 (3/5)
= - 0.4(log10 (0.4) / 0.301) – 0.6 (log10 (0.6) / 0.301)
= 0.97

IG(Temperature) =
= 0.94 – ((9/14) *0.76 + (5/14)*0.97)
= 0.94 – (0.49 + 0.35)
= 0.94 – 0.84
= 0.10

2
Wind

Entropy(Wind-Weak) =
= - (6/8) log2 (6/8) – (2/8) log2 (2/8)
= - 0.75 (log10 (0.75) / 0.301) – 0.25 (log10 (0.25) / 0.301)
= - 0.75 (- 0.12 / 0.301) – 0.25 (-0.6/0.301)
= (- 0.75)(-0.4) – (0.25) (-2)
= 0.3 + 0.5
= 0.8

Entropy(Wind-Strong) =
= - (3/6) log2 (3/6) – (3/6) log2 (3/6)
= - 0.5 (log10 (0.5) / 0.301) – 0.5 (log10 (0.5) / 0.301)
= - 0.5 (-0.301 / 0.301) – 0.5 (-0.301 / 0.301)
=1

IG(Wind) =
= 0.94 - ((8/14) *0.8 + (6/14)*1)
= 0.94 – (0.46 + 0.43)
= 0.94 – 0.89
= 0.05

The attribute for the root is Weather.