Department of Computer Science & Engineering

Artificial Intelligence and Machine Learning (AI&ML)

Question Bank – IA-2

1. Explain concept learning and also explain space of Find-S.

Definition: Concept learning - Inferring a Boolean-valued function from training examples of its input
and output

FIND-S Algorithm starts from the most specific hypothesis and generalize it by considering only
positive examples.

FIND-S algorithm ignores negative examples. – As long as the hypothesis space contains a hypothesis
that describes the true target concept, and the training data contains no errors, ignoring negative
examples does not cause to any problem.

FIND-S algorithm finds the most specific hypothesis within H that is consistent with the positive
training examples. – The final hypothesis will also be consistent with negative examples if the correct
target concept is in H, and the training examples are correct.

2. Compare the key difference between Find-S and Candidate Elimination algorithm.

FIND-S Algorithm

1. Initialize h to the most specific hypothesis in H

2. For each positive training instance x

For each attribute constraint ai in h

If the constraint ai is satisfied by x

Then do nothing

Else replace aii n h by the next more general constraint that is satisfied by x

Output hypothesis h
3. State and Explain version space representation Theorem.

Version Space representation Theorem.

Let X be an arbitrary set of instances and Let H be a set of Boolean-valued hypotheses defined over X.
Let c: X →{O, 1} be an arbitrary target concept defined over X, and let D be an arbitrary set of training
examples {(x, c(x))). For all X, H, c, and D such that S and G are well defined,

The below figure shows the version space for enjoysport concept learning including both general and
specific boundary sets.

From Version Space representation theorem, there exist a hypothesis s belongs to specific boundary set S,
and there exist a hypothesis g belongs to general boundary set G, a hypothesis h belongs hypothesis space
H holding the relationship “g is more-general-than-or-equal-to h and h is more-general-than-or-
equal-to s” forms a version space.

4. Consider the following set of well-formed formulas in predicate logic.

Convert these in to clause form and prove that hat e(mercus, caeser) using resolution theorem.
 Predicate Logic

1. Marcus was a man.

2. Marcus was a Pompeian.

3. All Pompeians were Romans.

4. Caesar was a ruler.

5. Also, All Pompeians were either loyal to Caesar or hated him.

6. Everyone is loyal to someone.

7. People only try to assassinate rulers they are not loyal to.

8. Marcus tried to assassinate Caesar.

5. Explain Candidate Elimination learning algorithm and Illustrate with Example

Candidate Elimination algorithm

 The Candidate-Elimination algorithm computes the version space containing all hypotheses from H
that are consistent with an observed sequence of training examples.

 It begins by initializing the version space to the set of all hypotheses in H; that is, by initializing the
G boundary set to contain the most general hypothesis in H as
G0 ← { <?, ?, ?, ?, ?, ?> }
and initializing the S boundary set to contain the most specific hypothesis as
S0 ← { <0, 0, 0, 0, 0, 0> }

 These two boundary sets delimit the entire hypothesis space, because every other hypothesis in H is
both more general than S0 and more specific than G0.

 As each training example is considered, the S and G boundary sets are generalized and specialized,
respectively, to eliminate from the version space any hypotheses found inconsistent with the new
training example.

 After all examples have been processed, the computed version space contains all the hypotheses
consistent with these examples and only these hypotheses.
Lets look into the algorithm steps :

Candidate elimination algorithm with an example

 Here are the training examples D

 CANDIDATE-ELIMINTION algorithm begins by initializing the version space to the set of all
hypotheses in H.
 Initializing the G boundary set to contain the most general hypothesis in H G0 = <?, ?, ?, ?, ?, ?>
 Initializing the S boundary set to contain the most specific (least general) hypothesis S0
<0,0,0,0,0,0>
 First training example — its a positive example and when its presented to the CANDIDATE-
ELIMINTION algorithm, it checks the S boundary and finds that it is overly specific and it fails to
cover the positive example. The boundary is therefore revised by moving it to the least more general
hypothesis that covers this new example
 No update of the G boundary is needed in response to this training example because G0 correctly
covers this example

 When the second training example is presented, it has a similar effect of generalizing S further to S2,
leaving G again unchanged i.e., G2 = G1 = G0

 Now the third training example — its a negative example and when its presented to the
CANDIDATE-ELIMINTION algorithm, it reveals that the G boundary of the version space is
overly general, that is, the hypothesis in G incorrectly predicts that this new example is a positive
example. The hypothesis in the G boundary must therefore be specialized until it correctly classifies
this new negative example.

 Given that there are six attributes that could be specified to specialize G2, why are there only three
new hypotheses in G3 ?
For example, the hypothesis h = (?, ?, Normal, ?, ?, ?) is a minimal specialization of G2 that
correctly labels the new example as a negative example, but it is not included in G3.
The reason is, this hypothesis is excluded as it is inconsistent with the previously encountered
positive examples.
 Now the fourth training example — its a positive example and when its presented to the
CANDIDATE-ELIMINTION algorithm, it further generalizes the S boundary of the version space.
It also results in removing one member of the G boundary, because this member fails to cover the
new positive example
  After processing these four examples, the boundary sets S4 and G4 delimit the version space of all
hypotheses consistent with the set of incrementally observed training examples. The entire version
space, including those hypotheses bounded by S4 and G4.

 This learned version space is independent of the sequence in which the training examples are
presented (because in the end it contains all hypotheses consistent with the set of examples).

As further training data is encountered, the S and G boundaries will move monotonically closer to each
other, delimiting a smaller and smaller version space of candidate hypotheses.

6. Explain ID3 Algorithm.

THE BASIC DECISION TREE LEARNING ALGORITHM (ID3)

The basic algorithm is ID3 which learns decision trees by constructing them top-down

ID3(Examples, Target_attribute, Attributes)

Examples are the training examples. Target_attribute is the attribute whose value is tobe predicted by
the tree. Attributes is a list of other attributes that may be tested by the learned decision tree. Returns a
decision tree that correctly classifies the given Examples.

 Create a Root node for the tree

 If all Examples are positive, Return the single-node tree Root, with label = +
 If all Examples are negative, Return the single-node tree Root, with label = -
 If Attributes is empty, Return the single-node tree Root, with label = most common value of
Target_attribute in Examples
 Otherwise Begin
 o A ← the attribute from Attributes that best* classifies Examples
o The decision attribute for Root ← A
o For each possible value, vi, of A,
 Add a new tree branch below Root, corresponding to the test A = vi
 Let Examples vi, be the subset of Examples that have value vi for A
 If Examples vi , is empty
 Then below this new branch add a leaf node with label = most common
value of Target_attribute in Examples
 Else below this new branch add the subtree ID3(Examples vi,
Targe_tattribute, Attributes – {A}))
 End
 Return Root

7. Define the terms

i) Version space ii)General Boundary iii) Specific Boundary

Definition: version space- The version space, denoted V SH,Dwith respect to hypothesis space H
and training examples D, is the subset of hypotheses from H consistent with the training
examples in D.

8. List and Explain the steps involved in decision tree representation.

Decision tree is a flowchart-like structure in which each internal node represents a test on a feature (e.g.
whether a coin flip comes up heads or tails) , each leaf node represents a class label (decision taken after
computing all features) and branches represent conjunctions of features that lead to those class labels.
The paths from root to leaf represent classification rules. Below diagram illustrate the basic flow of
decision tree for decision making with labels (Rain(Yes), No Rain(No)).
 Decision Tree for Rain Forecasting

Decision tree is one of the predictive modelling approaches used in statistics, data mining and machine
learning.

Decision trees are constructed via an algorithmic approach that identifies ways to split a data set based on
different conditions. It is one of the most widely used and practical methods for supervised learning.
Decision Trees are a non-parametric supervised learning method used for
both classification and regression tasks.

Tree models where the target variable can take a discrete set of values are called classification trees.
Decision trees where the target variable can take continuous values (typically real numbers) are
called regression trees.

9. Differences between
a. Training data and testing data.
b. Supervised, Unsupervised and reinforcement.

10. Explain appropriate problems for neural networks with their characteristics. List the properties of Neural
networks?