Pattern Recognition Practice Questions

from Chapters 1 and 2

March 2024

1 Chapter 1
1. What do you understand by pattern? Mention a few real life applications of
pattern recognition.
2. Discuss, with the help of example, what do you mean by feature vector.
3. Mention different approaches to pattern classification. Write a brief note on any
one of them.
4. Discuss in details about the advantages and disadvantages of statistical approach
to pattern classification.
5. Give a diagrammatic representation of the different components of pattern recog-
nition. Briefly discuss about the job of the classifier.
6. Discuss the importance of features to design a pattern recognition system.
7. Draw a block diagram showing the design cycle in pattern recognition system.
8. Mention the different learning techniques used in pattern recognition. Write a
short note on reinforcement learning.
9. Differentiate between supervised and unsupervised learning techniques.
10. What pattern classification approach does Bayes’ Decision Theory come under?
What learning technique does the same follow?

2 Chapter 2
1. Considering ‘c’ number of states of nature, frame the decision rule for the Bayes’
Classifier. Clearly describe the notations used.
2. Considering ‘c’ number of states of nature and ‘a’ number of actions, frame the
decision rule for the Minimum Risk Classifier.
3. Give the expression for the risk function associated with action α, given Feature
Vector X. What is Symmetrical / Zero-One Loss Function?

1
4. Discuss in details about the Bayes Decision Theory generalization criteria.
5. Suppose I can detect spam e-mails in my inbox. Assume that the word ‘offer’
appears in 80% of the spam messages in my account. Also, let’s assume that
‘offer’ appears in 10% of my desired e-mails. If 30% of the received e-mails
are considered as a spam, and I receive a new message which contains the word
‘offer’, how shall I classify the same on the basis of the Bayes’ Decision Rule?
Also find the probability that the new message is spam.
6. As we know, Covid-19 tests are quite common nowadays, but some results of
the diagnostic tests are not true. Let’s assume that a diagnostic test has 99%
accuracy, and 60% of all people have Covid-19. If a patient tests positive, what
will be your decision based on Bayes’ Decision Rule? Also find the probability
that the patient has Covid-19?
7. A person does not feel well and goes to a doctor. Assume that there are two
states of nature:
ω1 : the person has a common flu
ω2 : the person is really sick.
The doctor’s priors are P (ω1 ) = 0.9 and P (ω2 ) = 0.1. The doctor has two pos-
sible actions: ‘prescribe hot tea’ or ‘prescribe antibiotics’. But doctors generally
take some medical observations before making decisions - a reasonable observa-
tion is to perform a blood test. So we can consider the feature ‘x’ as the blood
test result, so that there can be the following possible feature values:
x1 : negative (i.e., no bacterial infection)
x2 : positive (i.e., bacterial infection).
However there is the probability that blood tests can give wrong results. In order
to account for this factor, let us assume the following class conditional probabil-
ities:
p(x1 |ω1 ) = 0.8, p(x1 |ω2 ) = 0.3, p(x2 |ω1 ) = 0.2 and p(x2 |ω2 ) = 0.7.
Again, let us assume the following cost (loss) matrix:
Discuss in details about what would be the prescription of the doctor when the

ω1 ω2
α1 0 10
α2 1 0

test report comes positive and when it comes negative for each of the following
cases:

(a) when the decision is solely based on the prior probabilities.

(b) when the decision is based on the posterior probabilities.
(c) when the decision is based on the conditional risk.

2
 8. Assume that W1 , W2 and W3 are the states of nature and β1 , β2 and β3 are the
actions. Let X be the feature vector of a given data. The loss incurred for taking
action βi if the true state of nature is Wj is given by

(i + j) if i ̸= j
λ(βi |Wj ) = (i + j)
 if i = j
8
3 5 2
Also assume P (W1 |X) = , P (W2 |X) = and P (W3 |X) = . Based
10 10 10
on the Minimum Risk Classifier, state which action should be taken?
9. Suppose a bank classifies customers as either good or bad credit risks. The bank
has observed that 1% of good credit risks and 10% of bad credit risks overdraw
their account in any given month. A new customer opens a checking account at
this bank. The bank believes that there is a 70% chance that the new customer
will turn out to be a good credit risk. Suppose that this customer’s account is
overdrawn in the first month. How does this alter the bank’s opinion of this
customer’s creditworthiness?
10. For a one-dimensional two-class problem with two actions α1 and α2 (α1 : choose
ω1 and α2 : choose ω2 ; where ω1 and ω2 are the two classes), we assume

3 2
P (ω1 ) = and P (ω2 ) =
5 5
and the conditional densities corresponding to a feature x:

p(x|ω1 ) ∼ N (0, 1) and p(x|ω2 ) ∼ N (2, 4)

where N (µ, σ 2 ) is Normal distribution with mean µ and variance σ 2 .

(a) Calculate the likelihood value for each class if x = 5.

(b) Calculate the evidence value if x = 5.
(c) Calculate the posterior probability value for each class if x = 5.
(d) Calculate the likelihood ratio value if x = 5.
(e) Calculate the likelihood ratio threshold for zero-one loss function.
(f) Calculate the likelihood ratio threshold for the following loss matrix.

ω1 ω2
α1 0 4
α2 2 0

3
 (g) Calculate the risks associated with the different actions if x = 5, consider-
ing the loss function as presented in (f).
11. Derive the decision rule of Minimum Error-Rate Classifier from that of Mini-
mum Risk Classifier considering the zero-one loss function.

12. What are the Discriminant Functions corresponding to the Minimum Error-Rate
Classifier and Minimum Risk Classifier?
13. Justify with the help of example that Discriminant Functions are not unique.
14. Write the pdf for the Multivariate normal density. Hence derive the same for the
Bivariate case considering statistically independent components of the feature
vector.
15. Suppose gi (X) denotes the Discriminant Function corresponding to class ωi ,
X being the Feature Vector. How will you represent the Decision Boundary
between classes ωj and ωk ? Hence construct the decision rule to classify an
incoming pattern into one of these classes.

16. The feature values corresponding to a two-dimensional feature vector X for ob-
jects belonging to two classes ω1 and ω2 are provided below:
       
2 4 4 6
ω1 :
−2 −4 0 −2

       
2 4 4 6
ω2 :
6 4 8 6

where
 
x1
X=
x2

follows the Bivariate Normal Distribution. Suppose the a priori probabilities are
P (ω1 ) = 0.5 and P (ω2 ) = 0.5 and the Covariance Matrix for both the classes is
given by
 
8/3 0
Σi = for i = 1, 2
0 8/3

Find out the equation of the Decision Boundary between the two classes ω1 and
ω2 .

4
17. Consider the multivariate normal density with mean µ, σij = 0 and σii = σi2 ,
that is, the covariance matrix is diagonal: Σ = diag(σ12 , σ22 , ..., σd2 , ).
(a) Show that the evidence is
" d  2 #
1 1X xi − µi
p(X) = Qd √ exp −
i=1 2πσi 2 i=1 σi

(b) Write an expression for the Mahalanobis distance from X to µ.

(c) Determine the evidence for a standard multivariate normal density, i.e.,
N (0, I).
18. Consider the three-dimensional Normal Distribution p(X|ω) ∼ N (µ, Σ), where
   
1 1 0 0
µ = 2 and Σ = 0 5 2 .
2 0 2 5

Find the probability density at the point x0 = (0.8, 0.2, 2)t .

19. Consider the following decision rule for a two-category one-dimensional prob-
lem:
Decide ω1 if x > θ; otherwise decide ω2 .
(a) Show that the total probability of error for this rule is given by
Z θ Z ∞
P (error) = P (ω1 ) p(x|ω1 ) dx + P (ω2 ) p(x|ω2 ) dx.
−∞ θ

(b) By differentiating, show that a necessary condition to minimize P (error)

is that θ satisfies

p(θ|ω1 )P (ω1 ) = p(θ|ω2 )P (ω2 ).

20. Let ωmax (X) be the state of nature for which P (ωmax |X) ≥ P (ωi |X) for all
i = 1, 2, ..., c.
1
(a) Show that P (ωmax |X) ≥ .
c
(b) Show that for the minimum-error-rate decision rule, the average probability
of error is given by
Z
P (error) = 1 − P (ωmax |X) p(X) dX.

c−1
(c) Use these two results to show that P (error) ≤ .
c
c−1
(d) Describe a situation for which P (error) = .
c

5
21. Suppose two equally probable one-dimensional densities are of the form
 
|x − ai |
p(x|ωi ) ∝ exp − for i = 1, 2 and bi > 0.
bi

(a) Write an analytical expression for each density, that is, normalize each
function for arbitrary ai and positive bi .
(b) Calculate the likelihood ratio as a function of the four variables.
22. In the two-category case, under the Bayes’ Decision Rule, the conditional error
is given by

P (error|x) = min[P (ω1 |x), P (ω2 |x)] (1)

and the full error by

Z ∞
P (error) = P (error|x) p(x) dx.
−∞

(a) Show that for arbitrary densities, we can replace Equation (1) by

P (error|x) = 2P (ω1 |x)P (ω2 |x)

in the integral and get an upper bound on the full error.

(b) Show that if we use

P (error|x) = αP (ω1 |x)P (ω2 |x) for α < 2,

then we are not guaranteed that the integral gives an upper bound on the
error.
23. Derive the equation of the decision boundary between two classes ωi and ωj
when the feature vector X follows Multivariate Normal Distribution under the
condition that the covariance matrix Σ is arbitrary but same for all the classes.
24. Derive the equation of the decision boundary between two classes ωi and ωj
when the feature vector X follows Multivariate Normal Distribution under the
condition that the covariance matrix Σ takes the form σ 2 I for all the classes
where I indicates the identity matrix. Hence show that:
(a) the decision boundary is orthogonal to the line joining the mean vectors of
the two classes ωi and ωj .
(b) the decision boundary passes through the mid-point of the line joining the
mean vectors of the two classes ωi and ωj when the classes have the same
prior probabilities.
25. Last monsoon, a fisherman caught 1, 000 fish from the Rupnarayan River con-
taining Rohu, Catla, and Hilsa fish. He told his son to place the Rohu fish in
the bucket labelled as R, the Catla fish in the bucket labelled as C, and the Hilsa

6
 fish in the bucket labelled as H. As a fisherman’s son, he knew the shininess of
Hilsa was much higher than that of Catla and Rahu. Also, the shininess of Rahu
is higher than that of Catla. So, the son chose to separate them based on their
shininess. For an arbitrary fish, based on his only guess on shininess, the proba-
bility that the fish will be Rahu, Catla, and Hilsa is respectively 0.33, 0.32, and
0.35. If he placed the fish in the correct bucket, then his father would not have
incurred any loss. If he placed a Hilsa fish in R or C, the loss would be 20 and
15. If he placed a Catla fish in R or H, the loss would be 10 and 5. If he placed a
Rahu fish in H or C, the loss would be 3 and 10. State in which bucket he should
place the fish based on the minimum risk classifier.
26. Last monsoon, a fisherman caught 1, 000 fish from the Rupnarayan River. One
day, his little daughter randomly picked 100 fish and checked that 45 were Rohu,
40 were Catla, and 15 were Hilsa. If the fisherman is forced to classify any
arbitrary fish taken from those 1000 fish based on his daughter’s observation
only, what decision should he take?
27. Last year, Subhasis appeared in the examination of PIP containing 100 questions.
The answer to the questions is either True or False. As usual, he had no clue
about the answer to any of the questions. The topper sarcastically told him that
among these 100 questions, the answer to 57 questions was False. Based on this
information only, if Subhasis wants to score good marks, what should be the
logical answer to all the questions and why?

28. Show that for the two-category classification, the Likelihood ratio between two
classes ω1 and ω2 has a threshold value, for taking a decision.
29. Show that for the two-category classification, the Likelihood ratio between two
classes ω1 and ω2 has a prior probability ratio value, for taking a decision where
(
1 if i ̸= j
λ(βi |Wj ) =
0 if i = j
.

30. Suppose a feature vector X is following the Multivariate Normal Density with
Σ = σ 2 I for every class. If all the classes have the same prior probability, then
show that the decision will be based upon the minimum Mahalanobis distance
between the feature vector X and the mean vector µ.
31. Suppose a feature vector X is following the Multivariate Normal Density with
Σ = σ 2 I for every class. The Discriminant Function for class ωi is given by:
1
gi (X) = − (X − µi )t (X − µi ) + ln P (ωi ).
2σ 2
Show that the decision boundary between the classes ωi and ωj is orthogonal to
the line joining their mean vectors µi and µj .

7
32. Suppose a feature vector X is following the Multivariate Normal Density with
Σ = σ 2 I for every class. The Discriminant Function for class ωi is given by:
1
gi (X) = − (X − µi )t (X − µi ) + ln P (ωi ).
2σ 2
Show that if the prior probabilities are same for the classes ωi and ωj , then the
decision boundary between them is an orthogonal bisector of the line joining
their mean vectors µi and µj .