ARTIFICIAL INTELLIGENCE

CLASS X
CODE : 417
CHAPTER : EVALUATION

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 EVALUATION
Evaluation is a systematic determination and assessment of a
subject’s merit, worth and significance, using criteria governed by a
set of standards. It can assist an organization, program, design,
project or any other intervention or initiative to assess any aim,
realisable concept/proposal, or any alternative, to help in decision-
making; or to generate the degree of achievement or value in regard
to the aim and objectives and results of any such action that has been
completed.
Evaluation Model: Evaluation model will decide that which evaluation
matrix will be used to evaluate the performance of a model because
this is the stage which will help decide that model is working well or
not. In the evaluation process, we evaluate the model by feeding the
testing data and then comparing the actual answer.
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Q. Write short note on Evaluation.

Ans. The last stage of Al Project cycle is evaluation. When

the model or project is ready after problem scoping, data
acquisition, data exploration and modelling, the final stage
is to just evaluate the model or project whether it is ready
or not for the action.

Evaluation helps to check whether the model is better than

the other one or not. It is the most important part of the
development process. It helps to determine the best model
for data processing. It also helps in how well the model will
work in future.
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 MODEL EVALUATION TERMINOLOGY

Prediction
The prediction is the output which is given by the
machine.

Reality
The reality is the real scenario observed when
the prediction has been made.
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

TO UNDER STAND THE PROCESS OF EVALUATION, WE
TO TAKE AND EXAMPLE FOR EVALUATION

Consider this scenario where you have an AI

prediction model which predicts the possibilities of
fires in the forest. The main aim of this model is to
predict whether a forest fire has broken out into
the forest or not. To understand whether the model
is working properly or not we need to predict to
check if the predictions made by the model is
correct or not.
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Condition 1 – Prediction – Yes, Reality – Yes (True Positive)
This condition arises when the prediction and reality both match with yes
(prediction – yes, reality – yes), if forest fire has broken out.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Condition 2 – Prediction – No, Reality – No (True Negative)
 If there is no fire in the forest and prediction predicted by machine
correctly as No as well as reality also no, this condition is known as True
Negative.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Condition 3 – Prediction – Yes, Reality – No (False Positive)
 There is no fire in reality but the machine has predicted yes incorrectly.
This condition is known as False Positive.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Condition 4 – Prediction – No, Reality – Yes (False Negative)
 The forest fire has broken out in reality but the machine has
incorrectly predicted No refers to False Negative condition.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Prediction: The output given by the machine after training and
testing the data is known as a prediction.
Reality: The reality is the real situation or real scenario in the
forest where the prediction has been made by a machine.
There are four cases and all have their own terms:

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 Confusion Matrix or Error Matrix
 Itis a comparison between prediction and reality. It
helps us to understand the prediction result. It is not
an evaluation matrix but a record that can help in
evaluation. The confusion matrix allows us to
understand the prediction result.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 Evaluation Methods

Evaluation methods are as follows:

1.Accuarcy
2.Precision
3.Recall
4.F1 Score

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 Evaluation Methods
Accuracy
The percentage of correct predictions out of all the observations is called
accuracy.
If the prediction matches with reality then it said to be correct.
There are two conditions where prediction matched with reality:
True Positive
True Negative
So the formula for accuracy is:

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 The total observations cover all the possible cases of prediction that
can be True Positive (TP), True Negative (TN), False Positive (FP), and
False Negative (FN).
Accuracy talks about how true the predictions are by any model. Now
let’s return to the forest fire.
Example:
If the model always predicts that there is no fire where in reality there
is a 2% chance of forest fire breaking out. So 98% of the model is right
but for these 2% also model predicts there is no fire. Hence the
elements of the formula are as follows:
True Positive = 0
True Negatives = 98
Total Cases = 100
Therefore, accuracy = (98 + 0) / 100 = 98%.
This returns high accuracy for an AI model. But the actual cases where
the fire broke out are not taken into account. Therefore there is a need
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)
to look at another parameter that takes account of such cases as well.
AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)
 Precision
The percentage of true positive cases versus all the cases where the prediction is true.
It takes into account the True Positives and False Positives.
Now return to the forest fire example, Assume that model predictions of forest fire is
irrespective of the reality. In this case, all the positive conditions will be taken into the
account i.e. True Positive (TP) and False Positive (FP). So in this case firefighters always
see if the alarm was True or False.
Recall the story of the boy who falsely cries out and complains about the wolf every
time, but when the wolf came in reality, no one rescued him. Similarly if Precision is low
(more false alarms) then the firefighters would get complacent and might not go and
check every time considering it a false alarm.
So, if Precision is high, means that True Positive cases are more, given lesser False
alarms.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 Recall
In the recall method, the fraction of positive cases that are correctly identified
will be taken into consideration. It majorly takes into account the true reality
cases wherein Reality there was a fire but the machine either detected it
correctly or it didn’t. That is, it considers True Positives and False Negatives.
As we have observed that the numerator in both precision and recall is same i.e.
True Positive. Where in the denominator, precision counts the False Positive while
recall considers False Negatives. In the following case False negative cases can be
very costly.
In forest fire case false negative cost us a lot and are risky too. If no alarm is given
when there is a forest fire, maybe the whole forest burn down.
Viral Outbreak is also one of False Negative case. Consider the case of covid 19, as it
spreads but the machine did not detect it then imagine it can affect so many lives.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 F1 SCORE
 F1 score is the balance between precision and recall.
 When we have 1 value (100%) for both Precision and Recall which is ideal solution.
 In that case the F1 score would be also ideal 1 (100%).
 It is known as the perfect value for F1 score.
 As the values of both Precision and Recall ranges from 0 to 1, the F1 score also ranges from 0
to 1.
 In conclusion, we can say that a model has good performance if the F1 Score for that model
is high.

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

Important Questions:
1.What is confusion matrix?
2.What is evaluation? Why do we need it in evaluation?
3.What is the difference between prediction and reality?
4.How can we calculate F1 score?
5.How can precision and recall be considered as a
correct measure for evaluation?
6.What do you mean by TP, FP, TN, FN?

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)

 THANK YOU
REFERENCE
CBSE
PREETI SAXENA
RK (GAURAV MATHUR & SONU LOHCHAB)
SUMITA ARORA
INTERNET SEARCH
&
MY PERSONAL TEACHING AND STUDYING EXPERIENCES

AMIT GAUR, TGT COMPUTER SCIENCE, KV ARTIFICIAL INTELLIGENCE(417)