Performance Metrics

Assignment

Supervised By

Dr Mohamed Abo Rizka

Prepared By
Saif Allah Mohamed Bakry
What are the performance metrics for Machine learning
Models?
o There are Two types of performance metrics

Performance Metrics For Classification

Performance Metrics For Regression

o Performance Metrics for Classification

Confusion Matrix

Accuracy

Precision

Recall

Specificity

F1 Score

AUC (Area Under ROC curve)

2|Page
Accuracy
It is the most frequently used performance metric for classification algorithms. It can
be defined as the proportion of correct predictions made to total predictions made.
We can easily calculate it using the confusion matrix and the following formula.
𝐓𝐏+𝐓𝐍
𝐀𝐜𝐜𝐮𝐫𝐚𝐜𝐲 =
𝐓𝐏+𝐅𝐏+𝐅𝐍+𝐓𝐍

Confusion matrix
It is the most basic way to evaluate the performance of a classification problem with
two or more types of classes as output. A confusion matrix is simply a table with
two dimensions: "Actual" and "Predicted," as well as "True Positives (TP)", "True
Negatives (TN)", "False Positives (FP)", and "False Negatives (FN)", as shown
below

Figure 1
Precision
In document retrieval, precision is defined as the number of correct documents returned by
our ML model. We can easily calculate it using the confusion matrix and the following
formula
𝑇𝑃
𝑷𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 =
𝑇𝑃 + 𝐹𝑃

3|Page
Recall
The number of positives returned by our ML model can be defined as recall. We can easily
calculate it using the confusion matrix and the following formula
𝑻𝑷
𝑹𝒆𝒄𝒂𝒍𝒍 =
𝑻𝑷 + 𝑭𝑵

Specificity
Specificity, as opposed to recall, can be defined as the number of negatives returned by our
ML model. We can easily calculate it using the confusion matrix and the formula below
𝑻𝑵
𝐒𝐩𝐞𝐜𝐢𝐟𝐢𝐜𝐢𝐭𝐲 =
𝑻𝑵 + 𝑭𝑷

F1 Score
The harmonic mean of precision and recall will be calculated using this score.The weighted
average of precision and recall is used to compute the F1 score. F1's best value is 1 and its
worst value is 0. The following formula can be used to calculate F1 score through the
following equation
𝟐 ∗ (𝐩𝐫𝐞𝐜𝐢𝐬𝐢𝐨𝐧 ∗ 𝐫𝐞𝐜𝐚𝐥𝐥)
𝑭𝟏 =
(𝐩𝐫𝐞𝐜𝐢𝐬𝐢𝐨𝐧 + 𝐫𝐞𝐜𝐚𝐥𝐥)

AUC (Area Under ROC curve)

AUC-ROC (Area Under Curve) is a performance metric for classification problems based
on varying threshold values. ROC is a probability curve, and AUC measures separability.
In other words, the AUC-ROC metric will tell us about the model's ability to distinguish
between classes. The higher the AUC, the more accurate the model.

4|Page
o Performance Metrics for Regression

Mean Absolute Error (MAE)

Mean Square Error (MSE)

R Squared (R2)

Mean Absolute Error (MAE)

It is the most basic error metric in regression problems. It is the sum of the absolute
difference between the predicted and actual values. In other words, we can use MAE to
determine how inaccurate the predictions were. MAE does not indicate the model's
direction, i.e. no indication of underperformance or overperformance. MAE can be
calculated with the following formula:
𝑴𝑨𝑬 = 1𝑛∑|𝑌 − 𝑌^|

Mean Square Error (MSE)

MSE is equivalent to MAE, but instead of using the absolute value, it squares the difference
between actual and predicted output values before summing them all. The distinction can
be seen in the following equation
𝑴𝑺𝑬 = 𝟏𝒏∑(𝒀 − 𝒀^)

R Squared (R2)
The R Squared metric is frequently used for explanatory purposes, and it indicates the
goodness or fit of a set of predicted output values to the actual output values. The following
equation will assist us in comprehending it.
𝟏 ∑𝒏𝒊 = 𝟏(𝒀𝒊 − 𝒀𝒊^)𝟐
𝟐
𝑹 =𝟏− 𝒏
𝟏
𝒏 ∑𝒏𝒊 = 𝟏(𝒀𝒊 − 𝒀𝒊)𝟐¯

5|Page