Scribd
Upload
108 views18 pages

Module 4 - Confusion Matrix-1

This document discusses confusion matrices, which are used to evaluate classification models. A confusion matrix displays the number of true positives, true negatives, false positives, and false negatives. It allows calculating performance metrics like accuracy, precision, recall, and F1-score. The document provides an example of a binary confusion matrix and explains how confusion matrices work for multi-class classification problems.

Uploaded by

tcr19me074
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
108 views18 pages

Module 4 - Confusion Matrix-1

This document discusses confusion matrices, which are used to evaluate classification models. A confusion matrix displays the number of true positives, true negatives, false positives, and false negatives. It allows calculating performance metrics like accuracy, precision, recall, and F1-score. The document provides an example of a binary confusion matrix and explains how confusion matrices work for multi-class classification problems.

Uploaded by

tcr19me074
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

MODULE 4

CONFUSION MATRIX
Introduction
• In machine Learning, Classification is the process of categorizing a
given set of data into different categories. In Machine Learning, To
measure the performance of the classification model we use the
confusion matrix.
• A confusion matrix is a matrix that summarizes the performance of a
machine learning model on a set of test data.
• It is often used to measure the performance of classification models,
which aim to predict a categorical label for each input instance.
• The matrix displays the number of true positives (TP), true negatives
(TN), false positives (FP), and false negatives (FN) produced by the
model on the test data.

• For binary classification, the matrix will be of a 2X2 table, For multi-
class classification, the matrix shape will be equal to the number of
classes i.e for n classes it will be nXn.
• A 2X2 Confusion matrix is shown below for the image
recognization having a Dog image or Not Dog image.
Actual
Dog Not Dog
True Positive False Positive
Dog (TP) (FP)
Predicted
False Negative True Negative
Not Dog (FN) (TN)
• True Positive (TP): It is the total counts having both predicted and
actual values are Dog.
• True Negative (TN): It is the total counts having both predicted and
actual values are Not Dog.
• False Positive (FP): It is the total counts having prediction is Dog while
actually Not Dog.
• False Negative (FN): It is the total counts having prediction is Not Dog
while actually, it is Dog.
EXAMPLE
Inde
x 1 2 3 4 5 6 7 8 9 10

Actu Not Not Not Not


al Dog Dog Dog Dog Dog Dog Dog Dog Dog Dog
Pred
icte Dog Not Dog Not Dog Dog Dog Dog Not Not
d Dog Dog Dog Dog

Res
ult TP FN TP TN TP FP TP TP TN TN

Actual Dog Counts = 6


Actual Not Dog Counts = 4
True Positive Counts = 5
False Positive Counts = 1
True Negative Counts = 3
False Negative Counts = 1
CONFUSION MATRIX

Actual
Dog Not Dog
True Positive False Positive
Dog (TP =5) (FP=1)
Predicted
False Negative True Negative
Not Dog (FN =1) (TN=3)
Performance metrics
• ACCURACY: Accuracy is used to measure the performance of the
model. It is the ratio of Total correct instances to the total instances.

For the above case:


Accuracy = (5+3)/(5+3+1+1) = 8/10 = 0.8
Precision
• Precision is a measure of how accurate a model’s positive predictions
are. It is defined as the ratio of true positive predictions to the total
number of positive predictions made by the model.

For the above case:

• Precision = 5/(5+1) =5/6 = 0.8333


RECALL
• Recall measures the effectiveness of a classification model in
identifying all relevant instances from a dataset. It is the ratio of the
number of true positive (TP) instances to the sum of true positive and
false negative (FN) instances.

• For the above case:


Recall = 5/(5+1) =5/6 = 0.8333
F1-SCORE
• F1-score is used to evaluate the overall performance of a classification
model. It is the harmonic mean of precision and recall,

For the above case:


• F1-Score: = (2* 0.8333* 0.8333)/( 0.8333+ 0.8333) = 0.8333
Specificity
• The number of samples predicted correctly to be in the negative class
out of all the samples in the dataset that actually belong to the negative
class.
Confusion Matrix for Multiple Classes

• Let us elaborate on the features of the multi-class confusion matrix


with an example. Suppose we have the test set (consisting of 191 total
samples) of a dataset with the following distribution:
• The confusion matrix obtained by training a classifier and evaluating
the trained model on this test set is shown below
• As usual, the diagonal elements are the correctly predicted samples. A total of 145 samples were correctly predicted out of the
total 191 samples. Thus, the overall accuracy is 75.92%.
• The confusion matrix can be converted into a one-vs-all type matrix (binary-class confusion matrix) for calculating class-wise
metrics like accuracy, precision, recall, etc.
• Converting the matrix to a one-vs-all matrix for class-1 of the data looks like as shown below. Here, the positive class refers to
class-1, and the negative class refers to “NOT class-1”. Now, the formulae for the binary-class confusion matrices can be used for
calculating the class-wise metrics.
• Similarly, for class-2, the converted one-vs-all confusion matrix will
look like the following:
Classwork
• We have a total of 20 cats and dogs and our model predicts whether it
is a cat or not.
• Actual values = [‘dog’, ‘cat’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’,
‘cat’, ‘dog’, ‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’,
‘dog’, ‘cat’]
Predicted values = [‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’,
‘cat’, ‘cat’, ‘cat’, ‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’,
‘dog’, ‘cat’]
• Construct confusion matrix and calculate the performance metrics.

You might also like