Scribd
Upload
10 views7 pages

Confusion Matrix

A confusion matrix is a tool used to evaluate the performance of a classification model by summarizing correct and incorrect predictions. Key metrics derived from the confusion matrix include Precision, Recall, F1 Score, Accuracy, and Specificity, which help assess the model's effectiveness in distinguishing between classes. Understanding these metrics is crucial, especially in cases of imbalanced datasets, to ensure reliable classification results.

Uploaded by

Akash Verma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views7 pages

Confusion Matrix

A confusion matrix is a tool used to evaluate the performance of a classification model by summarizing correct and incorrect predictions. Key metrics derived from the confusion matrix include Precision, Recall, F1 Score, Accuracy, and Specificity, which help assess the model's effectiveness in distinguishing between classes. Understanding these metrics is crucial, especially in cases of imbalanced datasets, to ensure reliable classification results.

Uploaded by

Akash Verma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Object 1

3
2

What is confusion matrix?


Confusion matrix is used to tell, how good the classification model got trained…simply, to
describe the performance of classification model.
Terminologies to know : True = Cat, False = Dog
True Positive : When you have Cat, Your model predicted as Cat.
True Negative : When you have Dog, Your model predicted as Dog.
False Positive : When you have Dog, Your model predicted as Cat.
False Negative : When you have Cat, Your model predicted as Dog.
TP = Actual is True,Predicted True
TN = Actual False, Predicted False
FP = Actual False, Predicted True
FN = Actual True, Predicted False
Precision : correctly classified positive / total predicted positive

Recall/Sensitivity : Correctly Classified Positive / Total Positive

F1 Score : It is the balance between Precision and Recall.

Accuracy : total number of two correct predictions (TP + TN) divided by the total number of a
dataset (P + N).
Specificity : correct negative prediction / Total negatives

1. Confusion Matrix:
A confusion matrix provides a easy summary of the predictive results in a classification problem.
Correct and incorrect predictions are summarized in a table with their values and broken down by
each class.

We can not rely on a single value of accuracy in classification when the classes are imbalanced. For
example, we have a dataset of 100 patients in which 5 have diabetes and 95 are healthy. However, if
our model only predicts the majority class i.e. all 100 people are healthy even though we have a
classification accuracy of 95%. Therefore, we need a confusion matrix.

2. Calculate a confusion matrix:


Let’s take an example:
We have a total of 10 cats and dogs and our model predicts whether it is a cat or not.
Actual values = [‘dog’, ‘cat’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘cat’, ‘dog’]
Predicted values = [‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘cat’, ‘cat’, ‘cat’]
Remember, we describe
predicted values as Positive/Negative and actual values as True/False.

Definition of the Terms:


True Positive: You predicted positive and it’s true. You predicted that an animal is a cat and it
actually is.
True Negative: You predicted negative and it’s true. You predicted that animal is not a cat and it
actually is not (it’s a dog).

False Positive (Type 1 Error): You predicted positive and it’s false. You predicted that animal is a
cat but it actually is not (it’s a dog).
False Negative (Type 2 Error): You predicted negative and it’s false. You predicted that animal is
not a cat but it actually is.

Classification Accuracy:
Classification Accuracy is given by the relation:

Recall (aka Sensitivity):


Recall is defined as the ratio of the total number of correctly classified positive classes divide by the
total number of positive classes. Or, out of all the positive classes, how much we have predicted
correctly. Recall should be high.
Precision:
Precision is defined as the ratio of the total number of correctly classified positive classes divided
by the total number of predicted positive classes. Or, out of all the predictive positive classes, how
much we predicted correctly. Precision should be high.

Trick to remember: Precision has Predictive Results in the denominator.

F-score or F1-score:
It is difficult to compare two models with different Precision and Recall. So to make them
comparable, we use F-Score. It is the Harmonic Mean of Precision and Recall. As compared to
Arithmetic Mean, Harmonic Mean punishes the extreme values more. F-score should be high.

Specificity:
Specificity determines the proportion of actual negatives that are correctly identified.

Example to interpret confusion matrix:


Let’s calculate confusion matrix using above cat and dog example:
Classification Accuracy:
Accuracy = (TP + TN) / (TP + TN + FP + FN) = (3+4)/(3+4+2+1) = 0.70
Recall: Recall gives us an idea about when it’s actually yes, how often does it predict yes.
Recall = TP / (TP + FN) = 3/(3+1) = 0.75
Precision: Precision tells us about when it predicts yes, how often is it correct.
Precision = TP / (TP + FP) = 3/(3+2) = 0.60
F-score:
F-score = (2*Recall*Precision)/(Recall+Precision) = (2*0.75*0.60)/(0.75+0.60) = 0.67
Specificity:
Specificity = TN / (TN + FP) = 4/(4+2) = 0.6
4. Summary:
• Precision is how certain you are of your true positives. Recall is how certain you are that
you are not missing any positives.
• Choose Recall if the occurrence of false negatives is unaccepted/intolerable. For example,
in the case of diabetes that you would rather have some extra false positives (false alarms)
over saving some false negatives.
• Choose Precision if you want to be more confident of your true positives. For example, in
case of spam emails, you would rather have some spam emails in your inbox rather than
some regular emails in your spam box. You would like to be extra sure that email X is spam
before we put it in the spam box.
• Choose Specificity if you want to cover all true negatives, i.e. meaning we do not want any
false alarms or false positives. For example, in case of a drug test in which all people who
test positive will immediately go to jail, you would not want anyone drug-free going to jail.
We can conclude that:
• Accuracy value of 70% means that identification of 3 of every 10 cats is incorrect, and 7 is
correct.
• Precision value of 60% means that label of 4 of every 10 cats is a not a cat (i.e. a dog), and 6
are cats.
• Recall value is 70% means that 3 of every 10 cats, in reality, are missed by our model and 7
are correctly identified as cats.
• Specificity value is 60% means that 4 of every 10 dogs (i.e. not cat) in reality are miss-
labeled as cats and 6 are correctly labeled as dogs.

You might also like