Receiver Operating Characteristic
Curve: Overview
Compiled and Prepared by
Umesh Raj Aryal
School of Public Health
Karnali Academy of Health Sciences
Jumla, Nepal
The educational materials used in this presentation were collected from different published sources
including ChartGPT, hence, I would like to appreciate and acknowledge all those resources.
Decision Matrix
a c
b d
sensitivity = a / (a + b) : The probability of getting test positive for those who have diseases positive
specificity = d / (c + d) : The probability of getting test negative for those who have no diseases
Predictive Positive Value= a/a+c: The probability of getting disease positive for those who have tested positive
Negative Predictive Value=d/b+d: The probability of getting the disease negative for those who have tested negative
false negative = b / (a + b): The probability of getting the test negative for those who have disease-positive
false positive = c / (c + d) : The probability of getting the test positive for those who have disease-negative
accuracy = (a + d) / (a + b + c + d)
Example: Prostate-Specific Antigen (PSA) Density and Prostate Cancer
Monaghan TF, Rahman SN, Agudelo CW, Wein AJ, Lazar JM, Everaert K, et al. Foundational Statistical Principles
in Medical Research: Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value. Medicina
(Kaunas). 2021 May 16;57(5):503. doi: 10.3390/medicina57050503
Relationship between Sensitivity, Specificity , Positive Predictive
Value (PPV,) and Negative Predictive Value (NPV)
Sensitivity and Specificity
are intrinsic properties of a test, meaning they do not change based on the prevalence of the
condition in the population.
Positive Predictive Value (PPV) and Negative Predictive Value (NPV)
are affected by the prevalence of the disease in the population:
•If the disease is rare (low prevalence), even a test with high sensitivity and specificity will have a
lower PPV because there could be more false positives than true positives..
•Conversely, when the disease is common, PPV increases, but
NPV decreases because the test is more likely to identify true positives correctly but might miss true
negatives
Relationship between Sensitivity, Specificity , Positive Predictive
Value (PPV,) and Negative Predictive Value (NPV)
Higher prevalence leads to a higher PPV (more true positives relative to false
positives), but a lower NPV (more false negatives relative to true negatives).
Lower prevalence leads to a higher NPV (more true negatives relative to
false negatives), but a lower PPV (more false positives relative to true
positives).
Key Messages
When prevalence is high, PPV increases, meaning positive results are more likely to be correct.
When prevalence is low, NPV increases, meaning negative results are more likely to be correct.
Receiver Operating Characteristics Curve
It illustrates the diagnostic ability of a binary classifier system as its
discrimination threshold is varied.
The curve is plotted with two metrics:
True Positive Rate (Sensitivity) on the y-axis.
False Positive Rate (1 - Specificity) on the x-axis.
The ROC curve helps to visualize the trade-off between sensitivity and
specificity at various threshold settings of the test
Key Terms
•True Positive Rate (Sensitivity):
•The proportion of actual positives correctly identified.
•False Positive Rate (1 - Specificity):
•The proportion of actual negatives incorrectly identified as positives.
•Threshold:
•The cut-off point where the test classifies a result as positive or negative.
•Different thresholds lead to different true and false positive rates.
A random guess would result
in a diagonal line from the
bottom-left to the top-right,
indicating an AUC of 0.5. The
closer the curve is to the top-
left corner, the better the
model’s performance.
If you flip a coin to decide whether a patient
has a disease or not , that’s a random guess,
since the decision has no relation to any
medical data or symptoms.
Source : https://medium.com/@ilyurek/roc-curve-and-auc-evaluating-model-performance-c2178008b02
A ROC curve connects coordinate
points with 1 - specificity (= false
positive rate) as the x-axis and
sensitivity as the y-axis at all cut-off
values measured from the test
results. When a strict cut-off point
(reference) value is applied, the
point on the curve moves downward
and to the left (Point A). When a
loose cut-off point value is applied,
the point moves upward and to the
right (Point B). The 45° diagonal line
serves as the reference line, since it
is the ROC curve of random
classification.
Source: Nahm FS. Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol.
2022 Feb;75(1):25-36. doi: 10.4097/kja.21209. Epub 2022 Jan 18.
How to Read an ROC Curve?
A point closer to the top-left corner represents a better classifier.
This means high sensitivity (few false negatives) and low false positive
rates (high specificity).
A diagonal line from the bottom-left to the top-right represents a classifier
that performs no better than random chance
The Area Under the Curve (AUC) is a numerical summary of the ROC
curve.
An AUC of 1 indicates perfect classification, while an AUC of 0.5
indicates random guessing.
Interpretation of AUC
For a diagnostic test to be meaningful, the AUC must be greater than 0.5. Generally, an
AUC ≥ 0.8 is considered acceptable.
Source: Nahm FS. Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol.
2022 Feb;75(1):25-36. doi: 10.4097/kja.21209. Epub 2022 Jan 18.
If the entire ROC curve lies below the diagonal line,
this indicates that the distribution of the biomarker is
opposite of the usual convention (e.g., lower values of
the biomarker are associated with diseased patients)
Try to Interpret Biomarker vs Prostate Cancer
Source: https://cancerprevention.qmul.ac.uk/index.php/using-roc-curves-to-identify-the-most-accurate-test/
Thank You
Thank you
