DIAGNOSTIC
STUDY
Patarawan Woratanarat, MD, PhD
Research question
◦ Uncertainty about diagnostic test
◦ New test
◦ Cheaper, simple test
◦ Differentiate, classify severity
◦ Diagnostic threshold
◦ Treatment threshold
◦ Compare with a gold standard
Study design
◦ Cross-sectional study
◦ Cross-sectional study with randomization for test
◦ In case of sequence matter e.g. Arthroscope vs MRI
◦ Setting, location, dates
Population
◦ Reference population
◦ Study population
◦ Uncertainty of diagnosis e.g. groin pain with fever (septic hip vs.
transient synovitis of the hip)
◦ Only one population included whole spectrum of the disease
◦ Avoid patients vs normal control
moderate
Patient who were suspected to have disease
Test
Gold standard
Test
◦ Explain details
◦ Procedure: sufficient detail to allow replication
◦ Test performance
◦ Interpreter (if presents)
◦ Definition and rationale for positive, and negative results
◦ e.g. ESR > 40 mm/hr
◦ Emphasize
◦ No reader/interpreter of test known the results of gold standard (blind
comparison, independent reader)
Stard-statement.org, 2015
Question
CRP
ESR
Test (ESR) vs wbc
Gold standard
Gold standard
◦ Rationale for selection, acceptable?
◦ Explain details
◦ Procedure: sufficient detail to allow replication
◦ Gold standard performance
◦ Interpreter (if presents)
◦ Definition and rationale for positive, and negative results
◦ e.g. rim enhancement in MRI
◦ Emphasize
◦ No reader/interpreter of test known the results of the test (blind
comparison)
◦ Test should not be parts of gold standard
◦ Perform gold standard in all participants unless it is highly invasive, then
use silver standard
Verification bias
Gold
Positive
standard
Test
Negative
Example
◦ The diagnosis of primary bone tumor
◦ Test = MRI known results
◦ Goal standard
◦ Bone biopsy & pathology
◦ If no biopsy, use silver standard (x-ray) with clinical follow-up
Data collection
◦ Participant recruitment
◦ Specimen collection
◦ Logistic method for test and gold standard
◦ Data record form
Statistical analysis
◦ Sensitivity
◦ Specificity
◦ Positive predictive value
◦ Negative predictive value
◦ Accuracy
◦ Likelihood ratio of positive test
◦ Likelihood ratio of negative test
◦ Odds ratio with 95% confidence interval
◦ Area under receiver operating characteristic (ROC) curve
Exercises
Test results Septic arthritis Aseptic arthritis Total
Positive 7 (100) 5 (35.7) 12
Negative 0 (0) 9 (64.3) 9
Total 7 14 21
Sensitivity =
Specificity =
Positive predictive value =
Negative predictive value =
Accuracy =
Likelihood ratio of positive test =
Likelihood ratio of negative test =
Vanaprasert N, et al. J Med Assoc Thai 2018; 101:S35-S40.
Test results Septic arthritis Aseptic arthritis Total
Positive 7 (100) 5 (35.7) 12
Negative 0 (0) 9 (64.3) 9
Total 7 14 21
Diagnostic value Test, mean
(95% confidence interval)
Sensitivity (%) 100 (59-100)
Specificity (%) 64.3 (35.1-87.2)
PPV (%) 58.3 (27.7-84.8)
NPV (%) 100 (66.4-100)
Accuracy (%) 76.1 (52.8-91.8)
LR of positive test 2.8 (1.39-5.65)
LR of negative test 0
ROC area 0.82 (0.69-0.95)
Example printout from STATA
Pretest probability
◦ A 28-year-old man ◦ A 78-year-old woman
◦ High level of anxiety ◦ 10 days after THR
Probability of pulmonary embolism
Low Pretest probability High
Likelihood ratio LR of +ve results = 102/251 = 18.3
14/630
◦ PIOPED study: pulmonary embolism LR+ = T+D+/T+D-
= sense/(1-spec)
Ventilation-perfusion Angiogram
scan (high-prob) (gold standard)
positive negative
positive 102 14
negative 149 616
Total 251 630
Likelihood ratio LR of -ve results = 149/251 = 0.61
616/630
◦ PIOPED study: pulmonary embolism LR- = T-D+/T-D-
= (1-sense)/spec
Ventilation-perfusion Angiogram
scan (high-prob) (gold standard)
positive negative
positive 102 14
negative 149 616
Total 251 630
Likelihood ratio Fagan Nomogram
◦ LR+ = sensitivity / (1- specificity)
◦ LR - = (1-sensitivity) /specificity
Increase LR+ LR-
Minimal 1-2 0.5-1
Small 2-5 0.2-0.5
Moderate 5-10 0.1-0.2
Large > 10 < 0.1
Posttest probability Fagan Nomogram
◦ Odds(post) = Odds(pre) x LR
= (0.5/(1-0.5) x 18.3
= 18.3
◦ Posttest prob = odds(post)/(odds post+1)
= 18.3/(18.3+1)
= 94.8
SARS-CoV-2 Antigen Self Test Nasal Test characteristics (ROCHE)
Assay format Lateral flow test / immunochromatographic
Sensitivity 91.1% (Ct value ≤ 30)
Specificity 99.6%
Calculate LR+ = 0.911/(1-0.996) = 227.8
LR- = (1-0.911)/0.996 = 0.089
https://diagnostics.roche.com/global/en/products/params/sars-cov-2-antigen-self-test-nasal.html#productSpecs
Woratanarat, et al 1 August 2021. 1.33 pm.
Sample size calculation
. power oneproportion 0.8 0.7, continuity
◦ Based on
◦ Alpha error 0.05 Performing iteration ...
◦ The power of the study 0.8 Estimated sample size for a one-sample proportion test
Score z test
◦ Sensitivity Ho: p = p0 versus Ha: p != p0
Study parameters:
◦ Accuracy
alpha = 0.0500
power = 0.8000
◦ ROC curve 0.8 delta = -0.1000
p0 = 0.8000
pa = 0.7000
Estimated sample size:
N= 147
From STATA 15.0
Results
◦ Participants
◦ Flow diagram
◦ Demographic data included distribution of severity of disease
Excluded =
Results
Disease ROC
Sensitivity Specificity PPV NPV Accuracy LR+ LR- OR
Test area
Yes No (95%CI) (95%CI) (95%CI) (95%CI) (95%CI) (95%CI) (95%CI) (95%CI)
(95%CI)
(N = ) (N = )
Test 1 N (%) N (%))
Test 2 N (%) N (%)
Test 3 N (%) N (%)
Discussion
◦ What we found
◦ How they were different from the other studies
◦ Strength
◦ Limitation
◦ Variation of test
◦ Variation of interpretation
◦ Ability to diagnose
◦ Highly skilled
Discussion
◦ Clinical application
◦ Severe disease: LR move away from 1
◦ Mild disease: LR move toward 1
◦ Practice: setting
◦ Patients: meet inclusion & exclusion criteria
◦ The test threshold (stop testing)
◦ The treatment threshold (start treatment)
Example
Patients VP scan Posttest probability of
pulmonary embolism
A 28-year-old High prob. 82% Rx
man Intermediate prob. 23% + test
Low prob. 10% stop
A 78-year-old High prob. 97% Rx
woman Intermediate prob. 74% Rx /+ test
Discussion
◦ Sequential tests
◦ E.g. PE, Lab, x-ray
◦ Do not use related test e.g. ESR, CRP
◦ Clinical prediction rules (combination of tests, parallel test)
◦ Benefit vs. risk by
◦ Randomizing patient into tests
◦ FU to determine target outcomes
Summary of diagnostic study
◦ Methodology
◦ whole spectrum, blind, independence
◦ Results: LR
◦ Discussion
◦ reproducibility, applicability, changed strategy, benefit/risk
QUESTIONS?
