VALIDATION OF LABORATORY TEST
DEPARTMENT OF MICROBIOLOGY AND IMMUNOLOGY
Presenter: Daniel Mwandu
Facilitator: Dr. Doreen Komori
Date: 22th Feb, 2024
Venue: Microbiology Lab
Presentation Outline
• Introduction
• Important of methods validation
• Pre- validation
• Methods validation parameters
• Post validation monitoring
INTRODUCTION
• Validation is the process of testing a
measurement procedures to asses it performance
if inline with acceptance criteria.
• Laboratory method validation is a process by
which a method is determined to meet the
requirement for the intended use.
• It ensure accuracy, precision and reliable results
• It help to determine the magnitude of error.
• It ensure good customers services
ISO 15189:2022 , Clause 7.3.3 The laboratory
shall validate the following
• Non standard methods
• Laboratory designed/ developed methods
• Standard methods used outside their intended.
• Validated methods subsequently modified
Reason for establishment of new methods
• Clinical need of new analyte
• Improve diagnosis and treatment due to
short TAT.
• Improve accuracy and precision
• Reduce cost
• Clause 6.4.3. state that “ the laboratory shall
verify the equipment's/methods to conforms the
specified acceptability criteria.
• Clause 6.4.4. state that “each equipment/ test
method should be used as specified by
manufacturer, unless validated by laboratory.
• Clause 7.3.2.verification
• Clause 7.3.3 validation
3/4/2024 8
Performance characteristics
VALIDATION
Performed for inhouse/modified
methods
It normally it done by manufacturer
to establish the acceptance criteria
VERIFICATION
Confirm the performance claims
by manufacturer
But I’m using an FDA approved kit
3/4/2024 9
FDA
APPROVED
Conduct verification only
Perform validation if some of the
parameters were not validated
Non -APPROVED
Conduct validation and
verification
Tests to validate
• Waived test(simple test approved by FDA) i.e.
UPT – No need of validation
• Non-waived test(complex test) validation is
needed i.e. Stain techniques.
• Unmodified FDA approved- no need of validation
• Modified /non-FDA approved need validation
Important of methods validation
• It ensure accuracy and reliable results
• Understanding of uncertainty in the
methods.
• Provide confidence in the method
through measuring robustness and
ruggedness
Pre- validation
Issue to consider prior validation.
Methods purpose and scope
Regulatory guidelines eg SOPs
Method development
Parameter acceptance criteria
A calibrated equipment to be validated.
Materials: eg reagents, reference standard
Methods validation parameters
Measurement
type
Description
01 Trueness Reference intervals
02 Accuracy measure the closeness of the test results to the
true value
03 Precision Repeatability, measure degree of agreement
04 Linearity
(Reportable
range)
Ability to produce results, that are directly
proportional to the analyte concentration within a
specified range
05 Uncertainty Measure Standard deviation.
06 Sensitivity Detection
07 Specificity Ensure selectivity, without interference
Methods validation parameters
Measurement
type
Formula
01 Trueness Reference interval
02 Accuracy (TP+TN )/ (TP+TN+FP +FN)x100%
03 Precision (TP )/ (TP + FP)x100%
04 Linearity Y= mX + C.
05 Uncertainty(MoU) CV=SD/x
06 Sensitivity TP/(TP+FN) X100%
07 Specificity TN/(TN+FP) X100%
Trueness, Accuracy, and precision
Precision
• To perform precision test is running 20 of one
sample depending on your schedule,
• Analysis and interpret statics measure including
mean, SD,CV, 2SD range and SD goal
• The precision test will pass if the calculated SD
is below the SD goal
Accuracy
• It need 40 different samples
• Sample are tested in comparison of new method
with existing accuracy method(gold standard)
 Measure the statics(Linearity) includes
 Correlation coefficient(R) greater 0.9
 Slope for 95% CI is 1.000
 Intercept for 95% CI is 0.000
Reportable range/linearity
• Assess the lowest and highest reliable results,
• Used especially when there is two calibrators
• Run 5 sample with known concertation in
triplicate
• Assess Linearity statics, then point out error
rate, and evaluate if allowable
• Error allowable is are proximity to reportable
range
Sensitivity and Specificity
• Done by comparing new method with gold
standard method.
• It involved running greater 100 sample in each
methods.
• Obtained results should be tabulated in two way
table
• Then, sensitivity and specificity will be calculated
Examples
• A validation of a new rapid syphilis test kit, YYY was
conducted at your laboratory. 500 specimen known to
positive were tested and 479 results came out positive
and others negative. A total of 506 known negative
sample were tested and a total of 490 results were
negative and others were positive.
• The acceptability criteria for both Diagnostic sensitivity
and diagnostic specificity is set at 98%.
• Use a contingency table to calculate the validation
results.
• What would be your decision after obtaining the
validation results?
GOLD STD TEST
NEW TEST POSITIVE NEGATIVE TOTAL
POSITIVE 479 16 495
NEGATIVE 21 490 511
TOTAL 500 506 1006
• Sensitivity=TP/D+X100%=95.8%
• Specificity=TN/D-X100%=3.2%
MoU
• It a systematically assess the factors influencing
the measurement
• It measure the actually parameter that should be
included with any measurement results obtained
• It measure existing doubt(degree from margin)
• It tell us about the quality of testing methods
• Errors are not the same as uncertainty
Post validation monitoring
• To provide validation report which should
includes performance specifications, results
obtained, and final statement(performance was
achieved or not)
• Proficiency Testing eg EQA
• System Performance Monitoring eg monitoring
of deviation like LJ charts.
summary
References
• ISO 15189:2022
• Analytical performance in accordance with CLSI guidelines,
Update of 2017.
• ICH Q2(R2) Validation Of Analytical Procedures Scientific
Guideline, December 2023.
• Published Articles.

Validation of laboratory tests with important guidelines

  • 1.
    VALIDATION OF LABORATORYTEST DEPARTMENT OF MICROBIOLOGY AND IMMUNOLOGY Presenter: Daniel Mwandu Facilitator: Dr. Doreen Komori Date: 22th Feb, 2024 Venue: Microbiology Lab
  • 2.
    Presentation Outline • Introduction •Important of methods validation • Pre- validation • Methods validation parameters • Post validation monitoring
  • 3.
    INTRODUCTION • Validation isthe process of testing a measurement procedures to asses it performance if inline with acceptance criteria. • Laboratory method validation is a process by which a method is determined to meet the requirement for the intended use. • It ensure accuracy, precision and reliable results • It help to determine the magnitude of error. • It ensure good customers services
  • 5.
    ISO 15189:2022 ,Clause 7.3.3 The laboratory shall validate the following • Non standard methods • Laboratory designed/ developed methods • Standard methods used outside their intended. • Validated methods subsequently modified
  • 6.
    Reason for establishmentof new methods • Clinical need of new analyte • Improve diagnosis and treatment due to short TAT. • Improve accuracy and precision • Reduce cost
  • 7.
    • Clause 6.4.3.state that “ the laboratory shall verify the equipment's/methods to conforms the specified acceptability criteria. • Clause 6.4.4. state that “each equipment/ test method should be used as specified by manufacturer, unless validated by laboratory. • Clause 7.3.2.verification • Clause 7.3.3 validation
  • 8.
    3/4/2024 8 Performance characteristics VALIDATION Performedfor inhouse/modified methods It normally it done by manufacturer to establish the acceptance criteria VERIFICATION Confirm the performance claims by manufacturer
  • 9.
    But I’m usingan FDA approved kit 3/4/2024 9 FDA APPROVED Conduct verification only Perform validation if some of the parameters were not validated Non -APPROVED Conduct validation and verification
  • 10.
    Tests to validate •Waived test(simple test approved by FDA) i.e. UPT – No need of validation • Non-waived test(complex test) validation is needed i.e. Stain techniques. • Unmodified FDA approved- no need of validation • Modified /non-FDA approved need validation
  • 11.
    Important of methodsvalidation • It ensure accuracy and reliable results • Understanding of uncertainty in the methods. • Provide confidence in the method through measuring robustness and ruggedness
  • 12.
    Pre- validation Issue toconsider prior validation. Methods purpose and scope Regulatory guidelines eg SOPs Method development Parameter acceptance criteria A calibrated equipment to be validated. Materials: eg reagents, reference standard
  • 13.
    Methods validation parameters Measurement type Description 01Trueness Reference intervals 02 Accuracy measure the closeness of the test results to the true value 03 Precision Repeatability, measure degree of agreement 04 Linearity (Reportable range) Ability to produce results, that are directly proportional to the analyte concentration within a specified range 05 Uncertainty Measure Standard deviation. 06 Sensitivity Detection 07 Specificity Ensure selectivity, without interference
  • 14.
    Methods validation parameters Measurement type Formula 01Trueness Reference interval 02 Accuracy (TP+TN )/ (TP+TN+FP +FN)x100% 03 Precision (TP )/ (TP + FP)x100% 04 Linearity Y= mX + C. 05 Uncertainty(MoU) CV=SD/x 06 Sensitivity TP/(TP+FN) X100% 07 Specificity TN/(TN+FP) X100%
  • 15.
  • 16.
    Precision • To performprecision test is running 20 of one sample depending on your schedule, • Analysis and interpret statics measure including mean, SD,CV, 2SD range and SD goal • The precision test will pass if the calculated SD is below the SD goal
  • 17.
    Accuracy • It need40 different samples • Sample are tested in comparison of new method with existing accuracy method(gold standard)  Measure the statics(Linearity) includes  Correlation coefficient(R) greater 0.9  Slope for 95% CI is 1.000  Intercept for 95% CI is 0.000
  • 18.
    Reportable range/linearity • Assessthe lowest and highest reliable results, • Used especially when there is two calibrators • Run 5 sample with known concertation in triplicate • Assess Linearity statics, then point out error rate, and evaluate if allowable • Error allowable is are proximity to reportable range
  • 19.
    Sensitivity and Specificity •Done by comparing new method with gold standard method. • It involved running greater 100 sample in each methods. • Obtained results should be tabulated in two way table • Then, sensitivity and specificity will be calculated
  • 20.
    Examples • A validationof a new rapid syphilis test kit, YYY was conducted at your laboratory. 500 specimen known to positive were tested and 479 results came out positive and others negative. A total of 506 known negative sample were tested and a total of 490 results were negative and others were positive. • The acceptability criteria for both Diagnostic sensitivity and diagnostic specificity is set at 98%. • Use a contingency table to calculate the validation results. • What would be your decision after obtaining the validation results?
  • 21.
    GOLD STD TEST NEWTEST POSITIVE NEGATIVE TOTAL POSITIVE 479 16 495 NEGATIVE 21 490 511 TOTAL 500 506 1006
  • 22.
  • 23.
    MoU • It asystematically assess the factors influencing the measurement • It measure the actually parameter that should be included with any measurement results obtained • It measure existing doubt(degree from margin) • It tell us about the quality of testing methods • Errors are not the same as uncertainty
  • 24.
    Post validation monitoring •To provide validation report which should includes performance specifications, results obtained, and final statement(performance was achieved or not) • Proficiency Testing eg EQA • System Performance Monitoring eg monitoring of deviation like LJ charts.
  • 25.
  • 26.
    References • ISO 15189:2022 •Analytical performance in accordance with CLSI guidelines, Update of 2017. • ICH Q2(R2) Validation Of Analytical Procedures Scientific Guideline, December 2023. • Published Articles.