Quality Control

IT IS A BASIC SAFETY PRACTICE TO
MONITOR ANALYTICAL QUALITY OF
MEASUREMENT.

To detect changes from stable day to day
operation and eliminate reporting of results with
medically important errors.`
OBJECTIVES
1.To determine how to establish the
  analytical goal and quality control
  scheme /schedule in your lab
2.To identify the Quality control
  charts and quality control roles
3.To identify the new rules of west
  guard.
4. Overview of the CLSI guide lines
  C24.
                                        2
FIRST OBJECTIVE


 To determine how to establish the
analytical goal and quality control !
  scheme /schedule in your lab




                                        3
Data we needed to design Quality
            Control

1.Commercial QC material

2.Manufacturers’ kit inserts .

3. Doing it for yourself is, as usual,
 the safest method.

4.EQAS programs .
                                         4
Designing the internal quality control
                  protocol
Define the level of quality that the laboratory wants to.1
 attain for a determined test (the analytical quality
(.specification

.know the stable analytical performance for this test . 2

a control rule (control limits and number of controls per . 3
( .run

:Assure quality of results. 4
 a. analytical imprecision and bias
 b. EQAS


                                                                5
Proficiency testing
# Inter laboratory Comparison Programs Help
 Improve Laboratory Performance.

#
Define Analytical Goal


  Two main strategies for analytical
  quality specifications based on
.calculation of imprecision and bias




                                       7
?How to establish your analytical goal

The underlying principle of ‘measurement*
uncertainty’ is that a laboratory should know how
.precisely they can measure any particular analyte



Two main strategies for analytical quality*
specifications based on biology have been
evaluated for imprecision and bias (in combination
with imprecision), respectively




                                                     8
Challenges to set analytical Goal
                       External                            Internal

Permenant
            External                                       Internal
                       :Method                             :Implementation
Permenant

            :Method
                       Analytical principle                Choice of conditions
                                                           :Implementation

            Analytical Equipment
                       principle                           In house' equipment'
                                                           Choice of conditions

                    Reagents
            Equipment                                       ,In house' reagents'
                                                           In house' equipment'

            Reagents)choice      of producer(              ,.Time, Temperature,
                                                            In house' reagents'    Volume, etc

            )choice of:producer(
                       Standardization                     .:StandardizationVolume, etc
                                                            Time, Temperature,

            :Standardization                               :Standardization

            Traceability of calibration of
                      Traceability           calibration   Calibration function
                                                           Calibration function

Variable
Variable    :Batches   :Batches                            ::Performance
                                                            Performance

            )Reagents Reagents
                    ) (variability   (variability           in house' reagents'
                                                           in house' reagents'

                      Calibrators
            Calibrators                                    ..Training, Maintenance,
                                                            Training, Maintenance, etc   etc

                   Consumables
            Consumables                                    ''Control with 'trouble-shooting
                                                            Control with 'trouble-shooting
                                                                                                 9
                                                           Documentation
                                                           Documentation
10
11
Relevance to customers

All these
complicated
processes to
provide the
appropriate
quality for our
patient cares




                                      12
Setting Goals For Q.C
              Performance
1.Maximum allowable number of unacceptable
results, due to an out of control error conditions.
2.Maximum allowable probability of reporting
unacceptable results.
3.Minmum acceptable probability of detecting an
out of control error condition.
4.Maximum acceptable probability of false
rejection.
  *Main aim is to Maximize probability to detect
an out of control condition for measurement
procedure , while minimize probability for false
Q.C alerts.
                                                      13
Analysis of Control Materials
A stable control which mimics
 patient’s sample is analyzed (DAY
 TO DAY OR SET TO SET(

Need data set of at least 20 points,
 obtained over a 30 day period

Calculate mean, standard
 deviation, coefficient of variation;
 determine target ranges
CLIA proficiency testing criteria for acceptable
analytical performance:
Objective

2.To identify the Quality control
  charts and quality control roles




3.To identify the new rules of west
  guard.
                                      21
Monitoring QC Data
Develop Levey-Jennings chart.

Plot control values each run, make
 decision regarding acceptability of run.

Monitor over time to evaluate the
 precision and accuracy of repeated
 measurements

Review charts at defined intervals, take
 necessary action, and document
Levey-Jennings Chart
A graphical method for displaying
 control results and evaluating whether
 a procedure is in-control or out-of-
 control

Control values are plotted versus time

Lines are drawn from point to point to
 accent any trends, shifts, or random
 excursions
Levey-Jennings Chart




The Levey-Jennings chart usually has the days of the month plotted on
the X-axis and the control observations plotted on the Y-axis.
On the right is the Gaussian or "bell-shaped" curve turned on its side to
show the correlation of the curve to the chart (ie, fewer data points
should appear on the upper and lower extremities of the chart, since the
"bell" is thinner farther from the mean).
By observing the data plotted in the L-J chart, we can determine if test
results are in control and accurate, or if test results are not in control
and consequently unacceptable.                                             24
.
Levey-Jennings Chart
           Calculate the Mean and Standard Deviation;
           Record the Mean and +/- 1,2 and 3 SD Control
           Limits
3SD+115
2SD+ 110
1SD+ 105
Mean 100
1SD- 95
2SD- 90
3SD- 85
      80
           1   2   3   4   5   6   7   8   9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

                                                          Day
Levey-Jennings Chart -
             Record Time on X-Axis and the Control Values on Y-
             Axis
                                 115
                                 110
   Control Values (e.g. mg/dL)



                                 105
                                 100
                                  95
                                  90
                                  85
                                  80
                                       1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24


                                            (Time (e.g. day, date, run number
Levey-Jennings Chart -
Plot Control Values for Each Run


                              115
                              110
Control Values (e.g. mg/dL)



                              105
                              100
                               95
                               90
                               85
                               80
                                    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

                                         (Time (e.g. day, date, run number
Levey-Jennings Chart -
 Record and Evaluate the Control Values

3SD+ 115
2SD+ 110

1SD+ 105

Mean 100
       95
1SD-
2SD-   90

3SD-   85

       80                                  Day
            1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Look for
                                                     assignable
         In control              Out of control !      cause !


  UCL                                                       Problem
                                Natural                    corrected
                      6σ        Variation
Target
= Mean
                           3σ
  LCL

                                                    Time
   Samples             Natural variation
“Allows determination of whether
 an analytical run is “in-control” or
 “out-of-control”
Westgard Rules are a multi role QC
               procedure

1-West-gard rules: (Regular twice entry for Q.C)
 3s, 1 2s, 2 2s , R 4s, 4 1s, 10x (& modifications 1
                      ) 8x, 12x

Recent west-gard rules, fit better and are easier-2
to apply in situations where 3 different control
:materials are being analyzed
                2of3 2s, 3 1s, 6x & 9x
A related control rule that is sometimes used-3
":looks for a "trend
 7T

                                                       31
Westgard Rules are a multirule QC procedure


    13s refers to a control rule that is commonly used
    . with a Levey Jennings chart
    A run is rejected when a single control
    measurement exceeds the mean plus 3s or the
    .mean minus 3s control limit




                                                         32
12s refers to the control rule that is commonly used with a
Levey-Jennings chart single control measurement exceeds
. the mean plus 2s or the mean minus 2s control limit
 This rule is used as a warning rule to trigger careful
inspection of the control data by the following rejection rules .




                                                                    33
22s - reject when 2 consecutive control
measurements exceed the same mean plus 2s
. or the same mean minus 2s control limit




                                            34
R4s - reject when 1 control measurement in a group exceeds the
. mean plus 2s and another exceeds the mean minus 2s




                                                                 35
41s - reject when 4 consecutive control
measurements exceed the same mean plus 1s
. or the same mean minus 1s control limit




                                            36
10x - reject when 10 consecutive control measurements fall
. on one side of the mean

 :some modifications
8x - reject when 8 consecutive control measurements fall on
 one side of the mean
12x - reject when 12 consecutive control measurements fall
. on one side of the mean




                                                              37
In situations where 3 different control materials are
being analyzed, some other control rules fit better and
:are easier to apply, such as
2of32s - reject when 2 out of 3 control measurements
exceed the same mean plus 2s or mean minus 2s control
limit


                       ;




                                                          38
31s - reject when 3 consecutive control
measurements exceed the same
mean plus 1s or mean minus 1s
.control limit




                                          39
some modification:
6x - reject when 6 consecutive control
measurements fall on one side of the
mean.




                                         40
A related control rule that is sometimes used,
looks for a "trend" where several control
measurements in a row are increasing or
 decreasing
7T - reject when seven control measurements
trend in the same direction, i.e., get progressively
higher or progressively lower.
there are two types of errors, random and
 systematic
the multirule combines the use of two types of
rules to help detect those two types of errors.




                                                       41
Corrective action

1- Determine the type of error occurring on
the basis of the rule violated.

2-Refer to trouble-shooting guides to identify
possible causes for the type of error indicated
by the control rule that was violated.




                                                  42
There are two types of errors, random and systematic
the multi rule combines the use of two types of rules to help
detect those two types of errors:




           Type of Error             Control rule that detects it

          Random error                                       13s, R4s


        Systematic error   2s, 4 1s, 2of3 2s, 3 1s, 6x, 8x, 9x, 10x, 2
                                                          12x, cusum




                                                                         43
44
45
46
47
Correct the problem, then analyze control- 3
.samples again to assess control status
4- Repeat or verify the results on the patient
samples once the method has been demonstrated
to be in-control.
5- Consult a supervisor for any decision to report
patient results when a run is out-of-control.
                                                     48
OBJECTIVE

Overview   of the CLSI guide lines
                .C24




                                     49
?Who is CLSI

Clinical and Laboratory Standards Institute
ANSI-accredited, global, nonprofit standards•
development organization
CLSI has over 2,000 members – organizations such•
as IVD
manufacturers, hospital laboratories, reference
,laboratories, universities
professional associations, and government agencies




                                                     50
CLSI –C24:Q.C planning process

Define Quality requirement inn the form of.1
.Allowable total error
.Select suitable Q.C material.2
Obtain estimates of methods of impersion and bias.3
.Identify traditional control rules.4
Predict performance in terms of probabilities for.5
rejection (including false rejection),through the
.available charts/graphs
Set goals for Q.C performance as probability of error.6
detection of 90%,and a probability of false rejection
.of 5% chance
Detect critical systemic errors using suitable.7
.graphical tools
                                                          51
Calculating Sigma Level of critical
               systemic error


=Critical systemic error
         total allowable error - Bias] )
              %[mean)/Impercision


=SIGMA-METRIC
Critical systemic error + 1.65



                                           52
CLSI EP23 Guideline


 Laboratory Quality Control Based on Risk
Management—Proposed Guideline

guidance to enable labs to develop effective, cost-efficient
:QC protocols to
 Reduce negative impact of test system’s-
.limitations
 Monitor immediate and extended test-
.performance


                                                               53
54
55
For each risk, a mitigation strategy is found that will
.reduce the residual risk to an acceptable level


Sum of all QC elements (manufacturer provided and
laboratory added) becomes the laboratory’s QC plan
.specific to this device and the laboratory environment




                                                          56
CLSI EP23 Guideline



Doesn’t replace surrogate QC, but
incorporates surrogate QC to
address the potential for certain
risks

Utilizes a risk management
approach to developing a
.customized QC plan

Provides a scientific basis for
justifying QC strategies (useful for
(lab inspectors
                                       57

Quality assignment(1)

  • 1.
    Quality Control IT ISA BASIC SAFETY PRACTICE TO MONITOR ANALYTICAL QUALITY OF MEASUREMENT. To detect changes from stable day to day operation and eliminate reporting of results with medically important errors.`
  • 2.
    OBJECTIVES 1.To determine howto establish the analytical goal and quality control scheme /schedule in your lab 2.To identify the Quality control charts and quality control roles 3.To identify the new rules of west guard. 4. Overview of the CLSI guide lines C24. 2
  • 3.
    FIRST OBJECTIVE Todetermine how to establish the analytical goal and quality control ! scheme /schedule in your lab 3
  • 4.
    Data we neededto design Quality Control 1.Commercial QC material 2.Manufacturers’ kit inserts . 3. Doing it for yourself is, as usual, the safest method. 4.EQAS programs . 4
  • 5.
    Designing the internalquality control protocol Define the level of quality that the laboratory wants to.1 attain for a determined test (the analytical quality (.specification .know the stable analytical performance for this test . 2 a control rule (control limits and number of controls per . 3 ( .run :Assure quality of results. 4 a. analytical imprecision and bias b. EQAS 5
  • 6.
    Proficiency testing # Interlaboratory Comparison Programs Help Improve Laboratory Performance. #
  • 7.
    Define Analytical Goal Two main strategies for analytical quality specifications based on .calculation of imprecision and bias 7
  • 8.
    ?How to establishyour analytical goal The underlying principle of ‘measurement* uncertainty’ is that a laboratory should know how .precisely they can measure any particular analyte Two main strategies for analytical quality* specifications based on biology have been evaluated for imprecision and bias (in combination with imprecision), respectively 8
  • 9.
    Challenges to setanalytical Goal External Internal Permenant External Internal :Method :Implementation Permenant :Method Analytical principle Choice of conditions :Implementation Analytical Equipment principle In house' equipment' Choice of conditions Reagents Equipment ,In house' reagents' In house' equipment' Reagents)choice of producer( ,.Time, Temperature, In house' reagents' Volume, etc )choice of:producer( Standardization .:StandardizationVolume, etc Time, Temperature, :Standardization :Standardization Traceability of calibration of Traceability calibration Calibration function Calibration function Variable Variable :Batches :Batches ::Performance Performance )Reagents Reagents ) (variability (variability in house' reagents' in house' reagents' Calibrators Calibrators ..Training, Maintenance, Training, Maintenance, etc etc Consumables Consumables ''Control with 'trouble-shooting Control with 'trouble-shooting 9 Documentation Documentation
  • 10.
  • 11.
  • 12.
    Relevance to customers Allthese complicated processes to provide the appropriate quality for our patient cares 12
  • 13.
    Setting Goals ForQ.C Performance 1.Maximum allowable number of unacceptable results, due to an out of control error conditions. 2.Maximum allowable probability of reporting unacceptable results. 3.Minmum acceptable probability of detecting an out of control error condition. 4.Maximum acceptable probability of false rejection. *Main aim is to Maximize probability to detect an out of control condition for measurement procedure , while minimize probability for false Q.C alerts. 13
  • 14.
    Analysis of ControlMaterials A stable control which mimics patient’s sample is analyzed (DAY TO DAY OR SET TO SET( Need data set of at least 20 points, obtained over a 30 day period Calculate mean, standard deviation, coefficient of variation; determine target ranges
  • 20.
    CLIA proficiency testingcriteria for acceptable analytical performance:
  • 21.
    Objective 2.To identify theQuality control charts and quality control roles 3.To identify the new rules of west guard. 21
  • 22.
    Monitoring QC Data DevelopLevey-Jennings chart. Plot control values each run, make decision regarding acceptability of run. Monitor over time to evaluate the precision and accuracy of repeated measurements Review charts at defined intervals, take necessary action, and document
  • 23.
    Levey-Jennings Chart A graphicalmethod for displaying control results and evaluating whether a procedure is in-control or out-of- control Control values are plotted versus time Lines are drawn from point to point to accent any trends, shifts, or random excursions
  • 24.
    Levey-Jennings Chart The Levey-Jenningschart usually has the days of the month plotted on the X-axis and the control observations plotted on the Y-axis. On the right is the Gaussian or "bell-shaped" curve turned on its side to show the correlation of the curve to the chart (ie, fewer data points should appear on the upper and lower extremities of the chart, since the "bell" is thinner farther from the mean). By observing the data plotted in the L-J chart, we can determine if test results are in control and accurate, or if test results are not in control and consequently unacceptable. 24 .
  • 25.
    Levey-Jennings Chart Calculate the Mean and Standard Deviation; Record the Mean and +/- 1,2 and 3 SD Control Limits 3SD+115 2SD+ 110 1SD+ 105 Mean 100 1SD- 95 2SD- 90 3SD- 85 80 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Day
  • 26.
    Levey-Jennings Chart - Record Time on X-Axis and the Control Values on Y- Axis 115 110 Control Values (e.g. mg/dL) 105 100 95 90 85 80 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 (Time (e.g. day, date, run number
  • 27.
    Levey-Jennings Chart - PlotControl Values for Each Run 115 110 Control Values (e.g. mg/dL) 105 100 95 90 85 80 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 (Time (e.g. day, date, run number
  • 28.
    Levey-Jennings Chart - Record and Evaluate the Control Values 3SD+ 115 2SD+ 110 1SD+ 105 Mean 100 95 1SD- 2SD- 90 3SD- 85 80 Day 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  • 29.
    Look for assignable In control Out of control ! cause ! UCL Problem Natural corrected 6σ Variation Target = Mean 3σ LCL Time Samples Natural variation
  • 30.
    “Allows determination ofwhether an analytical run is “in-control” or “out-of-control”
  • 31.
    Westgard Rules area multi role QC procedure 1-West-gard rules: (Regular twice entry for Q.C) 3s, 1 2s, 2 2s , R 4s, 4 1s, 10x (& modifications 1 ) 8x, 12x Recent west-gard rules, fit better and are easier-2 to apply in situations where 3 different control :materials are being analyzed 2of3 2s, 3 1s, 6x & 9x A related control rule that is sometimes used-3 ":looks for a "trend 7T 31
  • 32.
    Westgard Rules area multirule QC procedure 13s refers to a control rule that is commonly used . with a Levey Jennings chart A run is rejected when a single control measurement exceeds the mean plus 3s or the .mean minus 3s control limit 32
  • 33.
    12s refers tothe control rule that is commonly used with a Levey-Jennings chart single control measurement exceeds . the mean plus 2s or the mean minus 2s control limit This rule is used as a warning rule to trigger careful inspection of the control data by the following rejection rules . 33
  • 34.
    22s - rejectwhen 2 consecutive control measurements exceed the same mean plus 2s . or the same mean minus 2s control limit 34
  • 35.
    R4s - rejectwhen 1 control measurement in a group exceeds the . mean plus 2s and another exceeds the mean minus 2s 35
  • 36.
    41s - rejectwhen 4 consecutive control measurements exceed the same mean plus 1s . or the same mean minus 1s control limit 36
  • 37.
    10x - rejectwhen 10 consecutive control measurements fall . on one side of the mean :some modifications 8x - reject when 8 consecutive control measurements fall on one side of the mean 12x - reject when 12 consecutive control measurements fall . on one side of the mean 37
  • 38.
    In situations where3 different control materials are being analyzed, some other control rules fit better and :are easier to apply, such as 2of32s - reject when 2 out of 3 control measurements exceed the same mean plus 2s or mean minus 2s control limit ; 38
  • 39.
    31s - rejectwhen 3 consecutive control measurements exceed the same mean plus 1s or mean minus 1s .control limit 39
  • 40.
    some modification: 6x -reject when 6 consecutive control measurements fall on one side of the mean. 40
  • 41.
    A related controlrule that is sometimes used, looks for a "trend" where several control measurements in a row are increasing or decreasing 7T - reject when seven control measurements trend in the same direction, i.e., get progressively higher or progressively lower. there are two types of errors, random and systematic the multirule combines the use of two types of rules to help detect those two types of errors. 41
  • 42.
    Corrective action 1- Determinethe type of error occurring on the basis of the rule violated. 2-Refer to trouble-shooting guides to identify possible causes for the type of error indicated by the control rule that was violated. 42
  • 43.
    There are twotypes of errors, random and systematic the multi rule combines the use of two types of rules to help detect those two types of errors: Type of Error Control rule that detects it Random error 13s, R4s Systematic error 2s, 4 1s, 2of3 2s, 3 1s, 6x, 8x, 9x, 10x, 2 12x, cusum 43
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
    Correct the problem,then analyze control- 3 .samples again to assess control status 4- Repeat or verify the results on the patient samples once the method has been demonstrated to be in-control. 5- Consult a supervisor for any decision to report patient results when a run is out-of-control. 48
  • 49.
    OBJECTIVE Overview of the CLSI guide lines .C24 49
  • 50.
    ?Who is CLSI Clinicaland Laboratory Standards Institute ANSI-accredited, global, nonprofit standards• development organization CLSI has over 2,000 members – organizations such• as IVD manufacturers, hospital laboratories, reference ,laboratories, universities professional associations, and government agencies 50
  • 51.
    CLSI –C24:Q.C planningprocess Define Quality requirement inn the form of.1 .Allowable total error .Select suitable Q.C material.2 Obtain estimates of methods of impersion and bias.3 .Identify traditional control rules.4 Predict performance in terms of probabilities for.5 rejection (including false rejection),through the .available charts/graphs Set goals for Q.C performance as probability of error.6 detection of 90%,and a probability of false rejection .of 5% chance Detect critical systemic errors using suitable.7 .graphical tools 51
  • 52.
    Calculating Sigma Levelof critical systemic error =Critical systemic error total allowable error - Bias] ) %[mean)/Impercision =SIGMA-METRIC Critical systemic error + 1.65 52
  • 53.
    CLSI EP23 Guideline Laboratory Quality Control Based on Risk Management—Proposed Guideline guidance to enable labs to develop effective, cost-efficient :QC protocols to Reduce negative impact of test system’s- .limitations Monitor immediate and extended test- .performance 53
  • 54.
  • 55.
  • 56.
    For each risk,a mitigation strategy is found that will .reduce the residual risk to an acceptable level Sum of all QC elements (manufacturer provided and laboratory added) becomes the laboratory’s QC plan .specific to this device and the laboratory environment 56
  • 57.
    CLSI EP23 Guideline Doesn’treplace surrogate QC, but incorporates surrogate QC to address the potential for certain risks Utilizes a risk management approach to developing a .customized QC plan Provides a scientific basis for justifying QC strategies (useful for (lab inspectors 57