PROTOCOL FOR THE LABORATORY

EVALUATION OF HCV MOLECULAR ASSAYS

Page 1

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 © World Health Organization 2018
Some rights reserved. This work is available under the Creative Commons Attribution Non-Commercial-Share Alike 3.0
IGO license (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

Under the terms of this license, you may copy, redistribute and adapt the work for noncommercial purposes, provided
the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO
endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the
work, then you must license your work under the same or equivalent Creative Commons license. If you create a
translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was
not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this
translation. The original English edition shall be the binding and authentic edition”.
Any mediation relating to disputes arising under the license shall be conducted in accordance with the mediation rules
of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules).

Cataloguing-in-Publication (CIP) data: CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To
purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries
on rights and licensing, see http://www.who.int/about/licensing.

Third-party materials: If you wish to reuse material from this work that is attributed to a third party, such as tables,
figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain
permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned
component in the work rests solely with the user.

General disclaimers: The designations employed and the presentation of the material in this publication do not imply
the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city
or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on
maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or
recommended by the World Health Organization in preference to others of a similar nature that are not mentioned.
Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

WHO and the WHO Prequalification Evaluating Laboratory, do not warrant or represent that the evaluations conducted
with the HCV test kits referred to in this document are accurate, complete and/or error-free. WHO and the WHO
Prequalification Evaluating Laboratory disclaim all responsibility for any use made of the data contained herein, and
shall not be liable for any damages incurred as a result of its use. This document must not be used in conjunction with
commercial or promotional purposes.

Page 2

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
TABLE OF CONTENTS
1. INTRODUCTION ................................................................................................ 5
1.1. Prequalification of In Vitro Diagnostics ............................................................................................................. 5
2. INTENDED AUDIENCE ........................................................................................ 5
3. STUDY OBJECTIVES ........................................................................................... 5
3.1. Overall Objectives ............................................................................................................................................ 5
3.2. Specific Objectives: ........................................................................................................................................... 6
4. WHO PREQUALIFICATION EVALUATING LABORATORIES ................................... 6
4.1. Training, performance evaluation and supervision ........................................................................................... 6
4.2. Safety ............................................................................................................................................................... 7
4.3. Storage of assays .............................................................................................................................................. 7
5. STUDY DESIGN .................................................................................................. 7
6. SPECIMEN PANELS ............................................................................................ 8
6.1. Analytical Performance Specimen Reference Panel .......................................................................................... 8
6.2. Clinical Performance Specimen Reference Panel .............................................................................................. 9
6.3. Characterization of the Specimen Panels .......................................................................................................... 9
7. LABORATORY TESTING .................................................................................... 10
7.1. Recording test results ..................................................................................................................................... 10
8. QUALITY CONTROL AND INTERPRETATION OF TEST RESULTS .......................... 11
8.1. Test kit controls .............................................................................................................................................. 11
8.2. Internal quality control ................................................................................................................................... 11
8.3. External quality control specimen .................................................................................................................. 11
8.4. Proficiency panels........................................................................................................................................... 11
8.5. Limits of acceptability ..................................................................................................................................... 11
8.6. Interpretation of results ................................................................................................................................. 11
9. ANALYSIS OF DATA ......................................................................................... 12
9.1. Invalid runs ..................................................................................................................................................... 12
9.2. Invalid individual specimen results (invalid IQC/calibrator) ............................................................................ 12
9.3. Performance characteristics from WHO specimen reference panel ................................................................ 12
9.3.1. Analytical performance ...............................................................................................................................12
9.3.1.1. Precision of measurement...........................................................................................................................12
9.3.1.1.1. Intra-assay variation (within-run) .......................................................................................................... 12
9.3.1.1.2. Inter-assay variation (within days) ........................................................................................................ 12
9.3.1.1.3. Inter-instrument variation ..................................................................................................................... 12
9.3.1.2. Limit of detection and limit of quantitation ...............................................................................................13
9.3.1.3. Cross-contamination....................................................................................................................................13
9.3.2. Clinical Performance ....................................................................................................................................13
9.3.2.1. Trueness of measurement ...........................................................................................................................13
9.3.2.2. Sensitivity and Specificity ............................................................................................................................14
Confidence intervals .......................................................................................................................................................15
9.3.2.3. Discrepant results ........................................................................................................................................15
10. OPERATIONAL CHARACTERISTICS ................................................................... 15
11. REPORT PREPARATION ................................................................................... 15
12. MATERIALS AND SUPPLIES .............................................................................. 15
13. ROLES AND RESPONSIBILITIES ......................................................................... 16
13.1. Responsibilities of the WHO Prequalification Evaluating Laboratory .............................................................. 16
13.2. Responsibilities of WHO ................................................................................................................................. 16
Page 3

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
14. OTHER DOCUMENTS AND TOOLS REQUIRED................................................... 16
REFERENCES .......................................................................................................... 17

Page 4

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
1. Introduction

1.1. Prequalification of In Vitro Diagnostics

The World Health Organization (WHO) Prequalification of In Vitro Diagnostics is coordinated through the
Prequalification-Diagnostics Assessment Team (PQDx). The aim of the WHO Prequalification of In Vitro
Diagnostics Assessment is to promote and facilitate access to safe, appropriate and affordable diagnostics of
good quality in an equitable manner. Focus is placed on products for high burden diseases and their
suitability for use in resource-limited settings.

The WHO prequalification of in vitro diagnostics process includes three main components:

 Review of a product dossier;

 Performance evaluation of the product;
 Inspection of the manufacturing site(s).

The performance evaluation will be conducted by a WHO Prequalification Evaluating site following a choice
of two different mechanisms described here. Performance evaluations conducted by a laboratory in List 1 will
be coordinated and cost covered by WHO. Performance evaluations conducted by a laboratory in List 2 will
be coordinated and cost incurred by the manufacturer.

This protocol describes the procedures required to perform an evaluation of HCV molecular technologies
submitted for WHO prequalification assessment. This protocol is not intended to replace validation and
verification studies that need to be conducted by the manufacturer in order to fulfil WHO prequalification
product dossier requirements.

Given the variety of molecular assays available, this protocol remains generic in nature and some sections
may be open to interpretation. Manufacturers are encouraged to contact WHO before the start of the
evaluation in order to verify that their preferred approach is in line with WHO expectations.

This protocol was developed in collaboration with the National Serology Reference Laboratory, Fitzroy,
Victoria, Australia.

2. Intended audience
This document is intended to provide WHO Prequalification Evaluating Laboratories and manufacturers with
the WHO performance evaluation procedure.

3. Study Objectives

3.1. Overall Objectives

The overall objective of the performance evaluation is:
Page 5

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 To evaluate commercially available HCV molecular assays (including qualitative and quantitative claims as
necessary) for detection of HCV RNA against a designated reference result.

3.2. Specific Objectives:

The specific objectives of the evaluation are to:
 assess the precision, of selected technologies using dilutions of two stock plasma specimens;
 assess, limit of detection, limit of quantification and robustness (if applicable);
 assess genotype detection;
 assess specificity using negative plasma specimens;
 evaluate the operational characteristics of selected technologies, e.g. ease of performance, specimen
type utility, suitability for use in extreme conditions (high/low temperatures, high humidity),
suitability for use in countries with limited infrastructure (no/limited electricity, no/limited clean
water, inadequate means of waste disposal;
 report and disseminate the findings of the evaluation.

4. WHO Prequalification Evaluating Laboratories

The performance evaluation will be exclusively conducted by a WHO Prequalification Evaluating Laboratory.
These laboratories have successfully undergone assessment through the WHO Alternative Laboratory
Evaluation Mechanism which includes:
 Submission of an Expression of Interest (EoI) by the laboratory,
 Stage 1 audit of the laboratory (assessment of EoI and specific quality management system (QMS)
documentation),
 Stage 2 (on-site) audit to assess compliance with WHO requirements
The list of WHO Prequalification Evaluating Laboratories can be accessed here.

The laboratory shall hold the following certification for quality management within the laboratory: ISO17025
(General requirements for the competence of testing and calibration laboratories), ISO15189 (Medical
laboratories: Particular requirements for quality and competence) or equivalent.

The person(s) listed in the EoI letter to WHO will act as the Principal Investigator (PI) for the work performed
by the WHO Prequalification Evaluating Laboratory.

4.1. Training, performance evaluation and supervision

The following issues are key to minimizing error and maximizing the value of this evaluation:
 Only personnel having received specific training for this evaluation will be employed in the evaluation;
 Accurate record keeping is crucial to the success of the evaluation and the PI will be responsible for
ensuring that all data required for the evaluation are recorded as agreed
 Worksheets should be prepared and tubes, test devices or plates labelled prior to commencement of
any run / assay;

Page 6

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
  Because objective, machine-generated, permanent results for some of the technologies available
may not be feasible, it is essential that the PI emphasizes the need for accurate recordkeeping;
 To minimize the risk of error, results will be directly exported from the platform wherever possible. If
this is not the case, results should be entered by one staff member and verified by another.

4.2. Safety
HIV, hepatitis B and hepatitis C and other viruses are transmissible by blood and body fluids. Therefore, all
types of specimens (including venous and capillary whole blood, serum/plasma, oral fluid, etc.) must be
handled as potentially infectious. Appropriate precautions to minimize infectious hazards must be taken at
all stages from the collection of specimens to the disposal of used materials from the laboratory. The WHO
Guidelines on HIV Safety Precautions, and Guidelines for the Safe Transport of Specimens (WHO/EMC/97.3)
and the site’s guidelines on laboratory safety should be carefully followed by the laboratory staff.

4.3. Storage of assays

All reagents must be stored as indicated in the instructions for use. Some assays may not need refrigeration.
Calibrated thermometers are placed at each location where reagents and specimens are stored, i.e. ambient,
refrigerator and freezer. Temperatures are recorded daily on forms designed for the purpose. The lot
numbers of the test kits received/used and their expiry dates are recorded on the individual run worksheets.

5. Study Design
The study will be conducted with two separate objectives, one investigating analytical performance and the
other investigating clinical performance. Although there are many performance characteristics that could be
investigated, this evaluation is risk-based and focuses on aspects of greatest importance to assuring the
safety and performance when such IVDs are used in WHO member states.

Analytical aspects that will be evaluated include the following performance characteristics:
• Precision of measurement
– Intra-assay variation (within-run if applicable),
– Inter-assay variation (within days)
– Inter-instrument variation (for point of care technologies with very low-throughput)
• Limit of detection (LOD), lower limit of quantification (LLOQ)
• Robustness
• Genotype detection

The evaluation of clinical performance will compare the assay's result with a reference method using
clinically derived specimens (specimens collected for routine testing at the evaluating site).

The evaluation will also include an assessment of the assays' operational characteristics in view of their
anticipated use in resource-limited settings. This assessment will include but is not limited to the following
characteristics:

 Skills and training requirements

 Maintenance and calibration requirements

Page 7

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
  Specimen requirements
 Electricity and water requirements
 Equipment required (including equipment provided and ancillary equipment that is required but not
provided)
 Storage requirements for reagents
 Shelf life of reagents (upon the time of manufacture)
 Time to result and hands-on time required (including number of steps)
 Laboratory logistics, including equipment footprint.

6. Specimen Panels

6.1. Analytical Performance Specimen Reference Panel

• At a minimum two stock specimens representing the most commonly occurring genotypes 1, 3 will
be used to construct the panel of specimens for the analytical stage of evaluation. Each stock specimen will
be diluted in defibrinated normal human plasma that is negative for HIV, HBV and HCV. Dilutions will be
prepared from each stock specimen: 102 IU/ml and 104 IU/ml to assess precision as described in Table 1. This
will yield a total of a minimum of 4 specimens.

• In order to conduct the assessment of the limit of detection, the WHO international standard
genotype 1a will be diluted to obtain, at a minimum, the following concentrations: 10 3, 102.5,102, 101.5, 101,
100.5, 100, 10-0.5 IU/ml. This assessment will require at a minimum, 24 replicates for each dilution member.

• The robustness experiment will be conducted with the genotype 1 stock specimen at a concentration
6
of 10 IU/ml and negative specimens in order to detect potential cross-contamination, if applicable to the
technology under evaluation.

• Specimens to demonstrate appropriate genotype detection should either be contributed by the

evaluating laboratory or procured commercially.

Concentrations may be modified to accommodate the dynamic range of the assay under evaluation.

Table 1 - Specimen Requirements for Analytical Evaluation

Genotype Concentration (IU/ml) Number of replicates
Between-run and within-run variation assessment
1 102 and 104 10+10
3 102 and 104 10+10

Limit of Detection and Quantification

1a, WHO International 103, 102.5,102, 101.5, 101, 100.5, 100 10-0.5 24
Reference Preparation
Robustness
1 or 3 Genotype 106 20
Page 8

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
Negative specimens 0 20
Total (plus an additional 20% to account for external controls and 272 (327 total)
errors)

Given that some of the experiments will share the same dilution for specific specimens, whenever possible
and depending on the assay under evaluation, specimens will be accommodated on the platform to maximize
the throughput and avoid the need to run the same dilution of specimen twice for different purposes. All
specimens will be prepared as a single use aliquot.

6.2. Clinical Performance Specimen Reference Panel

Clinically-derived specimens should comprise specimens collected from HCV-infected, RNA positive
individuals with a range of viral loads up to 107 IU/mL. Specimens with viral loads <104 IU/mL should be
actively sought as it will be these specimens with lower viral loads whose results will be most important
should a threshold HCV RNA concentration be nominated in the future for decision-making with respect to
assessing treatment failure. Table 2 represents the total number of clinical specimens required.

Table 2 - Total number of specimens required

Viral load – positive specimens Number of Specimens Number of tests required:

n/0.80*
LOD-10 000 Up to 50 63
10 000-10 000 000 Up to 50 63

Uninfected specimens 50 (minus negatives 63

used in precision study)

Total Up to 150 189

*Adjustment calculation to account for an assay failure rate of less than 15% and controls.

6.3. Characterization of the Specimen Panels

Evaluation Parameter Specimen type Specimen requirements

HCV RNA positive clinical stock
Precision
specimens  Known VL +/- genotype
HCV RNA positive clinical stock  Plasma
Robustness specimen; HCV RNA negative plasma  >= 2mL available
pool  Stored at <= -70°C
HCV RNA positive clinical stock  Fresh estimate of VL if
specimens; determination >12
Genotype detection
previously
2nd HCV RNA genotype panel procured
Page 9

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 from NIBSC

HCV RNA positive clinical stock

VL comparison
specimens
WHO international standard procured
LOD
from NIBSC
 Validated for use in the
routine assay
In-house or commercially manufactured
External QC specimen  Established acceptance
QC sample
limits
 VL 3 – 5x LOD

Clinically derived HCV positive specimens and HCV negative specimens for the clinical performance
evaluation will be characterized using the Roche cobas 6800 HCV viral load assay. This assay is calibrated
against the WHO HCV RNA international standard. The result obtained using this test will serve as the
reference result; no additional testing will be required. Discrepant resolution plan is described in 10.3.2.3. An
alternative reference method may be selected by the WHO Prequalification Evaluating Laboratory with prior
agreement from WHO.

7. Laboratory testing
Each product under evaluation will be used in accordance with the instructions for use (IFU) issued by the
manufacturer. The evaluating site will send a copy of the IFU to WHO upon delivery of the reagents and prior
to the commencement of the laboratory evaluation. The IFU must be reviewed against the IFU submitted to
WHO as part of the application or pre-submission form. If the IFU has been updated since this time, it is the
onus of the manufacturer to submit to WHO a letter detailing changes made prior to the start of the
laboratory evaluation. Records of the version used must be kept.

The interpretation of results for each assay under evaluation is made strictly according to the manufacturer’s
instructions within the IFU. Invalid runs and/or test results are recorded on the data collection sheets.

7.1. Recording test results

Wherever possible, all test results are saved and exported directly from the instrument to standardized test
result worksheets in Microsoft Excel spreadsheets for further data analysis.

A technician's appraisal is made of each assay under evaluation and is completed by the operator performing
the testing. This appraisal is comprised of questions addressing ease of the procedure, reading of results,
clarity of IFU, as well as records of any specific difficulties encountered during the evaluation. The

Page 10

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
technician’s appraisal table will be made available to WHO Prequalification Evaluating Laboratories as a
separate document.

8. Quality control and interpretation of test results

8.1. Test kit controls

Manufacturer-supplied positive and negative test kit controls will be run at the frequency indicated in the
IFU., in each test run when applicable and at the start of each testing session when applicable. Where
positive and negative test kit controls are not supplied by the manufacturer, the external quality control
specimen will act at the control specimen.

8.2. Internal quality control

Internal procedural controls should be incorporated into the design of most assays by the manufacturer.
These may take the form of extraction, and/or amplification, and/or detection controls, as indicated in the
IFU. Any internal quality control must be valid as per manufacturer’s instructions.

8.3. External quality control specimen

The evaluating site will supply a previously validated external quality control (QC) specimen which is tested in
single for each test run or once at the start of the day per instrument for single use devices. The QC
specimen represents an HCV positive specimen with a viral load between 10 2 and 103IU/mL. The QC
specimen will be made by the evaluating site or acquired commercially and validated by the site.

8.4. Proficiency panels

A proficiency panel of routine specimens comprised of high and undetectable specimens must be run
successfully for each assay by each operator before the evaluation commences. This may be the same panel
as that used at the time of assay demonstration by the manufacturer or for training purposes.

8.5. Limits of acceptability

All results on test kit controls and the QC specimen are documented. The QC sample result should fall
between the mean +/- 0.3 log10. Should the QC sample lie outside +/- 0.3 log10 , the run will be considered
invalid, in which case the run will be repeated, or troubleshooting should occur for instruments using single
use devices. Such problems should be recorded on the data sheets. The PI will be responsible for carefully
checking all data entry forms for legibility, accuracy and completeness.

8.6. Interpretation of results

The interpretation of results for each assay under evaluation is made strictly according to the
manufacturers’ instructions within the IFU. Invalid test results and errors will also be recorded.
Recording will be done either directly from the instrument printer (if applicable) or manually in the
data collection sheet. In order to avoid transcriptional errors, a digital record will be kept of the
results in the latter case.
Page 11

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
9. Analysis of data

9.1. Invalid runs

The number of invalid test runs will be recorded as the absolute number of invalid runs and as a
percentage of the total number of runs performed for the entire evaluation using all specimens.
Other types of readings indicating an invalid run may be possible depending on the platform under
evaluation. These will also be recorded.

9.2. Invalid individual specimen results (invalid IQC/calibrator)

The number of individual invalid specimen results is recorded. They are presented as a percentage
of the total number of specimens tested per platform for the entire evaluation.

9.3. Performance characteristics from WHO specimen reference panel

The following methods are used to calculate the performance characteristics for each assay under evaluation.

9.3.1. Analytical performance

9.3.1.1. Precision of measurement

Estimation of precision will require, at a minimum the testing of the specimens described in section 6.1. The
mean, standard deviation (SD) and percentage coefficient of variation (% CV) will be calculated and
compared.

9.3.1.1.1. Intra-assay variation (within-run)

Variation will be assessed by measuring, at a minimum, ten replicates of four QC specimens (two genotypes,
two different concentrations) in the same run. A run will be defined depending on the assay’s throughput: if
the platform can accommodate all specimens in a single run, i.e. in the same test plate, the specimens will be
run together. If the assay can only accommodate a smaller set of specimens, a run will be defined as a testing
session carried out from a single dilution and on the same instrument/module.

9.3.1.1.2. Inter-assay variation (within days)

Variation will be assessed by running, at a minimum, four QC specimens (two genotypes at two different
concentration levels) over ten different runs over five days (two runs per day; one session in the morning,
one session in the afternoon). This experiment will provide ten measurements for each of the four specimens.

9.3.1.1.3. Inter-instrument variation

Where possible, inter-instrument variation will be measured using 10 replicates of four QC specimens (two
genotypes, two different concentrations), on three to five instruments of the same brand.

Page 12

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 9.3.1.2. Limit of detection and limit of quantitation
The limit of detection (LoD) is the lowest concentration of analyte that can be consistently detected in 95%
of specimens tested under routine laboratory conditions and in a given specimen matrix. It defines the
analytical sensitivity [4]. The LoD will be therefore defined as the lowest viral concentration detected with a
positivity rate of 95%.

The lower limit of quantitation (LoQ) is defined as the lowest concentration of measurand that are
determined with acceptable precision, trueness and linearity; "acceptable" is defined by clinical applications
of the assay. [4] For the purpose of this protocol, the LoQ will be defined as the viral concentration detected
with a positivity rate of at least 50%.

In order to estimate the limits of quantitation and detection for each assay under evaluation, a minimum of
24 replicates of an eight member dilution series concentrating on the lower end of the manufacturers’ claims
for the dynamic range of the assay will be used. The 24 replicates will be separated in a minimum of eight
runs where applicable. The WHO International Standard HCV RNA preparation will be used for this purpose.

9.3.1.3. Cross-contamination
If applicable, the cross-contamination experiment will allow the determination of the well-to-well / device-
to-device cross-contamination rate of the platform. The robustness of the different platforms will be
assessed by running 20 positive HCV genotype 1 specimens alternating with 20 negative specimens. The
concentration of the positive samples will be 106 IU/ml.

9.3.2. Clinical Performance

Specimens used for the evaluation of the clinical performance will be clinically-derived and collected by the
testing laboratory. Details of the panel are described in Table 2.

9.3.2.1. Trueness of measurement

The trueness will be reported as bias or mean difference. The bias or mean difference is defined as the
difference between the expected test results and an accepted reference value obtained from an instrument
of methodology of a higher order. In this case, the reference value will be defined as the result obtained
using the reference method. The bias or mean difference will reflect the average difference between the viral
load results obtained with the reference method and the method under evaluation.

The level of agreement between the different specimens will be evaluated using the Bland-Altman analysis
(M. Bland, 1986); i.e. through a graphical representation of the plot of the difference between the
measurements using the two different methods (assay under evaluation and reference method) for each
data point against their mean. The limit of agreement is the 95% confidence interval of the difference
between the methods which is bias ±1.96 SD (standard deviation).

Page 13

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 9.3.2.2. Sensitivity and Specificity

Table 1 2x2 table for calculation of sensitivity and specificity

Results of reference testing

+ –
a b
+ a+b
Results of assay True positives False positives
under
evaluation
c d
– c+d
False negatives True negatives

a+c b+d

Sensitivity
Sensitivity will be calculated as the number of true positive results compared to true positives by the
reference method.

a
Sensitivity  (see Table 1)
ac
Sensitivity will be expressed as a percentage.

Specificity
Specificity will be calculated as the number of true negative specimens identified by the index method
compared to true negatives by the reference method.

d
Specificity  (see Table 1)
bd
Specificity will be expressed as a percentage.

Page 14

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
Confidence intervals
The 95% confidence intervals are calculated in order to assess the level of uncertainty introduced by sample
size. Exact 95% confidence intervals for binomial proportions will be calculated from the F-distribution (P.
Armitage, 2002) (B. Kirkwood, 2003).

9.3.2.3. Discrepant results

Discrepant results are defined as results that vary by more than 0.5 log10 from results obtained with the
reference testing results.

Those specimens with results that are consistent with the reference testing results undergo no further
testing. Where possible, specimens with test results discrepant from the reference testing will be retested by
the same operator on the assay under evaluation if sufficient specimen is available.

10. Operational Characteristics

The operational characteristics of the assay will be assessed by the laboratory technicians performing the
evaluation testing using a standard evaluation sheet in order to give an appraisal of the assay under
evaluation. Special attention should be paid to the IFU in order to evaluate whether these instructions are
sufficient for WHO Member State end-users. Comments on the IFU must be made in the report if it does not
meet an acceptable standard for any of the following criteria :clarity, presentation, content, safety
instructions.

11. Report preparation

For evaluations conducted in List 1 WHO Prequalification Evaluating Laboratories, the preliminary data
analysis and drafting of the report will be carried out by the laboratory according to a pre-defined report
template and sent to WHO in a timely manner. WHO will verify the data and draft report and send to the
authorized contact designated by the manufacturer for comment. Data generated and the report prepared
by a List 2 WHO Prequalification Evaluating Laboratory will be shared simultaneously with WHO and the
manufacturer. Manufacturers will have one month right of reply. After one month has elapsed, the report
will be accepted as final by WHO, regardless if comments are submitted. The final report will be prepared
and disseminated by WHO. A copy of the final report will be sent to the authorized contact designated by the
manufacturer and to the laboratory.

12. Materials and supplies

Manufacturers will provide the products and any equipment necessary for the evaluation free of charge.

Page 15

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
 13. Roles and responsibilities

13.1. Responsibilities of the WHO Prequalification Evaluating Laboratory

i. Ensure availability and maintenance of all specimen panels;
ii. Conducting the performance evaluation in accordance with this protocol and good laboratory
practice;
iii. Preparation of draft report of the laboratory evaluation;
iv. Advising WHO on operational characteristics of assays evaluated.
All source data, data analysis records and all correspondence are retained and archived for a period of at
least ten years.

13.2. Responsibilities of WHO

i. Technical advice to the PI;
ii. Technical and administrative management of the laboratory evaluation;
iii. Verification of the draft report, seeking comments from the manufacturer;
iv. Preparation and dissemination of the final report;
v. Formal contacts with the manufacturers.

Any publication by WHO of the results of these evaluations and the WHO recommendations derived
therefrom will, however, be accompanied by the following disclaimer:

The mention of specific companies or of certain manufacturers’ products does not imply that they are
endorsed or recommended by the World Health Organization in preference to others of a similar nature that
are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by
initial capital letters.

WHO and the WHO Prequalification Evaluating Laboratory, do not warrant or represent that the evaluations
conducted with the HCV test kits referred to in this document are accurate, complete and/or error-free. WHO
and the WHO Prequalification Evaluating Laboratory disclaim all responsibility for any use made of the data
contained herein, and shall not be liable for any damages incurred as a result of its use. This document must
not be used in conjunction with commercial or promotional purposes.

14. Other documents and tools required

Standard Operating Procedures

SOP_PQDx_224_Overaching Procedure for Molecular Evaluations

Master Templates
PQDx_274 PQT REPORT TEMPLATE FOR THE LABORATORY ASSESSMENT OF HCV MOLECULAR TECHNOLOGIES

Other Tools
Page 16

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx
PQDx_292_Technician’s appraisal of operational characteristics

References
1. B. Kirkwood, J. S. (2003). Essential Medical Statistics 2nd edition. Blackwell Science Ltd.
2. ISO/IEC. (2007). International vocabulary of metrology - Basic and general concepts and associated
terms (VIM). ISO/IEC Guide 99. Geneva, Switzerland: International Organization for Standardization.
3. M. Bland, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical
measurement. Lancet.
4. P. Armitage, G. B. (2002). Statistical Methods in Medical Research, 4th Edition. Oxford: Blackwell
Scientific Publications.

Page 17

PQDx_225 PROTOCOL FOR THE LABORATORY EVALUATION OF HCV MOLECULAR ASSAYS V 4.0
WHO/EMP/RHT/PQT/2018.xx