WEBINAR: Proficiency testing selection, participation, and

results evaluation in Microbiology and Chemistry testing

laboratories
Presented by: Janine Musso
INTRODUCTION TO NSI LAB SOLUTIONS
INTRODUCTION
Proficiency Testing Interlaboratory comparison Intralaboratory comparison
A formal, systematic process Involve laboratories voluntarily A comparison is conducted
where multiple laboratories sharing and comparing test when several analysts or
receive the same samples and results with each other. These technicians within an
independently perform tests comparisons can be organized organization perform testing or
on them. The results are then formally or informally, and they calibrations on the same or
collected and analyzed by an may or may not be similar artifact, using the same
ISO/IEC 17043 accredited coordinated by an external method, under specified,
provider. body. controlled conditions.

Same lab
ISO 17043
INTRODUCTION
Both PT and ILC aim to monitor and improve the quality and accuracy of
laboratory testing, but they have some distinct differences:
Aspect Proficiency Testing (PT) Interlaboratory Comparisons (ILCs)
Structure Highly standardized and formal Flexible and can be informal
Managed by an external provider Can be self-organized or facilitated by
Oversight
(ISO/IEC 17043 accredited) peers

Can be qualitative, statistical, and peer-

Analysis Statistical, external analysis
reviewed
Purpose Objective performance evaluation Collaborative learning and benchmarking
Generally lower and more variable
Cost Often higher due to formal structure
benchmarking
Compliance Required for accreditation Optional but beneficial to include
INTRODUCTION
Why is PT / ILC so important?
Proficiency testing (PT) is an essential tool for helping to ensure that your
laboratory gets its results right – both now and into the future. By assessing
the actual results produced by the laboratory, PT reflects on all aspects of
the laboratory’s quality assurance system.
One fundamental reason for your laboratory joining a PT scheme is to obtain
an external and independent assessment of its performance in conducting
specific tests or measurements – and it is also a key requirement within
international standards on laboratory competence.
This assessment either gives you, your customers, and regulators the
confidence that your processes are robust – or lets you know that there are
potential problems and pinpoints the corrections that need to be made.
INTRODUCTION
Why is PT / ILC so important? (continued)
Human and financial resources needed for PT/ILC participation, can be
important, but should not be limited to such a degree that there are risks that
a participant’s data may have errors, biases or significant differences
compared to their peers that remain undetected.
It is important to underline that the aim of PT/ILC participation is not just
about performing well or badly, but also about enabling the participant to
learn from their participation in PT schemes and/or ILCs and to use this
information to improve the quality of their measurements.
INTRODUCTION
Benefits of PT / ILC:
• ISO accreditation – Taking part in a PT scheme is essential to achieving
laboratory accreditation.

• Verifying methods and equipment calibration – Laboratories can use

PT/ILC results to verify their measurement processes - including methods,
sample handling, equipment, and calibration. They can check or, with
caution, derive their measurement uncertainty estimates, and they may
also use surplus PT/ILC materials to support internal quality controls.
INTRODUCTION
Benefits of PT / ILC (continued):
• Managing risks – PT and ILC helps protect consumers, brand reputation and
profitability by helping laboratories to manage early warning signs and
enabling a rapid resolution of problems.

• Training laboratory teams – PT schemes and ILC can be used as a

continuous personnel education and improvement tool, allowing managers to
identify and deliver training needs, and to develop individuals via exposure to
infrequently seen analyte concentrations and sample matrices.

• Comparing results – PT enables participants to compare their performance

with their peers across the world, as well as within and between an
organization's sites.
SELECTING A PT PROVIDER
The PT provider selected by the laboratory should, to the extent
possible:
a) Have the effect of checking all sample handling procedures, i.e., sample
receipt, sample preparation, testing, results review and reporting.
b) Provide samples that mimic customer samples, i.e., similar or equivalent
matrix that provides relevant challenges and matrix interference;
c) Fulfil ISO/IEC 17043 requirements.

(c) is the evaluation criteria for the competency of a Proficiency Testing Scheme provider –
accredited to ISO/IEC 17043 Conformity assessment – General requirements for the
competence of proficiency testing providers.
SELECTING A PT PROVIDER
When selecting a PT provider, the laboratory should consider the type of
target value offered. Target values are:
a) independently set by a reference method, or
b) set by overall consensus data, and/or
c) set by method peer group consensus data, or
d) set by a panel of experts.

“Target values” refers to the desired or expected result that laboratories aim
to achieve when participating in an PT program. It serves as a benchmark
against which the laboratory's own test results are compared to assess the
accuracy and reliability of their testing processes.
SELECTING A PT PROVIDER
The different types of target values mentioned in the clause represent how
these benchmarks are established:
1. Independently set by a reference method:
• The target values are determined by a well-established reference method,
which is considered the gold standard for the particular test or assay.
• Laboratories aim to achieve results that closely match those obtained by
the reference method.
2. Set by overall consensus data:
• Target values may also be established based on consensus data derived
from a large group of laboratories performing the same test.
• This consensus reflects the typical or expected range of results obtained
by these laboratories.
SELECTING A PT PROVIDER
3. Set by method peer group consensus data:
• Some PT programs set target values based on consensus data specific to
a peer group of laboratories using the same testing method or platform.
• This ensures that the benchmarks are tailored to the performance
characteristics of the method being used.
4. Set by a panel of experts:
• In certain cases, target values may be established by a panel of experts in
the field who review available evidence and provide recommendations
based on their expertise.
• These experts may consider factors such as clinical significance and
technical feasibility when setting target values.
SELECTING A PT PROVIDER
Other factors to consider:
• Do any PT schemes exist for the various areas of technical competence?
• Is the frequency of rounds per year sufficient?
• Is the selected PT scheme independent of any manufacturing or marketing
interests in equipment, reagents or calibrators in its field of operation?
• Regulatory requirements / import permits required.
• Technical support provided by PT provider. Consider the ease of access to
PT provider contacts, including timely and adequate responses to queries.
• Costs of participation.
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.1:
The laboratory shall have a procedure for monitoring the validity of results.
The resulting data shall be recorded in such a way that trends are
detectable and, where practicable, statistical techniques shall be applied to
review the results.
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.1 (continued):
Implementation:
• Obtain information from the selected PT Scheme regarding the enrollment
process and participation requirements. Document this information in the
laboratory’s internal procedures. This should include all relevant details
related to the handling of PT samples, submission of results, receipt of
results, and related activities.
• If the laboratory participates in ILC activities, documented procedures
should also cover all the above-stated requirements.
• Describe how and where results are recorded and describe the statistical
analysis tools and acceptance criteria used to evaluate the results.
• Consider additional accreditation requirements related to PT participation.
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.1 (continued):
This monitoring shall be planned and reviewed and shall include, where
appropriate, but not be limited to:
a) use of reference materials or quality control materials;
b) use of alternative instrumentation that has been calibrated to provide
traceable results;
c) functional check(s) of measuring and testing equipment;
d) use of check or working standards with control charts, where applicable;
e) intermediate checks on measuring equipment;
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.1 (continued):
This monitoring shall be planned and reviewed and shall include, where
appropriate, but not be limited to:
f) replicate tests or calibrations using the same or different methods;
g) retesting or recalibration of retained items;
h) correlation of results for different characteristics of an item;
i) review of reported results;
j) intralaboratory comparisons;
k) testing of blind sample(s).
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.2:
The laboratory shall monitor its performance by comparison with results of
other laboratories, where available and appropriate. This monitoring shall be
planned and reviewed and shall include, but not be limited to, either or both
of the following:
a) participation in proficiency testing;
b) participation in interlaboratory comparisons other than proficiency
testing.

NOTE: ISO/IEC 17043 contains additional information on proficiency tests

and proficiency testing providers. Proficiency testing providers that
meet the requirements of ISO/IEC 17043 are considered to be
competent.
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.2 (continued):
Implementation:
• Document a PT/ILC plan describing the frequency of participation over a
certain period of time, e.g., 5-year plan.
• Evaluate and review the outcome of results on a regular (routine) basis,
rather than sporadically. This allows for continuous monitoring of
performance trends and timely intervention if deviations are detected.
• Adjusting the PT/ILC plan can usually occur as outliers are obtained and a
problem is flagged, or during management review meetings when the
outputs of the discussion initiate a change.
• Confirm and retain evidence that the PT provider is ISO 17043 accredited
(can be included as part of supplier evaluation as per ISO 17025 Clause
6.6).
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.3:
Data from monitoring activities shall be analysed, used to control and, if
applicable, improve the laboratory's activities. If the results of the analysis of
data from monitoring activities are found to be outside pre-defined criteria,
appropriate action shall be taken to prevent incorrect results from being
reported.
Implementation:
• PT/ILC reports shall be reviewed (and documented in an internal
summary) to indicate compliance or outliers.
• Describe in a procedure how outliers are actioned (refer to ISO 17025
Clause 7.10 Nonconforming work and 8.7 Corrective actions).
ISO/IEC 17025:2017 REQUIREMENTS
ISO/IEC 17025:2017 Clause 7.7.3 (continued):
The review of PT/ILC data should provide an accurate reflection of the
laboratory's current performance. This requires selecting an appropriate time
frame for review that allows for the identification of recent trends and
patterns.
ISO/IEC 17025:2017 REQUIREMENTS
Examples of implementation:
a) Real-time monitoring dashboard:
• The laboratory utilizes a real-time monitoring dashboard that displays
PT/ILC results as they are received.
• Acceptability criteria are programmed into the dashboard, triggering
alerts for any results that fall outside the predefined limits.
• Laboratory personnel can proactively address issues as they arise,
ensuring timely intervention and corrective actions.
ISO/IEC 17025:2017 REQUIREMENTS
Examples of implementation (continued):
b) Monthly review meetings:
• The laboratory holds monthly meetings dedicated to reviewing PT/ILC
data for all tests performed.
• Acceptability criteria are established for each test, such as allowable
biases or coefficients of variation.
• Trends in performance are analyzed over the past month to identify any
patterns or deviations from expected results.
ISO/IEC 17025:2017 REQUIREMENTS
Examples of implementation (continued):
c) Quarterly performance reports:
• Quarterly performance reports are generated, summarizing PT/ILC
results for the previous three months.
• Acceptability criteria are applied to assess the laboratory's performance
against established benchmarks.
• Any trends or issues identified during the review are documented, and
corrective actions are proposed and implemented, as necessary.
ISO/IEC 17025:2017 REQUIREMENTS
Examples of implementation (continued):
c) Annual performance reviews:
• An annual comprehensive review is conducted to assess the
laboratory's overall performance based on PT/ILC data collected
throughout the year. Preferably this review is done during Management
Review meetings (see ISO 17025 Clause 8.9).
• Performance trends are analyzed, and comparisons are made to
previous years' data to evaluate progress over time.
• Lessons learned and best practices are identified, and improvements
are incorporated into the laboratory's quality management system.
ACCREDITATION REQUIREMENTS
Laboratories are required to participate in appropriate PT/ILC:
= The laboratory is to determine frequency and justify it through the level of
risk established (i.e., perform a risk assessment on frequency as per ISO
17025 Clause 8.5).

Therefore, monitoring activities may include (in order of preference):

a) External PT/ILC (ISO 17043 accredited)
b) Inter- and intralaboratory comparisons
c) Suitable alternative as agreed with the accreditation body

If there is no PT/ILC available, it is not a valid reason for non-participation

and a suitable alternative needs to be implemented. May include ILC,
CRMs, etc.
ACCREDITATION REQUIREMENTS
PT / ILC plan:
• The plan shall cover two accreditation cycles (e.g., 2020-2024).
• The plan shall cover all accredited activities in 1 accreditation cycle (over a
5-year period).
• Documented in QMS and reviewed in response to method changes,
equipment, personnel, with revision and approval after each change.
• PT plan and results shall be available for evaluation during assessments.
Plan must kept be current.
ACCREDITATION REQUIREMENTS
PT / ILC plan must include the following:
a) Parameters;
b) Proficiency testing type (PT, ILC; Intra-LC; use of a RM);
c) ID or no. of participants;
d) The name/s and or identification of the PT schemes which the laboratory
intends to participate;
e) Name and make of RM used;
f) The proposed measurement artifact / instrument;
g) The measurement parameters, including range and measurement points;
h) The minimum acceptance criteria for participation;
ACCREDITATION REQUIREMENTS
PT / ILC plan must include the following (continued):
i) How the reference or consensus value is to be established;
j) How the uncertainty of the reference value or the uncertainty of the
interlaboratory comparison is to be established;
k) Responsibility for issue of the PT / ILC report, and who will act as the referee in
the event there are only 2 participants and there is a disagreement in the
results;
l) Where applicable, the typical ranges that cover the scope of accreditation,
particularly where measurements at extremities may pose specific
measurement challenges i.e., high temperatures, and low pressures;
m) Any issues experienced with participating in PT and;
n) Frequency of participation per time period justified by the laboratory.
 ACCREDITATION REQUIREMENTS
PT / ILC plan example:
• Points a-h are documented on the PT/ILC plan.
• Points i-n are available in PT reports, internal PT/ILC procedures, etc.
Accreditation cycle: 2020 - 2024

*PT - Proficiency Testing; ILC – Interlaboratory comparison

Instrument / Test PT Scheme name/ILC Measurement
Test parameter PT type* Frequency Acceptance criteria
method participants range
E. coli PROC 6.2-TM-01 PT [PTS NAME] 10 – 1500 cfu/g 4× per year -2 < Z < 2

• Laboratories should establish clear criteria for assessing the acceptability of PT/ILC
results. These criteria may include allowable limits of deviation from target values,
precision requirements, or other performance indicators relevant to the specific tests
being assessed.
COMMON PROBLEMS ENCOUNTERED
• Failure to have a comprehensive PT/ILC plan.
• Failure to keep the PT/ILC plan up to date.
• Failure to adhere to the PT/ILC plan.
• Not using ISO 17043 accredited schemes when they are available.
• Participation with another laboratory and each laboratory draws up its own report with
different conclusions. (i.e., no procedure implemented before ILC).
• Inappropriate choice of matrix or reference material.
• Trying to pass off a PT/ILC for one test method as being adequate as competence for
another method.
• Failing to prepare a testing protocol before commencing the PT/ILC which means that
different laboratories test at different points and report differently, and statistical evaluation
is not possible).
• The PT/ILC report does not cover the required items listed in the accreditation
documents.
PT / ILC FREQUENCY
The laboratory should define the level and frequency of its participation
after careful analysis of its other quality assurance (QA) measures to ensure
the validity of the results (especially those that are able to disclose, quantify
and follow the development of bias of a stated magnitude).

The level and frequency of participation should be made dependent on the

extent to which other measures have been taken into account.
PT / ILC FREQUENCY
QA measures can include, but are not limited to:
• Regular use of certified reference materials and/or reference materials.
• Comparison of analysis by independent techniques.
• Participation in ILCs for method development/validation and/or reference
material characterization studies.
• Use of internal quality control (IQC) measures.
• Other inter/intra – laboratory comparisons e.g., analysis on blind samples
within the laboratory.
• Robustness of the metrological traceability chain (i.e., are instruments
calibrated under the same conditions as routinely used versus
assumptions on e.g., influence factors or secondary parameters).
PT / ILC FREQUENCY
The level of risk used to determine the frequency of participation can
consider the following aspects:
• Number of measurements undertaken.
• Frequency of tests at a different concentration level.
• Number of different calibration intervals.
• Turnover of technical personnel.
• Experience and knowledge of technical personnel.
• Source of metrological traceability (information and availability of reference
materials, national measurement standards, etc.).
• Known stability/instability of the methodology.
• Complexity and robustness of the methodology.
PT / ILC FREQUENCY
The level of risk determination (continued):
• Significance and final use of measurement data (e.g. forensic science
represents an area requiring a high level of assurance).
• When statements of conformity are required and changes in related
specifications are made.
• Risks and opportunities associated with the laboratory activities, in
particular those that will prevent, or reduce, undesired impacts and
potential failures in the laboratory activities and achieve improvement.
• Extent of validation and/or verification.
• Any requirements for frequency and type of PT participation from other
sources, e.g., legislation, customers, etc.
PT / ILC SAMPLE TREATMENT
PT/ILC samples shall be processed by personnel who routinely handle test
items and perform the analysis.
a) Assign PT responsibilities to routine personnel:
• Personnel rotation and training: Ensure that all laboratory personnel
who perform regular tests are also involved in processing PT/ILC
samples. For instance, an analyst who routinely performs a particular
test should handle PT/ILC samples for that same test.
• Training programs: Conduct training sessions to ensure that all
relevant personnel are familiar with the specific requirements and
protocols for handling PT/ILC samples. This includes areas such as
sample reception, sample preparation, analytical processes like
conducting the actual tests, and end-processes like data entry and
result reporting.
PT / ILC SAMPLE TREATMENT
b) Simulate routine workflow with PT / ILC samples:
• Integrated PT/ILC processing: Incorporate PT/ILC samples into the
regular workflow without distinguishing them from customer samples in
terms of handling and processing, i.e., they should process the PT/ILC
samples in the exact same manner.
• Blind testing: When possible, use blind PT/ILC samples, where the
laboratory personnel do not know which samples are for PT/ILC. This
ensures that they follow the standard procedures as they would with
regular customer samples.
PT / ILC SAMPLE TREATMENT
c) Documentation and record keeping:
• Simulate normal practices: Laboratory personnel should strictly follow
all documented procedures for handling PT/ILC samples, ensuring that
records are meticulously maintained, just as they would be for actual
customer samples. This includes logging sample receipt, preparation,
testing, and result reporting, thereby integrating PT/ILC processing
seamlessly into routine laboratory operations.
PT / ILC SAMPLE TREATMENT
Example scenario: PT/ILC sample to be tested for pesticide residue analysis
• Sample reception: The same personnel who receive and log samples also receive PT/ILC
samples. They check the integrity, label them appropriately, and document their receipt.
• Preparation: Technicians who prepare customer samples for analysis also prepare the
PT/ILC samples, following the same protocols for extraction, aliquoting, and storage, etc.
• Testing: The analysts who conduct routine pesticide residue analysis use the same
equipment and methods for PT/ILC samples as they do for customer samples.
• Quality Control: Implement routine quality control checks and calibration using the same
standards and controls for both customer and PT/ILC samples.
• Result recording: Results from PT/ILC samples are entered into the laboratory information
system by the same personnel who enter customer results, ensuring consistency and
accuracy.
• Review and submission: Technical signatories review the results for both customer and
PT/ILC samples, ensuring that any discrepancies are addressed before submission to the
PT/ILC provider.
WORKING RANGES AND PT / ILC
Working range: The working range of a test method is the span between
the lowest and highest concentrations of an analyte that
the method can accurately and reliably measure. This
range is determined during the validation process of the
test method, involving rigorous testing to confirm the
method’s precision, accuracy, and linearity within this span.

PT samples must be analysed and fall within the working range of the test
method. For example:
PT value

Working range:
10 ppm 1000 ppm
PERSONNEL COMPETENCY
Establishing personnel test method competency can generally be
divided into 3 parts:
a) Initial and on-going training as per a documented training program.
b) Witnessing / observation of the performance of a test method to
determine that the activity is being performed correctly.
c) Participating in PT/ILC to determine that the results obtained are
accurate.

Including both witnessing activities and participation in PT/ILC is a crucial

aspect to ensuring the continuous competence of personnel and the
laboratory as a whole unit.
PERSONNEL COMPETENCY
• Participating only in witnessing or only in PT/ILC is not sufficient as it only
provides one aspect of the competency evaluation, i.e., either correct
performance or accuracy, but not both.
• If a laboratory only participates in PT/ILC and excludes witnessing, and the
PT results are acceptable, it does not indicate that the method is being
performed correctly and will very likely lead to result deviations/failures in
future.
In other words: Competency = Correct performance + Accurate results
EVALUATING PT / ILC RESULTS
Example: Microbiology report snippet
EVALUATING PT / ILC RESULTS
Example: Microbiology report snippet (continued)
EVALUATING PT / ILC RESULTS
Example: Chemistry report snippet
EVALUATING PT / ILC RESULTS
Example: Chemistry report snippet (continued)
EVALUATING PT / ILC RESULTS
1. Review and Analysis:
a) Comprehensive review: Thoroughly review the PT report, including
results, statistical analysis, and performance evaluations.
b) Understand results: Ensure that the laboratory understands its
performance and any areas where it did not meet the acceptable criteria.

2. Root Cause Analysis:

a) Investigate issues: Conduct a root cause analysis for any questionable
or unacceptable results.
b) Identify causes: Determine the underlying reasons for any deviations,
such as procedural errors, equipment malfunctions, or human errors.
REFERENCE VS CONSENSUS VALUES
There are, as described in ISO 13528, essentially five methods available to
obtain the assigned value, a working estimate of the true value:
1. Formulation
2. Certified reference values
3. Reference values
4. Consensus values from expert laboratories
5. Consensus value from participants
REFERENCE VS CONSENSUS VALUES
In proficiency testing, both consensus values and reference values are used
to assess the performance of laboratories, but they are derived and used
differently. Here’s an explanation of the differences between these two types
of values:
1. Consensus Value:
• A consensus value is a value agreed upon by a group of participating
laboratories in a proficiency testing (PT) scheme. It is usually derived
from the statistical analysis of the results submitted by the participants.
REFERENCE VS CONSENSUS VALUES
a) Derived from:
• Statistical Calculation: It is typically calculated using statistical
methods such as the mean or median of all participants' results,
sometimes excluding outliers to improve reliability.
• Participant Agreement: It is based on the collective agreement or
average of the results from the participating laboratories.
b) Use:
• Performance Evaluation: The consensus value serves as a
benchmark to evaluate the performance of individual laboratories by
comparing their results against this collective value.
• Inter-Laboratory Comparisons: It helps in understanding the level of
agreement among different laboratories performing the same test.
REFERENCE VS CONSENSUS VALUES
2. Reference value:
• A reference value is a value that is established based on a higher
degree of certainty and accuracy. It is often determined using a
reference material or through a reference method, which is highly
accurate and traceable to national or international standards.
REFERENCE VS CONSENSUS VALUES
a) Derived from:
• Reference materials: It is derived from the use of certified reference
materials (CRMs) with known properties and values.
• Reference methods: It can also be determined using standardized methods
that are recognized for their high accuracy and reliability.
• Traceability: It is traceable to international or national measurement
standards, ensuring its high accuracy and credibility.
b) Use:
• Calibration and validation: The reference value is used for the calibration of
instruments and validation of methods to ensure they produce accurate
results.
• Quality Assurance: It provides a benchmark of the highest accuracy against
which laboratory results can be compared to assess their accuracy and
precision.
REFERENCE VS CONSENSUS VALUES
Key differences:
Aspect Consensus value Reference value
Derived from the collective results of Established from highly accurate
Source of value
participants in the PT scheme. reference materials or methods.
Generally, less accurate, as it reflects
Highly accurate and traceable to
Level of the central tendency of participants'
recognized standards, providing a more
accuracy results, which may include some degree
reliable benchmark.
of variability and error.
Used primarily for comparative purposes Used for calibration, method validation,
Purpose among participating laboratories to and as a standard for ensuring
gauge relative performance. measurement accuracy and traceability.
Determination Calculated using statistical analysis of Determined using certified reference
process participant data. materials or high-precision methods.
After collecting results from 50 A certified reference material with a
participating laboratories, the median known concentration of the analyte,
Example
value of these results is calculated to be traceable to a national standard,
the consensus value. provides the reference value.
ACCEPTANCE CRITERIA
There are generally 6 different types of evaluation techniques and
acceptance criteria used to evaluate PT results. The criteria selected
depends on the PT provider and may include the following:
a) Z-score (most commonly used and measurement uncertainty not taken into account)
b) Z’-score (standard uncertainty of the assigned value is taken into account)
c) Zeta-score (standard uncertainty of the assigned value and the participants result is taken into account)
d) En number (expanded uncertainty of assigned value and participants result is taken into account)
e) %CV
f) %Bias
The upper and lower acceptable limits are determined by the PT provider
and shall be implemented by the laboratory.
ACCEPTANCE CRITERIA: %CV

Percentage coefficient of variation is a statistical measure of the

%CV relative variability of data, often used PT to assess the precision and
consistency of laboratory results.

Indicates that the laboratories' results are close to each other,

Low %CV
demonstrating high precision and consistency.

Suggests greater variability among the laboratories' results, indicating

High %CV
potential issues with precision and consistency.
ACCEPTANCE CRITERIA: %BIAS
It is a measure used to assess the accuracy of a laboratory's results
by comparing them to a known or consensus value. It quantifies the
%Bias difference between the laboratory's result and the target value as a
percentage, helping to evaluate how close the laboratory's
measurement is to the true value.
Indicates that the laboratory’s results are very close to the target
%Bias close to 0%
value, demonstrating high accuracy.
Indicates that the laboratory’s results are higher than the target
Positive %Bias
value.

Negative %Bias Indicates that the laboratory’s results are lower than the target value.
ACCEPTANCE CRITERIA: Z-SCORE

It is a statistical measure that quantifies the number of standard deviations a

data point is from the mean of a data set. It is used to assess how far away a
Z-score
particular result is from the average, allowing for comparisons across different
data sets or different proficiency testing rounds.
Z=0 The data point is exactly at the mean.

Z>0 The data point is above the mean.

Z<0 The data point is below the mean.

Z<1 The data point is within one standard deviation of the mean.
The data point is one or more standard deviations away from the mean,
Z≥1
indicating it is relatively unusual.
ACCEPTANCE CRITERIA: Z-SCORE
Formula for Z-score:
The z-score for a data point 𝑋 is calculated using the formula:
ACCEPTANCE CRITERIA: Z-SCORE
Calculation:
• After all participating laboratories have submitted their results for a
particular test, the mean (𝜇) and standard deviation (𝜎) of these results are
calculated.
• Each laboratory's result is then converted to a z-score using the above
formula.
ACCEPTANCE CRITERIA: Z-SCORE
Simply put:
Performance evaluation: Think of the z-scores as a QC chart in your
• -2 ≤ z ≤ 2: The result is generally laboratory and the z-score represents the number
considered acceptable and indicates of standard deviations your result is from the true
value.
satisfactory performance.
3 3sd
2 2sd
• 2 < z < 3 or -2 > z > -3: The result is a
1 1sd
warning signal, suggesting that the

Z-score
0
laboratory should review its procedures
and check for potential issues. -1 1sd
-2 2sd
-3 3sd

• -3 ≤ z ≥ 3: The result is unacceptable,

indicating significant deviation from the
expected performance. The laboratory
should investigate and correct any
issues.
ACCEPTANCE CRITERIA: Z-SCORE
Example:
CAUSES FOR POOR PERFORMANCE
The reasons for obtaining a poor performance are "unfortunately" numerous, resulting in a
time consuming and complex investigation. Example include, but are not limited to:
a) Sample preparation (e.g., weighing, drying, extraction, digestion, clean-up, dilution, etc.)
b) Equipment failure or servicing problem
c) Human error (e.g., inappropriate training, transcription)
d) Calibration
e) Selection of measurement method
f) Calculation error
g) Reporting problem (e.g., format, units, detection, interpretation)
h) PT item problem
i) Sample transport and storage
j) Primary sampling
k) Sample tracking (e.g., labelling, chain of custody)
l) PT scheme provider problem
CAUSES FOR POOR PERFORMANCE
In order to identify the root cause of poor performance, it is important to
focus on the potential causes, such as:
a) clerical error
b) technical problem (e.g., method, equipment, training, internal quality
controls)
c) problem related to the PT scheme (e.g., inadequate scheme,
inappropriate evaluation)

It may be possible that after a thorough investigation, the origin of the poor
performance is not identified. A single poor performance could then be
attributed to random error. If it is a repeated poor performance, then the
analytical process should be questioned.
 This webinar is sponsored by NSI Lab Solutions

THANK YOU

JANINE MUSSO | +27 82 506 8050 | [email protected]

LAUREN STAINBACK | +1 800 234 7837 | [email protected]

www.michemdynamics.com