Importance of Proficiency

Testing/Interlaboratory
Comparison
Dr. Erik J.M. Konings, Nestlé Research Centre, Lausanne, Switzerland
“International Workshop and Training Program on Good Food Laboratory Practices” to be held on
November 15-19, 2016 in Mumbai.

Organized By
Food Safety and Standard Authority of India, Government of India
&
International Life Sciences Institute-India
&
Export Inspection Council, Government of India
 Agenda
1 Introduction

2 General requirements for Analytical Quality Assurance

3 Introduction to Proficiency Testing (process, statistical design, benefits)

4 Nestlé Proficiency Testing

5 AOAC INTERNATIONAL Proficiency Testing (to start as from March 2017)

 Poor Quality data can result in substantial
problems and high costs

Consequences of reporting data of poor quality for the end-users?

• High costs involved in repeated measurements to correct poor data;

• Faulty decision-making;
• Damage to reputation and credibility;
• Possible loss of business;
• Legal and financial liability from the use of poor data.
 The various tools helping laboratories to
demonstrate the reliability of their analytical data
 ISO/IEC 17025: General requirements for the
competence of testing and calibration
laboratories
• Chapter 5.9: Ensuring the quality of test and calibration results
 What is Proficiency Testing?

ISO/IEC 17043:2010 General requirements for Proficiency

Testing

“Proficiency testing schemes (PTS) are interlaboratory comparisons

that are organized regularly to assess the performance of analytical
laboratories and the competence of the analytical personnel.”
 Proficiency testing – The Process
P-test Provider Testing laboratory

Prepare Homogeneity and Ship Analyse

samples Stability testing samples samples

Conduct
Prepare Analyse Report
Root cause
report results results
analysis
How often does a lab has to participate?
 Proficiency testing – The Statistical design
How is the assigned value established?
The assigned value is the value selected as being the best estimate of the “true value” for the parameter under test. The
method used to determine the assigned value may vary depending upon the particular scheme and test material.

As per ISO 13528 “Statistical methods for use in proficiency testing by interlaboratory comparisons”, there are five ways to
determine the assigned value :
• Formulation
• Certified references values
• Results from one laboratory
• Consensus from experts laboratories
• Consensus value from participants (e.g. median)

For Qualitative tests, participant’s results are compared against the intended result based on formulation.

For Microbiology samples, participant’s results are transformed by converting them to log10 before the statistical analyses
is undertaken.
 Proficiency testing – The Statistical design
How is the standard deviation for proficiency assessment
(SDPA) determined?
The SDPA expressed the acceptable difference between the laboratory results and the assigned value. The method used to
determine the assigned value may vary depending upon the particular scheme and test material.

As per ISO 13528 “Statistical methods for use in proficiency testing by interlaboratory comparisons” there are five ways to
determine the SDPA:
• Prescribed value
• By perception
• From a general model
• From results of precision experiment
• From data obtained on a round of proficiency testing scheme
 Participant’s result converted in Z-score.
How is the z-score calculated?
z-score is a scaling that makes the errors of equal magnitude in different situations, (normalized different errors in
different methods) allowing to compare the performance of the laboratories observed in different tests and with
different analytes.

As per ISO 13528, it is the ratio between the deviation of each individual result from the assigned value, and the
standard deviation for proficiency assessment SDPA.
How do I interpret the z-score?
 What to do with unsatisfactory results?

ISO/IEC 17025 Management Requirements – 4.9 Manage non conformities

If anything in the lab is not conform to what it shall be, a procedure must be activated to:
• Take immediate actions (repeat analyses, contact customer if necessary, remove an instrument out of order)
• Evaluate the cause and take corrective actions (train people, replace out of spec instrument, etc)
• Evaluate consequence of the work done
• Verify that the corrective actions prevent similar problems
Example of FAPAS report

If the following criteria is not met:

u(xpt) < 0,3 *σpt
(standard uncertainty of the assigned value lesser
than 0.3 times the Standard Deviation of Proficiency
Assessment)

The uncertainty is not considered as

negligible and needs to be included in
the interpretation of the results. In
this case other statistical design
should be considered (e.g. use of z’-
score).
Example of FAPAS report
There are multiple benefits taking part to
Proficiency Testing

Demonstrate good performance of the laboratories

Detecting method related issued

Raising trust in supplier’s certificates

Comparing the various analytical methods

Generating reference materials for methods monitoring

Example of Method Related Issues detected in P-test
 Example of method performance comparison
Source: LGC report
 Nestlé Proficiency Testing
• Is an accredited ISO/IEC 17043 Proficiency Testing Provider, by the Swiss Accreditation Service.

• Comprehensive testing programs offering a wide range of analytes and matrices

• Work with external P-test providers to offer the most suitable P-test programme adapted to Nestlé
product specifications (analyte, matrix, concentration)

• Participation to the Nestlé Proficiency Tests programme is mandatory for the Nestlé laboratories

• For parameters and/or matrices not covered by the Nestlé P-Test program, each laboratory must also
participate in Proficiency test schemes provided by external organizations like FAPAS, LGC, Bipea etc.
accredited as a P-test provider based on ISO/IEC 17043
 Who are the Nestlé users of Proficiency Testing ?
900 laboratories
• Internal laboratories
• External laboratories (e.g. private,
governmental, supplier)

Worldwide located (90 countries)

Using around 600 test methods and 120

official methods

Delivering 128 million of analytical results

per year
 Nestlé P-test Program - key figures
CHEMICAL NUTRIENTS CHEMICAL CONTAMINANTS MICROBIOLOGY
3 domains (organised in collaboration with Fapas, LGC, Bipea) (organised in collaboration with LGC)

 Fats  Allergens  Probiotics

 Sugars  Antibiotics  Pathogens
 Vitamins  GMO  Hygiene indicators
244 analytical  Minerals  Melamine& cyanuric acid
determinations  Proximates  Metallic contaminants
 Fatty acids  Mycotoxins
 Nitrite/nitrate
 Pesticides
 Authenticity

 Infant formula  Oil  Milk powder

 Cereals  Spicy  Lyophilized
 Pet food  Fruit Purée
 Health care product  Milk powder
42 food products  Confectionery  Cereals
 Sweetened/Unsweetened  Meat
condensed milk
 Bouillon
 Water
 Beverages

41’000 analytical results /year

 How do we treat non-satisfactory P-test results at
Nestlé ?
• A root cause analysis (RCA) must be conducted after every non-satisfactory results i.e. for each unsatisfactory
and questionable result.
• Remedial action must be taken to solve the problem
• The investigation must be carried out immediately after each scheme round report has been published.
• Repeat samples are sent to the laboratory that failed to verify effectiveness of the corrective actions
• Two consecutive unsatisfactory results must trigger a specific action
• Key learnings and P-test best practices are shared with laboratories to prevent gaps reoccurrence

Example of key learning

The challenges from the Nestlé P-Test provider
perspective
Barrier to start new P-Tests

• Specific/exotic P-test demands

• Less than 8 results engaged, limitation of the statistical design
• Difficulties to meet criteria of homogeneity and stability check

Ensure samples delivery in every countries

• Increasing number of documentation requested by the customs

Laboratory gives special treatment to P-Test sample

• Testing multiple times and reporting an average

• Selection of senior technicians, not involved in everyday routine analysis, to
perform the analysis of P-Test samples
 AOAC Proficiency Testing

• AOAC INTERNATIONAL is an A2LA accredited Proficiency Testing Provider.

• First accredited program for analytical food laboratories in the United States, in
2001.

• Comprehensive testing programs offering a wide array of analytes and matrices

• AOAC’s proficiency testing samples arrive at the laboratory ready to analyze like
routine samples. The samples undergo rigorous homogeneity testing prior to
shipment
 Methods endorsed by CODEX

 Vitamin A ISO 20633:2015 AOAC 2012.10

 Total nucleotides ISO 20638:2015 AOAC 2011.20
 Pantothenic acid ISO 20639:2015 AOAC 2012.16
 Iodine ISO 20647 | IDF 234:2015 AOAC 2012.15
 Cr, Mo, Se ISO 20649 | IDF 235:2015 AOAC 2011.19
Implementation of Infant Formula and Adult
Nutritionals PT Program
• As of March 2017
• Nutrients:
• Fatty acids • Vitamin A
• Vitamin E • Myo-Inositol
• Vitamin B12 • Nucleotides
• Panthothenic acid • Ultra Trace Minerals (Cr, Mo, Se)
• Iodine

• Quarterly
• $650 per Shipment
• 30 days for analysis
Example of Method Performance Information
 Why is participating to Proficiency
Testing/Interlaboratory comparison important?
• Proficiency test is a key element of quality assurance to assess the
proficiency of laboratories
• Proficiency tests are pivotal to help demonstrate the reliability of
laboratory results. P-test helps to identify problems in laboratories
and initiate actions for improvement
• P-test plays a key role as it maintains confidence in laboratories
capability to perform analysis, irrespectfully whether or not
accredited to ISO 17025 (also interesting for third parties, e.g.
authorities, customers)
Acknowledgement
Thanks to Laura Coisne, Nestlé P-test Manager, and the
Nestlé P-test team for providing the information!

CONTACT:
[email protected]