Analytical Method Replacements
Regulatory-Approved Strategies and Case Studies
Stephan O. Krause, PhD
MedImmune
1
Replacing Analytical Methods
Presentation Agenda:
I. Introduction and Strategies
The Analytical Method Life Cycle (Process Map)
AMR background and rationale
General Strategies for Qualitative and Quantitative Methods
General test method performance characteristics compared
II. Analytical Method Replacement (AMR)
Case Studies
Krause/PDA, 2011 2
Analytical Method Life Cycle
AMR Studies
AMR Studies
Krause/PDA, 2011 3
Analytical Method Replacement Studies
Why do we need it ?
What is AMR ?
• AMR is the demonstration of comparable (“equivalent or better”) test method
performance of a modified/new method done during or after AMV completion.
• AMR should be demonstrated for methods replacing approved methods (in-house
licensed, compendial, or otherwise recognized).
Why is it important ?
• A continuous suitable test method performance must be assured for
safety/efficacy/quality to patients (linking to licensed specs and clinical data).
• This also assures a quality continuum (=> release) for the firm.
How exactly can we demonstrate AMR ?
• Follow ICH E9 and CPMPs Points to Consider guidelines.
• Demonstrate “equal or better” by testing for non-inferiority, equivalence, or
superiority depending on assay type and need (risk).
• Compare particular test method performance criteria per ICH Q2(R1).
Krause/PDA, 2011 4
Analytical Method Replacement
Suggested Performance Characteristics and Statistics
Potency or
Identification
ICH Q2(R1) Limit Test Limit Test Content (Purity
Test
Category (Qualitative) (Quantitative) or Range)
(Qualitative)
(Quantitative)
Accuracy Not Required Not Required T-test, TOST; T-test, TOST
Some Data could
be at QL level
Intermediate Not Required Not Required ANOVA, mixed ANOVA, mixed
Precision linear model, or linear model, or
other F-test other F-test
statistics statistics
Specificity Probability Probability and/or Not Required Not Required
and/or Chi- Chi-Squared for
Squared for Number of Correct
Number of Observations
Correct
Observations
Detection Not Required Depends on how Not Required Not Required
Limit DL was
established.
Probability
calculations may
be used
Krause/PDA, 2011 5
AMR Categories (from ICH E9)
Equivalence
Non-inferiority
Superiority
Krause/PDA, 2011 6
Krause/PDA, 2011
Equivalence Limit
-95% CI
Mean Difference
+95% CI
Demonstrating Equivalence
Equivalence Limit
7
Demonstrating Non-Inferiority
Non-inferiority Testing (ICH E9)
Current Method = Reference
Non-Inferiority Demonstrated
Desirable Direction/Range
Non-Inferiority Limit
Mean Difference
+95% CI
-95% CI
- Delta 0
Current Method No difference New Method
“Better Results” “Better Results”
Krause/PDA, 2011 8
Demonstrating Superiority
Superiority Testing (ICH E9)
Current Method = Reference
Superiority Demonstrated
Desirable Direction/Range
Superiority Limit (0)
Mean Difference
+95% CI
-95% CI
0
Current Method New Method
“Better Results” No difference “Better Results”
Krause/PDA, 2011 9
AMR – Qualitative Method Models
When comparing qualitative data, non-inferiority or superiority models
should be used and three possible outcomes are illustrated below.
Inferiority. A particular performance characteristic compared provides
significantly inferior results for the current method, therefore failing to
demonstrate AMR.
Non-inferiority. The new method performs at a comparable level. The new
method could be superior, equivalent, or insignificantly inferior. All three
outcomes are acceptable outcomes to demonstrate non-inferiority.
Superiority. The new test method is superior. When testing for superiority,
only this outcome is acceptable.
Krause/PDA, 2011 10
AMR – Quantitative Method Models
When comparing quantitative data (for accuracy/matching), two possible
outcomes are illustrated below:
No equivalence for accuracy: There is a significant difference between
results from both test methods. Both differences, lower and higher, are
statistically unacceptable outcomes. The new method may be acceptable
if specifications changes or other adjustments can be made.
Equivalence: The difference between both methods is insignificant and
the new method performs at a comparable level.
Krause/PDA, 2011 11
General Points to Consider for AMR
Studies
When using one of the suggested three AMR categories, the following
major points should be considered:
The comparison category should be explained and justified. For
example, a non-inferiority test may be suitable, if all outcomes (non-
inferiority, equivalence, and superiority) are acceptable, and if the
new method is superior in other aspects such as faster test results
and/or increased sampling/testing.
The AMR protocol should include a detailed study design and the
statistical test(s) to be used.
The pre-specified maximum allowable difference(s) should be
justified. The difference limit(s) should strike a balance among
possible opposing incentives: Impact on patient and/or manufacturing
versus AMR results are “comparable” (when they may not really be).
Krause/PDA, 2011 12
Demonstrating Non-Inferiority
Introduction
• A faster and technologically advanced method for sterility testing was
validated and compared to the compendial EP/USP Sterility Test.
• The non-inferiority comparison at the 95% confidence level (p=0.05) was
chosen with a pre-specified delta of –10% versus the compendial (current)
method.
• Justification: Non-inferiority, equivalence, and superiority are all acceptable
outcomes, and the increased testing frequency of daily (n=7 per week) for the
new sterility versus twice weekly (n=2 per week) for the EP/USP Sterility test
significantly increases the likelihood of detecting organisms with the new
method.
• Results/Conclusion: The 95% confidence level includes 0 (no difference) and
lies entirely to the right of the pre-specified delta of –10%. The comparison
results obtained indicate that the candidate method is not inferior to the
EP/USP sterility test method.
Krause/PDA, 2011 13
Demonstrating Non-Inferiority
Results for the Non-Inferiority Test: Candidate Method vs. USP Sterility
Method Positives Total Samples Positives-to-Fail
Ratios
Candidate 225 300 0.75
EP/USP 232 300 0.77
Statistical Results
Difference = p (new method) - p (EP/USP)
Estimate for difference: -0.024
95% lower confidence interval limit for difference: -0.08
Results/Conclusion:
The 95% confidence level includes 0 (no difference) and lies entirely to the
right of the pre-specified delta of –10%.
The comparison results obtained indicate that the candidate method is
not inferior to the EP/USP sterility test method.
Krause/PDA, 2011 14
Non-Inferiority of New Method (vs.
Current/Compendial) Demonstrated
Non-Inferiority Limit
+95% CI
-95% CI
-2.4%
Krause/PDA, 2011 15
Demonstrating Superiority
Introduction
From the previous example for non-inferiority: When the relative testing frequency
of our example of n=7 (new method) versus n=2 for the compendial method is
integrated in our comparison studies, the superiority of the new method could be
demonstrated.
Krause/PDA, 2011 16
Demonstrating Superiority
Results
Candidate Method (7x) vs. EP/USP Sterility (2x):
Sample Positives Total Probability 95% CI for Probability
Candidate 225 300 0.9999 0.9997 – 1.0000
EP/USP 232 300 0.947 0.921 – 0.967
Results/Conclusions:
Superiority at the 95% confidence level could be demonstrated because the new
method’s 95% confidence interval (0.9997-1.0000) for the positive-to-fail
probability (0.9999) lies entirely to the right of the 95% confidence interval
(0.921-0.967) of the compendial method’s positive-to-fail probability (0.947).
The superiority test was passed with a much greater relative margin than the
non-inferiority test. This is a good example why we should always consider
upfront which comparison study to select and how to defend our strategy in
regulatory submission.
Krause/PDA, 2011 17
Non-Inferiority of New Method (vs.
Current/Compendial) Demonstrated
Desirable Direction/Range
(Lower 95% CI > 96.7%)
New Method
(not drawn to scale)
USP/EP
Probability: 90% 91% 92% 93% 94% 95% 96% 97% 98% 99% 100%
USP/EP Method New Method
“Better” “Better”
Krause/PDA, 2011 18
Demonstrating Equivalence
Introduction
Because of anticipated supply problems for critical SDS-PAGE materials, it
was decided to develop and validate a capillary zone electrophoresis (CZE)
method that will replace the current (licensed) electrophoretic method.
The method performance characteristics for a quantitative limit test,
accuracy and intermediate precision, are compared.
For accuracy: A delta of plus/minus 1.0% was chosen for the equivalence
category between both impurity levels from the analysis of historical release
data with respect to the current release specifications (for SDS-PAGE).
Both methods were run simultaneously (side-by-side) for each of a total of
n=30 reported results were compared by two-sided matched-paired t-test
statistics with pre-specified equivalence limits of plus/minus 1.0% (% =
reported percent and not relative percent).
Krause/PDA, 2011 19
Demonstrating Equivalence
Results
Equivalence Test Results Comparing Current Method to CZE:
Sample Size (n): 30
Hypothesized Difference in Mean: 0%
Minus Delta: -1.0%
Plus Delta: +1.0%
SDS-PAGE Mean (n=30): 3.8%
CZE Mean (n=30): 5.1%
95% confidence interval of CZE results (vs. SDS-PAGE): 4.88-5.32%
Krause/PDA, 2011 20
Equivalence of New Method Not Demonstrated
(New method’s result are different)
Equivalence Unclear
(stat. different)
+ 95% CI
- 95% CI
Passes Equivalence
(stat. not different)
Equivalence Limit
Equivalence Limit
+ 95% CI
- 95% CI
Fails Equivalence
(stat. different)
Passes Equivalence
+ 95% CI
- 95% CI
(stat. different)
+ 95% CI
- 95% CI
Passes Equivalence
(stat. different)
+ 95% CI
- 95% CI
- 1.0% 0 + 1.0%
New Method No difference New Method
“Lower Results” “Higher Results”
Krause/PDA, 2011 21
Demonstrating Equivalence
Results
The 95% confidence interval of the CZE method (4.88-5.32) lies entirely
over the current assay mean (3.8%) plus the positive delta (3.8% + 1.0% =
4.8%). This means that the CZE results for our impurity are significantly
higher than our licensed method.
The expected drift in results is significantly higher than our pre-specified
limit that was based on the gap of our historical release results relative to
the release specifications.
Unless available from AMV studies, some additional data pairs for
impurity levels around the specification limit may need to be run.
Specifications may need to be changed for the use of CZE method.
Krause/PDA, 2011 22
