THE VALUE QUESTION
HOW TO REALLY MEASURE
• Is a medical technology good value for the
COST-EFFECTIVENESS cost?
– Are buying sufficient health for what we are
spending?
Henry Glick, Ph.D. * Comparison of costs and effects with other therapies for
the same disease
University of Pennsylvania * Comparison with other therapies for different diseases
32 Annual Symposium • Need to know the value of the outcome we
“Clinical Microbiology Update 2001” are assessing
– Is spending $5,000 to increase the accuracy of a
test by 1 percentage point good value or not?
IMPLICATIONS COMMON METHODS
• Decision analysis
• The need to make comparisons with other
e.g., Frazier et al. Cost-effectiveness of screening for
diseases and to know the value of the outcome colorectal cancer in the general population. JAMA
implies final outcomes more important than 2000;284:1954-61.
intermediate ones
• Randomized trials
– Years of life saved
– Quality-adjusted years of life saved (QALYS) • References: Economic Evaluation in Health
Care: Merging Theory with Practice. Ed. M
• Other outcomes (e.g., cost per correct Drummond and A McGuire. Oxford University
diagnosis) can be calculated, but will be of less Press, 2001.
use when arguing for the value of medical Cost-Effectiveness in Health and Medicine. Ed. MR
technologies Gold, et al. Oxford University Press, 1996.
DECISION ANALYSIS TYPES OF MODELS
• Formal technique for combining data • Decision trees
from several sources using simulation
– Graphic representation of a set of
models
equations
• Abstract relevant parts of reality
• Survival function models
• Forces the identification of options and
potential outcomes • State transition or Markov models
• Forces decision maker to make
assumptions explicit
1
Treat Vs. Test and Treat STEPS IN THE DEVELOPING A
DECISION TREE
Disease Test + $26,000
+ 0.9 5 QALYS
Test and 0.2 Test - $10,000
1. Develop the structure of the model (e.g.,
Treat 0.1 1 QALYS draw the decision tree)
Disease Test + $16,000
- 0.15 14 QALYS 2. Identify probabilities that different
0.8 Test - $1,000
events in the model occur
Treatment
0.85 15 QALYS
Decision Disease
+ $25,000
3. Identify the outcome values
0.2 5 QALYS
Treat
Empirically
4. Analyze the tree (evaluate the principal
Disease
analysis / analyses)
- $15,000
0.8 14 QALYS 5. Perform sensitivity analysis
Treat Vs. Test and Treat SOURCES OF PROBABILITIES
• Published reports of clinical and
Disease Test + $26,000
+ 0.9 5 QALYS
epidemiological studies
Test and 0.2 Test - $10,000 – Intervention Study
Treat 0.1 1 QALYS * Natural history of disease
Disease Test + $16,000
* Effect of intervention
- 0.15 14 QALYS
0.8
Treatment
Test - $1,000 – Intervention Study and Observational Study
0.85 15 QALYS
Decision Disease * Natural history: observational study
+ $25,000 * Intermediate effect of intervention: intervention study
Treat 0.2 5 QALYS
* Final effect of intervention: observational or intervention study
Empirically
Disease
-
• Consensus of expert opinion
$15,000
0.8 14 QALYS • Personal opinion
IDENTIFYING THE OUTCOMES RESULTS
• Costs
– Direct medical costs • For each strategy:
– Direct nonmedical costs – Expected costs
– Indirect costs – Expected outcomes
• Effects
– Survival • Comparison of costs and effects
– Years of survival – Difference in expected costs
– Quality-adjusted years of survival – Difference in expected effects
– Intermediate measures such as – Ratio of difference in expected costs and
sensitivity/specificity/ likelihood ratios effects
2
SENSITIVITY ANALYSIS POTENTIAL LIMITATIONS
• When events can repeat themselves
• Confidence intervals (e.g., when one can develop cancer this
– “Second-order” Monte-Carlo simulation year, next year, etc.), decision trees
may become too “bushy”
• Robustness of results • Trees generally don’t directly account
for timing of events (e.g., that one can
• Threshold analysis develop cancer this year, next year,
etc.)
• Markov or state transition models may
help overcome these problems
MARKOV OR STATE Colorectal Cancer States
TRANSITION MODELS No
Polyps
• Describe the disease process by simplifying it into a set Disease
of states
• Disease progression represented probabalistically as a
set of transitions among the states during fixed intervals Local
Death
of time (e.g., 6 month or 1 year intervals) CA
• Effects of an intervention (e.g., a diagnostic test to
detect polyps and cancer) described as a change in the
transition probabilities Regional Distant
CA CA
• Assess resource use and QALYS as a function of being
in a state for a period
STEPS IN DEVELOPING A RANDOMIZED TRIALS
MARKOV MODEL
• Considered the gold standard for evaluating the
1. Enumerate the states efficacy, effectiveness, and efficiency of medical
technologies
2. Define allowable state transitions
• Potential advantages:
3. Associate probabilities with the transitions – Provide head-to-head comparison between technology
under evaluation and alternative
4. Identify a cycle length
– Collect costs and outcomes in same individuals, and
5. Identify an initial distribution of patients within the states maintain correlations between costs and outcomes
6. Identify the outcome values • Potential disadvantages
– May reflect idealized rather than usual practice
7. Analyze the Markov model
– May not provide sufficient follow-up to determine final
8. Perform sensitivity analysis outcomes
3
STEPS IN CONDUCTING ECONOMIC FOUR ISSUES
EVALUATION IN RANDOMIZED TRIAL
• Step 1: Quantify the costs of care • What health care resource use should be
measured (data collection)?
• Step 2: Quantify outcomes
• How should the resources be valued?
• Step 3: Assess whether and by how much average costs
and outcomes differ among the treatment groups • How many subjects should be included in the
study (sample size)?
• Step 4: Compare magnitude of differences in costs and
outcomes and evaluate "value for the cost"
(e.g., by reporting a cost effectiveness ratio or the • For time-limited trials of therapies with longer-
probability that the ratio is acceptable) term effects: How should long-term results be
projected?
• Step 5: Conduct sensitivity analysis
WHAT HEALTH CARE RESOURCE USE HOW SHOULD THE
SHOULD BE MEASURED? RESOURCES BE VALUED?
• Ideal: All resource use • SOURCES OF U.S. UNIT COST DATA
– Hospital charges adjusted using cost to
• In practice: charge ratios
– Services that make up a large portion of the
difference in treatment between patients randomized – Internal hospital costing system data
to the different therapies under study
– Diagnosis related group payments (Hospital
* Estimate of the cost impact of the therapy stays)
– Services that make up a large portion of the total – Resource-based relative value units
(Physician services/procedures)
* Reduces likelihood that differences among unmeasured
services will lead to bias – Trial-specific costing exercise (e.g., time and
* Provides measure of variability of costs motion studies
HOW MANY SUBJECTS SHOULD BE SAMPLE SIZE DATA
INCLUDED IN THE STUDY? REQUIREMENTS
• Early 1990's approach: Select the larger of the
• Difference in costs (old and new)
sample sizes needed for estimating pre-specified
cost and effect differences • Difference in effects (old and new)
– i.e., what sample size was required to identify a $1000 • Standard deviation of costs (old and
difference in costs, and what was required to identify a new)
10% reduction in mortality
• Standard deviation of effects (old and
• Current approach: Estimate the number of study new)
subjects needed to rule out unacceptably high
upper confidence limits for the cost-effectiveness • Correlation of costs and effects (new)
ratio
4
15000
$6,277 HOW SHOULD LONG-TERM
$22,971 RESULTS BE PROJECTED?
12000
• For time-limited trials of therapies with potentially
Incremental Costs
9000 $4,407 long term effects, one should evaluate the costs
and outcomes that were observed during the trial
6000
– Maintain the same time horizons for costs and
$1,140
3000 outcomes observed in the trial
0 • Because the relative magnitude of incremental
costs and outcomes observed during the trial may
-3000
-0.10 0.40 0.90 1.40 1.90 2.40
not be reflective of the relative magnitude for
Incremental QALYS
longer periods of follow-up, one should also
Point Estimate CER: $5700
Means and S.D.s for identical project the results for longer periods
Correlations: 0.988; -0.988
COST EFFECTIVENESS RATIO AND 95% CONCLUSIONS
CI, 4S FOR DIFFERENT LENGTHS OF
FOLLOW-UP/PROJECTION
• Economic evaluations attempting to assess
_____________________________________________________
value for the cost
Years of
Follow-up Point Estimate 95% CI • Given need to compare interventions for a
_____________________________________________________
single disease and among diseases, a
2 $282,857 $45,577 to Dominated common outcome such as survival or quality
adjusted survival should be used
4 $12,074 Dominates to Dominated
• Common techniques for evaluation include
5.5 $15,258 Dominates to $122,772
decision analysis and randomized trials
10 $12,246 Dominates to $42,263 • Lots of opportunities exist for evaluating
20 $7,320 $681 to $21,841 microbiologic technologies
_____________________________________________________
