Introduction

Evidence-based medicine is defined as a systematic, quantitative,

preferentially experimental approach to obtaining and using medical
information.
Systematic reviews and meta-analyses are a critical foundation of evidence-
based medicine.
Essential tools for synthesizing evidence needed to inform clinical decision
making and policy.
 Systematic reviews : objective, reproducible method to find answers to a
research question by collecting all available studies related to that question and
reviewing and analyzing their results.
 Systematic reviews sometimes, use statistical techniques to combine data from
the examined individual research studies, and use the pooled data to come to
new statistical conclusions.
 This is called meta-analysis-represents a specialized subset of systematic
reviews.
Meta-analysis is a quantitative, formal, epidemiological study design used to
systematically assess the results of previous research to derive conclusions about that
body of research. Typically, but not necessarily, the study is based on randomized,
controlled clinical trials.

Conclusions produced by meta-analysis are statistically stronger than the analysis of any
single study, due to increased numbers of subjects, greater diversity among subjects, or
accumulated effects and results.

Meta-analyses are the most frequently cited form of clinical research.

History
Karl Pearson was the first researcher to use formal techniques to pool data from
different studies in 1904.
Synthesized data from several studies on typhoid vaccination

The rationale provided was :

“ Many of the groups are far too small to allow a definitive opinion being formed
at all , having regard to the size of the probable error involved”
In 1952 Hans Eysenck concluded that there were no favorable effects of
psychotherapy , initiating a raging debate which 25 years of evaluation research
and hundreds of studies failed to resolve
In 1978 to prove Eysenck wrong, Gene Glass statistically aggregated the findings
of 375 psychotherapy outcome studies
Glass coined the term ‘Meta-analysis’.
Challenges faced during
systematic review
Too many studies to search
•Publication bias: only studies with significant findings are published.
• Inconsistencies in findings because of flaws in study conduct or random errors.
Steps in conducting systematic
review
1. Formulate review question
2. Identification of relevant studies
3. Extraction of data
4. Assessment of bias in included studies
5. Synthesis of data
6. Interpreting the evidence
7. Writing up the review
8. Updating the review.
1. Formulate review question
•Address the variety of issues: incidence/ prevalence/
etiology of diseases / diagnostic accuracy/
effectiveness of interventions.
•Should be precise.
•Should specify key components: PICO
P= Participants
| = Interventions
C = Control / comparison
O = Outcome of interest.
2. Identify relevant studies
•Time consuming
• Eligibility criteria should be set for inclusion and exclusion.
• Criteria is set up with relevance to PICO components
Define the study design also
based on the research question.
1. Prevalence of diseases/ diagnostic accuracy of tests: cross sectional design
2. Aetiology of disease: cohort design
3. Effects of intervention: RCT
Searching for studies
SOURCES: MEDLINE, EMBASE, CENTRAL, LILAC, etc
• Can use MeSH terms to search articles.
•Search for unpublished literature and ongoing studies
• Unpublished literature/ grey literature: conference abstracts, dissertations, books, etc.
Studies in non English journals and small sample size studies to be selected too.
3. Data collection and extraction
WHAT DATA TO BE COLLECTED?
Data regarding;
• Eligibility of study
•Study methodology
•Details of participants
•No of intervention groups and details specific to interventions given
•Information regarding outcomes: definitions, how measured, all pre specified and unspecified
outcomes are to be collected and analysed.
• Information on ethical approval, funding, conflicts of interest, name and contact of authors.
Data extraction
Process of data recording into data collection forms.
Two reviewers should work independently: to reduce the risk of errors.
• Blinding of data extractors: to reduce the risk of bias.
•However routine blinding is not usually recommended.
Check for study duplication: same studies reported in more than one journal.
4. Assessing the presence and
risk of bias
•Assess for bias in study design, conduct, analysis or reporting of study.
Several methods available.
•For clinical trials: DOMAIN BASED EVALUATION recommended by Cochrane library.
Steps in conducting systematic
review
5. SYNTHESIS OF DATA: Meta analysis
6. INTERPRETING THE EVIDENCE
7. WRITING THE REVIEW: PRISMA guidelines
8. UPDATING THE REVIEW
Advantages of systematic
review
• Uses explicit methods which limits bias
• Draws reliable and accurate conclusions (best form of evidence)
• Very useful decision making tools for clinicians, researchers and for policy makers.
• Generation of new hypothesis about subgroups of study
• Increases the precision of the results.
Limitations of systematic review
• Location and selection of studies - Heterogeneity
• Loss of information on important outcomes
• Inappropriate subgroup analyses
Meta-analysis
ACCORDING TO GENE GLASS WHO FIRST DEFINED META ANALYSIS IN 1976,
meta analysis refers to a statistical analysis of a large collection of analysis results from individual
studies, for the purpose of integrating the findings."
Types of meta analysis
1. CUMULATIVE MA: new studies are added and MA repeated every time an new study is
published.
2. RETROSPECTIVE MA: commonly done.
3. PROSPECTIVE MA: criteria and protocol for selection is stated even before studies of interest
are published. (low bias)
Effect size in meta analysis
EFFECT SIZE:
measure of analysis
• Dependent variable
• Any standard index
• Eg: prevalence, incidence, odds ratio, relative risk, effects of intervention.
Subgroup analysis
•If a meta analysis is performed across heterogenous trials, it may be inappropriate to draw
conclusions from the pooled treatment effect.
•If the same trials are subgrouped and there is no heterogeneity within trials then valid
conclusions can be drawn using results from subgroup analysis.
If subgroup analysis demonstrate that the treatment is more or less effective for certain
subgroups of patients, interpretation of these subgroup analyses can provide valuable insight
into how the treatment should be used in clinical practice.
•Participants are divided into subgroups based on certain characteristics ( gender, ses) or trial
characteristics (geographic location) and then analysed
Sensitivity analysis
• Done to see if the estimate changes by changing some parameters.
To see how far the result is affected by changes.
Eg: estimates are checked before and after including low quality studies
Presentation of the result of ma:
FOREST PLOT
• Graphical representation of the results
• Always included in presenting the results.
Displays the effect size estimates and confidence intervals for each study included in MA.
•Studies to be ordered either according to;
effect size estimate/ magnitude
>Study weightage (precision)
➤Chronological order
Any other meaningful order
How to interpret
the result of meta
analyis??
Were apples
combined with
oranges??
Look for heterogeniety
Refers to difference between studies not due to chance.
•Types of heterogeneity: clinical ( pt characteristics, interventions, outcomes) and statistical (diff
in study design and quality).
Clinical heterogeneity always exists and can be identified without any calculation or tests.
• Statistical heterogeneity doesn't exist always and needs tests to identify them.
How to detect heterogeniety
1. REVIEW TABLES AND CHECK FOR THE TYPE OF PTS: for clinical heterogeneity. ( mixing of pts
with different diseases and treatment pattern)
2. EYEBALL TEST: look at the forest plot for overlapping of confidence interval. (Overlap + = no
heterogeneity).
3. STATISTICAL TEST: tells the extent of heterogeneity and its significance.
Fixed and random effect model
• FIXED EFFECT: differences among studies are purely due to chance.
• RANDOM EFFECT: differences among studies due to chance and other reasons also.
WHICH MODEL TO BE USED?
➤ when heterogeneity is absent: use fixed effect
>When heterogeneity is present: use random model Some researchers suggest to use both the
models irrespective of heterogeneity.
Ways to deal with heterogeniety
• Do not perform MA
•Do subgroup and sensitivity analysis: to find the reason for heterogeneity.
Do MA based on random model: use only when the reason for heterogeneity cannot be
explained.
Change the effect measure: may sometimes introduce artificial heterogeneity
Publication bias analysis: funnel
plot
• Happens when studies with positive and significant results are only selected.
•Tool to visually assess the possibility of publication or small study bias in meta analysis.
• Scatter plot of effect size over standard error of effect size.
• X axis: effect size
• Y axis: SE of effect size.
• Not recommended in a very small study MA (n
CONTOUR ENHANCED FUNNEL
PLOT:
• Funnel plot with additional contour lines of statistical significance
• Lines at p=0.01,0.00,0.05,etc..
Interpretation:
➤ if studies missing in area of non significance: PUBLICATION BIAS
If studies missing in areas of significance: OTHER REASONS
No studies in areas of significance : PUBLICATION BIAS
QUALITY ASSESSMENT OF
SYSTEMATIC REVIEW AND META
ANALYSIS
3 Commonest ways;
1. Overview quality assessment questionnaire
2. PRISMA checklist
3. The AMSTAR tool