METHODOLOGY
Research design
Basic strategies needed to develop accurate and interpretable evidence
Key design features:
1. Intervention
2. Control over extraneous variables
3. Masking
4. Comparisons
5. Location
6. timeframes
FEATURE KEY QUESTIONS DESIGN OPTIONS
Intervention Will there be an intervention? Experimental, quasi-experimental, non-
experimental
Control over EV How will EV be controlled? Matching, homogeneity, blocking,
crossover, randomization, statistical control
Masking From whom will critical information Open vs closed; single-blind,
be withheld? double-blind
Comparisons What type of comparisons be made to Within-subjects design,
illuminate key processes or between-subjects design,
relationships? external comparisons
Location Where will the study take place? Single site vs multiple site, field vs
controlled setting
Timeframes How often will data be collected? Cross-sectional, longitudinal design
When will information be collected – Retrospective, prospective design
looking backward or forward?
MASKING
Prevent awareness or expectancy bias
Increase objectivity
Blinding
1. Open study
2. Closed study
3. Single-blind study
4. Double-blind study
COMPARISONS
- Provide context for understanding the findings
1. Between two or more groups
2. One group’s status at two or more points in time
3. On group’s status at different circumstances
4. Based on relative rankings
5. With other studies
Between-subjects
1. Two or more groups of different people (1 and 4)
Within-subjects
2. Same study participants at different times or circumstances (2 and 3)
RESEARCH LOCATION
Site Selection
Whether site is appropriate for research question
Sufficient number and mix of people
Single site
Multiple site
Site visits
“fit” between what the researcher needs and what the site has to offer
Gatekeepers à characteristics and constraints
Gaining access
Trust
Progressive entry
Letter/information sheets:
Purpose
Why site was chosen
What research would entail
Ethical guidelines
Research gains
TIMEFRAMES
Cross-sectional
Longitudinal
Trend
Panel
Follow-up
Retrospective
Prospective
1. Cross-sectional
Data is collected once at a fixed point in time
Economical
Not applicable to change
Cohort comparison design
2. Longitudinal
Data is collected more than one point in time over extended period
Studying time-related processes
Determining time sequences
Making comparisons over time
Enhancing research control
1) Trend study
Samples are studied with respect to a given phenomenon
Different samples at repeated intervals
Patterns and rate of change over time
Predict future
2) Panel study
Same people are used to supplement data at two or more points in time
Panel – sample of subject providing data
Attrition – loss of subjects over time
How conditions and characteristics of time 1 affect conditions and characteristics of
time 2
3) Follow-up studies
Determines the subsequent development of individuals who have specific condition or
those who receive a specific intervention
3. Retrospective study
Collecting data on an outcome occurring in the present and then linking it to
antecedents
Cross-sectional
4. Prospective study
Information is first collected about a presumed cause and then the effect or
outcome is measured
PILOT STUDY
Feasibility study
Small scale version or trial run designed to test methods to be used in a larger, more rigorous study or
parent study
“lessons to learn from”
Quantitative RESEARCH DESIGN
QUANTITATIVE STUDIES
CAUSALITY
1. Temporal
2. Empirical relationship
3. NOT caused by third variable
4. Coherence
5. Consistency
6. Biologic plausability
COUNTERFACTUAL MODEL
Effect – difference between exposure and nonexposure
QUANTITATIVE DESIGNS
Research Design Terminology
Social Scientific Term Medical Research Team
Experiment, true experiment, experimental Randomized controlled trial, randomized clinical trial,
study RCT
Quasi-experiment, quasi-experimental study Controlled trial, controlled trial without randomization
Nonexperimental study, correlational study Observational study
Retrospective study Case-control study
Prospective nonexperimental study Cohort study
Group, condition Arm
1. EXPERIMENTAL DESIGN
1) Posttest only
2) Pretest – posttest
3) Factorial
4) Randomized block
5) Crossover
2. QUASI-EXPERIMENTAL DESIGN
1) Nonequivalent control group designs
2) Time series design
1. Interrupted time series
2. With withdrawal and reinstitution
3. Single subject experiments (N-of-1 studies)
3) Dose response design
3. NONEXPERIMENTAL DESIGN
1) Correlational
1. Retrospective
1. Case control
2. Prospective nonexperimental (cohort design)
3. Natural experiments
4. Path analytic studies
2) Descriptive
1. Descriptive correlational
2. Univariate descriptive
3. Prevalence study
4. Incidence study
EXPERIMENTAL DESIGN
Researchers are active agents
Isolation of a phenomena in a laboratory and controlling the condition under which they occur
Effective in determining causal relationships
Inherently prospective
NONEXPERMENTAL
Researchers are passive observers
Observational research
EXPERIMENTAL DESIGN
Manipulation
Doing something to research participants
Variation of IV
Intervention/treatment
Control
Counterfactual
No intervention
Randomization
Random assignment
Random allocation
Allocation concealment
Manipulation
1) Alternative intervention
2) Placebo
3) Standard methods of care
4) Diff dose or intensities
5) Wait-list control groups
1) Posttest only
1) After – only design
2) Pretest – posttest
1) Before – after design
2) Baseline data
3) Factorial
1) Manipulation of two or more variables
2) Main effects
3) Interaction effects
Auditory A1 Tactile B1
15 min B1 A1 B1 A2 B1
30 min B2 A1 B2 A2 B2
45 min B3 A1 B3 A2 B3
4) Randomized block
Levels by treatment
2 IV but one is stratifying/blocking variable
5) Crossover
Exposure of same subjects to more than one experimental treatment
Randomly assigned to different order of treatments
Counterbalancing
Carry-over effect
QUASI-EXPERIMENTAL DESIGN
With intervention
NO randomization
1) Nonequivalent control group designs
Comparison group, NO randomization
2) Time series design
Information is collected over an extended period and an intervention is collected during
that period
One group pretest – posttest
Interrupted time series
O1 O2 O3 X O4 O5 O6
a) With withdrawal and reinstitution
O1 O2 X O3 O4 X O5 O6 X O7 O8
O1 O2 X O3 O4 (- X) O5 O6 X O7 O8
b) Single subject experiments
N-of-1 studies
Based on responses of single respondent under controlled conditions
3) Dose response design
Different doses of treatment
NONEXPERIMENTAL DESIGN
Observational
No manipulation of IV
Technically
Ethically
Correlational
Interrelationship or association
Effect of a potential cause that cannot be manipulated
Retrospective
Case-control studies
Compare existing phenomenon with an antecedent cause
1. Prospective nonexperimental
Cohort design
Presumed cause à presumed effect
2. Natural experiments
Group exposed to a natural phenomena is compared with those nonexposed
Need not be natural phenomenon
3. Path analytic studies
Theories of causation based on nonexperimental data
Statistical procedures
Descriptive
Observe, describe, and document aspects of a situation as it naturally occurs
Starting point for theory development
1. Descriptive correlational
Describe relationships among variables
Whether men are less likely to bond with their newborns than women
2. Univariate descriptive
Frequency of occurrence of a behavior
Describe status of each variable
Women’s experience during menopause – symptoms, average age, percentage of women
using medications
3. Prevalence study
Determine prevalence rate of a condition at a particular point in time
Number of cases with the condition
or a disease at a given point in time xK
Number in the population at risk of being a case
4. Incidence study
Frequency of developing new cases
Number of new cases with the condition
or a disease at a given time period xK
Number in the population at risk of being a case
STRENGTHS AND LIMITATIONS
Research Design Strength Limitation
Reliability of cause-and-effect Impractical
EXPERIMENTAL relationships Artificial
Intervention effects Hawthorne effect
Less reliable than
QUASI - Practical
experimental
EXPERIMENTAL Enhances respondent
Rival hypothesis
participation
For phenomenon not
Causal relationships
amenable to experimentation
Self-selection
CORRELATIONAL Efficient in collecting large
Heavily relies on theoretical
amounts of data
basis
Realistic and practical
SCIENTIFIC MERIT:
1. Reliability
Accuracy and consistency of information obtained in the study
Methods used to measure variables
Statistical reliability
– probability that the same results would be obtained with a completely new sample of subjects
2. Validity
Whether findings are unbiased, cogent and well grounded
Answers ‘if the methods are really measuring the variables they purport to measure’
TRUSTWORTHINESS
1. Credibility
Extent that the research methods engender confidence in the truth of the data and in
the researchers’ interpretations of the data
2. Transferability
Extent to which findings can be transferred to other settings
Thick description
3. Confirmability
4. Dependability
