Quantitative research 1

NUR2 612 – September 17, 2023

Kalonde Malama, MPH, PhD
EBP model

If there is no evidence or insufficient

evidence: Conduct research
(Melnyk et al, 2005)
 1. Formulate a problem
Research process 2.
3.
Review the literature
Identify the gaps
4. Formulate a research question (and hypothesis)
5. Identify a theoretical framework
1. Conceptual 6. (Develop the intervention protocol)
phase

7. Select a research design

15. Communicate the findings
5. Dissemination 8. Identify a population
to different audiences 2. Design phase
phase 9. Design a sampling plan
16. If warranted, change/adapt
10. Specify methods to
practice
operationalise research variables

3. Empirical 11. Collect the data

4. Analytic phase 12. Prepare the data for analysis
13. Analyse the data phase
14. Interprete the results
PICOT Examples:
• Among caregivers of people with dementia,
• Population what is the effect of participating in an in-
person caregiver intervention compared an
• Intervention online caregiver intervention on quality of life
• Independent variable, exposure 6 months after enrollment?

• Comparison group • Among cancer patients, what is the effect of a

MBSR intervention on depression 1 year after
• Outcome intervention completion?
• Dependent variable
• Time • Among advanced nursing students, what is
the process by which students integrate
evidence in their practice?
Association
• Exposure: characteristic that may explain or predict the
presence of a study outcome.
• Outcome: characteristic that is being predicted

EXPOSURE OUTCOME
Research questions/objectives
• Can be worded as a question or as a
statement:
• What is the relationship between the functional
dependence level of renal transplant recipients
and their rate of recovery? Problem Research
• The objective of this study is to assess the
relationship between the functional dependence statement question
level of renal transplant recipients and their rate
of recovery.
A statement The specific
• Types of question articulating the queries the
• Therapy/intervention questions research problem research want to
• Diagnosis/assessment questions and indicating the answer in
• Prognosis questions need for a study addressing the
• Etiology (prevention) questions research problem
• Description questions

6
Research hypotheses
• Must suggest a predicted relationship between the independent variable
and the dependent variable
• Must contain terms that indicate a relationship (e.g., more than, different
from, associated with)
• Can be simple (i.e. predicted relation between one independent variable
and one dependent variable) or complex (i.e. predicted relationship
between two or more independent variables and/or two or more dependent
variables)
• Hypotheses can be directional (predicts the direction of a relation) or non-
directional (predicts the existence of a relation but not its direction)
• In statistics we talk about null (H0) vs alternate hypotheses (H1)

7
Jiang et al study
• Identify the problem statement in the article
• What is the research question and which PICOT elements are
provided?
• What type of research question is it?
• What is the hypothesis?
Quantitative research - descriptive vs
analytic studies
• Descriptive studies
• Primary purpose is the accurate portrayal of people’s characteristics or
circumstances
• Who, where, and when (person, place, time)

• Analytic studies
• Test hypotheses about independent (exposure) and dependent
(outcome) variable relations
• Why and how questions
Prevalence and Incidence
• Prevalence: the proportion of a population with a given condition at
a given point in time
= number of existing cases / population count

• Incidence: the rate of new cases with a given condition over a

specified period of time
= number of new cases over a given time period / population count of condition
free individuals (at risk of becoming a new case)
Prevalence and Incidence
 Prevalence and Incidence
Prevalence
increases as
incident cases are
added

Prevalence
decreases as cases
die or recover
Incidence and prevalence of HIV in the U.S.
from 1986 to 2003
Reflects disease
burden, can be used
for planning of
health care resources

Assessment of
disease etiology,
identification of
risk factors

13
 Overview of quantitative designs
Descriptive studies Analytic studies
1. Case study/series A. Experimental study designs
1. RCT
2. Cross-sectional or survey 2. Factorial design
(sometimes called prevalence 3. Crossover
study)
3. Longitudinal descriptive B. Quasi-experimental study designs
1. Non-equivalent control group
2. Pre-Post intervention design
3. Interrupted time series

C. Observational study designs

1. Cohort (prospective and retrospective)
2. Case-control
3. Cross-sectional or survey (sometimes called
correlational study)
 Covid-19 in Critically Ill Patients in the Seattle
Region – Case Series
Pavan K. Bhatraju, et al. N Engl J Med, 382(21):2012-2022, May 21, 2020

Background: Community transmission of coronavirus 2019 (Covid-19) was detected in the state of Washington in
February 2020.
Methods: We identified patients from nine Seattle-area hospitals who were admitted to the intensive care unit (ICU)
with confirmed infection with severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). Clinical data were
obtained through review of medical records. The data reported here are those available through March 23, 2020. Each
patient had at least 14 days of follow-up.
Results: We identified 24 patients with confirmed Covid-19. The mean (±SD) age of the patients was 64±18 years,
63% were men, and symptoms began 7±4 days before admission. CaseTheSeries – a descriptive
most common symptoms were study:
cough and
shortness of breath; 50% of patients had fever on admission, and 58% had diabetes mellitus. All the patients were
• Describe
admitted for hypoxemic respiratory failure; 75% (18 patients) needed mechanicalin-depth
ventilation.the
Mostcharacteristics
of the patients (17)
also had hypotension and needed vasopressors. No patient tested positive for influenza A, influenza B, or other
of one or a limited number of cases
respiratory viruses. Half the patients (12) died between ICU day 1 and day 18, including 4 patients who had a do -not-
resuscitate order on admission. Of the 12 surviving patients, 5 were(e.g. patients,
discharged home,health centres,from
4 were discharged villages)
the ICU
but remained in the hospital, and 3 continued to receive mechanical ventilation in the ICU.
• When little is known about a disease
Conclusions: …
or population
• Hypothesis generating
NUR1 - 312
Sampling and data collection in
quantitative studies
 Sampling: how participants are
identified/selected and how many are included
Sampling
Target Group researcher would like to
generalize findings to
population

Source/study Restricted group of interest from

which we draw our sample
population

Sample Study participants

Types of sampling
Probability sampling Non-probability sampling
• Random sampling: all individuals in the • Convenience sampling: group of
population have equal chance of being people who are readily accessible to
selected the researcher
• Stratified random sampling: the population is • Quota sampling: variant of
divided into strata (subgroups) and simple convenience sampling with the
randomization is applied within subgroups selection of a predetermined
number of units according to strata
• Systematic sampling: every nth unit is
selected
• Multi-stage sampling: groups are randomly • Purposive sampling
selected (e.g., schools), and then individuals • Snowball sampling
within groups are randomly selected
Slido
Slido
Sample size
• Factors that affect the sample size requirements in
quantitative designs
• Effect size (estimated effect of the IV on DV)
• Homogeneity of the study population/sample (smaller sample may be
needed for more homogeneous samples)
• Whether subgroup analyses are planned (e.g. female vs male)
• Extent to which missing data are expected (including due to loss to
follow up)

21
 Sample size calculation
Studies should provide information on how the sample
size was determined, usually using a sample size Jiang et al.
calculation including the following:

1. Estimated effect size: Authors expect to find a

medium effect size (Cohen)

2. Type 1 error: finding an association when there is

none – rejecting null hypothesis when should not
Alpha = 0.05

3. Type 2 error: finding no association when in fact there

is one – accepting null hypothesis when we should not
Beta = (1-Power) = (1-0.80) = 0.20

22
Variables and variable definitions
• Variable = any factor being studied, and which can be measured
• A dependent variable is the outcome being studied
• An independent variable is the characteristic being observed or manipulated
which is hypothesized to cause (or contribute to) the outcome being studied

• Conceptual definition: abstract or theoretical meaning of a

factor/concept of interest
• E.g. Obesity is the excessive accumulation of adipose tissue in the human
body

• Operational definition: the actual procedures by which a

factor/concept is measured
• E.g. Obesity is measured as a body mass index (weight in kg divided by
height in m2) greater than 30 kg/m2 for adults aged 19 years and over.

23
Interviewer administered questionnaire
• Advantages
• Face to face, by telephone, by video conferencing
• Can collect data on past and current exposures
• Interviewer can clarify meaning of questions

• Disadvantages
• Subject to recall bias and social desirability bias
• Cost and time
• Introduction of measurement error by interviewer

24
Self-administered questionnaire
• Advantages
• Can be done on paper or electronically
• Can collect data on past and current exposures
• Lower cost
• Can sometimes be combined with supervision whereby missing or
unclear answers are verified

• Disadvantages
• Low response rate (especially with long questionnaires)
• Missing data
• Difficult to get detailed or complex data
• Participant response fatigue can induce measurement error

25
Diaries
• Advantages
• Can be done on paper or electronically
• Good for experiences that are transient or of low impact
• Minimize recall bias

• Disadvantages
• Can only be used to collect present behaviors or experiences
• Large amount of data that needs to be processed, may be more
difficult to analyse

26
Direct in-person observation
• Advantages
• More objective
• Can be used for low-impact behaviors
• A lot of details can be captured

• Disadvantages
• Can only measure current behaviors or exposures
• Requires extensive training of observers
• Observers’ subjectivity can lead to measurement error
• Time consuming and expensive

27
Physical, biological measure on
participants
• Advantages
• Objective

• Disadvantages
• Cost
• Not available for many measures
• Attributes that are very variable may need to be measured multiple
times

28
How do researchers decide which data
collection methods to use?
• Choice depends on:
• Study design
• Amount and detail of data required
• Impact of what you are looking to measure on participants lives
(e.g. major surgery vs eating carrots)
• Sensitivity of the information sought
• Cost

• Choice often based on practical rather than theoretical

considerations

29
Jiang et al study
• Is it a descriptive or an analytic study?
• What is the target population? The study population?
• What type of sampling was used?
• What are the independent (exposure) and dependent (outcome)
variables?
• What data collection methods are used?
• Select one study variable and identify its conceptual definition
and its operational definition.
Internal and external validity
 Internal validity External validity
• How confident can I be that the • To what extent can the findings
estimated effect (association) is of my study be generalized to
a valid causal effect? (to what extent are they
applicable to):
• Target population
1) Confounding bias • Other populations
2) Selection bias
3) Information (measurement) • Features that increase the
bias internal validity of a study can
hamper its external validity
• E.g. well-defined inclusion and
exclusion criteria, blinding,
controlled environment
Bias
• Bias = any systematic sources of error in the design, conduct or
analysis of a study that results in a mistaken estimate of an exposure’s
(IV) effect on an outcome (DV)
• Bias can preclude finding a true effect or can lead to an inaccurate estimate
(underestimation or overestimation) of the true association between exposure
and an outcome

A study finds that adults who drink alcohol are 2.5 times more likely to develop
lung disease
(Risk ratio = 2.5 with a 95% confidence interval of 1.9 to 3.1)

33
Confounding bias
• Bias of the estimated effect of IV on DV due to the association of the
exposure with other factors that influence the occurrence of the
outcome
• i.e., when there is a third factor involved that is associated with the IV and DV

• Also known as confounding factor, lurking variable, extraneous

variable, or confounder

• Results in a biased estimate of the effect of the exposure on the

outcome
 A confounder cannot be an
Confounding intermediate between the
exposure and the outcome.

IV ? DV
Exposure Outcome

A confounder is also associated with

the exposure being studied but is
A confounder is predictive of the
not a proxy or surrogate for the
outcome even in the absence of the
exposure.
Confounding exposure.
variable

35
 Are there any smokers among people
who drink?
confounding variable= cigarette smoking

Confounding

RR=2.5 (95% CI: 1.9 to 3.1)?

36
Confounding : example
? Postpartum
Immigrant status depression

Education
Income
No partner/no support
Visible minority
Pregnancy complications
Lack of prenatal care
Host country language ability
37
Confounding : example
Diabetes
prevention ? Diabetes
education Knowledge
intervention for
Indigenous youth

Age
Education
Parents’ education level
Parents diabetes status
38
How to eliminate confounding bias in
quantitative studies?
• Before data collection (study design strategies):
• Randomization (creates equal groups on known and unknown
confounders)
• Restrict your population (e.g., exclude participants who are smokers in the coffee –
CVD example) (Homogeneity)
• Matching (case-control study) (e.g., recruit participants as ‘pairs’ matched
by smoking status)

• After data collection (data analysis strategies):

• Statistical control (must have a measure of the confounding variable)
• Intention to treat analysis (in RCT)
Selection bias (Selection threat)
• Bias in an effect estimate (association) due to the manner in
which subjects are selected for the study and which results in
pre-existing differences between groups under study

• Selection bias occurs at:

• the stage of recruitment of participants
• and/or during the process of retaining them in the study
41

Selection bias
Reference
population Heart Disease
(study pop) yes no
yes
Heavy
coffee Sampling fractions
drinker appear similar for all
no 4 cells = no selection bias

Study
sample

STUDY SAMPLE
Jeff Martin, UCSF
42

Selection bias
Reference
population Heart Disease
(study pop) yes no
Exposed and diseased
yes group has a lower
Heavy probability of being
coffee included in the study = selection
drinker bias
no

Study
sample

STUDY SAMPLE
Jeff Martin, UCSF
How to address selection bias?
• Not easy to fix
• Best avoided at the design stage: sampling approach
• During the study: maximising retention of study participants

• Can collect data to ‘estimate’ magnitude/direction of selection bias

and do sensitivity analysis
• e.g., collect data from a sample of non-respondents, and use this to do
sensitivity analysis

43
Information (measurement) bias
• Bias in the estimation of an effect (association) arising from
measurement errors
• The quality of the information is different between comparison groups on the
independent and dependent variables

• Examples:
• Outcome is measured/reported differently among intervention and control
groups in RCT

• How can this be best addressed?

• Must be addressed when designing study and through strict adherence to
study protocol, e.g., Standard Operating Procedures (SOPs)
• Blinding/masking in RCT
Statistical vs clinical significance
A study finds that adults who consume alcohol are 2.5 times more likely to
develop lung disease
Risk ratio = 2.5 with a 95% confidence interval of 1.9 to 3.1

• Statistical significance tells us whether an effect estimate (association) is not due

to chance alone

• Clinical significance tells us whether an effect estimate (association) is big

enough to make a difference for patients/populations
• What if: risk ratio = 1.2 with a 95% confidence interval of 1.1 to 1.3
• There is no equivalent to the p-value of statistical significance to determine clinical
significance
• Depends on many factors, including the population, the cost of the intervention/treatment,
how prevalent the exposure if, etc.
Causality
• Causality is never based on statistics

• Causality must be inferred

• Based on evidence
• Can rely on a set of criteria
• Subjective

• See video on The Question of Causation (14 mins)

Hill’s criteria to assess if an estimated effect
(association) is a causal effect
• Temporality = exposure precedes outcome
• Strength = size of the association
• Consistency = results are replicated
• Experimental evidence = removing/changing exposure has an
effect on outcome
• Biologic gradient = exposure the risk
• Coherence = consistent with existing knowledge
• Plausibility = common sense
• Alternate explanations considered
• Specificity = it’s the only cause

47
 All guidelines are
Reporting guidelines accessible via mycourses

• EQUATOR network, an international initiative that seeks to improve

the reliability and value of published health research literature

• Reporting guidelines promote clear reporting of methods and results

to allow critical appraisal of the manuscript.
• Randomised controlled trials: CONSORT guidelines
• Observational studies in epidemiology: STROBE guidelines
• Systematic reviews and meta-analyses: PRISMA guidelines
• Qualitative research: SRQR and COREQ guidelines

• When appraising a study, you can rely on the study design specific
guidelines to ensure all elements are reported

48
Critical Appraisal Skills Program (CASP) Checklists

• Checklists to help healthcare professionals more easily and

accurately perform critical appraisal across different study designs

• For each checklist, questions help you think about aspects of a study
in a structured way, and organized in 3 sections:
• Are the results of the study valid? (Section A)
• What are the results? (Section B)
• Will the results help locally? (Section C)

• Questions you can record as ‘Yes’, ‘No’ or ‘Can’t tell’. There are
prompts below the questions that highlight the issues to consider, as
well as a space to record the reasons for your answers.

49
What is next
• Next week: Experimental and quasi-experimental studies
• Preparatory readings (see myCourses), including articles used for in
class appraisal exercises
• Take a look at the CASP checklist and CONSORT reporting guidelines
for randomised controlled trials

• Reminders:
• Literature search assignment due Friday Sept 20

50