Primary research methodology: quantitative
Dr Sunny Chan
23rd Jan 2025 1
Contents to be covered
Quantitative
• True experimental design
• Quasi experimental design
• Non-experimental design
2
Types of
research
design
Adopted from
Portney and
Watkins (2015) 3
True experimental design
• Two major characteristics:
– Presence of a control group
– Random assignment (allocation) to groups
• Usually called Randomized Controlled Trial (RCT)
• Best design for determining cause and effect
4
True experimental design
Sample
Randomization
Treatment Control
group group
5
True experimental design – different types
• Pretest-posttest control group design
• Two-group pretest-posttest design
• Multi-group pretest-posttest design
• RCT with follow-up design
• RCT with cross-over design
6
Pretest-posttest Control group design
Sample
Randomization
Treatment Control
group group
Pretest Pretest
Treatment period Control period
Posttest Posttest 7
Pretest-posttest Control group design
Control group
• No treatment control:
– Do not receive any treatment
• Placebo control:
– Receive a treatment that appears to be like the
active treatment, but without therapeutic effect
8
Placebo control
sbac065.pdf (uwe.ac.uk)
9
Two-group pretest-posttest design
Sample
Randomization
Treatment Treatment
group 1 group 2
Pretest Pretest
Treatment 1 period Treatment 2 period
Posttest Posttest 10
Two-group pretest-posttest design
• Similar to the pretest-posttest control group design
• However, the comparison group receives a second
form of intervention
• Decides which is the better treatment
• Does not show that treatment works better than no
treatment
11
Multi-group pretest-posttest design
• A variation of the pretest-posttest control group
design
• Allows researchers to compare several (more than 2)
treatment and control conditions
12
RCT with follow-up design
Sample
Randomization
Treatment Control
group group
Pretest Pretest
Treatment period Control period
Follow-up Posttest Posttest Follow-up 13
RCT with follow-up design
• Similar to pretest-posttest control group design
• However, addition of follow-up period after the first
posttest
• To evaluate long-lasting effects of a treatment
• Time between post-test and follow-up can vary (e.g.,
1 month, 6 months, 1 year……)
14
RCT with follow-up design
A randomized controlled trial on the comparative effectiveness of mindfulness-b
ased cognitive therapy and health qigong-based cognitive therapy among Chines
e people with depression and anxiety disorders (nih.gov) 15
RCT with cross-over design
Sample
Randomization
Treatment Control
group group
Pretest Pretest
Treatment period Control period
Posttest Posttest
Wash-out period
Pretest Pretest
Posttest Control period Treatment period Posttest 16
RCT with cross-over design
• Similar to pretest-posttest control group design
• However, the treatment and control group will switch
after the first posttest
• The original control group will receive treatment, but
in later stage
• Must have a washout period – long enough so that
the effects of the treatments disappear/wear off
17
RCT with cross-over design
• Good
– All subjects can receive all treatments (ethics)
• Bad
– Difficult to determine appropriate time for
washout
– Carryover effects
– Time consuming
18
RCT with cross-over design
Impact of mindfulness-based cognitive therapy on counselin
g self-efficacy: A randomized controlled crossover trial - Scie
nceDirect (uwe.ac.uk)
19
RCT design – Advantages
• Can establish cause and effect (causal relationship)
• With control group: to control confounding variables
• With randomization: groups are more likely to be
“similar”
20
RCT design – Disadvantages
• Large sample size
• Compliance and attrition issues (particular for long-
term follow-up)
• Expensive
• Generalizable? Applicable to clinical settings? Due to
strict inclusion/exclusion criteria
21
How much you know so far?
22
Question 1
• Which of the following study design is the most
appropriate if you want to test whether OT
splintage is better than bandage therapy in
treating upper limb fracture?
• A: Cross-sectional design
• B: Pretest-posttest control group design
• C: Two-group pretest-posttest design
• D: Multi-group pretest-posttest design
23
Question 2
• You randomized the stroke patients into an individual-based
training group and a group-based training group. You
measured ADL function (Barthel Index) and found that the
individual-based group had significantly more improvement in
ADL function (+ 7 points) than the group-based training group
(+2 points). What conclusion is appropriate?
24
Question 2
A. Individual-based training is more effective than group-
based training in improving ADL in stroke patients.
B. Individual-based training is effective in improving ADL in
stroke patients.
C. Group-based training is less effective in improving ADL
in stroke patients.
D. All of the above.
E. None of the above. 25
Quasi-experimental design
• Two major characteristics:
– NO control/comparison group OR/AND
– NO Random assignment (allocation) to groups
26
Quasi-experimental design – different types
•One group pretest-posttest design
•Interrupted time series
•Historical controls
•Non-equivalent Pretest-posttest control group
design
27
One group pretest-posttest design
Sample
Treatment
group
Pretest
Treatment period
Posttest 28
One group pretest-posttest design
•The weakest design among all quasi-experimental
designs
•Disadvantages:
– Lack of control group
– Maturation effect
– History effect
– Testing/practice effect
29
Interrupted time-series design
Pre-intervention Post-intervention
30
Interrupted time-series design
•One treatment group
•Multiple pretests and posttests after intervention
31
Interrupted time-series design
•Advantages:
– Lessen the effects of maturation, testing/practice
– More confidence in baseline and outcome data
than one group pretest-posttest design
32
Interrupted time-series design
•Disadvantages:
– More time consuming
– No control group: no control over history
– Difficult to get stability of measurements in the
pretest phase in some cases
– Difficult to apply to subjects with acute
conditions
33
Historical controls
•Use of a control group who received a different
intervention during an earlier time period
•The protocol in one study acts as a control for
later studies
34
Historical controls – example
• Objective: To compare the effects of AI therapy and
conventional training on ADL performance in patients
with stroke
•AI therapy group:
• 80 subjects will receive AI therapy for the next 3 months.
•Conventional training group:
• Check the medical records in the past 2 years and find out
from the rehab notes the information on improvement in
ADL performance before and after conventional training.
35
Historical controls – Advantages
•Having a comparison group
•Alternative when ethical concerns may preclude a
true control group
36
Historical controls – Disadvantages
• Different subject characteristics of the
treatment and historical control groups (no
randomization, subjects from the past)
• Concerns with how the data were collected
from historical controls (measurement
error/bias)
• Exposure to different factors for the two groups
– Different study environment
– Different time of the year.
37
Non-equivalent pretest-posttest Control
group design
Sample
Non-Randomization
Treatment Control
group group
Pretest Pretest
Treatment period Control period
Posttest Posttest 38
Non-equivalent pretest-posttest Control
group design
• Similar to true experimental design
• BUT…..the group allocation is NON-random
39
Non-equivalent pretest-posttest Control
group design
Advantages:
• Strongest quasi-experimental design
• Some control over history, maturation,
testing/practice, and instrumentation effects
• An alternative to RCT.
Disadvantages:
• Comparison groups may not be equivalent (What
are the consequences?) 40
How much you know so far?
41
Question 3
• An occupational therapist wanted to determine whether an
innovate OT training is effective in improving ADL function
for people with stroke
• Each subject received 4 weeks of innovative training
• ADL function was measured by the same occupational
therapist using Barthel Index once before and after
treatment
• What is the study design? 42
Non-experimental designs
• NO intervention is provided as part of the study
(no manipulation)
• Data are collected as they naturally exist
• Also called observational research
43
Observational research
• 2 different types according to the timing
Cross-
Longit
sectio
udinal
nal
44
Longitudinal design
45
Longitudinal design
•You follow a cohort of subjects over time
•Measurements are taken periodically from the
same individuals
•More important if you study the specific patterns
of change with age
46
Longitudinal design
Advantages:
• Ability to record the actual change in the same individuals
Disadvantages:
• Time consuming
• More expensive
• Threat to internal validity
• Testing effect
• History
• Attrition (especially in long term studies)
47
Cross-sectional design
48
Cross-sectional design
• Data is collected at ONE point in time or over
a short period of time “in the present”
• Each variable is usually only measured once
for each subject
• Preferred if main interest is to describe the
typical characteristics at various stages of life
49
Cross-sectional design
Advantages:
• Easy and less time consuming
• Not influenced by testing or history
Disadvantages:
• Cannot examine the actual changes
• Cohort effects
50
Observational research
• 2 different types according to the purpose
Descr Explo
iptive rative
51
Descriptive research
• Document / describe conditions, attitudes, or
characteristics of individuals
•No hypothesis tested
•Examples:
– What is the normative value of handwriting function for
children at age 12?
– Basis for interpretation of assessment results
– Basis for prescription of treatment
52
Explorative research
• Systematic investigation of relationships among two or more
variables.
• Involve testing hypothesis
– Example: relationship between ADL performance and QOL
• The results are useful for generating hypothesis for experimental
studies to prove causality
– Example: Design a future experimental study to assess the
effects of an innovative ADL intervention for improving QOL
53
References
Atkinson, P. and Hammersley M. (1994). Ethnography and participant observation. In: N. Dnezin and Y. Lincon
(eds.). Handbook of qualitative research. Thousand Oaks, CA: Sage.
Braun, V.. and Clarke, V. (2006). Using thematic analysis in psychology, Qualitative Research in Psychology, 3, pp. 77-
101.
Chan, S.H.W., Chan, W. W. K., Chao, J. Y. W. and Chan, P. K. L. (2020). A randomized controlled trial on the
comparative effectiveness of mindfulness-based cognitive therapy and health qigong-based cognitive
therapy among Chinese people with depression and anxiety disorders. BMC Psychiatry, 20(1), pp. 590–
590.
Chan, S.H.W., Yu, C. K.-C. and Li, A. W. O. (2021). Impact of mindfulness-based cognitive therapy on counseling self-
efficacy: A randomized controlled crossover trial. Patient Education and Counseling, 104(2), pp.360–368.
Giorgi A., (1997). The theory, practice, and evaluation of the phenomenological method as a qualitative research
procedure. Journal of Phenomenological Psychology, 28(2), pp. 235-260.
Graneheim, U,.H. and Lundman, B. (2004).Qualitative content analysis in nursing research: concepts, procedures,
and measures to achieve trustworthiness. Nurse Education Today, 24, pp. 105-112.
Holstein, J.A. and Gubrium, J. E. (2003). Three approaches to quantitative content analysis. Qualitative Health
Research. 15(9), pp.1277-1288.
Neill, E., Rossell, S. L., Yolland, C., Meyer, D., Galletly, C., Harris, A., Siskind, D., Berk, M., Bozaoglu, K., Dark, F., Dean,
O. M., Francis, P. S., Liu, D., Phillipou, A., Sarris, J. and Castle, D. J. (2022). N-Acetylcysteine (NAC) in
Schizophrenia Resistant to Clozapine: A Double-Blind, Randomized, Placebo-Controlled Trial Targeting
Negative Symptoms. Schizophrenia Bulletin, 48(6), pp.1263–1272.
Portney, L.G. and Watkins, M.P. (2014). Foundations of Clinical Research. Applications to Practice. (3rd International
ed.). Essex: Pearson Education Inc.
54
