Qualitative Research Methods

Lecture 2
dr. John Gelissen

Formulating research questions and conceptual model

Why research questions? . . . . . . . . . . . . . . . . . . . . .
Example I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Often made errors while formulating research questions.
Example III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example VI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Unit of analysis and Unit of Observation. . . . . . . . . . .
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Conceptual model . . . . . . . . . . . . . . . . . . . . . . . . . .
Example conceptual model . . . . . . . . . . . . . . . . . . . .
Basic research designs
Controlling for. . . . . . . . . . . .
Dimensions of research designs
Experimental design I. . . . . . .
Example . . . . . . . . . . . . . . .
Experimental design II . . . . . .
Quasi-experimental design . . .
Example . . . . . . . . . . . . . . .
Non-experimental design . . . .

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

Designing and selecting samples

Introduction . . . . . . . . . . . . . . . . . . . . . . . .
Probability sampling . . . . . . . . . . . . . . . . . .
Sampling in Qualitative Research I. . . . . . . . .
Sampling in Qualitative Research II . . . . . . . .
Define Study Population. . . . . . . . . . . . . . . .
Criteria for sample frames. . . . . . . . . . . . . . .
Example sample matrix for purposive sampling

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

2
. 3
. 4
. 5
. 6
. 7
. 8
. 9
10
11
12
13
14
15

.
.
.
.
.
.
.
.

16
17
18
19
20
21
22
23
24

.
.
.
.
.
.
.

25
26
27
28
29
30
31
32

Formulating research questions and conceptual model

Page 2 / 32

Why research questions?

Why?

Limit danger of research getting out of control

Connect to problem formulation of commissioner of research or specific theory
Make it possible to criticize the research

Problem: few methodological guidelines for formulating research questions; fundamental distinction:

Research question(s): central question(s) to be answered

Research objective: indicates relevance of the research (Goal of this study is. . . , in order to. . . )

Qualitative Research Methods Lecture 2

Page 3 / 32

Example I
Research objective:
The objective of this study is to gain insight in the reasons for students in the Social
Sciences to experience the Research Methods courses as difficult, in order to improve the
teaching in Research Methods and to gain higher success rates for the Faculty of Social
Sciences.
Research question:
Why do Social Science students experience the Research Methods courses as difficult?
Qualitative Research Methods Lecture 2

Page 4 / 32

Example II
General research question:
What is the public opinion concerning Muslim Fundamentalism in the Netherlands, and
what are the causes of this public opinion?
Specific research questions:

1.
2.
3.

To what extent is there a negative public opinion?

Which associations exist between the opinions of people and their social background
characteristics?
How can these associations be explained?

Qualitative Research Methods Lecture 2

Page 5 / 32

Often made errors while formulating research questions

Incompleteness
Research objective is missing (esp. applied research)
Relevant research questions are missing
Vagueness
Research question is too ambitious
Research objective and research questions are not well matched
Unit of analysis is not clear

Examples. . .
Qualitative Research Methods Lecture 2

Page 6 / 32

Example III
Incomplete research question:
How can the problem of vandalizing youth in Tilburg be solved?
Research question is missing here; this question refers to objective of research: how can instead of
what causes the youth to vandalize?

Better: ask descriptive and explanatory questions!

1.
2.
3.
4.

What is the background and life history of offenders?

Where, when and under what conditions occurs vandalism?
What motivates offenders?
What function does vandalism constitute in the lives of young persons?

Qualitative Research Methods Lecture 2

Page 7 / 32

Example IV
Research question and research objective are not well matched:
Objective: With our research we want to contribute to the strategy of a leading European firm
that develops, produces and markets high-technology electronic products of relatively short
product life cycle. This firm needs to be able to develop new products and launch them on the
market successfully. Therefore, it is interested in an assessment of whether its current
resource-based capabilities are sufficient to do so.
Research question: What is the effect of a firms marketing activities on attaining sufficient
market share for new products?
Better: To what extent are the current resource-based capabilities of Firm X sufficient to
successfully introduce new products on the market?

Qualitative Research Methods Lecture 2

Page 8 / 32

Example V
Research question and research objective are not well matched:
Objective: . . . in order to enable to board of the Christian-democratic party to anticipate the
expected negative electoral consequences following the party point of view concerning new nuclear
power plants.
Research question: which factors determine that somebody is in favor of new nuclear power
plants?
Results will not contribute to achieving the research objective; better: what are the electoral
consequences and how do they come about?

Qualitative Research Methods Lecture 2

Page 9 / 32

Example VI
Vagueness: the objective of this research is to study the policies concerning alcohol abuse by
students in Tilburg . . . Why, so what??
Too ambitious: with this research I intend to structurally improve Dutch universities.
Unit of analysis is not clear: which differences exist in the degree of cooperation, and how can
these differences be explained? . . . Whose cooperation???

Qualitative Research Methods Lecture 2

Page 10 / 32

Unit of analysis and Unit of Observation

Those units on which relations between theoretical constructs are conceptualized

NOT the theme of the study!
Central in research questions and conceptual model
Unit of observation: Unit on which measurement/observation is done
Both: bearers of characteristics/cases in data matrix
Examples: persons, dyads, groups, households, departments, firms, countries, relations within a
network

Qualitative Research Methods Lecture 2

Page 11 / 32

Example

Theoretical
Case
(UoA)

Individual
Team
Department
National firm
Multinational

Observation (UoO)
Individual Team
Department National firm
identical
identical
identical
identical

Multinational

identical

Qualitative Research Methods Lecture 2

Page 12 / 32

Example
Job motivation of employee

Job performance of employee

Average job motivation in Department

Job motivation of employee

Job performance of employee

Average job motivation in Department

Performance of Department

Job motivation of employee

Qualitative Research Methods Lecture 2

Page 13 / 32

Conceptual model
Graphical display of a-priori theoretical expectations or a-posteriori theoretical conclusions within a
research study

Aspects of conceptual model

Unit of analysis
Theoretical concepts
Associations between theoretical concepts

Why important?

Guides selection of research questions

Guides selection of research design
Framework for interpreting empirical findings or reporting theoretical conclusions

Qualitative Research Methods Lecture 2

Page 14 / 32

Example conceptual model

(Stack & Gundlach, 1992)

Race (Black/White)

Listening to country music

Suicide attempt

Living in Poverty
Experiencing divorce
Gun possession
1

Qualitative Research Methods Lecture 2

Page 15 / 32

Page 16 / 32

Basic research designs

Controlling for. . .
How can we control/hold constant ?
1. By statistical control

Measure confounding variables, then include as control variables in statistical analysis

(important remark: no measurement of confounding variables means no control!)

2. By research design

Make systematic comparisons between groups that are as identical as possible, except for the
characteristic that presumably has an effect
Systematic comparison important for qualitative research
Rule: no comparison, no conclusion!

Qualitative Research Methods Lecture 2

Page 17 / 32

Dimensions of research designs

1.
2.
3.
4.

Number of individuals or groups to be included

Number of measurement moments
How are groups constructed
Intentional manipulation by researcher, natural occurrence, or existing variation in data?

Qualitative Research Methods Lecture 2

Page 18 / 32

Experimental design I
If well executed, most adequate design for causal inference
Also called: classical experimental design, randomized experiment, standard design, true
experiment
Laboratory experiment and field experiment
How?

1.
2.
3.
4.

Randomly split group of people in two (or more) groups

Perform pre-test
Experimental group is manipulated, control group is not manipulated
Perform post-test

Qualitative Research Methods Lecture 2

Page 19 / 32

Example
Does smoking marihuana make you less intelligent? Experimental design:
Assignment

Group
Exp.

Pretest
(Y)
Average
score
IQ test : 112

Manipulation
(X)
smoke
white
widow

Posttest
(Y)
Average
Score
104

Average
score
IQ test : 112

not
smoke
Marihuana

Average
Score
IQ test: 112

Randomized
Cont.

Qualitative Research Methods Lecture 2

Page 20 / 32

Experimental design II
If well executed, the following confounding factors threats to the internal (causal) validity are
controlled

History
Testing
Instrumentation
Mortality/attrition
Maturation
Selection bias
Statistical regression to the mean

Qualitative Research Methods Lecture 2

Page 21 / 32

10

Quasi-experimental design

Almost identical to true experiment, but without random assignment

The researcher does manipulate the independent variable
Dangers: selection bias; threats to internal validity may even work differently in experimental and
control group

Qualitative Research Methods Lecture 2

Page 22 / 32

Example
Assignment

Group
Exp.

Pretest
(Y)
Average
score
IQ test : 112

Manipulation
(X)
smoke
white
widow

Posttest
(Y)
Average
Score
104

Average
score
IQ test : 112

not
smoke
Marihuana

Average
Score
IQ test: 106

NON-random
Cont.

Qualitative Research Methods Lecture 2

Page 23 / 32

11

Non-experimental design
Research design in which 1) the researcher does not manipulate the independent variable and 2)
random assignment is not used
Cross-sectional designs

Longitudinal designs

Possible to extend with retrospective elements

Repeated cross-sectional desing/trend design
Cohort design
Panel design

Case study designs (special case: natural experiment)

All the above designs are highly relevant for qualitative research!
Qualitative Research Methods Lecture 2

Page 24 / 32

Page 25 / 32

Designing and selecting samples

Introduction

After choice of research design, choice of sampling strategy

Many different procedures
First, basic strategy in quantitative research, then sampling strategies in qualitative research
REMEMBER: Random sampling is NOT THE SAME as random assignment/randomization:
Random sampling is the process of randomly selecting samples from a population of potential
participants.
Random assignment is the process of randomly allocating individuals to the conditions of an
experiment.

Qualitative Research Methods Lecture 2

Page 26 / 32

12

Probability sampling

Goal: generalize from sample to population

Sample frame: list of all units of analysis in population
Probability sample: every unit of analysis has a known probability of being selected

Simple random sample:

Most basic strategy in quantitative research

Every unit has same known probability of becoming selected
Sample fraction n/N, with n=sample size and N=population size
List all units and randomly sample, e.g. 30%

Qualitative Research Methods Lecture 2

Page 27 / 32

Sampling in Qualitative Research I

Sometimes, we use random sampling in qualitative research:

documents

Sometimes, sampling is not necessary because population is completely available

Often, sample frame is not (completely) available
Non-probability sampling: Probability of entering sample is not known, because sample frame is
not (completely) available

Qualitative Research Methods Lecture 2

Page 28 / 32

13

Sampling in Qualitative Research II

Core sampling methods:

Criterion based/Purposive sampling

Theoretical sampling
Opportunistic/Convenience sampling
Snowball sampling

Key features:

Usage of prescribed selection criteria.

Usually small in size (Important: issues when determining sample size, R&L, p. 84)
Sampling is often done sequentially or iteratively

Qualitative Research Methods Lecture 2

Page 29 / 32

Define Study Population

Key questions:
1. Who or what is to be sampled?
2. What is the appropriate information source sample frame from which they are to be selected?

Specify characteristics of collective units required

Specify characteristics of the individuals within them
Which population will, by virtue of their proximity to the research question, will yield the richest
and most relevant information? Who should be excluded? Do we need to include supplementary
populations for comparison?

Qualitative Research Methods Lecture 2

Page 30 / 32

14

Criteria for sample frames

Important questions:

Does sample frame provide sufficient details to inform selection?

Does sample frame provide comprehensive and inclusive basis for sampling?
Does sample frame provide sufficient number of potential participants?
Practical considerations

Types of sample frames:

Existing lists
Specifically generated for study

Qualitative Research Methods Lecture 2

Page 31 / 32

Example sample matrix for purposive sampling

Cross-national focus group design to study beliefs about social security
To be applied in France, Germany, UK and the Netherlands
Table 1. Sample matrix with quotas for selection of participants.

Group 1*
Unemployed
Low education

Group 2*
Employed
Low education**

Group 3
Employed
Medium education**

Group 4
Employed
High education**

4 men
2 with children
below school
leaving age
2 aged > 50 years

4 men
2 with children
below school
leaving age
2 aged > 50 years

4 men
2 with children
below school
leaving age
2 aged > 50 years

4 men
2 with children
below school
leaving age
2 aged > 50 years

4 women
4 women
4 women
4 women
2 with children
2 with children
2 with children
2 with children
below school
below school
below school
below school
leaving age
leaving age
leaving age
leaving age
2 aged > 50 years
2 aged > 50 years
2 aged > 50 years
2 aged > 50 years
*Groups 1 and 2 possibly also include migrants.
** The combination of being employed and educational attainment will also allow the examination of social
class differences.

Data collection and analysis

Qualitative Research Methods Lecture 2

Page 32 / 32

15