RANDOM GROUP DESIGN

CONTINUED….
 STRETCHING EXERCISE (187)

1.WHAT ASPECT OF THE EXPERIMENT DID BUSHMAN (2005) CONTROL BY

USING MANIPULATION?
2 WHAT ASPECT OF THE EXPERIMENT DID BUSHMAN CONTROL BY HOLDING
CONDITIONS CONSTANT?
3 WHAT ASPECT OF THE EXPERIMENT DID BUSHMAN CONTROL
• BY USING BALANCING?
 BLOCK RANDOMIZATION

• A COMMON PROCEDURE FOR CARRYING OUT RANDOM ASSIGNMENT IS

BLOCK RANDOMIZATION.
• BLOCK RANDOMIZATION BALANCES SUBJECT CHARACTERISTICS AND
POTENTIAL CONFOUNDING'S THAT OCCUR DURING THE TIME IN WHICH THE
EXPERIMENT IS
CONDUCTED, AND IT CREATES GROUPS OF EQUAL SIZE.
 EXAMPLE

• WE HAVE AN EXPERIMENT WITH FIVE CONDITIONS (LABELED, FOR

CONVENIENCE, AS A, B, C, D, AND E).
• ONE “BLOCK” IS MADE UP OF A RANDOM ORDER OF ALL FIVE CONDITIONS:
• ONE BLOCK OF CONDITIONS → RANDOM ORDER OF CONDITIONS
ABCDE CAEBD
• IN BLOCK RANDOMIZATION, WE ASSIGN SUBJECTS TO
CONDITIONS ONE BLOCK AT A TIME. IN OUR EXAMPLE WITH FIVE
CONDITIONS, FIVE SUBJECTS WOULD BE NEEDED TO COMPLETE
• THE FIRST BLOCK WITH ONE SUBJECT IN EACH CONDITION. THE
NEXT FIVE SUBJECTS WOULD
BE ASSIGNED TO ONE OF EACH OF THE FIVE CONDITIONS TO
COMPLETE A SECOND BLOCK, AND SO ON. IF WE WANT TO HAVE
10 SUBJECTS IN EACH OF FIVE CONDITIONS, THEN THERE WOULD
BE 10 BLOCKS IN THE BLOCK-RANDOMIZED SCHEDULE.
 ADVANTAGES

• FIRST, BLOCK RANDOMIZATION PRODUCES GROUPS OF EQUAL

SIZE. THIS IS IMPORTANT BECAUSE THE NUMBER OF
OBSERVATIONS IN EACH GROUP AFFECTS THE RELIABILITY OF THE
DESCRIPTIVE STATISTICS FOR EACH GROUP, AND IT IS DESIRABLE
TO HAVE THE RELIABILITY OF THESE MEASURES COMPARABLE
ACROSS GROUPS
• SECOND, BLOCK RANDOMIZATION CONTROLS FOR TIME-
RELATED VARIABLES. BECAUSE EXPERIMENTS OFTEN TAKE
A SUBSTANTIAL AMOUNT OF TIME TO COMPLETE, SOME
PARTICIPANTS CAN BE AFFECTED BY EVENTS THAT OCCUR
DURING THE TIME THE EXPERIMENT IS CONDUCTED.
• IN BLOCK RANDOMIZATION, EVERY CONDITION IS TESTED
IN EACH BLOCK SO THESE TIME-RELATED VARIABLES ARE
BALANCED ACROSS THE CONDITIONS OF THE
EXPERIMENT.
• IF, FOR EXAMPLE, A TRAUMATIC EVENT OCCURS ON A COLLEGE
CAMPUS WHERE AN EXPERIMENT IS BEING CONDUCTED, THE
NUMBER OF PARTICIPANTS WHO EXPERIENCED THE EVENT WILL
BE EQUIVALENT IN EACH CONDITION IF BLOCK RANDOMIZATION
IS USED. WE ASSUME, THEN, THAT THE EFFECTS OF THE EVENT
ON PARTICIPANTS’ PERFORMANCE WILL BE EQUIVALENT, OR
AVERAGED, ACROSS THE CONDITIONS.
• THE BEAUTY OF BLOCK RANDOMIZATION IS THAT IT
WILL BALANCE (OR AVERAGE) ANY CHARACTERISTICS
OF PARTICIPANTS (INCLUDING THE EFFECTS OF TIME-
RELATED FACTORS) ACROSS THE CONDITIONS OF AN
EXPERIMENT.
THREATS TO INTERNAL VALIDITY
AND SOLUTION
• INTERNAL VALIDITY IS THE DEGREE TO WHICH DIFFERENCES IN
PERFORMANCE ON A DEPENDENT VARIABLE CAN BE ATTRIBUTED CLEARLY
AND UNAMBIGUOUSLY TO AN EFFECT OF AN INDEPENDENT VARIABLE, AS
OPPOSED TO SOME OTHER UNCONTROLLED VARIABLE.
• THESE UNCONTROLLED VARIABLES ARE OFTEN REFERRED TO AS THREATS
TO INTERNAL
• VALIDITY. THESE THREATS ARE POTENTIAL ALTERNATIVE EXPLANATIONS
FOR A STUDY’S
FINDINGS. RESEARCHERS CONTROL THREATS TO INTERNAL VALIDITY IN ORDER
TO MAKE A CLEAR CAUSE-AND-EFFECT INFERENCE ABOUT AN INDEPENDENT
VARIABLE.
• SEVERAL PROBLEMS IN EXPERIMENTAL RESEARCH THAT CAN
RESULT IN THREATS TO INTERNAL
VALIDITY, ARE DISCUSS NOW AND METHODS TO CONTROL THESE
THREATS.
 TESTING INTACT GROUPS

• RANDOM ASSIGNMENT IS USED TO FORM COMPARABLE GROUPS

IN THE RANDOM GROUPS DESIGN. THERE ARE TIMES, HOWEVER, WHEN NON
COMPARABLE
GROUPS ARE FORMED EVEN WHEN RANDOM ASSIGNMENT APPEARS TO HAVE
BEEN USED.
• THIS PROBLEM OCCURS WHEN INTACT GROUPS (NOT INDIVIDUALS) ARE
RANDOMLY ASSIGNED TO THE CONDITIONS OF AN EXPERIMENT.
• INTACT GROUPS ARE FORMED PRIOR TO THE START OF THE EXPERIMENT.
FOR EXAMPLE, THE DIFFERENT SECTIONS OF AN INTRODUCTORY
PSYCHOLOGY COURSE ARE INTACT GROUPS. STUDENTS ARE NOT RANDOMLY
ASSIGNED TO DIFFERENT SECTIONS OF INTRODUCTORY PSYCHOLOGY
(ALTHOUGH SOMETIMES SCHEDULING CLASSES SEEMS RANDOM).
• STUDENTS OFTEN CHOOSE TO BE IN A PARTICULAR SECTION BECAUSE OF
THE TIME THE CLASS MEETS, THE INSTRUCTOR, FRIENDS WHO WILL BE IN
THE CLASS, AND ANY NUMBER OF OTHER FACTORS. IF A RESEARCHER WERE
TO RANDOMLY ASSIGN DIFFERENT SECTIONS TO LEVELS OF AN
INDEPENDENT VARIABLE, A CONFOUNDING DUE TO TESTING INTACT GROUPS
COULD OCCUR.
 EXAMPLE,

• STUDENTS WHO CHOOSE TO TAKE AN 8 A.M. SECTION MAY DIFFER FROM STUDENTS WHO

PREFER A 2 P.M. SECTION. RANDOM ASSIGNMENT OF THESE INTACT GROUPS TO

EXPERIMENTAL
CONDITIONS IS SIMPLY NOT SUFFICIENT TO BALANCE THE SYSTEMATIC DIFFERENCES
AMONG THE INTACT GROUPS.
THESE SYSTEMATIC DIFFERENCES BETWEEN THE TWO INTACT GROUPS ARE ALMOST
GUARANTEED TO THREATEN THE INTERNAL VALIDITY OF THE EXPERIMENT. THE SOLUTION
TO THIS PROBLEM IS SIMPLE—DO NOT USE INTACT GROUPS IN A RANDOM GROUPS DESIGN.
 BALANCING EXTRANEOUS VARIABLES

• A NUMBER OF FACTORS IN AN EXPERIMENT MAY VARY AS A RESULT OF

PRACTICAL CONSIDERATIONS WHEN CARRYING OUT THE STUDY.
• FOR EXAMPLE, TO COMPLETE AN EXPERIMENT MORE QUICKLY, A
RESEARCHER MIGHT DECIDE
TO HAVE SEVERAL DIFFERENT EXPERIMENTERS TEST SMALL GROUPS OF
PARTICIPANTS.
THE SIZES OF THE GROUPS AND THE EXPERIMENTERS THEMSELVES BECOME
POTENTIALLY
RELEVANT VARIABLES THAT COULD CONFOUND THE EXPERIMENT.
• FOR EXAMPLE, IF ALL THE INDIVIDUALS IN THE EXPERIMENTAL GROUP WERE
TESTED BY ONE EXPERIMENTER AND ALL OF THOSE IN THE CONTROL GROUP
WERE TESTED BY ANOTHER EXPERIMENTER, THE LEVELS OF THE INTENDED
INDEPENDENT VARIABLE WOULD BECOME CONFOUNDED WITH THE TWO
EXPERIMENTERS. WE WOULD NOT BE ABLE TO DETERMINE WHETHER AN
OBSERVED DIFFERENCE BETWEEN THE TWO GROUPS WAS DUE TO THE
INDEPENDENT VARIABLE OR TO THE FACT THAT DIFFERENT EXPERIMENTERS
TESTED PARTICIPANTS IN THE EXPERIMENTAL AND CONTROL GROUPS.
• POTENTIAL VARIABLES THAT ARE NOT DIRECTLY OF INTEREST TO THE
RESEARCHER BUT MAY
BE SOURCES OF CONFOUNDING ARE CALLED EXTRANEOUS VARIABLES .
• BLOCK RANDOMIZATION CONTROLS EXTRANEOUS VARIABLES BY BALANCING
THEM ACROSS
• GROUPS.
 SUBJECT LOSS

• THE LOGIC OF THE RANDOM GROUPS DESIGN REQUIRES THAT THE GROUPS
IN AN EXPERIMENT DIFFER ONLY BECAUSE OF THE LEVELS OF THE
INDEPENDENT VARIABLE.
• FORMING COMPARABLE (I.E., SIMILAR) GROUPS OF SUBJECTS AT THE
BEGINNING OF AN EXPERIMENT IS AN ESSENTIAL CHARACTERISTIC OF THE
RANDOM GROUPS DESIGN.
• IT IS EQUALLY IMPORTANT THAT THE GROUPS BE COMPARABLE EXCEPT FOR
THE INDEPENDENT VARIABLE AT THE END OF THE EXPERIMENT.
• MECHANICAL SUBJECT LOSS OCCURS WHEN A SUBJECT FAILS TO
COMPLETE THE EXPERIMENT
• BECAUSE OF AN EQUIPMENT FAILURE (IN THIS CASE, THE EXPERIMENTER IS
CONSIDERED
• PART OF THE EQUIPMENT).
• WHEN SUBJECTS BEGIN AN EXPERIMENT BUT FAIL TO COMPLETE IT
SUCCESSFULLY, THE INTERNAL VALIDITY OF THE EXPERIMENT CAN BE
THREATENED.
• IT IS IMPORTANT TO DISTINGUISH BETWEEN TWO WAYS IN WHICH SUBJECTS
CAN FAIL TO
COMPLETE AN EXPERIMENT: MECHANICAL SUBJECT LOSS AND SELECTIVE
SUBJECT LOSS.
• MECHANICAL LOSS IS A LESS CRITICAL PROBLEM THAN SELECTIVE SUBJECT
LOSS BECAUSE THE LOSS IS UNLIKELY TO BE RELATED TO ANY
CHARACTERISTIC OF THE SUBJECT.
• WHEN MECHANICAL SUBJECT LOSS OCCURS, IT SHOULD BE DOCUMENTED.
THE NAME OR SUBJECT NUMBER OF THE DROPPED SUBJECT AND THE
REASON FOR THE LOSS SHOULD BE RECORDED. THE LOST SUBJECT CAN
THEN BE REPLACED BY THE NEXT SUBJECT TESTED.
• SELECTIVE SUBJECT LOSS IS A FAR MORE SERIOUS MATTER. SELECTIVE SUBJECT LOSS OCCURS
• (1) WHEN SUBJECTS ARE LOST DIFFERENTIALLY ACROSS THE CONDITIONS OF THE EXPERIMENT;
• (2) WHEN SOME CHARACTERISTIC OF THE SUBJECT IS RESPONSIBLE FOR THE LOSS
• (3) WHEN THIS SUBJECT CHARACTERISTIC IS RELATED TO THE DEPENDENT VARIABLE USED TO

ASSESS THE OUTCOME OF THE STUDY. SELECTIVE SUBJECT LOSS DESTROYS THE COMPARABLE
GROUPS THAT ARE ESSENTIAL TO THE LOGIC OF THE RANDOM GROUPS DESIGN AND CAN THUS
RENDER THE EXPERIMENT UNINTERPRETABLE.
 DEMAND CHARACTERISTICS AND
EXPERIMENTER EFFECTS
• DEMAND CHARACTERISTICS REPRESENT ONE POSSIBLE SOURCE OF BIAS DUE
TO PARTICIPANTS’ EXPECTATIONS (ORNE, 1962).
• DEMAND CHARACTERISTICS REFER TO THE CUES AND OTHER INFORMATION
THAT PARTICIPANTS USE TO GUIDE THEIR BEHAVIOUR IN A PSYCHOLOGICAL
STUDY.
• POTENTIAL BIASES ALSO CAN ARISE DUE TO THE EXPECTATIONS OF THE
EXPERIMENTERS.
• THE GENERAL TERM USED TO DESCRIBE THESE BIASES IS EXPERIMENTER
EFFECTS
• (ROSENTHAL, 1963, 1994A).
• EXPERIMENTER EFFECTS MAY BE A SOURCE OF CONFOUNDING IF
EXPERIMENTERS TREAT SUBJECTS DIFFERENTLY IN THE DIFFERENT GROUPS
OF THE EXPERIMENT IN WAYS OTHER THAN THOSE REQUIRED TO IMPLEMENT
THE INDEPENDENT VARIABLE.
• RESEARCHERS CAN NEVER COMPLETELY ELIMINATE THE PROBLEMS OF
DEMAND CHARACTERISTICS AND EXPERIMENTER EFFECTS, BUT THERE ARE
SPECIAL RESEARCH DESIGNS THAT CONTROL THESE PROBLEMS.
RESEARCHERS USE A PLACEBO CONTROL GROUP AS ONE WAY TO
CONTROL DEMAND CHARACTERISTICS.
• THE USE OF PLACEBO CONTROL GROUPS IN COMBINATION WITH A DOUBLE-
BLIND PROCEDURE CAN CONTROL FOR BOTH DEMAND CHARACTERISTICS
AND EXPERIMENTER EFFECTS.
• A DOUBLE-BLIND PROCEDURE, BOTH THE PARTICIPANT AND THE
OBSERVER ARE BLIND TO
• (UNAWARE OF) WHAT TREATMENT IS BEING ADMINISTERED.
 ESTABLISHING THE EXTERNAL
VALIDITY OF EXPERIMENTAL FINDINGS
• EXTERNAL VALIDITY REFERS TO THE EXTENT TO WHICH FINDINGS FROM A
RESEARCH STUDY CAN BE GENERALIZED TO INDIVIDUALS, SETTINGS, AND
CONDITIONS BEYOND THE SCOPE OF THE SPECIFIC STUDY
• CONDUCTING FIELD EXPERIMENTS IS ONE WAY THAT RESEARCHERS CAN
INCREASE THE
• EXTERNAL VALIDITY OF THEIR RESEARCH IN REAL-WORLD SETTINGS.
• ULTIMATELY, THE EXTERNAL VALIDITY OF RESEARCH FINDINGS IS LIKELY TO
BE ESTABLISHED MORE BY THE GOOD JUDGMENT OF THE SCIENTIFIC
COMMUNITY THAN BY DEFINITIVE EMPIRICAL EVIDENCE.
• THE EXTERNAL VALIDITY OF EXPERIMENTAL FINDINGS ALSO CAN BE
ESTABLISHED THROUGH
PARTIAL REPLICATION. PARTIAL REPLICATIONS ARE COMMONLY DONE AS A
ROUTINE PART OF THE PROCESS OF INVESTIGATING THE CONDITIONS UNDER
WHICH A PHENOMENON RELIABLY OCCURS.
• RESEARCHERS CAN ALSO ESTABLISH THE EXTERNAL VALIDITY OF THEIR
FINDINGS BY DOING
CONCEPTUAL REPLICATIONS. WHAT WE WISH TO GENERALIZE FROM ANY ONE
STUDY ARE CONCEPTUAL RELATIONSHIPS AMONG VARIABLES, NOT THE
SPECIFIC C CONDITIONS, MANIPULATIONS, SETTINGS, OR SAMPLES.
MATCHED GROUPS DESIGN
• MATCHED GROUPS DESIGN MAY BE USED TO CREATE COMPARABLE GROUPS
WHEN THERE ARE TOO FEW SUBJECTS AVAILABLE FOR RANDOM ASSIGNMENT
TO WORK
• EFFECTIVELY.
• MATCHING SUBJECTS ON THE DEPENDENT VARIABLE (AS A PRETEST) IS THE
BEST
APPROACH FOR CREATING MATCHED GROUPS, BUT SCORES ON ANY MATCHING
VARIABLE
MUST CORRELATE WITH THE DEPENDENT VARIABLE.
• AFTER SUBJECTS ARE MATCHED ON THE MATCHING VARIABLE, THEY SHOULD
THEN BE
RANDOMLY ASSIGNED TO THE CONDITIONS OF THE INDEPENDENT VARIABLE.
• THE MATCHED GROUPS DESIGN IS A GOOD ALTERNATIVE WHEN NEITHER THE
RANDOM
• GROUPS DESIGN NOR THE REPEATED MEASURES DESIGN CAN BE USED EFFECTIVELY.
• THE LOGIC OF THE MATCHED GROUPS DESIGN IS SIMPLE AND COMPELLING. INSTEAD
OF TRUSTING RANDOM ASSIGNMENT TO FORM COMPARABLE GROUPS, THE
RESEARCHER MAKES THE GROUPS EQUIVALENT BY MATCHING SUBJECTS.
• ONCE COMPARABLE GROUPS HAVE BEEN FORMED BASED ON THE MATCHING, THE
LOGIC OF THE MATCHED GROUPS DESIGN IS THE SAME AS THAT FOR THE RANDOM
GROUPS DESIGN
 NATURAL GROUPS DESIGN
• INDIVIDUAL DIFFERENCES VARIABLES (OR SUBJECT VARIABLES) ARE SELECTED RATHER

THAN MANIPULATED TO FORM NATURAL GROUPS DESIGNS.

• THE NATURAL GROUPS DESIGN REPRESENTS A TYPE OF CORRELATIONAL RESEARCH IN

WHICH RESEARCHERS LOOK FOR COVARIATIONS BETWEEN NATURAL GROUPS VARIABLES

AND DEPENDENT VARIABLES.
• CAUSAL INFERENCES CANNOT BE MADE REGARDING THE EFFECTS OF NATURAL GROUPS

VARIABLES BECAUSE PLAUSIBLE ALTERNATIVE EXPLANATIONS FOR GROUP DIFFERENCES

EXIST.
• RESEARCHERS IN MANY AREAS OF PSYCHOLOGY ARE INTERESTED IN INDEPENDENT

VARIABLES THAT ARE CALLED INDIVIDUAL DIFFERENCES VARIABLES, OR

SUBJECT VARIABLES.
• AN INDIVIDUAL DIFFERENCES VARIABLE IS A CHARACTERISTIC OR TRAIT THAT
VARIES ACROSS
• INDIVIDUALS.
• RELIGIOUS AFFILIATION IS AN EXAMPLE OF AN INDIVIDUAL DIFFERENCES

VARIABLE. RESEARCHERS CAN’T MANIPULATE THIS VARIABLE BY RANDOMLY

ASSIGNING
PEOPLE TO CATHOLIC, JEWISH, MUSLIM, PROTESTANT, OR OTHER GROUPS. INSTEAD,
RESEARCHERS “CONTROL” THE RELIGIOUS AFFI LIATION VARIABLE BY
SYSTEMATICALLY SELECTING
• INDIVIDUALS WHO NATURALLY BELONG TO THESE GROUPS. INDIVIDUAL
DIFFERENCES VARIABLES SUCH AS GENDER, INTROVERSION–EXTRAVERSION,
RACE, OR AGE ARE IMPORTANT INDEPENDENT VARIABLES IN MANY AREAS OF
PSYCHOLOGY