PSYCH 2061

September 17, 2021

Week 2: Job Analysis, Recruitment, and selection

Job Analysis
Example job description with Job specifications

What is job analysis?

¨ Systematic process of gathering, documenting, and analyzing data about what a person
does in a given job
¨ Work-oriented analysis: Outcomes describing the tasks performed to accomplish those
outcomes
¨ Worker-oriented analysis: General aspects of the job (i.e., perceptual, interpersonal,
sensory, cognitive, and physical activities)

¨ Two key outcomes of a Job analysis

• To describe and specify
¨ KSAOs
• Knowledge – What does someone need to know to perform a job
• Skills – Level of proficiency
• Abilities – More enduring characteristics
• Other characteristics (e.g., personality differences)

- We need to help HR make an informed decision

- Have a good job description and have specification
- Know who your hiring
Legal issues of job analysis
¨ Objective evidence of the skills and abilities required for effective performance
¤ Job requirements in Canada have the potential to discriminate against
individuals.
n Example?
¤ Bona fide Occupational Requirements (BFORs) are standards for setting job
requirements: They must be reasonably related to legitimate work-related
duties. Employers must show that it is impossible to accommodate individuals
sharing the characteristic on which the discrimination is based without imposing
undue hardship upon the employer.

Job Analysis Methods

¨ Interviews
¤ Subject matter experts (i.e., SMEs)
¤ Solicit insights from multiple organizational members, but should be voluntary
and free from any coercion
¨ Direct observation
¤ Job shadowing to obtain knowledge about the job
¨ Structured Job Analysis Questionnaires and Inventories (not important)

Critical incident technique

Generates behaviourally focused descriptions of work activities; describe the incident in detail
to address the following points
¤ Effective and ineffective observable work behaviours that correspond with
superior or inferior performance
¤ Basis for behaviourally anchored rating scales
n The situation leading up to the incident (antecedents)
n Actions or behaviours of the person involved
n Results or outcomes of the action

How does one choose a specific job analysis method?

¨ Operational status: Is the method tested and valid
¨ Quality of outcome: Will the method yield high-quality results?
¨ Availability: Is it accessible to the organization (i.e., off-the-shelf)?
¨ Standardization: Can you compare the results to other organizations, sectors, and job
evaluations?
¨ User acceptability: Is the method overly burdensome?
¨ Training requirement: How much training is needed and available for the chosen
method?
¨ Sample size: From how many employees must data be collected for the method to
provide reliable results?
¨ Cost: What are the costs of a method (materials, consultant fees, person-hours)
Recruitment
¨ Attracting the right people for the job
¤ Goal is to maximize the size of the qualified candidate pool
¤ Person-job fit: Necessary KSAOs
¤ Person-organization fit: Similar values and morals

Goals of Recruitment
¨ Adequate recruitment should result in:
¤ Self-selecting out
¤ Person-job and person-organization fit
n Important from the perspective of both the prospective employer and
employee

Recruitment Sources
¨ Networking with family or friends
¨ Help wanted ads
¨ Direct recruitment by employer
¨ Recruitment firms
¨ Job fair
¨ Internet sources

Realistic Job Preview

¨ Job applicants are told about both the positive and negative aspects of a job
¤ Goal: Calibrate an applicant’s expectations for the job

Do Realistic Job Previews work?

¨ Meta-analysis (Earnest, Allen, & Landis, 2011)
o Voluntary turnover, r = – .07
o Organizational honesty, r = .11
o Role clarity, r = .10
o Attraction, r = -.10
¨ Are these associations meaningful?
¨ Realistic Job Previews are best delivered:
o Verbally and post-hire

Job Analysis Recruitment Selection

Selection - Screening applicants

Applicant Screening
¨ Commonly used methods to screen applicants
¤ Application forms
¤ Biographical data
¤ Resumes/Work experience
¤ Reference checks
¤ “Digital footprints” obtained through social media and networking Web sites
(e.g., Facebook, Twitter; LinkedIn; Google)

Weighted Application Blanks

¨ Choosing a criterion to differentiate desirable from undesirable candidates (e.g., job
tenure)
¤ Reliable, representative, predictable, measureable, relevant, uncontaminated
and bias-free
¨ Identifying criterion groups (e.g., characteristics of employees who stay longer than a
year)
¨ Assigning scores to different response options (e.g., higher score based on average
tenure of former jobs)
Evaluating WAB
Strengths: Invasive
Weaknesses: negative applicant reactions

Resumes
¨ A brief, accurate (hopefully), written description of yourself
¤ Fabrication and embellishment are two components of resume fraud
n Some estimates suggest a quarter to a third of candidates lie on their resume
¨ Few studies have evaluated resumes as a whole
¤ Job-relevant experience, academic achievement, and education listed in resumes are best
predictors of future job success
¤ Employers make (inaccurate) inferences of people’s personality, and use such judgements to
guide their decisions (Cole et al., 2009)

References
¨ Reference: Expression of an opinion regarding an applicant’s: ability, previous
performance, work habits, character, potential for success
¨ Reference Check: Process of confirming the accuracy of resume and job application data
¤ 98% of employers use reference checks (Meinert, 2011)

Do References Work?
¨ Referees are lenient
q Fewer than 1% of applicants are rated below average! Why?
 q Implicit assumption that referee knows the person and is in a good position to evaluate
the candidate
q Poor inter-rater reliability
q Higher correlation between two letters written by the same person for two
different people than between letters written by two people for the same person

Digital footprints
¨ Predictive validity is mixed

Selection – cognitive ability and personality tests

Employment Testing
¨ Ability Tests
o Cognitive
o Physical
¨ Personality Assessments
¨ Interviews
¨ Work samples and simulation tests

Types of cognitive ability

q Verbal: oral and written comprehension and expression
q Quantitative: number facility and mathematical reasoning
q Reasoning: problem sensitivity, deductive/inductive reasoning, and originality
q Spatial: spatial orientation and visualization
q Perceptual: speed/flexibility of closure, perceptual speed
q General tests of cognitive ability tap into our ability to learn, which is why they
consistently emerge as the best predictor of performance

¨ Why do cognitive ability tests predict job performance?

¤ Assess learning speed (e.g., job knowledge acquisition)
¤ Assess decision-making effectiveness (e.g., ability to integrate and weight
multiple sources of information)
¨ Cognitive ability test scores exhibit higher levels of criterion validity for complex jobs.

Strengths: Low case validity

Weaknesses: evoke negative reactions

Personality testing
¨ Personality traits
¤ A set of characteristics or properties that influence, or help to explain, an
individual’s behaviour (Hall & Lindzey, 1970)

Personality traits
 ¨ The Big Five model of personality

¨
¨ Myers-Briggs type indicator
¨ Classifies personality according to types instead of along a continuum
¨ Very popular among organizations/businesses
¨ Issues with validity and reliability?

Personality Surveys and Faking

¨ Ways to minimize faking:
o Provide a faking warning
o Use a forced-choice format
¨ Instructions: Make sure you select one response as the Most similar to you and one
response as the Least similar to you. Only one bubble should be filled in beside each
Most and Least response row.

Evaluating Personality Tests

Strengths:
¨ Predicts best when based on a job analysis
¨ Easy to administer and minimal adverse impact
Weaknesses:
¨ Susceptible to faking
¨ Validity low in some cases

Selection - Interviews and work samples

The Interview
¨ Most commonly used method to select employees
¤ For a long time, they were the sole means of selecting candidates for companies
¨ Structured vs. unstructured Interviews

Problems with the unstructured interview

 ¨ Lack of job-relatedness
¨ Primacy effects (i.e., thin slicing)
¨ Social cues that are irrelevant to KSAOs
¨ Negative information bias
¨ Interviewer-interviewee similarity

The Structured Interview

q Behavioral Description Interviews vs Situational Interviews
q Goal: Measure job-relevant KSAOs while minimizing sources of bias
¨ Behavioral description interviews
e.g., This job involves persuading employees to follow our safety rules. Tell us about a time in
the past when you had to persuade an employee to do something.
¨ Situational Interviews
e.g given a situation and how you would react

Components of a Structured Interview

1. Question sophistication
2. Question consistency
3. Evaluation standardization
Because of this, they are:
ü More reliable
ü More valid
ü More legally defensible
Criterion Validity (with respect to job performance)
o Structured Interview r = .34

Work Samples
¨ Assesses an applicant’s ability to perform a work-related skill at this time
¨ Applicant performs actual job-related tasks
q Secretary types a letter
q Data scientist performs statistical analyses
q Computer scientists partake in a coding challenge
¨ Should be used if we expect those hired to be able to do a particular task before starting
a job

Evaluating Work Samples

Advantages:
¨ Criterion validity
o Verbal work samples (ρ = .48)
o Motor work samples (ρ = .43)
¨ Incremental validity over and above cognitive ability
¨ Positive reactions from applicants – high face validity
Disadvantages:
 ¨ Expensive to create/administer
¨ Tend to assess maximum rather than typical performance

AI-ML and selection

¨ Estimated ~33% companies have adopted AI-ML tools to aid their recruitment and
selection processes (Stephan et al., 2017)
¤ Scoring asynchronous video interviews (e.g., HireVue)
¤ Scraping social media profiles (e.g., Entelo)
¤ Gamifying the selection process (e.g., Pymetrics)

¨ Potential benefits
¤ Decision-making speed and efficiency = $$$
¤ Sophisticated algorithms might help uncover new sources of information in ways
that could improve selection decisions
¨ Potential limitations
¤ Prediction quality depends on the quality of data being used by algorithms
¤ Difficult to justify how selection decisions were made (i.e., black box problem)

Decision-making
¨ Once you obtain a test score for each applicant on each assessment, you need to make a
decision about who should get the job
¨ Methods used to make decisions
1. Top-down selection
2. Cut-off scores
3. Judgmental and statistical composites

Top-Down Selection
¨ Rank order applicants in terms of their score from highest to lowest
o Give a job offer the applicant who scored the highest, if he/she rejects the offer,
you then offer it to the next highest ranked candidate
¨ Used when selection is based on one predictor

Cut-off Scores
¨ Which applicants can perform at a minimally acceptable level?
¨ Applicants below a cut-off point are rejected
o For example, a grade of 50/100 is a cutoff
• If you score lower than a 50, you fail

Composite methods
¨ Collect both statistical and judgmental data about applicants e.g., Selection might have
involved
o Unstructured interviews and a reference check (judgmental data)
o Cognitive ability test (statistical data)
 ¨ Judgmental composite: Form an overall impression and make a decision without any
particular formula
¨ Statistical composite: The ratings or scores are then combined using a formula or
regression equation to produce an overall score for each applicant
o Weighted vs. unweighted statistical composites

Decision Making Models

¨ When several selection tools are used to hire employees; how can we combine the
information statistically?
1. Multiple Regression
2. Multiple Cut-Off
3. Multiple Hurdle

Multiple Regression
¨ Rank candidates based on their regression score
¨ Predicted job Performance = b1interview + b2reference check + b3cognitive ability test +
constant
¨ Predicted job Performance = (0.3 * applicant score on interview) + (0.1 * applicant
score on reference check) + (0.5 * applicant score on cognitive ability test )+ constant

¨ Problem with this model?

o Assumes that a high score on one predictor will compensate a low score on
another predictor

Multiple Cut-Off
¨ A cut-off score is set for each predictor
¤ e.g., Minimum cognitive ability score and minimum job-specific skill
¨ A candidate is rejected if they fall below the cut-off score on any one of the predictors
¤ e.g., Someone with exceptional cognitive ability that scores below the job-
specific skill would be rejected

Multiple Hurdle
¨ Similar to multiple cut-off – candidates must pass a certain cut-off on each predictor
¨ BUT…
o Applicants are screened out as soon as they fail to meet the cut-off score one
any one predictor

Utility Analysis
¨ How can you evaluate the effectiveness of your decisions?
¨ Utility analysis: The degree to which the use of a selection system improves the quality
of individuals selected beyond what would have happened if you had not used that
selection system
Taylor-Russell Model
¨ To demonstrate the effectiveness of a selection system you need:
1. Validity coefficient
2. Base rate – The proportion of applicants who would be successful at the job
without the selection procedure
3. Selection ratio – The proportion of applicants who were hired for the position
¨ These allow you to calculate a success rate