library / lib4d61a2728663736e
Structured_Interviewing_Raising
In a sentence
A highly structured employment interviewing technique, built on job analysis and standardized scoring, can raise the psychometric properties of the interview to the level of traditional cognitive aptitude tests.
For decades, the employment interview has been condemned by researchers as unreliable, invalid, subjective, and legally vulnerable. This study demonstrates that structuring the interview through six concrete steps—job-analysis-based questions, identical questions for every candidate, anchored rating scales, a trained interview panel, consistent administration, and rigorous attention to fairness and documentation—transforms the interview into a psychometrically sound selection device. In a field study of 149 entry-level production hires, the structured interview achieved high interrater reliability (r = .88), strong predictive validity (corrected r = .56), demonstrated fairness for minorities and women, and produced substantial economic utility. The technique's validity rivals or exceeds paper-and-pencil cognitive aptitude tests and far exceeds the traditional interview (mean r = .14), while offering managers an involving, defensible, and credible role in hiring. This is essential reading for anyone who selects employees and wants to keep the interview they love while giving it the rigor of a test.
The four lenses
- Science
- Statistics
- Systems
- Strategy
Tags
The model
A causal model in which interview design levers (job-analysis-based questions, question standardization, anchored scoring, panel rating, consistent administration, guideline adherence) produce standardization and job-related cognitive content, which raise the interview's psychometric properties (reliability, validity, fairness) and ultimately selection utility.
Job-Analysis-Based Question Developmentdesign lever
The design practice of deriving all interview questions from a systematic job analysis that identifies important, prerequisite knowledge, skills, abilities, and other requirements at the appropriate complexity level of the target job.
Question Standardization Across Candidatesdesign lever
The practice of asking every candidate the identical set of questions with no prompting or follow-up (questions may only be repeated), ensuring each applicant is evaluated on the same stimuli.
Example-Anchored Rating Scalesdesign lever
A predetermined scoring system for each question in which good, marginal, and poor answers are defined with concrete example responses scaled to job requirements, enhancing objectivity and consistency of judging candidate responses.
Trained Interview Panel Ratingdesign lever
The use of multiple trained raters, typically three (supervisors and a personnel representative), who independently record and rate each candidate's answers, reducing idiosyncratic biases and order or contrast effects of single interviewers.
Consistent Administration Processdesign lever
The practice of administering the interview uniformly to all candidates, with the same member asking all questions, no between-interview discussion of candidates, extensive note taking, equal item weighting, and a nonstressful setting.
Fairness and Documentation Guideline Adherencedesign lever
Explicit attention to job relatedness, test fairness, and documentation in accordance with professional (Validation Principles) and legal (Uniform Guidelines) standards, including adverse impact analyses and validity evidence.
Overall Process Standardizationpsychological state
The mediating state in which the combined design levers produce a highly standardized selection procedure that minimizes subjectivity, inconsistency, idiosyncratic bias, and susceptibility to order and contrast effects across candidates and raters.
Job-Related Cognitive Contentpsychological state
The mediating property whereby interview items tap job knowledge and general cognitive ability requirements at the appropriate complexity level, making the interview function much like an orally administered content-valid ability measure.
Interrater Reliabilityoutcome metric
The degree of consistency and agreement among the panel raters' scores of candidate answers, indicating that the interview measurement is stable and reproducible across independent judges.
Predictive Validityoutcome metric
The strength of the relationship between structured interview scores and subsequent job performance, reflecting how well the interview forecasts on-the-job effectiveness for hired candidates.
Test Fairnessoutcome metric
The absence of unfair differential prediction of job performance across demographic subgroups (race, sex), assessed by equality of regression slopes and by whether a common line does not underpredict for protected groups.
Selection Utilityoutcome metric
The estimated economic value gained from using the structured interview over random selection, expressed in dollar terms as a function of validity, selection ratio, and the standard deviation of job performance in dollars, net of development and administration costs.
How they connect
- job analysis based questions → predicts job related cognitive content
- question standardization → predicts process standardization
- anchored scoring → predicts process standardization
- panel rating → predicts process standardization
- consistent administration → predicts process standardization
- panel rating → predicts interrater reliability
- process standardization → predicts interrater reliability
- process standardization → predicts predictive validity
- job related cognitive content → predicts predictive validity
- interrater reliability → predicts predictive validity
- guideline adherence → predicts test fairness
- job analysis based questions → predicts test fairness
- predictive validity → predicts selection utility
- job related cognitive content → correlates test fairness
The story
The reader A hiring manager, HR professional, or I-O practitioner who wants to select the best employees using the interview—a tool they trust and want to keep using.
External problem
The traditional employment interview is unreliable, of low validity, and legally vulnerable, so hiring decisions based on it are poor and defensible only with difficulty.
Internal problem
They feel torn between their belief in interviewing and the constant research warnings that interviews don't work, leaving them uncertain their hiring choices are sound or fair.
Philosophical problem
It is wrong to make consequential, discriminatory-risk hiring decisions on a subjective, unstandardized process when a rigorous, fair, evidence-based alternative is available.
The plan
- Develop interview questions from a rigorous job analysis of important, prerequisite requirements.
- Ask the identical set of questions of every candidate without prompting or follow-up.
- Anchor rating scales with concrete example answers for good, marginal, and poor responses.
- Assemble and train an interview panel to independently record and rate answers.
- Administer the process consistently to all candidates and average ratings equally.
- Document job relatedness, fairness analyses, and validity evidence per testing guidelines.
Success
- Hiring decisions are reliable, valid, and predictive of actual job performance.
- Selection is demonstrably fair to minorities and women and legally defensible.
- The organization gains substantial economic utility from better hires.
- Managers stay involved in hiring in a role they find engaging and credible.
At stake
- Continued reliance on subjective, low-validity interviews yields poor hires.
- Selection decisions remain exposed to bias and legal challenge.
- The organization loses money through weak workforce productivity.
- Managers keep trusting a haphazard process that researchers rightly criticize.
Questions this book answers
- Can the employment interview be made reliable and valid?
- What specific procedures raise the psychometric quality of the interview?
- Does a highly structured interview predict job performance as well as cognitive aptitude tests?
- Is a structured interview fair to minorities and women?
- Does the structured interview provide economic utility to the organization?
Glossary
- Job-Analysis-Based Question Development
- The degree to which interview questions are systematically derived from a job analysis identifying important, prerequisite knowledge, skills, abilities, and other requirements at the appropriate job complexity level.
- Question Standardization Across Candidates
- The extent to which every candidate is presented with the identical set of interview questions, with no prompting or follow-up beyond repetition.
- Example-Anchored Rating Scales
- The presence and quality of a predetermined scoring system defining good, marginal, and poor answers for each question using concrete example responses scaled to job requirements.
- Trained Interview Panel Rating
- The use of multiple trained raters who independently record and rate each candidate's answers to reduce idiosyncratic bias and rating errors.
- Consistent Administration Process
- The uniformity with which the interview is delivered to all candidates, including a single questioner, no between-candidate discussion, note taking, equal item weighting, and a nonstressful setting.
- Fairness and Documentation Guideline Adherence
- The extent to which the interview process explicitly attends to job relatedness, fairness, and documentation per professional and legal testing standards.
- Overall Process Standardization
- The overall degree to which the combined design features produce a uniform, low-subjectivity selection procedure minimizing bias and contextual measurement error.
- Job-Related Cognitive Content
- The extent to which interview items tap job knowledge and general cognitive ability requirements at the appropriate complexity, functioning like an orally administered content-valid ability measure.
Related in the library