peopleanalyst

library / lib16570e59fc4b828c

Standardized Survey Interviewing - Minimizing Interviewer Error

In a sentence

A practical, evidence-grounded guide to conducting standardized survey interviews that minimize the measurement error interviewers introduce into survey data.

Standardized Survey Interviewing distills decades of methodological research into a clear, actionable framework for treating survey interviewing as rigorous scientific measurement rather than art. Fowler and Mangione document precisely how interviewers—through the way they read questions, probe answers, record responses, relate to respondents, and set performance standards—can inflate standard errors, reduce reliability, and bias estimates. Drawing on a large-scale experiment testing interviewer training and supervision plus a rich body of prior research (notably Cannell's work), the authors show which levers actually matter: writing questions that can be asked and answered consistently, training respondents in their role, communicating high accuracy standards, selecting and training interviewers, and—critically—supervising the question-and-answer process via tape recording and monitoring. The book is essential for anyone who collects quantitative data from people, offering cost-effective, empirically justified strategies to produce more valid and precise survey estimates.

The four lenses

  • Science
  • Statistics
  • Systems
  • Strategy

The model

A causal model in which design levers (question design, training, supervision, selection) and contextual conditions shape interviewer behaviors and respondent orientation, which in turn drive standardization and ultimately survey data quality (interviewer-related error, bias, precision).

Question Design Qualitydesign lever

The degree to which survey questions are fully scripted, understood identically by all respondents, specify acceptable response forms, and minimize the need for discretionary interviewer probing or clarification.

Interviewer Trainingdesign lever

The amount, content, and technique of general interviewer training in standardized interviewing skills prior to data collection, especially supervised practice in reading, probing, and recording.

Interviewer Supervisiondesign lever

The process of gathering information about how interviewers conduct the question-and-answer process during data collection, evaluating it, and providing feedback, notably via tape recording or telephone monitoring.

Interviewer Demographic Characteristicscontextual condition

Observable or inferable interviewer attributes such as age, gender, race, ethnicity, religion, education, and social status that may alter the stimulus situation for respondents.

Standardized Interviewer Behaviorbehavioral pattern

The extent to which interviewers read questions as worded, probe nondirectively, record answers without discretion, and remain interpersonally neutral during the question-and-answer process.

Respondent Role Orientation and Standardspsychological state

The respondent's understanding of the standardized interview rules and internalized standards for effortful, accurate, complete reporting, shaped by interviewer-set context and expectations.

Interviewer-Respondent Relationshipcontextual condition

The tone and quality of the rapidly established relationship between interviewer and respondent, ideally a warm but professional, task-focused, and trusting relationship enabling free communication.

Question Probing Demandbehavioral pattern

The degree to which a question routinely requires interviewers to probe beyond simply repeating the question in order to obtain an adequate answer, creating opportunities for inconsistency.

Measurement Standardizationoutcome metric

The degree to which the measurement process is identical across interviewers and respondents so that differences in answers reflect real differences between respondents rather than differences in how answers were produced.

Interviewer-Related Erroroutcome metric

Variation in survey answers that can be statistically associated with the interviewers who obtained them, indicating a failure of standardized measurement and reducing precision and validity.

Survey Data Qualityoutcome metric

The reliability, validity, and precision of survey-based estimates, reflected in low bias, low standard errors, and answers that faithfully measure the population characteristics of interest.

How they connect

  • question design quality influences probing demand
  • probing demand predicts standardized interviewer behavior
  • probing demand predicts interviewer related error
  • question design quality predicts interviewer related error
  • interviewer training predicts standardized interviewer behavior
  • interviewer training influences data quality
  • interviewer supervision predicts standardized interviewer behavior
  • interviewer supervision predicts interviewer related error
  • interviewer training moderates interviewer supervision
  • standardized interviewer behavior predicts measurement standardization
  • respondent role orientation predicts data quality
  • standardized interviewer behavior influences respondent role orientation
  • measurement standardization predicts interviewer related error
  • interviewer related error predicts data quality
  • interviewer characteristics moderates interviewer related error
  • interviewer respondent relationship correlates data quality

The story

The reader A researcher, survey manager, or data collector who wants to produce accurate, valid, precise quantitative descriptions of a population from interview data.

External problem

Interviewers introduce inconsistency and error into survey answers, inflating standard errors and reducing reliability and validity.

Internal problem

The researcher feels uncertain whether their data truly measure what they intend, and anxious that human variability undermines their conclusions.

Philosophical problem

Treating interviewing as unteachable art is wrong; measurement in social science can and should be rigorous and standardized.

The plan

  1. Design questions that are fully scripted, understood identically, and specify acceptable responses.
  2. Train interviewers in the four standardized techniques with supervised practice.
  3. Have interviewers train respondents and set high accuracy standards.
  4. Select interviewers for reading/writing skills, controlling demographics only when topic-relevant.
  5. Provide two to four days of training including supervised practice.
  6. Supervise the question-and-answer process via tape recording or telephone monitoring with structured feedback.

Success

  • More precise estimates with lower standard errors and less bias.
  • Reliable, replicable data whose differences reflect real respondent differences.
  • Interviewers who consistently apply high standards and respondents who understand their role.
  • Cost-effective quality improvements relative to increasing sample size.

At stake

  • Data less valid and precise than they should be, with hidden interviewer-related error.
  • Inability to replicate findings because data collection procedures cannot be described.
  • Wasted resources on large samples while ignoring cheaper error-reduction levers.
  • Biased estimates driven by interviewer inconsistency and improvisation.

Questions this book answers

What is a standardized survey interview and why does standardization matter for measurement?
What is interviewer-related error and how is it detected and measured?
Which interviewer behaviors most threaten standardized measurement?
How can question design reduce interviewer effects?
How do interviewer selection, training, and supervision affect data quality?

Glossary

Question Design Quality
The degree to which survey questions are fully scripted, consistently understood, and specify acceptable responses, minimizing discretionary interviewer action.
Interviewer Training
The pre-data-collection instruction of interviewers in standardized interviewing techniques and problem-solving skills, especially supervised practice.
Interviewer Supervision
Post-training gathering, evaluating, and feeding back information about interviewer performance, focused on the question-and-answer process.
Interviewer Demographic Characteristics
Interviewer attributes discernible or inferable by respondents that may alter the interview stimulus.
Standardized Interviewer Behavior
Interviewer adherence to reading as worded, nondirective probing, discretion-free recording, and interpersonal neutrality.
Respondent Role Orientation and Standards
Respondent understanding of interview rules and internalized standards for accurate, complete, effortful reporting.
Interviewer-Respondent Relationship
The tone and quality of the rapidly established dyadic relationship enabling cooperation and free communication.
Question Probing Demand
The extent to which a question routinely requires discretionary probing beyond repeating the question to obtain an adequate answer.