peopleanalyst

library / lib9e47f067f1b116a7

SURVEY & QUESTIONNAIRE DESIGN_ Collecting Primary Data to Answer Research Questions (55)

In a sentence

A practice-oriented guide to designing surveys and questionnaires that collect valid, reliable primary data to answer well-formulated research questions.

This concise, practice-focused ebook walks researchers, students, and managers through the full arc of survey design—from formulating a sharp research question and hypothesis, through writing clear factual and non-factual questions, choosing appropriate levels of measurement, coding responses, and ensuring validity and reliability, to laying out a self-completion questionnaire. Drawing on the authors' years of teaching survey methods at University College Cork and grounded in established survey-methodology literature (Fowler, Dillman, Sudman & Bradburn, Oppenheim), it pairs concrete examples, exercises, video links, and a sample small-business questionnaire to help readers avoid the common pitfalls that produce confused respondents and useless data. If you want to collect primary data that actually answers your question, this book gives you the practical decision rules to do it.

The story it tells the reader

The reader A researcher, student, or manager who wants to collect primary data that genuinely answers their research question.

External problem

They need to design a survey that produces valid, reliable, analysable data.

Internal problem

They feel uncertain and overwhelmed about whether their questions are good enough and fear collecting useless data.

Philosophical problem

It's wrong to assume anyone with common sense can write a questionnaire—poor design wastes effort and misleads.

The plan

  1. Formulate and clarify your research question, hypothesis, and objectives.
  2. Review the literature and choose a deductive or inductive approach.
  3. Design clear, standardised factual and non-factual questions.
  4. Choose appropriate levels of measurement and code responses systematically.
  5. Test the instrument for validity and reliability.
  6. Lay out the questionnaire for clarity and high response.

Success

  • A valid, reliable questionnaire that yields analysable data and answers the research question.

At stake

  • Confused respondents, unreliable answers, low response rates, and data that cannot answer the research question.

Model of the world · 12 constructs · 14 relations

A framework linking survey design levers (research question clarity, question wording, measurement choices, coding, layout, testing) through respondent cognitive states (comprehension, ability, willingness) to data quality outcomes (validity, reliability, response rate).

Design levers

  • Question Wording Quality
  • Research Question Clarity
  • Level of Measurement Choice
  • Instrument Pre-testing and Field-testing
  • Coding Quality
  • +1 more

Intermediate states & behaviors

  • Respondent Comprehension
  • Respondent Ability to Answer
  • Respondent Willingness to Answer

Outcomes

  • Data Validity
  • Data Reliability
  • Response Rate
Consolidated shape of the book’s model — full constructs and relationships below.

Research Question Claritydesign lever

The degree to which the research question, hypothesis, and objectives are clearly formulated, specific, and well-aligned so they can guide the design and types of questions, the sample, and the data needed for the survey.

Question Wording Qualitydesign lever

The extent to which individual survey questions are short, simple, standardised, free of jargon, double-barrels, leading, negative, assumptive, and sensitive wording, so each respondent experiences the same clear question.

Level of Measurement Choicedesign lever

The choice of nominal, ordinal, interval, or ratio scale for each question, which determines the precision of data captured, the analysis available, and the difficulty respondents face in providing the information.

Coding Qualitydesign lever

The rigour with which responses are translated into numbers using mutually exclusive, unambiguous, consistently applied codes, with appropriate pre-coding, post-coding, missing-data codes, and logic/range checks to prevent transcription and coding decision errors.

Questionnaire Layout Qualitydesign lever

The quality of the questionnaire's physical and visual organisation—cover letter, clear instructions, good first question, meaningful order, consistent answer spaces, white space, and avoidance of awkward formats—that motivates respondents and eases completion.

Instrument Pre-testing and Field-testingdesign lever

The extent of pre-testing (expert reviews, focus groups, cognitive testing, interviewer debriefing, observational interviews, behaviour coding) and field-testing (pilot tests, dress rehearsals) used to detect problems and minimise measurement error before full distribution.

Respondent Comprehensionpsychological state

The degree to which a respondent understands the question being asked in the way the researcher intended, reflecting the first golden rule that wording, vocabulary, and clarity allow a consistent interpretation across respondents.

Respondent Ability to Answerpsychological state

The respondent's capacity to provide an accurate answer, affected by task difficulty, cognitive ability, whether they actually know the information, the reference period, and ability to recall information from memory.

Respondent Willingness to Answerpsychological state

The respondent's willingness to give correct and valid answers, reduced by sensitive or prestige-biased topics such as income, weight, alcohol or drug use, and increased by salient, non-threatening question framing.

Data Validityoutcome metric

The extent to which the answers obtained correspond to what they are intended to measure, encompassing content, face, criterion, and construct validity so the instrument measures the right concept for the research hypothesis.

Data Reliabilityoutcome metric

The accuracy or precision of the measuring instrument—its capacity to extract the same response from respondents with similar characteristics—assessed via test-retest, internal consistency, alternative form, and split ballot methods.

Response Rateoutcome metric

The proportion of intended respondents who complete and return the questionnaire, influenced by question length, complexity, sensitivity, layout, motivation, and the balance between multiple short questions and long complex ones.

How they connect

  • research question clarity → influences question wording quality
  • research question clarity → influences measurement level choice
  • question wording quality → predicts respondent comprehension
  • question wording quality → predicts respondent willingness to answer
  • measurement level choice → influences respondent ability to answer
  • respondent comprehension → predicts data reliability
  • respondent comprehension → mediates data validity
  • respondent ability to answer → predicts data validity
  • respondent willingness to answer → predicts data validity
  • coding quality → predicts data reliability
  • questionnaire layout quality → predicts response rate
  • question wording quality → influences response rate
  • instrument testing → predicts data validity
  • instrument testing → predicts data reliability

Possible measures & feedback loops

A candidate team / org survey built from this book’s model — exploratory operationalizations, not validated instruments. Where a construct maps to a validated measure in Principia, we’ll point to that instead.

Research Question Clarity

expert clarity rating; checklist compliance score

self-report suitability: medium

Question Wording Quality

guideline-violation count; expert quality rating

self-report suitability: low

Level of Measurement Choice

per-item scale classification

self-report suitability: low

Coding Quality

coding error rate; inter-coder agreement

self-report suitability: none

Questionnaire Layout Quality

layout checklist score

self-report suitability: low

Instrument Pre-testing and Field-testing

number of testing methods used; revisions made

self-report suitability: medium

Respondent Comprehension

cognitive-interview comprehension coding; misinterpretation frequency

self-report suitability: medium

Respondent Ability to Answer

item non-response rate; response latency

self-report suitability: medium

Respondent Willingness to Answer

sensitive-item non-response rate; refusal rate

self-report suitability: low

Data Validity

criterion correlation; content coverage rating

self-report suitability: none

Data Reliability

test-retest correlation; internal consistency

self-report suitability: none

Response Rate

completed returns / distributed

self-report suitability: none

Preview the survey →

Frameworks & instruments in this book

  • Design the survey around clearly stated objectives derived from the research question.
  • Keep questions short, simple, and free of jargon, double-barrels, leading and assumptive wording.
  • Match question type to the data needed and the respondent's capacity to answer.
  • Code in detail and consistently so categories are mutually exclusive and collapsible.
  • Pre-test and field-test the instrument to minimise measurement error.
  • Lay out self-completion questionnaires for clarity, consistency, and respondent motivation.

Several of these are operationalized as tools in the People Analytics Toolbox.

Topics

Related in the library