Somebody built the engagement survey in a workshop — twelve items, a Likert scale, a name — and the organization has been steering on its scores ever since. Nobody has checked whether the items hang together, whether they measure one thing or three, or whether the scale reads differently across groups you compare. Decisions are riding on an instrument that has never been inspected.

DeVellis and Thorpe's Scale Development is the field's standard walkthrough of a claim most survey-writers never confront: a scale is a measurement instrument for an unobservable construct, and its quality is an empirical question, not a matter of whether the items sound right. Reliability, in their classical-test-theory framing, is the proportion of score variance that is true-score rather than noise — estimated by coefficient alpha and its relatives — and it is earned through item quality, content sampling, and an adequate development sample, then verified, not assumed. Carmines and Zeller's slim Reliability and Validity Assessment supplies the distinction that keeps the whole exercise honest: reliability is threatened by random error, validity by systematic error, and the two are different failures. A scale can be beautifully consistent and consistently measure the wrong thing; a high alpha is where validation starts, not where it ends.

Watkins's Exploratory Factor Analysis adds the caution that applies directly to dimensionality checks: EFA is a century old and still routinely misapplied, largely because researchers accept software defaults. The classic trap is the Kaiser eigenvalue-greater-than-one rule for deciding how many factors a scale contains — a convenient default that Watkins's evidence-based-practice review treats as a first estimate at best. So when a dimensionality report gives you a Kaiser factor count, the literature's own advice is to read it alongside the first-factor share and the item loadings rather than as a verdict. That is the right posture for the whole report: psychometric statistics are diagnostics that tell you which items to keep, review, or drop — they do not certify that the construct is the one you named.

The textbooks end with formulas and an exhortation to go compute them; here you upload items and a response matrix and the statistics — alpha, omega, eigenvalues, item-level flags, and Mantel-Haenszel DIF across groups — run in deterministic code, with the model confined to narrating numbers already computed.