What the Exit Data Can't See

During the Second World War the U.S. military had a problem and a pile of data. Bombers were being lost over Europe, and the planes that made it home came back peppered with bullet holes — concentrated on the wings and fuselage, sparse around the engines. The obvious move was to armor where the damage was. Abraham Wald, a statistician at Columbia's Statistical Research Group, told them to do the opposite: armor the engines — the places with no holes.¹

His reasoning is one of the cleanest pieces of thinking in the history of data. The holes weren't showing where bombers got hit. They were showing where a bomber could get hit and still come home. The planes shot through the engines weren't in the data set, because they were at the bottom of the Channel. The damage map was a map of survivable damage, and reading it straight pointed the armor at exactly the wrong place.

Every exit survey is that damage map.

They say: ask people why they leave

It is the most natural instinct in retention work. People are quitting; ask the quitters why. Stand up an exit survey, run exit interviews, code the reasons, rank them, and brief the leadership team: our top drivers of attrition are compensation, manager, and growth. It feels like going straight to the source — they're on their way out, the thinking goes, so they'll finally tell the truth.

And then the company armors the bullet holes on the planes that came back.

The data is a map of who you already lost

Here is the principal issue. Exit data is silent on almost everyone who matters, because three layers of selection sit between the truth and the chart. First, it only contains people who already left — the decision is made, the save is impossible, the intelligence arrives too late to act on the person who gave it. Second, among leavers, it only contains those who chose to respond — and the angriest and the most checked-out are often exactly the ones who skip the exit survey or give it a shrug, so the responses skew toward the conscientious and the diplomatic. Third, the ones who do answer are talking to an employer who controls their reference and their rehire eligibility, so the reasons get sanded down to the safe ones. Comp and a better opportunity are easy to say on the way out. My manager was the reason and everyone knows it is not.

So the exit chart describes a thrice-filtered population — people who left, who answered, and who felt safe being honest — and the most valuable group in the whole story is the one it structurally cannot contain: the people who are looking right now, still at their desks, still saveable, and not yet in any exit file. You are studying the survivors of a selection process and calling it the cause of the losses.

Measure the living, not the dead

The fix is not to throw out exit data. It is to stop treating it as the primary instrument and to measure the population you can still do something about.

That means moving the measurement upstream, to current employees: validated quit-intention items fielded while people are still here, the push and pull factors that predict leaving before it happens, surfaced through a channel safe enough that the honest answer is sayable — which, for people who still have to come to work tomorrow, means real protection, not a promise. It means treating exit data as a hypothesis generator, weighted for who's missing, never as the unbiased voice of the departed. And where you do use it, model the selection explicitly: the people who respond to an exit survey differ systematically from those who don't, and there is a whole statistical literature on correcting for exactly that kind of non-random selection rather than pretending it away.²

The honest version is more work than emailing a survey to people who already quit. It requires measuring the people who haven't — safely, before they go — which is harder, slower, and the only version that can actually change the outcome instead of just explaining it after the fact.

Why it's worth raising your voice about

Because the cost of the survivorship version is invisible and large. A company reads its exit data, sees compensation at the top, and spends a cycle on pay adjustments — while the real engine of its regretted attrition, the thing the best people leave over and never write on an exit form, goes unmeasured and unaddressed, quarter after quarter. The data didn't lie about what was on it. It lied by being silent about what wasn't, and silence doesn't show up as a bar on the chart.

So when someone hands you the ranked reasons people leave, ask Wald's question: who isn't in this data, and would they have said something different? The people still at their desks, quietly updating their resumes, would say something different — and they're the only ones you can still keep. The exit interview is the bullet-hole map of the planes that came back. The intelligence you actually need is on the ones that didn't.

---

Measurement-first method, useful whether or not you ever work with us. Measuring the saveable population — protected quit-intention signal from current employees, upstream of the exit — rather than doing forensics on the departed is the posture behind the Principia measurement program; the protection that makes current-employee honesty possible is its own argument in Safe to Say. A sibling of the dashboard-trap pieces Correlation Isn't a Driver, The Benchmark Trap, and The Law of Small Numbers. Every footnote names a real, checkable work.

Footnotes

1. Abraham Wald's wartime survivability analysis for the Statistical Research Group (1943); the scholarly reconstruction is Marc Mangel & Francisco J. Samaniego, "Abraham Wald's Work on Aircraft Survivability," Journal of the American Statistical Association 79, no. 386 (1984): 259–267 — the insight that damage observed on returning aircraft marks where a plane can be hit and survive, so armor belongs where the returning planes show no damage. ↩

2. James J. Heckman, "Sample Selection Bias as a Specification Error," Econometrica 47, no. 1 (1979): 153–161 — the foundational treatment of non-random selection into a sample as a bias to be modeled and corrected, not ignored; the work for which Heckman shared the 2000 Nobel Memorial Prize in Economics. ↩