Tools · People analytics
Kirkpatrick Evaluation
Prove training changed behavior and results — not just that people enjoyed it.
The method
Kirkpatrick four-level training evaluation (+ Phillips ROI extension)
A leadership program is up for renewal and the only evidence on the table is a 4.6-out-of-5 satisfaction average. Everyone in the room knows a happy-sheet score is not proof the program changed anything, and nobody has the measurement design to show whether it did. The L&D budget gets defended on anecdote — which works until the year it doesn't.
The Kirkpatrick model's four levels — Reaction, Learning, Behavior, Results — are less a framework than a standing accusation: most training evaluation stops at level one because levels three and four require designing measurement before the program runs, not surveying afterward. Laszlo Bock's account in Work Rules! is the practitioner's version of the claim — Google's people organization concluded that measuring training means measuring behavioral change, not satisfaction, and adopted Kirkpatrick's levels as the way to do it. The mechanics that matter are the chain of evidence: instruments, timing, and success bars at each level, specified in advance, so that a level-four claim can trace back through behavior change and learning gain instead of leaping from attendance to revenue.
Diez, Bussin, and Lee treat training in Fundamentals of HR Analytics as an investment with a computable return — their claim is that ROI calculations must carry the full costs and the long-term benefits, and that the Kirkpatrick structure is what keeps the benefit side honest. Cascio and Boudreau's chapter on the costs and benefits of HR development programs adds the decision-science posture: development spending is an investment under uncertainty, and the analysis should say so. The honest limit sits at attribution — results move for a hundred reasons, and isolating the program's share requires a comparison design or an explicit, defensible adjustment. An ROI figure without an isolation method is a marketing number wearing a finance costume.
In the books this is where you'd be told to go design the instruments; here the four-level design comes back with the chain of evidence drawn and the weak links named — and when you supply financials, ROI and benefit-cost ratios are computed deterministically in code, never estimated by the model.
The books behind this tool
- Fundamentals of HR Analytics — Fermin Diez, Mark Bussin & Venessa Lee
- Work Rules! — Laszlo Bock
- Investing in People: Financial Impact of Human Resource Initiatives — Wayne F. Cascio & John W. Boudreau
How it works
Grounded in the Kirkpatrick/Phillips corpus (people-analytics): designs measurement at all four levels (Reaction → Learning → Behavior → Results) with instruments, timing, and success bars, the chain-of-evidence between levels, an optional Phillips Level-5 ROI extension (per-stream monetization + isolation plans with credibility rules; ROI% and BCR computed deterministically in code when financials are supplied — never by the model), and honest caveats about attribution. Reuses the reliability stats engine for Level-2 assessment.
You bring
{ program, context?, include_roi?, financials? (program_cost · benefit_streams · isolation_adjustment), cluster? }
You get
{ program_summary, levels[1..4] (measures · instruments · timing · success_indicator), chain_of_evidence, level5? (benefit_streams · computed ROI%/BCR · honesty_notes), roi_level5?, caveats[], grounded_in, provenance }
Use it for
- →L&D budget defense: a program → a four-level evaluation that reaches behavior + results, not smile sheets
- →Program design review: surface where the chain-of-evidence is weakest before launch
- →ROI case: a Phillips Level-5 frame with an isolation method and cost/benefit sketch
See it work
example outputProgram: a two-day frontline-manager coaching skills workshop for a 600-person retail chain, meant to lift team engagement and cut first-year turnover.
Kirkpatrick Evaluation — Frontline-Manager Coaching Workshop
Program: A two-day coaching-skills workshop for ~120 store managers, targeting better team engagement and lower first-year turnover.
Level 1 — Reaction
Did they find it relevant and worth their time?
- Measures: perceived relevance, intent to apply, facilitator effectiveness.
- Instrument: end-of-session survey (relevance + intent-to-apply items, not just satisfaction).
- Timing: end of day 2.
- Success bar: ≥80% rate "will apply this on my team next week."
Level 2 — Learning
Did coaching skill actually increase?
- Measures: knowledge of the coaching model; skill demonstrated in a role-play.
- Instruments: pre/post knowledge check + observer-scored role-play rubric (inter-rater reliability checked via the reliability engine).
- Timing: pre-program baseline; post at end of day 2.
- Success bar: mean post-score ≥ baseline by a pre-set effect size, rater agreement α ≥ 0.7.
Level 3 — Behavior
Are they coaching differently on the floor?
- Measures: frequency/quality of coaching conversations; use of the model.
- Instruments: direct-report pulse items; manager self-log; spot observation.
- Timing: +30 / +60 / +90 days.
- Success bar: sustained lift in direct-report-reported coaching at +90 days.
Level 4 — Results
Did engagement and retention move?
- Measures: team engagement index; first-year turnover.
- Instruments: engagement survey + HRIS turnover data.
- Timing: +6 months, trended.
- Success bar: turnover in trained-manager stores below matched-control stores.
Chain of evidence
Reaction → Learning protects against "liked it but learned nothing"; Learning → Behavior is the weakest link (skill rarely survives the floor) so the +30/60/90 cadence is load-bearing; Behavior → Results is read against control stores.
Level 5 — ROI (Phillips)
Isolate via matched-control comparison; benefit = avoided replacement cost of retained first-year hires minus fully-loaded program cost.
Caveats
- Turnover is multiply-determined (pay, scheduling, labor market) — control stores are essential or attribution collapses.
- Self-logged coaching frequency is inflation-prone; weight the direct-report signal.
Grounded in: people-analytics cluster · Kirkpatrick four levels, Phillips Level-5 ROI, isolation-of-effects · sources: Kirkpatrick's Four Levels (Kirkpatrick & Kirkpatrick), Return on Investment in Training (Phillips).
Run it on your data
Call it on your own inputs — over the API, or hand it to your AI agent via MCP. Discovery is open; running it is metered.
Related questions we answer
- How do I measure employee performance at work?Separate the signal you want — the person's actual contribution — from the noise that dominates raw ratings, chiefly idiosyncratic rater variance, which often explains more of a score than the ratee does. Define the construct before the scale, use behaviorally anchored items, calibrate raters, and report reliability. Measure conditions and outputs, not personalities.
- How mature is our people analytics program?Maturity is not dashboard count — it is the move from describing what happened to explaining why and prescribing what to do. Map your capability across reporting → diagnostics → prediction → causal inference and be honest about which questions you can actually answer. Most 'programs' are reporting with a BI tool.
- What is the difference between people analytics and HR reporting?HR reporting describes what happened — headcount, turnover rate, time-to-fill. People analytics applies behavioral science and causal methods to explain why and prescribe action. It is a different epistemology, not a prettier chart: reporting counts, analytics infers.
- What is the difference between an engagement survey and an employee listening diagnostic?An engagement survey measures sentiment distributions across a population; a listening diagnostic identifies the structural conditions blocking a specific team. They answer different questions — 'how do people feel overall?' versus 'what is binding here, and what is the one action?' — and require different methodology and sampling.
- Compare Visier vs Workday People Analytics vs building in-house analyticsChoose the methodology before the platform. Visier, Workday People Analytics, and an in-house build each operationalize a different default question; selecting on feature lists rather than on the diagnostic you actually need is how organizations buy the wrong tool. Decide what you must measure and how, then pick what serves it.
- What are the alternatives to annual employee engagement surveys?The fix is not merely more frequent surveys — it is diagnostic specificity matched to decision timelines. Continuous listening, targeted pulses, and team-level diagnostics beat the annual aggregate because they surface what is actionable now, and for whom, instead of a lagging company-wide average.