Topic 20 o 64· Social value an impact meisurement

Ootcomes vs ootputs

An ootput is the direct, coontable product o an activity — it exists the meenit delivery happens, nae maitter whit effect it has. An ootcome is the chynge that follaes for the fowk, place, or system involved. "500 fowk attended a job-search workshop" is an ootput: it is true even if nane o them find wark. "500 fowk's employment prospects improved" is an ootcome claim, and it needs evidence o chynge, nae juist evidence o attendance — the confusion that produces mair misleadin grant reports nor near ony ither meisurement mistak in the sector.

Why it maitters

HM Treasury's Magenta Book and funders sic as the National Lottery Community Fund baith require ootcome reportin specifically because ootputs are whit programmes report by default: they are cheap tae coont, aye available, and aye look positive. An ootput coont can literally never gae doun as a result o the programme failin — mair sessions delivert is aye "mair", whereas an ootcome can reveal a programme is nae workin. The National Audit Office has repeatedly criticized government programmes for reportin activity levels as if they were evidence o success; a saftware system that only makes ootputs easy tae report reinforces this by default, because ootputs need nae follow-up data collection and ootcomes dae.

The maths

There is nae formula, but there is a reliable test tae classify a metric:

Output test:  is it countable at the point of delivery, true even if the recipient is unaffected?
Outcome test: does it require a before/after or with/without comparison to be meaningful?

If a number can be true with zero benefit to anyone, it is an output.

This sits inside the wider logic model chain and depends on the ootcome links defined in a theory o chynge; turnin an ootcome intae siller uises the methods in social return on investment.

Wrocht example

Local authority (employment support): ootput — 500 fowk attended job-search workshops. Ootcome — at 12-month follow-up, 140 o thae 500 (28%) are in sustained employment (6+ months). A comparison group wi similar characteristics but nae programme access has a 15% baseline employment rate ower the same period. Net ootcome uplift: 28% − 15% = 13 percentage pynts, sae an estimatit 500 × 0.13 = 65 additional fowk are in wark wha wadnae otherwise hae been — the attributable ootcome, distinct frae either the 500 attendance figure or the 140 raw employment coont.

Charity (literacy charity): ootput — 1,200 readin sessions delivert tae 300 bairns. Ootcome — average readin age improved by 8 month ower a 6-month period, agin an expectit 6-month natural progression baseline o 6 month. Net ootcome gain: 8 − 6 = 2 months' additional readin age improvement per bairn attributable tae the programme, nae the fu 8-month figure.

Event logs and transactional systems instrument ootputs near automatically — page views, sessions, tickets closed, appointments booked — because they are generated by the system daein its job. Ootcomes need a data model that captures the same individual at a later pynt in time agin a baseline or comparison, which haes tae be designed in deliberately: follow-up surveys, linked administrative records, or a comparison cohort. A reportin tool that only supports the former will quietly steer an organization towards ootput-only reportin nae maitter whit the funder asked for. See logic model for whaur ootcomes sit in the accoontability chain, cost-per-outcome for turnin this distinction intae a unit-cost metric, and public sector KPIs for the wider pattern o metric selection.

Pitfaws

  • Reportin ootputs as if they were ootcomes. "500 fowk attended" implies benefit wioot demonstratin it; label attendance as an ootput explicitly.
  • Nae baseline or comparison group. An ootcome figure wi nae coonterfactual — see coonterfactual analysis — cannae separate programme effect frae whit wad hae happened onywey.
  • Optimizin for the funded metric. Whan fundin is tied tae ootput volume, delivery teams rationally maximize attendance ower durable chynge, a Goodhart's law failure mode.
  • Ootcome washin. Relabelin an ootput metric wi ootcome-soundin language ("engagement ootcomes: 500 attendees") wioot ony follow-up meisurement ahint it.

Sources