Topic 20 of 64· Social value and impact measurement
Outcomes and Outputs: Di Difference Between Wetin You Do and Wetin E Change
Output na di direct, countable product of activity — e dey exist di moment delivery happen, no matter wetin effect e get. Outcome na di change wey follow for di people, place, or system wey dey involved. "500 people attend job-search workshop" na output: e true even if none of dem find work. "500 people employment prospects improve" na outcome claim, and e need evidence of change, no be only evidence of attendance — dis confusion dey produce more misleading grant reports pass almost any other measurement mistake for di sector.
Wetin make am matter
HM Treasury Magenta Book and funders like National Lottery Community Fund both require outcome reporting specifically because outputs na wetin programmes dey report by default: dem cheap to count, dem dey always available, and dem always look positive. Output count no fit ever go down because programme dey fail — more sessions wey dem deliver na always "more", but outcome fit show say programme no dey work. National Audit Office don criticize government programmes many times for how dem dey report activity levels as if na evidence of success; software system wey only make outputs easy to report go reinforce dis one by default, because outputs no need follow-up data collection and outcomes need am.
Di maths
Formula no dey, but test wey reliable dey to classify metric:
Output test: you fit count am for di point of delivery, e true even if di recipient no affected?
Outcome test: e need before/after or with/without comparison before e go mean something?
If number fit true with zero benefit to anybody, na output.
Dis one sit inside di wider logic model chain and e depend on di outcome links wey theory of change define; to turn outcome into money, use di methods for social return on investment.
Example wey dem work out
Local authority (employment support): output — 500 people attend job-search workshops. Outcome — for 12-month follow-up, 140 of dose 500 (28%) dey sustained employment (6+ months). Comparison group wey get similar characteristics but no access to di programme get 15% baseline employment rate over di same period. Net outcome uplift: 28% − 15% = 13 percentage points, so estimated 500 × 0.13 = 65 additional people dey work wey for no dey work otherwise — na di attributable outcome, wey different from both di 500 attendance figure and di 140 raw employment count.
Charity (literacy charity): output — 1,200 reading sessions wey dem deliver to 300 children. Outcome — average reading age improve by 8 months over 6-month period, against expected natural progression baseline of 6 months for 6-month period. Net outcome gain: 8 − 6 = 2 months additional reading age improvement per child wey programme cause, no be di full 8-month figure.
How e connect to software engineering
Event logs and transactional systems dey instrument outputs almost automatically — page views, sessions, tickets wey dem close, appointments wey dem book — because di system wey dey do im work dey generate dem. Outcomes need data model wey capture di same individual for later point in time against baseline or comparison, wey dem must design in on purpose: follow-up surveys, linked administrative records, or comparison cohort. Reporting tool wey only support di first one go quietly push organization go output-only reporting no matter wetin di funder ask for. See logic model for where outcomes sit inside di accountability chain, cost-per-outcome for how to turn dis distinction into unit-cost metric, and public sector KPIs for di wider pattern of how to select metrics.
Wahala dem wey you suppose avoid
- To report outputs as if dem na outcomes. "500 people attend" imply benefit without to demonstrate am; label attendance as output clearly.
- No baseline or comparison group. Outcome figure wey no get counterfactual — see counterfactual analysis — no fit separate programme effect from wetin for happen anyway.
- To optimize for di funded metric. When funding tie to output volume, delivery teams go rationally maximize attendance instead of durable change, na Goodhart law failure mode.
- Outcome washing. To relabel output metric with outcome-sounding language ("engagement outcomes: 500 attendees") without any follow-up measurement behind am.
Sources
- HM Treasury, Magenta Book (2020). https://www.gov.uk/government/publications/the-magenta-book
- National Lottery Community Fund, outcomes reporting guidance. https://www.tnlcommunityfund.org.uk/
- National Audit Office, value-for-money report methodology. https://www.nao.org.uk/