Getting assigned to the prime rib station at the Yellowstone National Park cafeteria was like winning the golden ticket. Every Sunday during the summer of 1998, I would cross my fingers that my name was scribbled next to ‘PR’ on the hand-written schedule. During my first shift ‘On the Rib,’ I was told that, as soon as all four of the lumps of meat were gone, I was done for the day. We were supposed to sell them in 200, 300, and 350-gram cuts, carefully measured to ensure that nobody got a gram more or less than they should. Always one for purposeful experimentation, I thought I might try to sell my four slabs of meat a bit faster and get back to exploring the park (which was my real reason for being there). So, I began eyeballing my cuts of beef instead of measuring them. Literal happy campers became even happier when they got 550 grams of mediocre American beef for the price of 350 grams. And I got off work two hours earlier than my colleagues at the pizza station when I’d sold all the prime rib for the day.
I’ve been reminded of this phenomenon by a few assignments this year – MSD programmes are not immune to the potential pitfalls of Goodhart’s Law.
- Chasing different targets: MSD practitioners often criticise direct delivery programmes for chasing targets at the beneficiary level, but one MSD programme was dangerously close to doing something similar at the market system level. Whilst it is useful to know how many partnership agreements you have – and it is important to signal to staff that one is not enough – elevating this metric to a numerical target creates perverse incentives. A reflection on what the purpose of those agreements were for and how they intended to trigger impact helped the team realise that they were chasing targets, but ones that weren’t delivering impact to their target group.
- Means versus ends: The DCED Standard for Results Measurement provides programmes and their funders with a tried-and-tested measurement standard. But in some cases, when achieving a high audit score becomes the end, rather than the means, of good measurement, what does it all add up to? In pursuing measurement purity there is a risk that programmes lose sight of what they set out to do in the first place.
- Frameworks versus frames of mind: Filled out forms are only as good as the thinking that has gone into them. As more organisations adopt and adapt MSD principles into their structures and operations, we are seeing a proliferation of individualised guidance – arguably a Good Thing. Yet, a colleague with more grey hair than myself observed that he worries when he sees an ‘empty library’ of formats waiting to be filled in. MSD is not a recipe – you can’t take a semi-baked donut, add a splash of WhoDoesWhoPays and an oversized cup of results chains to get impact at scale. Put another way, toolification cannot replace strategic and critical thinking and creative intervention.
So what? What can programmes – or cafeteria managers – do to combat the effects of Goodhart’s Law? Like most difficult things, there is no simple, tidy answer – no unifying Theory of Everything or the One Perfect Thing to Measure. But there are a few principles that may help mitigate the potential impact of Goodhart’s Law. First, awareness. Being aware of the signals that metrics can send is the first part of recognising a problem. We all respond to incentives – in this, MSD programmes and prime rib specialists are no different. Second, don’t put all your measurement eggs in one basket – reliance on a single metric, whether the number of MOUs signed or lumps of meat sold, is likely to over-incentivise that behaviour. Rather, a type of Balance Scorecard Approach, which combines a few key metrics, may help keep Goodhart and his law at bay. And lastly, keeping your eye on the prize – reminding yourselves of what it is you’re actually trying to achieve, whether that’s increasing incomes or getting rid of snakes – is a dull, but effective technique to overcome tendencies to chase targets, whatever they may be.