By Alan Gibson & David Elliott
In the context of more UK resources being devoted to aid, there is a political requirement for better assessment of performance to produce more rigorous evidence of impact. One response to this will be the establishment of a new UK Centre of Excellence for Evaluation of International Development (the proposal is available for download reference below). Evidently, this Centre will, develop “cutting edge methods”, produce “high quality” evaluations, build “networks of expertise” and more besides.
The logic behind this may seem clear and defensible. However, we’d see a number of caveats that should be considered:
The ‘Right Evidence’ caveat
The faith DFID is placing in the academic, research community to produce the right evidence and answers is striking but a tad brave. William Easterly, Professor of Economics at New York University and definitely someone that DFID should be listening to is quite vocal on the limitations of ‘smart scientific solutions’ in development – which is after all the essence of the Centre’s mission in life [http://nyudri.org/events/past-events/annual-conference-2012-debates-in-development/]. Believing that this kind of Centre, with this kind of brief, is going to generate the type of evidence that is sought is optimistic (at best).
The ‘Horse and Cart’ caveat
Research and evaluation methods that emerge outside a sound knowledge of ‘good practice’ in development can produce answers that so completely ‘miss the point’ that they drive development practice backwards. For example, sustainability and scale-ability are key objectives for development – and systemic change is at the heart of both of these aims. If evaluation methods don’t attempt to understand and assess this clearly, but instead focus on final end-benefits (say at a household level), practitioners will be encouraged to simply ‘buy impact’ by delivering goods and services directly to people (not sustainable of course, but who cares?). Evaluation practice has to be shaped by development objectives – and not the other way around.
The ‘Practitioners, Academics and Elephants’ caveat
The above caveats highlight an ‘elephant in the room’ reality about evaluation in the UK, namely the damaging schism that exists between development academics and practitioners. The latter (contractors, commercial and other) are, by inference, seen here to be rather dumb ‘do-ers’, incapable of much serious thinking and slightly untrustworthy anyway; the former are the big thinkers, revealers of truth and insight. These are caricatures of course but they lie at the heart of the motivation for this Centre and, like any caricature, are wide of the mark. In our own field, private sector development, academic researchers from the UK have contributed virtually nothing to progress (intellectual or practical) in the last 20 years. The idea that such individuals and organisations – as they currently stand – will offer useful additional knowledge to our field through better impact assessments is close to absurd. Underlying this state of affairs is the incentive structure for academics (and I speak as an ex-academic): what incentive do they have to be relevant to development practice? Very little just now – and the parallel sets of arrangements for funding from DFID maintain this divide between academics and implementers.
I can only imagine the foreboding among development practitioners as they contemplate armies of academics crawling over their projects – especially the very high transaction costs (loss of management time and focus) in trying to deal with them – with the added suspicion that it will lead to little added value for anyone (indeed, the opposite may often apply).
None of the above may be especially palatable – but, unfortunately, it is true. If not addressed, I fear the Centre – and similar such initiatives – will provide the appearance of doing ‘something’ but in reality will be an expedient gesture generating little to contribute to the development cause.