The Agenda for Change Global Hub supports its members to deliver systems change and document and share their experiences in the water, sanitation, and hygiene (WASH) sectors. As part of that overall effort, the Hub contracted a team from the Springfield Centre and Aguaconsult to test an approach to assessing systems change by applying it to three WASH cases. This case tests the approach by applying it to WaterSHED’s work on improving sanitation service delivery to rural households in Cambodia.
Approach
Programmes that try to facilitate systems change do not intervene directly to improve service delivery levels – for example, by digging wells or building toilets. Instead, they focus on addressing the underlying issues that have prevented the system from working well. The idea is that if the performance of key system factors – things like finance, monitoring, coordination, and information – can be improved, it will lead to improvements in WASH service delivery levels.
There are two ways of improving the performance of key system factors. One is a programme doing something themselves to directly improve the performance of a factor, whilst the other is trying to get permanent public, private, and civil society system actors to change their behaviours to enable better factor performance. To achieve more sustainable change, most system change programmes take the latter approach. In effect, this leads to a chain reaction of performance and behaviour changes: a programme’s support leads permanent system actors to change their behaviour, which improves the performance of key system factors, which in turn triggers further behaviour changes, which improves service delivery.
The approach being tested addresses five key questions which together tell the story of how effective systems change programmes’ work has been. It captures the depth of key system changes by assessing how much performance has changed (both in key system factors, and at service delivery level). It captures the sustainability and scale of system changes by assessing the ownership, scale and resilience of the behaviour changes that drove performance changes. It assesses attribution by examining the relationships between changes, and by looking at what else might have caused changes that occurred.
Findings
WaterSHED’s focus was on improving two key system factors, namely targeting (a better and more affordable latrine package and purchasing process, geared to rural households’ needs and aspirations), and information (more persuasive and accessible information for rural consumers about the benefits of improved sanitation, and about where to purchase toilets and how to install them). WaterSHED’s theory was that the combined effect of better demand generation, through improved information, and better demand fulfilment, through improved targeting, would lead more rural consumers to purchase toilets, and hence improve sanitation coverage.
Initially, WaterSHED worked with private sanitation enterprises to improve both targeting and information. WaterSHED invested in early research and development in partnership with other agencies, including iDE, Lien Aid and World Toilet Organisation, to develop a more affordable ‘core’ latrine package, which WaterSHED shared with enterprises. They also supported enterprises with information about rural consumers and technical and business management skills through training, coaching, and technical advice, and engaged local leaders as sanitation sales agents, trained them, and linked these sales agents to enterprises.
As a result of this support, enterprises in rural areas began producing and selling all the component parts needed for a latrine as a single “latrine core” package that could be delivered to customers’ homes, and self-installed. Enterprises also contracted sales agents trained by WaterSHED on a commission basis, to conduct door-to-door direct marketing and run village-based sales events promoting their products. These behaviour changes in sanitation enterprises led to two key performance changes. Firstly, the new latrine package drastically increased the convenience and reduced the cost of purchasing and constructing a relatively ‘high-end’ toilet for rural households. Secondly, through village product displays and personal visits from sales agents, households could now easily access accurate and relevant information about toilets and purchasing options.
The ownership and scale of enterprises producing and selling the latrine core package – the behaviour change that improved targeting – were very good. Evidence for resilience was less rich, but it seems likely that enterprises themselves are more resilient than they were before WaterSHED’s support. The behaviour change that improved information – enterprises promoting and selling toilets through sales agents – also reached good scale. However, ownership and resilience were not strong. Enterprises struggled to recruit, train, and manage a rural salesforce, and relied heavily on WaterSHED staff to do their marketing.
WaterSHED recognised this threat to sustainability and realised that to mitigate it, they would need to take a different approach to improving the information available in the system. The team had observed that the districts with the best latrine sales had a commune or village leader who was motivated to promote improved sanitation. About three years after it first started piloting interventions, WaterSHED decided to pivot to a new focus on improving commune leadership development, hoping they could leverage this pattern to improve sanitation.
To do this, WaterSHED developed a leadership training programme called Civic Champions. The training aimed to provide commune leaders with the tools, strategies, and skills needed to promote local development in their communes. Unusually for public leadership development programmes in Cambodia, participants had to pay for a place, learned by doing, and were assessed by the outcomes they achieved. Change in sanitation uptake was used as the proxy measurement for gauging the change in participants’ leadership skills. WaterSHED ran the programme jointly with district government staff who they trained to act as facilitators and coaches. The availability of a better training programme and, to some extent, improved training skills among the facilitators, led government staff to gradually take on greater ownership of the Civic Champions programme with each iteration, eventually running an entirely government-led iteration seven years after WaterSHED’s initial pilot.
Numerous pieces of research found that the Civic Champions programme led to improved leadership skills among participants. For example, perseverance, levels of cooperation across party-lines, and confidence improved following participation in Civic Champions. The depth of change in leadership development is significant. There are also strong early signs of ownership and of scale albeit these are preliminary at this stage. The National Department of Training is currently leading the Civic Champions programme in seven of the eight provinces in the programme area, is testing online rollout of the training, and is planning, subject to the results of the current iteration, to integrate Civic Champions into the National School of Local Administration and roll it out nationwide. A potential threat to resilience is an identified skills gap in facilitators, though this is somewhat mitigated by the government contracting experienced WaterSHED core team members to support them with technical advice and ensuring quality.
Civic Champions’ successes in leadership development led, as it was hoped they would, to commune leaders providing improved sanitation information to rural households in their communes. This new approach to ensuring rural households had access to better information about toilets proved successful, relative to providing that information through sales agents. Information from public leaders is generally more trusted as it is seen as less biased than information from sales agents, and it proved to be more sustainable and scalable too. As of September 2020, over 90% of commune councillors in the programme area had participated in the Civic Champions programme, and although not all of them provide information about toilets to rural households, most likely do. This is remarkable scale. It is too early to assess ownership well, but Civic Champions demonstrated good ownership of sanitation promotion strategies during the training, and sales in Civic Champions’ communes remained elevated for at least a year after the training programme finished.1 The resilience of this behaviour change depends primarily on whether the national government runs the Civic Champions training programme sustainably.
In line with WaterSHED’s theory of change, Civic Champions’ provision of improved information (thanks to improved leadership development) and better product targeting by enterprises have jointly led to the desired behaviour change at the service delivery level: more rural households buy toilets. This has happened at scale – arguably WaterSHED’s work more than doubled the number of toilets that would have been sold without any intervention2– and signs of ownership are also strong. Resilience depends on the sustainability of changes in leadership development, information and targeting, among other factors. The key performance change at service delivery level is simply more toilets being sold (improved sanitation coverage) at a better price, albeit with variations. WaterSHED data shows that sanitation coverage increased from 29% to 77% from 2011 to 2020 in the programme area.3 Whilst other factors likely contributed to this remarkable increase, numerous independent evaluations concluded that the changes WaterSHED facilitated drove an accelerated rate of latrine uptake, over and above what would have happened without intervention.
WaterSHED’s role, then, was not to drive change but to address the underlying issues that had previously prevented other local actors from driving change, thereby facilitating them to do so. This meant WaterSHED focused its resources on identifying and supporting actors to change their behaviour, rather than on providing solutions directly.
WaterSHED has demonstrated that prioritising sustainability need not come at the cost of impact. The depth of performance changes achieved in leadership development, targeting and information, and ultimately in latrine affordability and sales, is remarkable. Just as importantly, sustainability and scale have also been achieved. Despite the potential risks to resilience, overall WaterSHED’s system strengthening efforts have clearly contributed to dramatically improved sanitation coverage in a way that is scaled and likely sustainable.
Reflections on the learning approach
This approach to understanding system strengthening enabled us to map the links clearly, systematically, and comprehensively between WaterSHED’s system strengthening efforts and improved service delivery outcomes. Doing so in a detailed way led to new insights on where WaterSHED’s efforts had contributed to deep, sustainable, and scaled changes in the system, as well as where vulnerabilities – or simply information gaps – are. However, this is not a ‘quick and easy’ approach; it requires a strong understanding of an organisation’s work and of key concepts in systems thinking. Using the approach to assess two other cases will provide a more rounded picture of how valuable it is likely to be as a monitoring and adaptive management tool.