The Agenda for Change Global Hub supports its members to deliver systems change and document and share their experiences in the water, sanitation, and hygiene (WASH) sectors. As part of this overall effort, the Hub contracted a team from the Springfield Centre and Aguaconsult to test an approach to assessing systems change by applying it to three WASH cases. This case tests the approach by applying it to the Rural Access to New Opportunities in Water, Sanitation and Hygiene (RANO WASH) project in Madagascar, which is being implemented by three Agenda for Change members – CARE, CRS, and WaterAid – and two private sector members, Sandandrano and BushProof.
Approach
Programs that try to facilitate systems change do not solely intervene directly to improve service delivery – for example, by digging wells or building toilets. Instead, they focus on addressing the underlying factors and system actor behaviours that have prevented the system from working well. The idea is that if the performance of key system factors – things like finance, monitoring, coordination, and information – can be improved, it will lead to improvements in WASH service delivery levels.
There are two ways of improving the performance of key system factors. One is a program doing something themselves to directly improve the performance of a factor, whilst the other is trying to get permanent public, private, and civil society system actors to change their behaviours to enable better factor performance. To achieve more sustainable change, most system change programs take the latter approach. In effect, this leads to a chain reaction of performance and behaviour changes: a program’s support leads permanent system actors to change their behaviour, which improves the performance of key system factors, which in turn triggers further behaviour changes, which improves service delivery.
Box 1: Definitions of behavior and performance change |
The approach being tested analyses the behaviour of actors and the subsequent performance of factors defined in Box 1.
It addresses five key questions which together tell the story of how effective systems change programs’ work has been. It captures the depth of key system changes by assessing how much performance has changed (both in key system factors, and at service delivery level). It captures the sustainability and scale of system changes by assessing the ownership, scale and resilience of the behaviour changes that drove performance changes. Finally, it assesses attribution by examining the relationships between changes, and by looking at what else might have caused the changes that occurred.
Findings
RANO WASH is a $33 million USAID-funded project implemented by Agenda for Change members CARE, CRS, and WaterAid and two Malagasy private companies, BushProof and Sandandrano. The six- year project is organized according to three strategic objectives: 1) improved governance and monitoring at national and sub-national levels; 2) increased access to water and sanitation through public-private partnerships, and 3) increased practice of good hygiene behaviours and use of WASH services.
This case study focuses on assessing the system changes achieved based on a subset of factors in the strategic objective area for increasing access to water, namely i) management models for service delivery; ii) information and skills to procure and manage a delegated service provider, and iii) finance for new water supply infrastructure development. These factors have been prioritized because an initial review of RANO WASH’s work suggested that much of the system changes that have already occurred and can be logically mapped against a theory of change to contributing to improved service delivery levels, have resulted from these areas. Moreover, it uses the Atsinanana region as illustrative because of the relatively high performing service delivery outcomes after several years of implementation compared to other regions in which RANO WASH operates.
According to baseline data, 13.5% of people had access to basic drinking water sources in Atsinanana, with just under 1% having access to safely managed services. 28% of infrastructure was not functioning and payment for water services was practically non-existent. The 1999 Water Code and ensuing decrees formally allowed for two types of management models for rural service delivery – community-based management and delegated private sector management. In Atsinanana, only 26% of water committees were considered functional, and there was no systematic support for community managed water supply schemes in Madagascar. One strategy to address this has been to encourage private sector participation in managing infrastructure, and thus, various development partners over the last 20 years have promoted domestic private sector operators. As of 2018, the Ministry of Water and Sanitation reported that 46 of 1,695 water schemes nationwide, or approximately 3%, are operated by private service providers. In the Atsinanana region, prior investment from RANO WASH consortium partners contributed to 10% (9 of 90) of communes having a private manager. The ability of local government to procure and manage a delegated contract was very low and only one example of a private operator investing equity in infrastructure was found.
To increase the number of households using basic and safely managed water services, RANO WASH carried out a range of activities with public officials at national and sub-national levels, potential private operators, and future consumers to increase the number and diversity of service delivery management models for rural water services, provide information and training on how to manage such contracts, and advocate for increased private investment through a ‘co-invest, build, and operate’ model. All these activities led to three key behaviour changes in public agencies and private companies.
Firstly, communes, or local governments, tendered for private operators to implement the new management model of ‘co-invest, build, and operate’. Selected private operators then partially invested in the new infrastructure they would subsequently manage, and, with support from RANO WASH, marketed new connections. The scale of these behaviour changes was found to have reached the intended number of communes with viable areas for private water operators in Atsinanana. Signs of ownership and resilience of the three behaviour changes, however, were mixed.
Behaviours of key actors and the subsequent performance improvements in the factors assessed – management models, skills, and finance – have all contributed to behaviour changes at the service delivery level. Simply put, more private operators supply water services, and more customers pay for and use basic or improved water services for all their water needs. From a baseline of nine communes with private water operators, an additional eight have entered ‘co-invest, build, operate’ contracts. This has led to over 66,000 people accessing basic or safely managed services, and nearly four hundred outstanding connection requests. The behaviour change of private water operators was assessed to have reached relatively good scale, whereas their technical and financial capacities to continue to supply services without further support from RANO WASH was less certain. Challenges also remain regarding the key consumer behaviour change of paying to connect and use piped water, both in terms of reaching a greater number of users and sustaining the promotional processes and financing that enabled those who have connected.
We agree with the midterm review team who stated that “there is no doubt that RANO WASH’s efforts have led to an increase in access to water supply and sanitation services, particularly with regard to water supply”, particularly through “popularizing and operationalizing the PPP approach.” However, this study indicates that RANO WASH’s influence goes beyond co-financing for infrastructure. By reconstructing a theory of change and examining the evidence available, our findings suggest the project has positively contributed to some changes in the eco-system of factors necessary to improve service delivery arrangements, information, and finance, that then led to increased access for tens of thousands of Malagasy people. Reviews and the team itself recognize that RANO WASH capitalized on the prior work of other development partners, such as the World Bank, GRET, UNICEF, and the EU, to value continuity while also refining the contracting and procurement processes. In addition, RANO WASH’s predecessor project, RANO HP, was the first USAID investment that explored the concept of privately managed water supply schemes, albeit at a much smaller geographic scale and with less emphasis on systems strengthening. Lastly, it appears that several of the businesses benefiting from RANO WASH’s interventions leveraged skills and experience gained from earlier development projects.
A few aspects of the project approach are important to highlight as key contributors to the systems changes achieved. RANO WASH has demonstrated that working simultaneously at national and sub- national levels is feasible and delivers results, particularly in the context of a service delivery option that requires not just approvals at multiple administrative levels, but the skills and information to manage a more complex service delivery arrangement. Working at multiple levels of the water ‘system’ is a key tenet of systems strengthening as it directly addresses the inter-related nature of factors and actor behaviors that are required to be in place, not only at the level at which services are delivered (i.e., the water supply scheme run by the new private operator).
Another dimension is the time required to achieve both changes in the system factors targeted by RANO WASH and the subsequent service delivery improvements. Although RANO WASH has a six-year duration, it builds on prior work by both individual organizations on some of the key factors, even if they were not articulated as such, and past USAID investments in supporting private sector participation in managing water services. Given the extremely low levels of service delivery and challenging context in Madagascar, more time, rather than less, will be required to sustain and expand those changes. Timelines of a decade or more are in line with international good practice of how systems change evolves over time.
The consortium structure leverages the experience of three different Agenda for Change members, as well as two private businesses, offering an example of collective action by design. Beyond consortium collaboration, RANO WASH’s approach, as mentioned above, of working simultaneously at multiple government levels has been a key success factor in this case. The limited extent of decentralization in Madagascar and the legal implications of a delegated management contract itself, requires RANO WASH to work at national, sub-national, and local government levels. Given the involvement of all three parties in the tendering, approval, and management of such contracts, coordination and collaboration is not a nice to have, but a fundamental factor that is essential to success in this case. Although commune level universal coverage was not an explicit goal of the project, RANO WASH still has devoted considerable effort at addressing overall water governance at multiple levels and trying to strengthen the relationships between different agencies at commune, region, and national level. This goes beyond supporting communes to tender and supervise private operators and seeks to equip local elected officials and technical staff with the skills and information they need to be able to budget, plan and monitor all water and sanitation-related activities in their jurisdictions.
In summary, RANO WASH’s work to facilitate privately managed water services in the Atsinanana region has demonstrated that working to change behaviours of local actors to improve key factors in the WASH system contributed to improved service delivery for 66,000 people to date. The changes in management models for service delivery, skills, and information on managing a delegated contract, and finance for new infrastructure development facilitated by RANO WASH, have a clear and evidenced link to improved water services for previously unserved households.
1 These were also prioritized using the external midterm review’s assessment of the priority areas in which the reviewers felt that RANO WASH had contributed to more substantially than other parts of its portfolio.
2 See JMP data for service ladder information: Service ladders | JMP (washdata.org)
3 Carter, Richard and Ryan, Peter, 2016. ‘Financing requirements and management approaches for sustainable rural water services for all in Madagascar.’ UNICEF. Available at: F & M Madagascar RWS draft final (unicef.org). Accessed 4 March 2022.
4 SIMS / MSIS, 2018. ‘RANO WASH: Baseline study’. Unpublished.
5 Ibid
6 CARE International Madagascar, 2021. ‘RANO WASH: Midterm review report’. Available at: RANO WASH Rural Access to New Opportunities in Water, Sanitation, And Hygiene Mid-Term Review Report – CARE | Evaluations (careevaluations.org)
7 TetraTech, 2021. ‘Midterm performance evaluation of Madagascar Rural Access to New Opportunities in Water, Sanitation, and Hygiene Activities.’ Available at: Mid-term performance evaluation of Madagascar rural access to new opportunities in water, sanitation, and hygiene (RANO WASH) activity, October 2021 – Madagascar | ReliefWeb
8 https://www.ircwash.org/resources/all-systems-go-background-note-wash-systems-symposium