Unpicking system change

Transcript:

Hi folks, this is Rachel Shah from the Springfield Centre and this video is to talk about some recently published guidance on how to assess systemic change and whether and how well it fits with existing guidance on market systems development, which some of you might know better as ‘making markets work for the poor’ or M4P or MSD. I’m going to use the language of MSD in this video.

So, this paper was published recently. It’s called “The antidote to systemic change frameworks: six practical steps to assess systemic change and”, as a bonus, “improve your strategy”. It was written and published by Jake Lomax who’s an associate of the Springfield Centre and somebody I know well and work with frequently. It lays out six steps, divided into the system snapshot and the system dynamics, as you can see in the diagram, six steps to assess systemic change. It claims to offer systemic change programmes a straightforward way to assess systemic change using primarily internal resources.

So, if you came across this you might have thought, “that’s absolutely what we need. We are a systemic change programme, we want to achieve systemic change and therefore sustainability and scale of impacts for our target group but we’re struggling with a straightforward and yet credible way to assess it in our programme.” But if you read it, and as you dug into it, you might have found some unfamiliar language and wondered whether or not this is compatible with MSD.

Some of you who are MSD practitioners might have found yourself wondering as you read the antidote paper, “What about all the other frameworks we already have?” MSD is a well-established approach to development and there are a lot of frameworks laid out as part of the process and the antidote paper doesn’t really refer to them. So, it begs the question, “If I apply this guidance, will it clash with anything that I’m already using? Is it really for me if I’m in an MSD programme? How well do these two sets of guidance fit together?”

Well this video will answer those questions. My point of view is that they do complement each other very well and that, although the antidote paper uses unfamiliar language, it builds on the concepts in MSD, complements existing guidance and adds to our measurement toolbox, so I hope in the video to show you how you can make use of it in your programmes.

The antidote paper uses a slightly different language than that which those of us who work in MSD might be used to and familiar with. So, in MSD, systems are described in terms of a core supply and demand, supporting functions and rules. There’s a really useful and valuable imprecision built into this language and it’s possible to quickly and simply describe complex systems and to identify which parts of them are important for the programme and where the opportunities to intervene might be. Without this kind of useful imprecision, it would be really easy to end up with paralysis by analysis trying to map out every single aspect of the system and losing oneself, unable to really diagnose what’s going on, what’s critical and start designing interventions.

However, this useful imprecision can become an obstacle when it comes to measurement. As many of you hear measurement professionals out there will have seen for yourselves, when a concept is imprecise you sometimes get one of two measurement problems: either people don’t do the measurement – they just freeze because they can’t really figure out or remember what exactly it is that they’re meant to be assessing when they encounter that broad and slightly imprecise concept – or they do assessment but they miss a lot of really important information without realising it because they forget about all the other things are incorporated into the concept.

So, to take a classic example from an agricultural programme, if I ask, “Has the maize function changed?”, some people would just be stuck because they wouldn’t know how to answer that

question. They wouldn’t know “What information do I need to collect? What do I need to assess to answer that question – Has the maize function changed?”

Others might give a piece of information like, “Yes 35% of smallholder farmers now use hybrid seeds compared to just 10% when we started.” This is really important information of course, but there’s a lot of information about the maize function that’s still missing and sometimes to answer that one question, a lot of that other information might be sort of hidden behind the imprecision of the concept. So, for example, we might need to know about other inputs, if they’ve been prioritised fertilizers, say, or irrigation. We might need to know about what technique has being used for planting. We might need to know about business models, so if farmers have moved into contract farming for example, that’d be a different business model and really critical to understand as part of the maize function. We might need to know if there’s been a change to actors, perhaps gender roles have changed or perhaps new actors have entered the market and, of course, we definitely want to know how all of these changes have impacted on the output of that function on the maize itself. Has the quality changed? Has the yield changed?

So, you can see the idea of the function actually holds a lot of information in it and that’s really valuable. We don’t need to be digging down to every single aspect of what a function means in diagnosis but when it comes to measurement it can be useful to break it out. So to avoid these problems, the antidote paper, which is very much measurement focused, uses language that’s a bit more precise and concrete. It’s designed to support assessment so the language reflects the things that we need to measure empirically.

So instead of talking about functions, the antidote paper talks about:

• Actors
• Actions
• Resources
• Behaviour change

Actors are very simply the people in the system who supply or demand the function. Actions are what they do to supply and demand the functions – so all of the things that is sourcing inputs producing the maize and then selling it on, are all incorporated into it. Resources is of course the stuff that they use to do it, so the inputs that they use to produce the maize but also that the resources that the people who demand use as well. So, for example, finance perhaps to buy maize. And then really, really critically, resources also includes the things that the function actually supplies, so for the infrastructure function it’s going to be infrastructure, for the maize function it’s going to be maize, for the information function it’s going to be information. It’s what it’s actually providing to the market. And then behaviour change, really simply, is just to change any of those things which is obviously important for interventions to be able to have a kind of catch-all category for anything that changes.

So that’s the language that the antidote paper uses and, although it might be slightly unfamiliar and might seem even potentially opposed to MSD, it’s not at all. It actually fits very neatly with it. It’s just taking a more actor and action-oriented look at a system because of measurement, because of the importance of assessment and how it can be really useful to break down some of the concepts that we use in MSD to more precise kind of categories in order to help us with measurement.

Okay so let’s look at the six steps and how they fit in with existing MSD guidance. Step one in the antidote paper is ‘Define: what is in your system?’ and on the to do list that’s associated with the step, we’re asked to list the actions, actors and exchange-based connections in the system. In MSD, we start our programmes by looking for a system that has relevance feasibility and an opportunity for our identified target group. We then describe the system in which that opportunity lies and identify the important parts of that using a donut. So, core, supply and demand, supporting functions and rules, as we’ve just discussed. Programmes might also at this point use the value chain map to show connections, and then we diagnose using the diagnosis funnel, looking at revalidating which market systems are important to the poor, looking at how is the system not working for the target group, why is the system not working and then looking at root causes. So these diagnosis activities are part of any MSD programme and they inform what’s being asked for here in step one.

So once you’ve done diagnosis, you’ve actually got the information that you need to do step one:

• list actions – well you’ve already done diagnosis you’ve already listed the important functions and, as we’ve said, actions is just a more broken down way of looking at functions
• list actors – you will have already listed all the supply side and demand side players, segmented them in relevant ways for your programme objective, and looked at who actually performs the supporting functions and who sets the rules so you should already know actually who the actors are on both the supply and demand side for the core, for the supporting functions and for who sets the rules so that information is already there.
• list connections – well, there’s two ways that a programme’s probably done this once it’s done diagnosis. It’s probably done a value chain map so that maps the exchange-based connections in the system, and then it’s also probably looked at the relationship between supporting functions and the core, so how the functions relate to the core and then become their own systems that can be analysed with their own supporting functions and rules.

So, all the information from diagnosis gives you the information that’s needed to complete step one. So why has the antidote paper added this in as a step after diagnosis? Well step one is really just asking for a more detailed mapping of the system than we usually do in diagnosis, so it breaks a function down into its different parts and shows exactly how the different actors are connected. So, for example, to take the maize example again, maize would be broken down into sourcing all the necessary inputs, producing the maize and then finally, supplying the maize to buyers. That’s on the supply side, and then, of course, on the demand side, there’s the actions of actually accessing it and purchasing the maize. So really it’s just asking for a little bit more detail here, as we discussed when we talked about the differences in language, sort of like a value chain map but with more detail and including supporting functions. That can be a really useful thing to do as a foundation for later measurement because it means that you can easily point, once you’ve broken it down to that level, to exactly where something’s going wrong and who all the actors connected to that bit of the system that’s going wrong are. But even if you don’t do that, even if you just stick with your donut that you’ve done in diagnosis, you’ve got your functions, you’ve got your players on both sides and you’ve got your value chain map and you understand how the supporting systems are related to the main system, then really you’ve got all the information you need to actually do the rest of the steps,  the next five steps in this, and to assess systemic change, just using the information that you collected in diagnosis.

So, step one, done. Whether you just use your diagnosis information or whether you break it down to a greater level of detail as is being asked for here, you can tick it off and move on to step two.

Step two is ‘Describe: Who is doing what and how are they doing it?’

The point of this is to record how things are being done in the system. As the to do list for this step shows, this means describing how things are being done now, who (if anyone) is already doing things the way you want them done, and a bit of information about the players (like how many of them there are, what their market share is, or what their reach is, that kind of thing). So, again, this is a very familiar activity for MSD practitioners. As part of diagnosis, you do this anyway, and then this information is then cemented and any gaps are filled in before vision is formed in the ‘who does / who pays?’ exercise. In this exercise, for each function, the programme records who performs a function (who does it) and who resources it (who pays for it).

But simply writing who does / who pays and filling in this table doesn’t quite cover all the things that we’re being asked to do here in step 2 of the antidote paper. The reason for that, again, is that the antidote paper is very much assessment-oriented. It’s giving us guidance on how to assess systemic change. The ‘who does / who pays’ exercise is diagnosis-oriented in the sense that it’s really meant to help people get a quick snapshot of who the players are, what they’re doing, and to think really deeply about what their incentives and capacities are so that the programme can find out what the problems in the system are, look for the opportunity to intervene and shape their vision. That’s obviously really important in the diagnosis phase, but as MSD practitioners well know, the diagnosis is also the time when we collect the data that provides the baseline of what the system looks like now, so your diagnosis will become your baseline for system change assessment.

So what the antidote paper is trying to do here in step two is give us some guidance on what kind of information we need to collect at this stage to draw out from our diagnosis information so that we’ve got a really useful baseline, and a really rigorous and credible baseline, so that when we come to later do system change assessment, we’re able to do that on the basis of a really strong foundation. So, to do that a lot of programmes will use an expanded ‘who does / who pays’ table.

Here’s an example of an expanded ‘who does / who pays’ table from a real programme. It’s slightly crowded on the slide because it would normally be done in Excel, so it’s just showing one supporting function which is information about vacancies. So, this was a labour market, a jobs programme. As you can see, you’ve got the supporting function of information about vacancies and then the definition explaining that it’s supply of information about the jobs that are available to job seekers in the defined target group. And then you’ve got the players who do this function, who perform it, segmented in relevant ways. And then you’ve got a bit more information – so you can see it’s been expanded out here. Now this information is really useful for diagnosis but it’s absolutely critical for measurement. So, if you’re going to use your diagnosis information as a baseline, which we do in MSD, then you really want to be recording some of this empirical data about the different players.

So to take the top example, we’ve got public job boards, which are done by the government agency, we’ve got the number in the system (about 117) across the system boundaries as they have to find it, and it’s reaching about 65 percent of the job seekers. And then we’ve got a bit of information about how they do it – so this is to do with how they actually do that action, what is the behaviour like? So here we’ve got any employer can post a job and it’s a physical job board, so it’s driven by the employers. And then other notes – you might have other notes about the actors, so for example, in that very bottom example of employment agencies, it’s got these two employment agency names saying that they represent about 5% of the market share between them. So sometimes you might want to name specific actors, depending on your market. And then you go on to ‘who pays’ and you’ve got a little bit less information here but this is just showing who resources it. So again, for the public job boards, the government resources it (the specific body is named), you’ve got the number in the system (one per district) and which is, again, that’s one government body per district, and then how they do it and any other notes that are relevant to that.

So, for MSD practitioners, this is not asking a lot. Many of you might already be doing it. It’s basically just an expanded ‘who does / who pays’ table where all the information that you’re collecting in diagnosis is collated and if there are any gaps they’re filled in and then they’re there, ready for your systemic change assessment. So that’s step two done.

Step three is ‘Performance: how well are they doing it?’

The to do list is, again, written in language that might not be familiar but nonetheless this is something that MSD programmes already do. The idea of performance is certainly something that is a big part of our analysis in MSD. So we’ve already talked about ‘who does / who pays’ but there’s another column in the ‘who does / who pays’ exercise that is used for programmes to analyse the performance of functions. In diagnosis, we usually just look at each function to say, ‘is it inadequate, mismatched or absent?’ but here, because again, there’s an assessment focus, we’re being asked to go a little deeper. So this is information again that you’ll be collecting anyway as part of your diagnosis activities but we want to expand that column to go into recording all of the information that we’re later going to use as the baseline for our systemic change assessment.

This looks similar to your ‘who does / who pays’ table and in fact I’ve actually just cut out the ‘who does / who pays’ bits in the middle to be able to fit it onto the slide. So it just kind of carries on in that excel table and essentially you’ve still got your inadequate / mismatched / absent, but then you’ve got a bit more detail on the competitiveness and inclusivity, essentially the performance of the function for each of the different segments of players. So, to take the example at the top for public job boards, we’ve got some notes here. So the quality and timing of the supply of information is poor, adverts are mostly expired, the officials who manage job boards know nothing about the employers or the jobs so they aren’t able to kind of provide any further information to job seekers. So that’s the kind of notes, and then you’ve got empirical data, so this is where you’re going to start recording your indicators – some information that will become your indicators later for systemic change assessment and that here is going to be your baseline. So, for example, we’ve got here the percentage of the target group who access the public job boards and that’s quite high, the number of adverts, the percentage actually, of adverts that are live (as assessed at the baseline pictures – that’s only 40 percent, that’s pretty poor), the cost of access (it’s free). So here we’re looking at quality, quantity, price or timing of whatever it is that they’re producing, so here they’re producing information and supplying that to the market. What’s the quality of the information? How much information is there? What’s the cost of accessing that? What’s the timing? Is it relevant? Are they live? And then you’ve got a final column which is inclusivity. Now this is just to emphasise an inclusivity issue. For example, there might be something specific that you want to highlight there, like perhaps if it’s in a context where women aren’t as mobile, and they’re not able to get to the places where these public job boards are then you might want to put notes on that in the inclusivity column.

Okay, so now you’ve done steps one to three and you’ve got your baseline data. So, what the antidote paper reminds us of is that we need to regularly repeat these system snapshots. In other words, we need to repeat steps one to three at regular intervals. So how often should we do this? Well for most programmes, annually is about right. What this just means that we do is that we take a step back and look at how the system as a whole is changing. So you’ve got your ‘who does / who pays’ and performance data for your baseline picture – let’s say in January 2018 – and then in January 2019, you want to take a system snapshot again – who’s doing now, who’s paying now, what’s the performance now? – and then again in January 2020, you want to take another system snapshot – what does the system look like now, who are the actors, how are they doing things, how many of them are there, what’s their market share? All that information that’s in those detailed columns in the expanded ‘who does / who pays’ and performance tables will show you how things are changing, especially when you compare them from one to the other, how things are changing over time. Now this is a really, really valuable aspect of assessing system change.

Often in programmes we tend to just look at interventions and how an intervention is changing the system and how sustainable that is and how much it’s reached scale and that’s really important, but it’s also important to take a step back and just look at the system as a whole and say: “how were things working?” and then: “how are they working now?” And as you compare those system snapshots, you get a different angle, you get a helicopter view on the system as a whole and how things have changed. This is important because interventions might interact in different ways, they might work together and complement one another, and you don’t necessarily get that whole system view from an intervention lens. But when you take this helicopter view or, as the antidote paper calls, it a system snapshot, then you get that whole system view of how things are changing. A really important and valuable aspect of assessing systemic change.

Step four is: ‘Define: who needs to do what differently?’

This just means articulating what kind of behaviour changes you want to see for each player in the system and then being clear about how they’re connected, which most programmes do by putting them into a results chain that shows the links between them. It’s important in making the results chain to think about sustainability and what will need to be done on an ongoing basis in order to secure sustainability. There’s no disagreement at all here between the antidote guidance and existing MSD guidance. In MSD, we do this by moving from ‘who does / who pays’ to Vision – who will do, who will pay – based on our diagnosis information and our understanding of incentives and capacities. Once we’ve done that and got a vision for who will do and who will pay for the core function in each of our supporting functions and relevant rules, we then come up with a strategic framework that shows how all our interventions might work together to change the systems that we’re targeting as a whole. And then we have individual intervention results chains which show really specifically all the expected causal links between an intervention and eventually impact, laying out all the behaviour changes for different players in between. So this is all really standard MSD practice and, once you’ve done all of that, you’ve covered step four.

Step five is: ‘Describe: what is stopping them and how will this be overcome?’

This is basically an analysis of incentives and capacities. There are two parts to step five. The first part is the analysis of incentives and capacities that you do before the intervention, and the second part is a check and revision that you do while the intervention is ongoing. In MSD, incentives and capacities are embedded in every aspect of the process, from diagnosis through to vision and intervention design, in your results chain and in your choice of partners, your will/skill analysis. What the antidote paper encourages us to do in the to do list is a deeper dive into incentives and capacities, to be more specific in order to assess systemic change. So this can be really useful, particularly in the design and pilot of interventions.

The guidance it offers here is drawn from a paper that I co-authored with Jake (Jake, the author of this antidote paper) called ‘Unpacking incentives and capacities’. I’ll briefly explain it here and then if you want to apply this guidance you can check out the paper for more details. Unpacking incentives and capacities is all about looking in more detail at what the blockers and enablers of behaviour change are. So, let’s say, for example, that the intervention is to get more broadcast media which provides information to micro and small enterprises via radio shows. There may be social and political reasons to do that and there may also be financial reasons to do that so there may be incentives which broadcast media companies have to actually air these kind of radio shows. So why hasn’t this already emerged in the market? Why haven’t they done it so far? There’s obviously some kind of a blocker to this behaviour change. Well it may be that they don’t actually know that there’s a financial incentive because they don’t have the market research, they don’t know about the audience size that they could actually be tapping into if they were to air these kind of shows. But it may be more than that. Perhaps, even if they did know, they still wouldn’t do it because it’s such an unfamiliar thing, if they’re used to being geared towards urban audiences and this would be a more rural audience. And it’s also difficult to go out and do really interactive investigative journalism that provides relevant information to micro and small enterprises in rural areas so familiarity and difficulty might be affecting their incentives as well. And perhaps there’s a capacity issue as well – if  it’s unfamiliar it may also be that they don’t have the skills on their team for this kind of level of investigative journalism. So as you can see, what this does is it enables you to go beyond just saying that a given actor or a given set of actors has an incentive or doesn’t have capacity to actually saying very specifically “what are the enablers of their behaviour change?” and “what are the blockers?” This is essentially a way of kind of looking at, let’s say, a seesaw. You want that seesaw to tip towards the enablers but you recognise that there’s a real weight weighing down – although they do have a financial incentive, they do have a social and political incentive, it hasn’t been able to be so leveraged so far.

So, the other thing that we’re being asked to do in the antidote paper here is to ask, really explicitly, what is needed to overcome the blockers and, therefore, to leverage the enablers. So, for example, they don’t have information about having a financial incentive so perhaps what’s needed to overcome this blocker is market research. The programme might consider what kind of support would it take for the media companies to get the market research that would convince them that they have a financial incentive and that they would therefore start investing in making these radio shows. Technical assistance might be provided in order to overcome the familiarity, or rather the unfamiliarity blocker and the fact that it’s difficult and also this might help to coach and improve the skills of the investigative journalists so that they’re better able to boost their capacity and do this. And perhaps in another intervention in the future they’ll think about skills training for these journalists that would actually really embed that and address that capacity issue. So, these are all what the antidote paper calls ‘change resources’. It’s just the answer to the question “what is needed to overcome the blockers?” The antidote paper calls it change resources – it’s the  things that are needed in the system to be provided to the actor to overcome the blockers and leverage the enablers.

The second half of step five is to describe what actually happened post-intervention. So now we’re getting into real assessment stuff and, if we look at the to do list, this is really asking us to answer a series of questions. It’s basically: ‘who changed their behaviour, how and why?’ or, to put it in the language of the antidote paper, ‘was the thing that we thought was needed to overcome the blockers actually supplied and, basically, did it work?’ So, this is really an assessment of what actually happened. In MSD, this is familiar, of course, we would ask those questions as well – ‘who changed their behaviour, how and why? – normally in the measurement against the results chain. The way the antidote paper lays it out here is to give some more detailed ways to look at this by zooming in on the specific blockers and specific levers of behaviour change. So what the antidote paper is asking us to do here is to look at what actually happened by first looking at whether what we thought would be needed was actually supplied – did the radio company get market research, did they get technical assistance? Also, to look at did they change their behaviour – so did the seesaw tip in the other direction, did they actually air the radio shows? And then, why? – trying to really understand their actual incentives and their actual capacity and what drove that behaviour change and whether it really aligned with what you thought would happen.

Why is this a valuable way of doing it, this more detailed and structured way of looking at what actually happened? Well, of course, for intervention design, it can be really valuable to zoom in on exactly what’s working and what isn’t and so this can really help you with your strategy. But from an assessing systemic change point of view it helps a lot with thinking about attribution because, when you’ve got this level of detailed understanding of, not just that people changed their behaviour but why they changed their behaviour, and when you can say we thought that if this was supplied it would make a difference to behaviour change, if market research of technical assistance were supplied, it would make a difference to behaviour change and it did and we’ve got evidence that, not only they changed their behaviour but why they changed their behaviour, it can really help to build a strong contribution story for the programme’s contribution to systemic change.

So the sixth and final step is: ‘Endurance: will the changes last?’

Of course, for MSD practitioners, this is very familiar because sustainability is central to market systems development. In MSD, we tend to do this by looking at the behaviour changes that were introduced and then asking whether they will last. There are various frameworks and a very common and familiar one is ‘Adapt, Adopt, Expand, Respond’ – AAER. So what this looks at is will the behaviour change that was introduced, such as producing radio shows for micro and small enterprises, be adopted with programme support, be adapted independently by the media companies themselves, expand to reach scale in the system so there’s loads of different radio shows coming out and competing with each other and creating a thriving environment, and then will the system respond and actually embed this change in the system itself?

The antidote paper takes a slightly different approach, although is trying to achieve exactly the same thing. It’s a bit more of a detailed to-do list here but, to sum up, it’s asking: ‘will the change resources last?’ Now that’s unfamiliar language again but let me explain quickly how this is different because it’s not very different, even though it’s expressed in a bit of an unfamiliar way. When the antidote paper asks: ‘will the change resources last?’, it’s really asking: ‘will the thing that caused the behaviour changes last?’ So, if I map out the example that we’ve been talking about in a zoomed in level of detail, the behaviour change is: ‘radio programmes about rural enterprise are aired’. This is the market system change that the intervention is intending to facilitate across the system and so the Development Programme has done their analysis of incentives and capacities and come up with the enablers and blockers of that behaviour change and they’ve dug down to the root cause and said that, ‘okay the reason this hasn’t emerged in the market already is because of a lack of market research and a lack of technical assistance that’s being supplied to these media companies and that would actually overcome those blockers and leverage those incentives.’ So, we want to know whether this is going to be sustainable. Perhaps the programme has partnered with some media companies or with some radio shows and actually provided that market research and technical assistance to them as part of their support. But how do they know that the programme about rural enterprise is going to go on being aired, not just with their partners, but with other radio programmes in the system as well?

Well, in MSD, one of the ways of looking at sustainability would be to use the AAER framework, so you would actually look at the behaviour change itself – the programme about rural enterprise being aired – and say: ‘has it been adopted, has it been adapted?’ and then: ‘has it expanded beyond the pilot to other players in the system?’ and: ‘has the system responded?’ In the antidote paper, we’re being asked to do something a little bit different, which is to look at the thing that drove that behaviour change and see whether it will last. So the market research – is it going to be derailed in any way, is the behaviour change going to be derailed by something happening to the market research (for example, it becoming dated), or are the people who are really driving this change in the radio show departing to another role? So, this is just looking at the thing that drove the behaviour change and saying will that in any way become dated or depart or be depleted, destroyed, deteriorate, degrade or become deficient over time and, therefore, it will stop being put into that chain and, therefore, the behaviour change itself won’t last.

So, these are both just different ways of analysing sustainability. They’re both valid. They’re different ways of looking at the same thing and, if you want to get really technical about it, they’re actually related because, of course, if the media company adapts and starts investing in market research, then that will stop the market research from getting dated. Or if the system responds and consultants start offering market research as a service to media companies, then that too could stop the market research getting dated because it would always be offered up-to-date and fresh. One of the key things is, if you think that you’ve sort of provided a one-off support, to ask really whether that one off resource (market research or information or technical assistance) will derail the sustainability of the behaviour change over time. And another way that these are really related is that, if you want to prevent that from happening, then you’re going to want to look at the function in the system that provides that thing that drives behaviour change. So, in this case, the function in the system would be the provision of market research and technical assistance. So, for example, consultants provide technical assistance and market research to radio shows and media companies that then prevents the derailing and then, of course, that can be analysed using adapt, adopt, expand, respond. So, as you can see, these overlap quite a lot and they’re just different ways of looking at the same thing, which is sustainability – both quite valuable.

So, now we’ve done the six steps, let’s put it all together. There are four stages related to this process which fit very well into normal MSD processes in a programme. Firstly, you’ve got the three steps for the system snapshot – so building on diagnosis, take a snapshot really of what’s in your system, who’s doing what and how they’re doing it, and what is the performance of the different parts of the system. Then you’ve got your system strategy and intervention plans, so looking at what are the desired behaviour changes, what are the incentives and capacities, and what do we think will overcome any blockers. And then the real assessment starts. So, this is regular intervention assessments – who’s actually changed their behaviour, why and will it last? Sustainability is such an important part of any systemic change assessment. And then you go back to the first three steps and do another system snapshot. So you should take regular system snapshots, annually is good for many programmes, and ask those same questions again of what’s in your system, who’s doing what, how are they doing it, and what is the performance? And, of course, it wouldn’t be MSD if it wasn’t iterative so the first system snapshot provides the baseline for these regular system snapshots and these regular system snapshots, which tell you about how the system is changing, provide information for your system strategy and intervention planning.

So, there you have it. Putting it all together in the context of an MSD programme. So, in conclusion, I think this is a valuable addition to the measurement toolbox for MSD practitioners. It gives us useful guidance on how to more clearly and explicitly define and assess systemic change. It builds on, complements and enhances existing guidance. I don’t think it’s in conflict with it at all, but it does give us a different language to facilitate assessment and measurement of systemic change which is something that MSD programmes often stumble on so I think it’s really helpful for that.

What I think is one of the biggest things that it adds is really detailed guidance of what information we need, specifically, to be collecting for assessment purposes for systemic change. It actually lists out in quite a lot of detail what kind of information to collect and that can help us build those expanded ‘who does / who pays’ tables and expanded performance tables and put indicators against it and then use that for reporting purposes, as well as for shaping strategy so that’s really valuable. And it suggests some new tools for doing more granular analyses of incentives and capacities and of sustainability and, again, that can actually provide a bit of depth and a bit more specificity to measurement and assessment of systemic change.

So, is the antidote paper for MSE practitioners? I would say yes.

I hope this video has been helpful for you and that enables you to put this new guidance into practice in your programmes. I would love to hear from you if you’ve got any questions or thoughts or comments. I’m Rachel Shah (rshah@springfieldcentre.com) – feel free to email me and get in touch. Otherwise all the best for measuring systemic change.