Most Significant change - monitoring without indicators - feb 2005

Most Significant Change (MSC) is a participatory monitoring technique based on stories rather than indicators. MSC stories are about important or significant changes - they give a rich picture of the impact of development work and provide the basis for dialogue over key objectives and values of development programmes.

Rick Davies, one of the authors of MSC, explained how the emphasis on storytelling makes MSC different to formal monitoring techniques. Instead of introducing new professional skills, MSC takes advantage of everyday communication practices: “Every language has an expression for 'What’s new?'” said Rick.

MSC doesn’t replace other methods of monitoring and evaluation – in fact it works well in conjunction with methods like content analysis and quantitative analysis – but it comes into its own where outcomes are unexpected and meanings are disputed. Indicators just don’t help us to see what has changed in these situations.

Rick Davies initially developed MSC for a participatory rural development programme run by Christian Commission for Development Bangladesh (CCDB). Planning was devolved to regional committees but that also meant devolving decision-making about priorities and activities, making it more difficult to report on what was happening and why in a coherent way. MSC offered a way for staff at regional level to select, discuss and report the changes that were important to them.

MSC can be understood through two metaphors – of a newspaper, which picks out the most interesting or significant story from the wide range of events and background details that it could draw on. Alternatively Rick described it as an amoeba – exploring in many diverse directions simultaneously, but ultimately gradually ‘lumbering’ forward in one overall direction.

One way to explain MSC is through doing it, so discussion participants did a short exercise:

Stories ranged from an unexpected solution to a web design problem on the walk home from work to a rethink of planning a journey after being stuck on a bus for three hours during what was usually a half-hour journey.

The exercise demonstrated the great value and power in face-to-face communication – the storytelling and selection process in MSC enhances communication and encourages people to interact, rather than being a paper based exercise. It is a powerful way of communicating.

Rob Vincent from Exchange described using storytelling in a communication workshop that brought together project managers from 10 different countries: despite the different cultures and languages at the workshop, stories allowed people to share their experience in a a holistic and concise manner, in a way that was powerful and promoted dialogue.

See Introduction to Most Significant Change (PDF) and Rick Davies’ lunchtime discussion presentation (Powerpoint) for more details of the methodology.

One of the key stages of MSC is the selection of domains (areas of focus for monitoring). Rick Davies pointed out that it is always a strategic decision about what to measure and why – this is no different when using MSC. However the MSC domains offer a lot of flexibility: a domain for negative changes can be included to counteract bias towards positive stories; or an ‘open window’ domain – what else has changed? – can be included. The domains can even be left to emerge during the story selection process.

However, if extra domains are used, more stories will be generated and more time will be needed for discussion, negotiation and filtering.

Enough time for face-to-face discussion is essential to the MSC technique. Values become explicit through the conversations people have as they select which stories are most significant.

But there is flexibility in the story selection process: criteria for selection can be defined before the stories are gathered – meaning the history of a project or organisation will dominate; or the stories can be gathered first and then selected according to criteria that emerge during discussions.

The key point is that the selection process is documented – participants at all levels are expected to say why they chose one set of stories over another. Rick said that in practice, even in cases where there is obvious bias, diverse stories come through.

There are huge variations in practice of what sort of stories are encouraged and produced. It depends on the programme and who is included in the process – whether you work with programme staff or a group of beneficiaries like farmers for example. “Getting people to explain why something is significant can be a challenge in different cultures,” said Rick Davies.

You should work in the local language wherever possible, and then use translations when necessary. This might be more difficult in minority languages. Correct translation of the MSC concepts is also vital – for example, in a Latin American project, the word story has a connotation of fiction so an alternative was used.

All stories remain significant for their authors but some stories will have more significance at regional or national level – these are the ones that get filtered up through the levels of authority, up to the funders of a project or programme.

The stories can be verified through a visit or by talking to several participants to check that people do not invent fantasies or to see how people in the field influence outcomes by producing a particular type of story.

A meta level of content analysis can also be done – addressing questions such as what is the proportion of negative to positive stories.

Rick had a word of warning when using MSC to report to donors: provide contextual information along with each story – where it comes from, who gathered it, when – making it clear that a coherent valid method has been used to generate and select the stories. This will increase the chance of the stories being understood as relevant evidence.

Feedback is the weakest part in all monitoring and evaluation systems and MSC is no exception.

It is important to remember that the MSC technique emerged from research on organisational learning in non-governmental organisations (NGOs). As the stories filter up a hierarchy, the feedback from different levels of authority about how stories are selected informs participants about how their work is understood and valued.

It is also vital that people are explicit about their reasons for seeing a story as significant, since someone else may ‘read’ what they feel is an obvious reason into a story which was not the one intended. Culture also plays a key role in the interpretation of stories at different levels: “…if you are a cultural outsider, picking up on a negative message in a seemingly positive statement can be difficult.” said Rick.

The Guide to Most Significant Change is available free of charge on the MandE news website.

Rick Davies is an independent evaluation consultant. MandE news is one of Exchange’s partners. Exchange has supported the Guide to Most Significant Change.