M & E, theories of change and realistic evaluation 002 – Blamey and Mackenzie 2009
Two main approaches towards evaluation are discussed, mainly theories of change or programme theories of change and realistic evaluations.
The authors are from Scotland and they refer to work done in the UK.
It is a discussion of evaluations to improve planning and delivery of programmes to achieve behaviour change for the stakeholders and specific populations.
Theory of change relate to planning of programmes and follow different steps.
- Step 1 – determine long-term vision e.g. reduce smoking among participants/young women.
- Step 2 – think how these young women may behave in different settings.
- Steps 3 & 4 – stakeholders state different outputs or short-term outcomes of a programme (e.g. the women, the implementing organisation, the setting such as a school or club).
- Step 5 – determined the most effective activities to bring about change.
- Step 6 – stakeholders to explain reasons for the change in behaviours.
Realistic evaluation –steps for evaluator.
- Step 1 – discuss with programme implementers about nature of social programme and target group.
- Step 2 – tap into practitioners’ knowledge e.g. more or less motivated participants.
- Step 3 – build picture though information gathered (narratives and statistics)
- Step 4 – explain context, mechanism (factor that contribute to outcome) and outcome (CMO) – what works for whom in what circumstances.
- Programme theory – ‘the responses on the people to programme activities’ (little t theory) and when ‘descriptive’ (big T theory).
- Explanation theory – how an individual or group respond to a particular programme or intervention.
- For policy change – you need lots of information (big data) collected from different settings and different groups.
- For realistic evaluation – learn about programme delivery and most promising elements of a programme that contribute to behavioural change for strategic learning. It also identify what resources are needed for what activities that produce the desired outcomes.
- Realistic evaluations are significant for organizations to gain insights on their programme design, delivery model and programme effects on different groups or target populations.
- For policy change, you need realistic evaluation (micro or individual level) and theories of change – big size evaluations for individual, groups, organisation and community levels. This evaluation is time-consuming.
- Learning from different evaluations can assist in developing a monitoring and evaluation system that can produce meaningful strategic knowledge.
- How do you go about evaluating your programmes?
- What have you learnt from the monitoring and evaluation conducted by your own organisation?
- What have you learnt from evaluation conducted by an external agency?
- Thinking about one of your programmes or interventions, what mechanisms (elements) of the programmes work best to deliver on your aims or outcomes?