Lightening rods and pitfalls: lessons arising from the development of a realistic evaluation of Well Doncaster (a Well North pilot site)

Background Well North is a collaborative programme which is developing community-led interventions to improve the health of vulnerable people living in some of the most deprived areas of the North of England. These... [ view full abstract ]

Background

Well North is a collaborative programme which is developing community-led interventions to improve the health of vulnerable people living in some of the most deprived areas of the North of England. These interventions are intended to help people solve, rather than manage their health, social and economic problems. Denaby Main, a former mining community near Doncaster and identified as having high levels of need, was chosen as a Well North pathfinder site. An appreciative inquiry process revealed that residents and local staff felt Denaby had missed out on resources that could have supported the community. In realist terms a possible explanatory cause for poorer outcomes.

Developing the evaluation approach:

Through the appreciative inquiry a number of potential theories were identified and the evaluation team recognised that using Pawson and Tilley’s realistic evaluation would help us both understand what works, for whom in what circumstances. Moreover, insights arising from the evaluation would help us adapt our programme of activities. The community identified the need for a space to meet, and therefore a Bumping Space project was commissioned, to provide an opportunity to bring people together and engage them in community activity. This provided an ideal opportunity for us to operationalise realist methodology in a small-scale study. Given the philosophy of Well North we were keen to ensure stakeholder involvement in the evaluation and also began work to explore how realistic evaluation could combine with return on investment approaches.

Findings and discussion

In this paper, we will describe our approach to date and argue that the articulation of theory and its refinement (for our pilot evaluation of the Bumping Spaces project) has helped direct the project and strengthen programme management. We will present evidence from participants and project workers which allows us to better understand whether and how it works, who benefits the most, and in what circumstances. We will describe how we have sought to ensure stakeholder involvement by operationalising realistic precepts in a meaningful, practical and understandable way.

Provisional results have demonstrated considerable value in the realist evaluation approach. At the same time, we have identified challenges for extending the evaluation which merit discussion within the realist community. These challenges are: evaluation at different levels of programme operation (strategic, operational and project levels) with limited evaluation resource; communicating both the evaluation approach and its finding to range of stakeholders i.e. local politicians, council officer, community members and project workers and our work to combine a realistic evaluation approach with measurement of return on investment.