Using Realist Evaluation in intervention development: Key learning from a Realist Process Evaluation of a complex intervention in a pilot trial (Engager)

Background Engager is a care-coordination intervention to improve mental health outcomes of men near to and after release from prison who have common mental health challenges. A Realist Review (reported 1st CARES conference)... [ view full abstract ]

Background
Engager is a care-coordination intervention to improve mental health outcomes of men near to and after release from prison who have common mental health challenges. A Realist Review (reported 1st CARES conference) produced programme theory (PT) about what might work for this population and in what way. This informed development of the Engager intervention and its delivery platform (practitioner manual, training, and supervision). A pilot trial put the intervention PT into practice.
Realist Evaluation was used alongside other methods as part of a broader project of intervention development and testing. The Realist Evaluation aimed to explore whether the delivery platform was working in the way theorised, who for, and in what way, to inform changes to the PT, the delivery platform, and the intervention for the full trial.

Method
Realist interviews were carried out with men in the pilot trial, practitioners delivering Engager (Engager practitioners), and practitioners from other services. Engager practitioners took detailed notes about their delivery of the Engager intervention (what they did, who for, and in what way). Sessions between Engager practitioners and men receiving the intervention were recorded.
A five-level Framework Analysis (FA, Ritchie & Spencer, 1994) was used to analyse the data.

Results and Discussion: Learning from using Realist Evaluation for the process evaluation of the Engager pilot trial
1. The original PT was detailed and complex. The Realist Evaluation focused our understanding of the PT on the core ways the intervention is expected to bring about the desired outcomes for this population. We refer to this as core PT. This core PT informs the design of the process evaluation, the intervention model, and the delivery platform for the main trial.
2. For pragmatic purposes, using Realist Evaluation in intervention development required the Realist Evaluation to feed-in in two ways:
I. Real-time during pilot. During the pilot trial, pieces of data were selected for follow up in real-time using judgement based on incomplete, emerging data from a range of sources. For example, data suggesting changes that would prevent failure or promote success of the pilot trial and ability to test the model, such as data related to disengagement of practitioners in delivering the model. This allowed emergent issues to be discussed by the wider team across two sites. In this way, PT could be quickly updated and important changes made to the intervention during the pilot, such as adding monthly supervision for the mentalisation-based approach via Skype.
II. Main analysis post-pilot. The Framework Analysis refined the PT, the intervention model and the delivery platform for the main trial. ‘Core’ PT emerged from this process, such as the impact of release day work with men in the intervention on engagement in the community.
3. The Realist Evaluation supported understanding of contextual mediators and moderators of mechanisms firing. The practitioner manual outlined when and for whom the practitioners should use different intervention resources.