Ongoing learning and reflections from undertaking a realist evaluation of Schwartz Rounds: an healthcare intervention to promote compassionate care

The Realist Evaluation approach provides a theory-based method for setting out and testing causal explanations of how and why healthcare interventions work. Although work is underway (i.e. RAMESES project II), there are... [ view full abstract ]

The Realist Evaluation approach provides a theory-based method for setting out and testing causal explanations of how and why healthcare interventions work. Although work is underway (i.e. RAMESES project II), there are currently few agreed guidelines and reporting standards on how to conduct and evaluate ‘good’ realist evaluations.
In this poster, we identify the data collection and analytical steps we took and reflect upon the challenges faced and lessons learnt from conducting an evaluation of Schwartz Rounds . Specifically we focus upon our data collection and analysis work and the challenges we encountered.

Despite extensive reading about realist evaluation at the start of the study, we learnt most by doing.

1. We began the data collection process by interviewing programme architects in the USA and then key stakeholders who adapted it for a UK context, with the aim of developing our initial programme theories.

2. In 9 case study sites, we then undertook an iterative process of data collection to build explanations, which included ethnographic observation of rounds (n=46), panellist preparation meetings (n=29) and steering groups (n=27) and interviews with facilitators; audience and steering group members and panellists (n=175) who told stories to trigger reflection in Schwartz Rounds audience members.

3. These interviews were multi-staged and served different purposes at different phases - theory gleaning; theory refining; theory testing and consolidating interviews (Manzano 2016). We also drew upon our own expertise gained through observing key Schwartz Rounds stages to help us develop, test and refine our candidate programme theories (Context (C)-Mechanism (M)-Outcome (O) configurations.

4. We formally first analysed our interview data with our Rounds expert key informants for full and half nuggets of illustrative text and developed more than 30 CMOc.

5. We revisited key respondents and undertook repeat interviews with the same participants to refine, test and consolidate our CMOcs .

6. Further analysis cycles used an iterative analytical process to refine, combine and generate new CMOcs through inductive analysis of our interviews

Issues and practical challenges of applying realist evaluation methodology included:

(i) Defining and searching for nuggets of illustrative text and deciding which CMOc they best illustrated / fitted,
(ii) Using both observation field notes and interview data to test and refine our CMOs
(iii) Identifying core CMOc in such a multifaceted and complex intervention; what is core and how determined?
(iv) Appreciating that CMOc occur at different levels of analysis, are often connected and require linkage to account for these different levels and to identify full ripple effects (Jagosh et al 2015).

References:
Jagosh J. et al (2015) A realist evaluation of community-based participatory research: partnership, synergy, trust building and related ripple effects. BMC Public Health 15: 725.
Manzano A. (2016) The Craft of Interviewing in Realist Evaluation. Evaluation 1-19.

Funder is National Institute for Health Research (project number 13/07/49 ). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the [Health Services and Delivery Research Programme], NIHR, NHS or the Department of Health.