Why 'reinvent the wheel'? Using an a priori implementation theory to think about mechanisms in realist evaluation

As the ‘M’ in Context + Mechanism = Outcome configurations, mechanisms are central to realist evaluation and it is therefore critical to have clarity (and perhaps more uniformity) around how to think about them. This... [ view full abstract ]

As the ‘M’ in Context + Mechanism = Outcome configurations, mechanisms are central to realist evaluation and it is therefore critical to have clarity (and perhaps more uniformity) around how to think about them. This presentation will illustrate this potential union of realist evaluation and Contextual Interaction Theory to clarify thinking around mechanisms by drawing upon a realist evaluation currently underway in British Columbia, Canada exploring the implementation of provincially mandated nutritional criteria for foods and beverages sold in public schools.

Mechanisms, in realist evaluation, are considered to be the “engines of change”; those processes that occur between the roll-out of program resources and individuals’ decisions to change their behaviours . These processes of behaviour change have been said to be: (1) mostly invisible, (2) are influenced by context, and (3) produce outcomes (ie. behavioural change). These well-acknowledged characteristics of mechanisms are reflective of the complexity thinking underpinning a realist approach to evaluation. Some realist evaluations have tended towards identifying highly context-specific and varied mechanisms, leaving little room for developing ‘middle-range’ mechanisms that could, in practice, help alleviate some of the complexity around the concept of a mechanism. Ray Pawson, in his 2013 book “The Science of Evaluation: a Realist Manifesto”, proposed an ex nihilo (but evidence-based), generaliseable conceptual platform of program mechanisms that can be used as a frame for thinking about mechanisms across programs and sectors.

Another potential approach to thinking about mechanisms in a more broad way is to use an existing implementation theory. Realist evaluation literature has largely neglected this established area of study in the field of evaluation research originating from the 1970s. One major critique of implementation theories, however, has been their lack of parsimony. Implementation theories have often emerged from case studies of program or policy implementation, consisting of lengthy, highly context-specific and varied lists of factors influencing implementation.

One parsimonious implementation theory introduced by a cadre of Dutch and American researchers in the late 1980s, Contextual Interaction Theory (CIT), is a heuristic tool that takes into consideration these ‘lengthy lists’ of implementation factors. It has been applied in various fields from environmental to public health policy and programming. CIT posits that implementation occurs through individual actors who each have unique motivations, cognitive frames of references, and resources. Quality of implementation of a program is influenced by the context only in as much as the context influences individual actors’ own motivation, cognitive frames of references, and resources.

If the realist approach views mechanisms as “the process of how subjects interpret and act upon the intervention stratagem” after being provided programmatic resources, then there is potential to apply CIT from the realm of implementation research to use as an a priori guiding framework to help clearly identify program mechanisms.