The transferability of complex interventions: a case study of parenting programmes

Background Evaluation design generally focuses more on the strength of the causal relationship between intervention and outcomes (internal validity) than on the generalisability of these findings (external validity). ... [ view full abstract ]

Background
Evaluation design generally focuses more on the strength of the causal relationship between intervention and outcomes (internal validity) than on the generalisability of these findings (external validity). Consequently there are numerous examples of interventions demonstrated to be effective in one setting which fail in another. Despite considerable advances in understanding how interventions operate, there has been little work to analyse what shapes the transferability of interventions between settings, different target groups, or different problems.

Policy makers and practitioners frequently have to decide whether an intervention developed for a different setting, target group and/or problem would be appropriate for their needs, and research funders have to decide whether further evaluation is required for interventions being delivered in different circumstances. This is critical as research and public service interventions are globalised, especially in low-income countries (LICs) where there are scarce resources to confirm evidence from studies in high-income countries (HICs).

Aims
This paper will set out the rationale for identifying general principles for assessing the transferability of interventions and some of the challenges involved. We will explore which aspects of setting, target group or problem impact on effectiveness, and what makes interventions resilient to loss of effectiveness. We will illustrate this using a literature review of the transferability of parenting interventions from HICs to LICs.

Issues
A recent review (Gardner et al. 2015) found that four HIC parenting interventions were effective when transferred, even into culturally distant settings, but these only included two middle-ICs and no LICs. We will explore which kind of HIC parenting interventions seem effective in Africa, which contextual factors do or do not impede their effectiveness, and why.

Several conceptual issues will be addressed. Should ‘the intervention’ be defined in terms of form or function? Focusing on programme theory would suggest the latter, yet most programme designers prescribe content (e.g. Incredible Years, Triple P). Others define their programme by its goals and approach, intending to make it adaptable to local culture (e.g. International Child Development Programme).

What criteria should be used to define effectiveness, e.g. evaluation design, strength of outcomes, and /or size of effect? More fundamentally, concepts of effectiveness are socially constructed: should we restrict ourselves to the designers’ intended primary outcomes or include secondary, interim, or unanticipated but beneficial outcomes which might be more socio-culturally appropriate?

Analysing context generates most challenges. Realist evaluation has long acknowledged that some intervention factors can be defined as either mechanism or context, for instance the motivation of facilitators in parenting programmes. Even if this distinction is unproblematic, it is difficult to know which aspects of context are important in relation to a specific intervention and therefore need to be described.

Contextual factors need to be categorised to formulate general principles. The literature suggests three possible typologies: levels in the socio-ecological model; Pawson’s VICTORE checklist of dimensions of complexity; or elements of programme theory. We hope to identify which typology is most useful in exploring the transferability of parenting programmes.


Gardner,F. et al. 2015 Jou. Clinical Child Adolescent Psych. 1-14.