Research in daily practice (how we) just do it!

Aim: Two years after the implementation of a new system-based case management method, Youth Protection Amsterdam Area started an organization-wide implementation booster (270 staff working with 3200 families). At the same... [ view full abstract ]

Aim: Two years after the implementation of a new system-based case management method, Youth Protection Amsterdam Area started an organization-wide implementation booster (270 staff working with 3200 families). At the same time, an implementation and evaluation study was set up. The main aims of this - transdisciplinary - Learning Effectiveness Study are: 1) to get insight in program fidelity; 2) to study the effectiveness of the program; and 3) to combine research evidence and daily practice in order to enhance learning on caseworker level and organisational level; and 4) to improve the outcomes for families. 


In this presentation we share our experiences in developing and using program fidelity and meaningful outcome measures, co-created by professionals and implemented in clinical practice. The results are important for three reasons: 1) they provide insight in the ongoing learning process of professionals in child care and youth protection; 2) management and stakeholders have measures for quality assurance on an organizational level; and 3) the data are important input for a longitudinal effectiveness study.


Method: Measures for program fidelity and effectiveness were developed and designed together with professionals, and were validated through the change mapping method. Change mapping is a participatory method to plan, monitor and evaluate (part of) a program. It is specifically applicable to collaborate with users of an intervention, as it improves understanding and use of scientific knowledge in daily practice. By explicating relations between program elements and aimed impact, the process of change mapping becomes a learning activity for professionals. Three sessions were organized with implementation agents and 42 sessions with all users. 


Results: Alongside the program fidelity booster session, all professionals joined a change mapping session. Professionals became more aware of the underlying program theory and the use of scientific knowledge in daily practice. The sessions led to one integrated change map including variables on the level of professional’s behavior (concrete elements of the intervention) and variables on impact level. The variables are used to co-create fidelity and impact scales. These are implemented and used for reflection and adaptation of daily practice as well as for measuring impact. 


Preliminary conclusion/discussion: The Learning Effectiveness Study is a large-scale example of shared scales for clinical practice and research purposes. The change mapping sessions resulted in input for the measures of program fidelity and impact. With this input, both scales were adapted to fit the clinical practice and are now used on a daily basis. Professionals use these data for reflection on an individual and a team level, while researchers use them to examine overall implementation and quality assurance. Simultaneously, data regarding program fidelity and impact are gathered for the effectiveness study. Integrating research into daily practice is thought to be an important prerequisite for professionalisation of child and family care, but challenges abound. The social worlds of research and practice are separated in terms of language, time lines, aims and practices; administering questionnaires in clinical practice is experienced as burdening rather than supporting core tasks. This paper shows the potential of constructing scales that act as boundary objects, by being meaningful in both science and clinical practice.