In-Depth Organizational Evaluation of Child Welfare Agencies

Currently, services research has a body of research around organizational culture and how to effectively assess organizational status and change capacity. This session will present a mixed methods evaluation approach to... [ view full abstract ]

Currently, services research has a body of research around organizational culture and how to effectively assess organizational status and change capacity. This session will present a mixed methods evaluation approach to understanding and assessing sustainable change capacity in child welfare programs and agencies. The focus will be on evaluation methods that identify effective organizational change implementation strategies that improve outcomes for at-risk children and their families.

Systems theory and implementation science provides the theoretical framework for this evaluation approach. Child welfare services involve multiple, formal and informal, interconnected components – never just one organization. Systems change is a purposeful transformation of the ways in which organizations function and interact with one another on behalf of children and families. Implementation Science is the study of methods to promote the integration of research findings and evidence into policy and practice.

This presentation will introduce participants to strategies for conducting in-depth assessments of child welfare agencies using a participatory research approach. Data collection methods will be reviewed including assessment of organizational culture and climate through staff surveys; assessment of interagency collaboration levels using a standardized instrument, interviews and focus groups with agency leadership, case managers and supervisors; observations of case managers in court and during home visits; review of child and family outcome data; case record reviews, and focus groups with key external stakeholders including judges, Guardian ad Litems, and treatment providers. Data analysis methods will be shared including triangulation of the findings. Challenges and limitations of the evaluation will be presented including dealing with resistance by leadership and/or line staff, sharing “bad news”, and maintaining rapid feedback. The presentation will conclude with lessons learned including the importance of establishing boundaries around the scope of work, clarity about roles (evaluator vs consultant), dealing with unexpected findings, and the importance of individual and shared reflexivity.