Robust Evidence In Complex Environments: What Does It Mean?

A considerable investment has been made for the use of experimental and quasi-experimental evaluation designs in development projects. However, the use of complex, multi-layered analyses that allow for in-depth interpretation... [ view full abstract ]

A considerable investment has been made for the use of experimental and quasi-experimental evaluation designs in development projects. However, the use of complex, multi-layered analyses that allow for in-depth interpretation of findings is less often highlighted and yet every bit as important, particularly in complex developing settings such as post-conflict societies. In 2010, CARE developed its Common Indicator Framework to track changes in four domains related to girls’ learning: Attainment, Quality, Equality and Empowerment. This paper will reflect on how CARE used the Common Indicator Framework as an evolving roadmap to design robust evaluation systems for projects, investing in multi-layered analyses to assess progress in learning. The evidence generated indicates the need for nuanced interpretation of findings supported by the triangulation of quantitative and qualitative data in order to identify hidden areas of exclusion with a direct impact on learning, such as attendance, gender norms and violence. Learning trends are tracked at multiple levels, identifying not only the acquisition of academic skills through standardized assessments but also the development of leadership skills, gender equitable attitudes, and the evolution of the environment within the broader community and school, focusing on the enablers for future usage of knowledge and skills. The importance of a multi-layered approach is highlighted in countries where hard data largely emerge from development projects, thus requiring moving from simple tracking of average progress in the acquisition of basic literacy and numeracy to a sophisticated analysis of the broader progress at the individual and contextual levels.