The Application of the Realist Evaluation Method on Programme Impact Evaluations in the South African Public Sector

The purpose of the study is to explore whether the application of Realist Evaluation Method on impact evaluations could result in programme impact evaluations that are significant, have utility value and aid in policy making.... [ view full abstract ]

The purpose of the study is to explore whether the application of Realist Evaluation Method on impact evaluations could result in programme impact evaluations that are significant, have utility value and aid in policy making. This is the right question to investigate at this time because, The South African National Evaluation Policy Framework (NEPF)has been promulgated as a step in in the embedment and institutionalisation of a national system of evaluation. Such a system requires skillful implementation in order to lend credibility to evaluation findings.
However, the framework suggests a range of methodologies, research techniques and data sources that may be appropriate, notwithstanding that none of these varied methods have been subject to systematic rigour under practice within the South African government-wide program evaluation. Such a mixed pot of impact evaluation methodologies pose a risk of providing inconsistent results of evaluations, as well as inconclusive and less meaningful findings.

Therefore, this is an opportune time to ask whether a systematic application of Realist Evaluation Method on public sector impact evaluations result in evaluation findings that are significant, have efficacy and efficiency and positively impact policy making. The setting is also right for investigating this question because answers to questions such as “How have beneficiary lives changed as a result of a programme or policy?” remain inconclusive. The South African public sector under constrained economic environment is compelled to improve accountability and transparency in the use of limited public financial resources.

In order to bring new knowledge and understanding on impact evaluation of government programmes, this research endeavours to test the applicability of the Realist Evaluation Method to be utilised in impact evaluation in the South African public sector. The research will seek to test and prove scientifically whether the Realist Evaluation Method and its theory underpinnings can bring new knowledge and further insights and understanding on impact evaluation of government programmes in the South African context. Could the application of Realist Evaluation methodology on impact evaluations result in programme evaluations that are meaningful, have utility value and aid in policy making?

The specific objectives are: -
1. To get an in-depth understanding of the Realist Evaluation Method through a detailed review and analysis of the related literature.
2. Compare with international experiences on the use of Realist Evaluation methodical approach in conducting impact evaluations in the public sector.
3. To elicit from the commissioners of impact evaluations in the South African public sector their understanding and uses of the approach by establishing: -
•the methodologies and approaches used in past programme impact evaluations
•justifications given for the choice of methodology and approach in past and planned programme impact evaluations
•the utility value of the evaluation results in offering new insights in judging the merit, success and impact of an intervention
•the awareness of Realistic Evaluation methodology and how this approach differs from other known methodological approaches.
4. To establish the applicability of Realistic Evaluation Method as a methodological approach in conducting programme impact evaluations in the South African public sector.