Meta-evaluation of environmental education projects: rethinking the role and effectiveness of indicators

There is a relevant amount work on the assessment of governmental environmental education projects and programs. However, these tools do not usually include evaluation of the performance instrument itself. There is a lack of... [ view full abstract ]

There is a relevant amount work on the assessment of governmental environmental education projects and programs. However, these tools do not usually include evaluation of the performance instrument itself. There is a lack of research on approaches that support the evaluation of the evaluation, identifying the strengths and weaknesses of an evaluation, and drawing conclusions about its overall utility, accuracy, validity and feasibility. This main research aim is to present a meta-evaluation approach to analyze the efficiency and effectiveness of an assessment model of governmental environmental education programs. Meta-evaluation can serve a valuable function as a quality-control tool during the implementation of an assessment. A Brazilian case study in the energy sector is used to test the proposed approach and usefulness of the proposed methodology is demonstrated. The selected case to test the meta-evaluation approach is an assessment model of environmental education programs operating in oil and gas facilities. The assessment model is supported by monitoring indicators. The proposed meta-evaluation framework is based upon a participatory oriented approach and is structured through a checklist of criteria and meta-evaluation performance indicators, that analyses how appropriate the education assessment indicators are and allows an evaluation of overall assessment tool´s performance. The case study data were collected through a questionnaire survey and focus groups. Stakeholders evaluated model´s performance, covering several parameters: utility, accuracy, validity and feasibility. Stakeholders also conducted a detailed analysis on the relevancy and adequacy of the model indicators and assessment procedures, as well as its capacity to guide teams in the improvement of environmental educational programs and projects. The main results indicated that the analyzed assessment model was accepted by stakeholders as a relevant tool to foster improvements and innovations on education environmental projects, programs and public policies while many suggestions were given to the betterment of indicators and its descriptions. The meta-evaluation process also included governance procedures that pre-established how the collected suggestions about the analyzed assessment model could be judged and implemented. Lastly, a seminar was provided to share the meta-evaluation results to stakeholders in general, including the changes adopted in the assessment model for government environmental education programs. These results enlightened the question about the role and applicability of indicators considering the challenges of supporting government policy development and management strategies, and sustainability evaluation and reporting of strategic instruments (policies, plans and programs).

meta-evaluation, environmental education project, performance assessment, indicators