Measuring the effectiveness of report cards in the age of information

Environmental report cards are commonly used worldwide as assessment and communications products that synthesize data from scientists and volunteers and convert it into an image-rich format. They can have an important role in... [ view full abstract ]

Environmental report cards are commonly used worldwide as assessment and communications products that synthesize data from scientists and volunteers and convert it into an image-rich format. They can have an important role in Monitoring, Evaluation and Reporting programs, particularly when accessible community reporting is a desired outcome.

A successful report card depends on many factors including having clear goals and ways to measure performance against those goals. These goals should not only include environmental outcomes that the report card shows but also refer to the effectiveness in communicating desired messages. With many report cards now available online, a range of novel options exist to measure how successful they are at achieving their goals. These include the use of web analytics data, online surveys, user experience testing and tracking application downloads. Despite these options being available, few programs take full use of the opportunity to clearly measure the effectiveness of their report cards.

The Port Phillip Bay and catchments report card program fell into this category. It had clear goals that included providing clear information on environmental water quality, accessibility to a wide audience, accepted methodology and transperancy to users. While the goals were clear, the measures of success for this report card tended to be ad-hoc and qualitative (like getting positive feedback from users). However, a combination of web analytics and user experience testing undertaken since May 2014 have helped drive improvements to this report card. Online surveys are now also being used. It’s hoped that by putting in place a range of ways to measure success, we can use the ensuing findings to make report cards more effective reporting tools.