A Developmental Evaluation of Collective Impact in Colorado, USA

In 2014, a Colorado Foundation announced a ten-year, $100 million investment in delivery system and payment reform through a statewide collective impact initiative focused on accelerating the Triple Aim of better health,... [ view full abstract ]

In 2014, a Colorado Foundation announced a ten-year, $100 million investment in delivery system and payment reform through a statewide collective impact initiative focused on accelerating the Triple Aim of better health, better care, and lower cost. Collective impact is described by John Kania and Mark Kramer as, “the commitment of a group of important actors from different sectors to a common agenda for solving a specific social problem.” Michael Quinn Patton suggests that developmental evaluation is particularly well suited to support the development of innovation and adaptation in dynamic environments. Evaluators were engaged in spring 2015 to conduct a developmental evaluation of the initiative.

This session is particularly timely and relevant to the field of evaluation. Changes inherent in the 2010 Affordable Care Act have ignited innovations in health care delivery and financing at the local, state, and national levels. Health system partners are seeking new ways to shift away from a system that rewards episodic care to one that rewards providers for whole person care focused on quality and value. While the federal government leads the way in policy and financing innovation, state and local partners also have opportunities to make contributions to complex system change. Private organizations, including foundations, are looking for ways to accelerate the change process and disseminate learning about the process. Collective impact is seen by some as a way to harness the power of multiple partners around a common agenda. Because desired health system change involves a wide variety of partners that may impact the health of individuals and plays out in a rapidly changing context, developmental evaluation may be particularly well suited to help understand the ongoing impact various partners may have on the system.

This case study provides an opportunity to examine how a foundation, acting as one partner in a complex system, attempted to influence a number of factors related to how other system actors act and react. The evaluation focuses on the initiative’s context, mechanism (both resources and reasoning) and outcomes. Through presentation of the case study, the session audience will understand what it is like to participate in a complex developmental evaluation in real time – the types of data collected and how collected, the creation of multiple feedback loops to inform the work of initiative partners, sense making conducted by the evaluation team and by initiative partners, and presentation of results that is generative to the process. Woven into the case presentation will be the evaluator’s thoughts on evaluation quality as it applies to this example.