Using principles-focused evaluation with a complex, multi-grantee initiative

PDA used a principles-focused evaluation framework to guide our evaluation of a statewide initiative to prevent chronic disease.

Background

Evaluators have a growing number of approaches and methods available to meet the needs of increasingly complex initiatives. In evaluating a chronic disease prevention initiative that included state-level and local work across nearly two dozen communities, we partnered closely with our clients to create an evaluation plan, theory of change, and logic models. There was a LOT to evaluate! Despite the usefulness of some of these traditional approaches, we realized the complex nature of the program required another approach. This is when we introduced and dove into principles-focused evaluation in partnership with our client.

Approach

In a principles-focused evaluation framework, stakeholders develop a guiding framework embodied in principles that cross cut individual program components. The principles are evaluated, rather than the individual program components. This fit the situation because the primary evaluation challenge was to meaningfully define and measure how nearly two dozen grantees were working in different ways toward shared goals, and because the deliverables were flexible. A principles-focused evaluation also fit well because discussions with the program staff to co-construct the health equity statement laid the groundwork to develop their principles. To determine the principles, we began by advising our client that principles are guiding, not directive and that we needed language that was good enough to put into practice, but not necessarily perfect. We engaged in multiple rounds of review with the principles and made sure there was a shared understanding of the intent of each principle.

Chart of principles developed during Principles-Focused Evaluation

Our evaluation team then explored these guiding evaluation questions:

  1. To what extent are the identified principles meaningful to those meant to follow the principles?
  2. To what extent are the principles adhered to in practice?
  3. If adhered to, to what extent do the principles guide followers to the results they hope to achieve?

Results

Moving to a principles-focused evaluation allowed our client to see how the evaluation and program informed each other and how they can use the evaluation to make decisions moving forward. They brainstormed ways to put the principles into practice, including:

  1. Adding principles to the program website
  2. Integrating principles into documents
  3. Highlighting principles in webinars and trainings
  4. Using principles as a guide for RFPs, contracts, and meetings

Because the principles-focused evaluation resonated with the client, there were opportunities for the evaluation to have broader use and involved more stakeholders. We were invited to share evaluation results with our client’s grantees, rework the evaluation of the coalitions, and host a full day in-person session with grantees.  The health equity statement, which is reflective of the program principles, is now displayed on all of the program’s documents, alongside the program mission, vision, and logo.

Related Highlights

Using responsive evaluation to provide a coalition with actionable insights

PDA's responsive evaluation and data visualization techniques provided useful reports to Ohio’s Comprehensive Cancer Control Program.

Read More

Facilitating learning for a health equity-focused community grant program

PDA's evaluation helped the Hawai'i Community Foundation share successes and opportunities in serving priority populations.

Read More

Illuminating Key Facilitators and Challenges in Health Systems Change

PDA's evaluation helped health systems describe facilitators and challenges in identifying tobacco users and connecting them to treatment.

Read More


See More Highlights

Our Team

Our dedicated and talented team is united by our common desire to improve communities.

Harlan Luxenberg

Harlan Luxenberg

President & CEO

Julie Rainey

Julie Rainey

Vice President

Tia Bastian

Tia Bastian

Senior Evaluator I

Angie Ficek

Angie Ficek

Senior Evaluator I