Creative Evaluation Toolbox

How can an organization or a program create a better future? We think it always starts with the people and the stakeholders in an organization. We love a method called Appreciative Inquiry because the underlying assumption of this framework suggests that what we focus on becomes our reality. In organizations and programs, there are multiple realities and values. To shift and transition, the first step is to acknowledge and include all of these different realities and values. We work with our clients and stakeholders to create structured pathways for people to ask questions, talk about their reality and values. Along the way, we start to catalog all the organizational assets, including individual skills and team processes that function well. Then as a team, we come together to create a shared vision. In our experience, employees and stakeholders have more enthusiasm and motivation to change because they become part of the future possibilities and opportunities.

Focus groups are small, facilitated focused discussions specifically designed to uncover insights and guide future action. We use them for two precise reasons: (1) to help our clients understand a problem or situation, and (2) for pilot testing of ideas, materials, policies, services, or products. These tools look deceptively easy when they are, in fact, very complicated. Verifiable results require adequate preparation, planning, and analysis. People in focus groups should never be random; they should always have direct experience and common characteristics. Questions should be well-researched questions and consistent with the overall theme. The discussion flow should be carefully planned to create a non-threatening environment so that participants feel free to talk openly and give honest opinions. The final stage of analysis involves three layers of the discussion: the individual, the group, and the group interaction. Using these steps will provide insightful information, which is often fascinating.

Evaluation has an opportunity to be transformational. We utilize frameworks rooted in contemporary critical theory. Contemporary critical theory is a social theory-driven approach towards contemplating and critiquing policy alongside history, cultural trends, and popular discourse. The expressions of critical theory for policy evaluation that we use most frequently are critical race theory, the epistemology of intersectionality, truth as power, and the role of lived experience. These theories provide a pathway to analyze society through the lens of power and the social conditions that create inequity. When couples with realist or empowerment evaluation models, we work with stakeholders to dig beneath the surface, uncover the policy's assumptions, and reveal oppressive power dynamics. Utilizing these theories provides a framework for developing deep praxis by encouraging stakeholders to engage in creative reflection and thoughtful action to transform social norms and engage in social change. As part of the evaluation process, we measure how much the stakeholders have been transformed.

Participatory photography, or Photovoice, is all about letting people create images that tell their stories. For many people who live in marginalized or underrepresented communities, visual storytelling removes the need for words. Visual communication can build a bridge between communities and policymakers by using these images to share their community's progress or unmet needs. We use these images to create a shared account of a line of inquiry between stakeholders using the framework from participatory action research. Participatory and action-oriented in the sense that community members conduct data collection and constructive reflection through listening circles or other interviewing techniques. We analyze this data through systematic linkages and collective interpretation. We empower stakeholders through their collective action of selecting images and stories, exploring themes, understanding the analysis process, and disseminating how, where, and to whom the findings should be shared to impact community transformation and social change.

For our coalition and community of practice friends, we offer participatory monitoring and evaluation. This framework is such a tremendously empowering and creative process designed to engage stakeholders and encourage mutual learning. It also fosters trust and engagement as stakeholders make difficult decisions on how to decide what constitutes success, assess progress, and the best way to take action. Participatory monitoring engages the stakeholder in identifying the questions to ask about the project and the best ways to ask them. Stakeholders can also support making sense of the data, determining short-term performance (e.g., outputs and outcomes). Participatory evaluation engages stakeholders initially by naming or framing the problem to be addressed, how to achieve success, and develop a theory or practice. During and concluding the evaluation, stakeholders identify what to celebrate, what to adjust or change, and determine the overall impact.

Storytelling is a powerful tool to describe or make sense of a situation. Through the art of narration, people can find connections to events or conditions and are likely to retain more information because events are relational and not abstract. We use stories in a variety of ways. We combine storytelling and visualization aids to communicate findings to clients and stakeholders. We also use investigative research methods in the community for key informant interviews instead of a questionnaire or focus group. We find the individual narratives answer our questions and provide insight into barriers or successes that we did not know to ask and place the narrative story in context. Our clients end up with a much broader understanding of actual needs. We also support our clients with collective storytelling to move people to action for social change. In collaborative storytelling, we utilize publicly available data, vivid stories from the community to create a thematic account centered around shared values that people can unite around.