This week’s blog is from Esther Goodwin Brown, an independent consultant and Programme Manager at the Calouste Gulbenkian Foundation, UK Branch (CGF). Esther shares some reflections from a series of conversations with funders and other stakeholders about how to facilitate civil society to share wellbeing evidence from project evaluations.
Funders and charities regularly produce evaluations to learn from projects and understand where funding has been more or less effective. However, it is often challenging to share learning systematically or to be sure that evaluations will stack up to traditional ‘standards of evidence’ if they are shared externally.
To help the Centre understand the potential and challenges of sharing evaluation and its role in this space, I spoke to 12 funders and 8 stakeholders. I also drew on my own experience of designing and commissioning wellbeing evaluations at Calouste Gulbenkian Foundation. Here are some reflections from those conversations:
The potential for learning
- Civil society holds a lot of evidence that is often underestimated and that could be more widely shared in order to improve practice and understanding.
- Outcomes evidence is not useful in and of itself. Both funders and charities value evidence that describes how interventions are being delivered in practice and how they help to bring about changes for individuals and communities.
- Giving this learning a longer life and wider reach. There is potential for bringing wellbeing evaluations together in a useful way that meets to needs of small and medium-sized charities.
The challenges
- Wellbeing is broad theme. Given the range of organisations working (implicitly or explicitly) on wellbeing (in different contexts and through diverse means) it may be challenging to create a resource that appeals to the full range of organisations and issues.
- Traditional standards of evidence can be a barrier to sharing. When different types of evidence are shared, it should be done in a way that describes the context and approach, rather than just focusing on the sophistication of the evaluation methods used.
- Individual measures of wellbeing are sometimes over-prioritised. Evidence sharing should help to develop the sector’s understanding of the impact projects have on community wellbeing too.
- Shared evidence projects can be challenging. They can require substantial commitments in time and resources from those collecting and submitting evidence. The Centre should be proactive in building relationships with charities in order to facilitate engagement.
The opportunities
- Describing and visualising data can help bring the evidence to life. Thematic summaries and evidence maps could be used to help organisations understand what evidence is relevant to them and how their work makes a contribution.
- It is notoriously hard to benchmark outcomes. Funders and charities struggle to understand what a ‘good’ increase in an indicator looks like. Having a bank of evaluations that show what wellbeing improvements might look like, would help charities understand what to aim for and funders to better develop evaluation frameworks.
- Telling a story of impact. Many organisations don’t have the funding or capacity to conduct sophisticated evaluations but feel under pressure to demonstrate impact. Sharing wellbeing evaluations could help charities to find similar projects and use their learning to understand the potential impact of their own work.
- A long-term goal of research should be to remove the burden on small charities to continually produce evaluations and forecast impact. The Centre could support this by enabling them to look elsewhere at existing wellbeing evidence and what others are doing.
- An online resource could link to existing resources, in order to connect the sector and support an ‘ecology’ of evidence use.
There is an opportunity for the Centre to work with funders and practitioners to create a resource to bring these learnings together. Any resource developed must be fit for purpose and useful for this audience – supporting them to learn from each other and help build up the evidence base. It will also need to tackle the challenges around the diversity and quality of wellbeing evidence, and help guide the use of evidence in different settings.
