Wellbeing measurement in practice: Power2’s experience of using WEMWBs to evaluate impact
Our measurement guide contains a range of wellbeing measures including the widely tested Warwick-Edinburgh Mental Wellbeing Scales (WEMWBS) and accompanying Shorter Warwick Edinburgh Mental Wellbeing Scale (SWEMWBS), which capture multiple dimensions of mental wellbeing. The WEMWBS scales are designed to measure mental wellbeing – and evaluate programmes and services that aim to improve this – through a series of statements.
We have undertaken a rapid review of the evaluation literature that uses the WEMWBS scale and as a Centre, our role is to make knowledge visible and available, help others understand and measure wellbeing and its drivers, and also to share peer learnings and experience.
We recently shared homelessness charity the 999 Club’s reflections on their use of WEMWBS and now share children and young people’s charity, Power2’s experiences.
Power2 has been delivering programmes to support children and young people experiencing vulnerabilities and disadvantages since 2001. Here, Julie Randles talks us through the charity’s use of the Warwick-Edinburgh Mental Wellbeing Scale to demonstrate the impact of its programmes.
What did you do?
We support children and young people experiencing vulnerabilities and disadvantages to improve their wellbeing and engagement with school and learning so they can thrive and reach their full potential.
We do this through a variety of in-person individual and group programmes, such as Power2 Rediscover, Power2 Thrive and Teens and Toddlers. Our programmes use a strengths-based, trauma-informed and child-centred approach and are delivered by our highly-trained staff in schools and other settings throughout the North West, London and the West Midlands.
Evidencing impact – what and how
It is key for us to measure the wellbeing impact our services have on the children and young people we work with to evaluate their efficacy and inform our future design and delivery.
For over 10 years we’ve used proxy wellbeing measures such as improved self-confidence and self-esteem. We wanted to build on this by introducing an external, wellbeing-specific outcome measurement framework, adding validity to our outcomes for funders.
We chose to use the Warwick-Edinburgh Mental Wellbeing Scale to do this because:
- It’s easy to use and to interpret, particularly with the interpretation guidance provided.
- Licences are free for charities.
- Most importantly, the language used is friendly and doesn’t risk retraumatising already-traumatised participants.
We chose to use the standard 14 question scale, rather than the shorter 7 question scale, because we wanted to have a fuller picture of the mental wellbeing of our participants so we could better tailor our support.
We’ve been using the Warwick-Edinburgh Mental Wellbeing Scale since September 2021, when we rolled out our Power2 Rediscover programme following its 2020 pilot and evaluation. We use the measures at the beginning and end of shorter programmes (10 or 16 weeks), and termly for longer-term programmes. These are conducted at baseline and post-intervention with participants (young people).
What went well and what could be improved?
As an organisation we find the scale very straightforward to use, both for us and for our participants.
Our data team finds it simple to load individual participants and their results, and particularly appreciates that as soon as the second questionnaire’s responses are uploaded the comparative results are provided instantly. They also appreciate the extensive support documents provided, and how easy it is to interpret the results.
Their only caveat is that it would be helpful to be able to add more than two questionnaires to be able to measure progress in greater detail over time, and ultimately compare trends with others. National WEMWBS data is available albeit it does not chart the differences individual interventions or organisations make and rather plots average scores at points in time.
Our frontline practitioners find the Scale easy to use, and appreciate how it allows them to understand how the young people they support see themselves as opposed to how their referrers – usually their teachers – see them.
Our programme participants tell us that they find the wording clear and, on the whole, easy to understand. However we nearly always have to explain what optimistic means because it’s not a word they’ve come across before, and they can also be confused about the question around being interested in other people because they’re not sure how to interpret that for themselves. The wording of the scale has to be replicated faithfully, so we are unable to adapt it, and this extra level of explaining needed is something we are mindful of.
We also find – in common with many self-evaluation frameworks – that some young people often mark themselves higher on the framework when they first start working with us because they think that’s what we want to hear. This means that their results at the end, once they’ve developed a trusting relationship with us, can look artificially low to us.
Overall we are extremely pleased with how we’re able to use WEMWBS to demonstrate the impact of our programmes.
One of the main referral criteria to us is a teacher’s concerns for a young person who is struggling. As such, the majority of our participants are at a low level of wellbeing when they start working with us. We’re delighted to see what a difference our support makes; for example, last year 79% of participants on our Power2 Rediscover programme (n=87) saw meaningful positive change in their wellbeing by the end of their time with us.
Recommendations for others
- Choose a measure that is appropriate for and proportionate to your context so that results are meaningful.
- Make the most of accompanying guidance and documentation to make sure you’re using the measure effectively and interpreting the results correctly.
- Be aware of artificially low scores at baseline with self-evaluation frameworks – where participants tell you what they think you want to hear- and build this into your understanding and interpretation. For example, using practitioner observation of where this has happened.
- Consider a mixed-methods approach, using qualitative and quantitative data to report outcomes and add richness to your reporting.
- Using an established measure can give your scores more validity and robustness and allow for comparison with organisations doing similar work.
Julie Randles is CEO of Power2, a charity supporting children and young people to improve their wellbeing so they can thrive and reach their full potential.