Our thoughts - Catherine Mitchell

This month, Catherine Mitchell, the Centre's Organisational Learning Lead, reflects on barriers to evaluation practice and the steps we can take to create 'good thinking environments' for all.
 

This week’s Just One Question asks practitioners ‘what one thing would make evaluation really useful for you right now?’ The top responses so far are ‘more time to think about it’ and ‘less pressure to do different things for different funders.’ Only 2% have said that they currently find evaluation really useful as it is.

This is troubling, although on the first point, it’s no surprise that time to think about evaluation feels particularly hard to come by at the moment. We last asked this question as part of the ‘Asking Good Questions’ survey back in February 2020 - right before the pandemic hit - and ‘time to think about it’ was high up there in the responses, although not the top. With the pandemic and lockdown placing more pressure on practitioners and youth organisations, it’s no surprise that time feels like one of the biggest barriers to evaluation practice right now.

February 2020:

April 2021:

Context and other barriers considered, this ‘time to think’ challenge has been sticking with me recently. Specifically, I am wondering if and how we can better help tackle this challenge through our work at the Centre, and in collaboration with our partners. This question comes after a period of already doing a lot to try to adapt our programmes and the support that we offer, as my colleague Jo reflected back in February. We’ve been breaking training down into more digestible chunks, holding extra spaces to introduce and discuss some of the core elements of our work and how it relates to the current context, and (of course) moving content online. As we continue to make decisions about where we prioritise time and resources for programme support over the coming months, I’m wondering how we could best support practitioners with finding time to think about evaluation. Can we?

My sense is that yes, we can - to a certain extent. There are factors both within and outside of our control. For example, we’ve been developing a neat collection of tools that can support practitioners and organisations to self-assess and think about improvements to their practice - from the Confidence Framework for design and delivery of youth social action, to the Programme Quality Assessment that focuses on intentional staff practice that supports social and emotional learning, to the Listening Fund diagnostic that helps organisations to reflect on how they listen to young people. Responding to the pandemic has meant making these tools more digestible, able to be broken down, and (again) to be used online. But what happens when it feels like there’s no time to sit down and have a go at one of these, or to engage with the training and learning that goes alongside them? What other thinking spaces and opportunities can we help to create?

Sometimes it will be more about creating ‘permission’ to stop and just talk about a particular topic, with peers or with an external facilitator or ‘coach’. We can offer these spaces and the facilitators, but that ‘permission’ also needs to come internally from the organisation. For that, the content needs to feel relevant and applicable.

It might also be about just salvaging an hour to do that piece of reading that’s been sitting on the to-do list for a month and that could - maybe - prompt some more ideas or provide some new knowledge to draw on in your work. Doing this is hard, with no guaranteed or clear ‘output’, and it often means compromising something else, like other bits of your job or your own personal time. It’s also hard to prioritise these things above more urgent and pressing demands.

It’s also not just about what we’re doing (programmes, tools, training) but how we’re doing it. We need certain conditions in order to do our best thinking, and these need to be in place consistently. ‘Needing time to think’ is often associated with not feeling confident about the topic - we want to assess our options first, do some research, learn more about the thing we’re doing or how others are experiencing it. To encourage confident thinking (and doing), we must ensure that opportunities to think about evaluation feel safe, welcoming, inclusive, and accessible. The components set out by Nancy Kline in the link above provide some helpful principles to follow when doing this, such as giving attention by listening with palpable respect and without interruption and embracing difference by welcoming divergent thinking and diverse group identities. Alongside this, we also need to think carefully about the language and the platforms that we use, and how (and why) we bring people together.

I’m keen to make sure we are always putting these quality principles for ‘good thinking environments’ into practice when we come together to think and explore with colleagues in the sector. For this to be effective, however, it also requires individual organisations' leaders and funders to help create these environments for their teams and partners back in the ‘office’, whilst also finding practical ways to fit ‘thinking about evaluation’ into the busy day-to-day.

Things are still pretty volatile. For all the good intentions, it only takes one staff member to leave or become furloughed, or one crisis that means a training session has to be missed, to lose momentum with learning and quality improvement efforts and make ‘getting back on the evaluation horse’ feel even harder. At the same time, we often see what can happen when practitioners are supported and equipped to act on data and learning from their evaluations, to share with peers, and to drive forward improvements to their provision. We need to think really carefully about how we build foundations and infrastructure for this kind of good thinking to happen and to be acted on if we are to make evaluation feel useful and like an effective way of capturing learning from the past year (of which there is a lot). I think we’ve all got a role to play in that ambition.