Just one question
Taking inspiration from the success of ‘TeacherTapp’, in response to COVID-19 we are now using an adapted version of our existing ‘Asking Good Questions’ survey tool to ask youth practitioners to respond to one key multiple-choice question each week.
What we do
How do I get involved?
Choose two words to describe how you’re feeling about your work with young people right now.
[12/03/21 - 18/03/21]
We’ve asked this question twice before, first in July 2020 and again in September 2020. We know that as lockdown conditions have eased, tightened, and eased once more, practitioners' feelings towards their work with young people has varied significantly. We were interested in gauging the current ‘mood’, given that lockdown conditions are slowly set to be lifted over the coming months.
This week we heard from 75 practitioners, and received a total of 104 responses.
The comparison graph below shows how responses have changed since mid-July (presented as a % of responses).
Looking across the last eight months, we can see a slight decrease in overall positivity from practitioners towards their work with young people. Whilst feelings of motivation, frustration and excitement are the most prevalent feelings at all three time points, reported excitement has remained lower, with 21% and 20% of respondents reporting this in September and March respectively, vs 24% in July. Levels of frustration have also increased, from 15% in July, 19% in September, to 22% reporting feeling frustrated in our latest survey. More positively, practitioner’s motivation has increased significantly, with 22% of respondents feeling motivated, compared to just 16% in September.
Respondents related frustrations to the fact that it is ‘very difficult to plan ahead’ and there is a continuous need to be ‘responsive’:
“Young people respond really well, but our systems and processes are not as fleet of foot as they need to be to meet their needs”.
This notion of constantly needing to respond but struggling to match the pace of change and young people’s needs, was echoed in respondents’ own ‘something else’ answers to better convey what they are feeling: including words such as ‘tired’, ‘powerless’, and ‘uncertain’. Two respondent’s also alluded to impact of ongoing capacity and financial challenges on their feelings towards their work, particularly being perceived to have slipped to the ‘bottom of government spending priorities’ and youth services funding being cut.
These negative feelings were part of conflicting or ‘tumultuous’ emotions, and one respondent suggested:
“Most of the suggested words could apply at differing times and in different ways but the underlying feeling is the importance of the work.”
Despite some of the frustrations and continuing feelings of uncertainty, it is clear that as a reduction of social distancing measures appears to be in sight, many practitioners feel less anxious around their work with young people. Reported nervousness had continued to decrease, from 11% back in July, to just under 4% in this week's responses. Whilst confusion (understandably) peaked in September with the ever-changing rules around face-to-face delivery, just 2% of respondents described feeling confused this week, suggesting there is much more clarity in the guidelines around delivery. Feelings of preparedness have also gone up, with just 5% of respondents reporting feeling prepared in September, vs. 11.5% this week. It suggests that tools such as the NYA COVID-19 Guidance have been well received and utilised. It also suggests that as the road map out of lockdown progresses and organisations have consolidated some of their learning over the past year, organisations do feel more ‘ready to go’. As one respondent reflected:
“Now is the time to be excited about creating new opportunities alongside young people”
Whilst overall this week’s responses suggest that practitioners are feeling more confident around the practicalities of delivery and motivation has increased, levels of frustration are still high, and feelings of stress are hovering at around 10%. We know that the next six months or so are not going be easy, and we will continue to focus on how we can support organisations - through the information and tools we share, and conversations we facilitate - to keep motivation and excitement levels high, and reduce levels of stress or confusion.
Has digital delivery widened your geographical reach?
[05/03/21 - 11/03/21]
Comments from previous surveys indicated that some practitioners were finding that they are now reaching a wider range of young people through their provision, due to moving some or all activities online. We were interested in finding out whether this was the case more widely.
This week 52 practitioners shared a response.
From this week’s survey responses, the picture is fairly split; 42% of respondents report that they are now reaching young people from further afield, whilst 40% share that their geographical reach is the same. 12% already had a wide geographical reach, and a small percentage don’t know or provided a different answer.
The potential for increasing reach in this way will vary, of course, depending on an organisation’s type of provision, capacity, and resources. One participant commented that their organisation’s funding limits them to a particular geographical area, so they would not be able to widen their reach even if it was practically feasible to do so. Another participant does not do so much digital delivery, so this question is less applicable for them.
On the converse, one respondent shared that digital delivery has enabled them to develop links with projects from around the country, meaning that the young people they support are now joining online groups from other areas of the UK - and beyond. This is supported by an increased ability to link staff up across geographical areas for networking (an opportunity that also came through clearly in a previous survey about areas of work that are working better online). Another person added that they have been engaging groups at an international level.
Finally, a wider geographical reach might not be based solely on digital provision - one participant commented that they have managed to increase capacity of locality based staff, which could also be said to increase their geographical reach.
Are the young people who are engaging with your online provision now the same ones who were engaging at the start of lockdown?
[26/02/21 - 04/03/21]
This week we received 52 responses. Participants could select one answer each.
The question for this week builds on responses and feedback from a number of previous surveys:
- In two early surveys from late May last year, 56% of respondents reported that fewer young people were engaging with online provision compared to face-to-face, and there were mixed experiences of whether the same or different young people were engaging regularly each week.
- Then, in November, 47% of respondents told us that they did not feel that they were reaching young people most in need of their organisation’s provision.
- Finally, last month in February, 47% of respondents and many comments reported that young people were less keen to join activities online due to digital fatigue.
A major concern shared at the start of lockdown was around the limitations of building and maintaining relationships with young people, whilst pre-pandemic provision was not possible and sustained engagement felt more challenging.
Almost a year into lockdown restrictions, the picture of young people’s engagement with provision continues to be mixed. As one respondent commented, “some [young people] have been retained, some have dropped off and some young people have come on board.” 20% of respondents report that they are currently still engaging with the same group(s) of young people that they were at the start of lockdown, and 50% report that they are engaging both the same groups and new young people. Responses this week indicate some consistency and continuity, although we cannot draw conclusions from these figures alone (more might be learned through organisational user and engagement data, for example).
17% are engaging the same groups, but have found that overall, engagement has dropped off. One person commented “numbers attending have reduced substantially” and another also noted that “some levels of engagement [are] unclear in the more 'anonymous' world of social media.”
A small percentage (6%) are now engaging totally different groups. As would be expected, it is not possible to move all types of provision and engagement online - those who primarily worked with young people through detached work, for example, are not easily able to move this engagement online. For some, adapting provision has led to new groups engaging:
“Because we have changed our offer as the first offering was not working. We are attracting new groups .”
At the same time, there are large groups of young people not accessing provision:
“We are seeing [an] increasing number of young women engaging whereas take up from young people with refugee status has tapered off. This is particularly at the referral stages of our services and provision. We suspect that this has to do with the digital provision changing accessibility for certain groups but are looking into this further in the next few months. In terms of our group work provision we see smaller numbers engaging but relatively more young women participating.”
“We're actually reaching a far more diverse group of young people than before - care-experienced, different geographical regions - all as a result of the move online. It's superb to see. And we've focused on designing our programmes to be accessible, recognising some of the barriers young people face. Young people have been calling in, have been joining by text, video and all sorts of things. That has made a huge difference - accessibility is at least as much about design as it is device.”
Challenges with accessibility continue, but comments such as those above demonstrate ways in which practitioners and organisations are prioritising and responding to these through their work.
Are there any features of your remote provision that are working better than face-to-face delivery?
[19/02/21 - 25/02/21]
This week we received 56 responses from 52 people.
42% of all respondents report that none of their online provision is working better than it would be were it being delivered face-to-face. This week’s comments conveyed a general sense that whilst there were some benefits to online, such as reducing travel costs or increasing organisational capacity to deliver through lockdowns, most provision was better suited to being delivered face-to-face. However, there was also a general consensus that online is ‘better than nothing’, and most comments offered both positives and negatives to the current situation. This came through strongly from the 29% who selected the ‘something else’ response.
“It’s not working better, it has just added another dimension to the work.”
A number of respondents did report specific activities that, in their experience, are working better online. These include: family support (4%); group activities like sports, cooking or art (6%); peer-to-peer support (8%); targeted group work - for example, with young carers (8%); and 1:1 support, counselling, or mentoring (12%). One person also added ‘outreach and inclusion work’, and another expanded with detail on their work with young carers:
“Work with young carers - less anxiety around leaving the house - less anxiety around joining a new provision - no outlay on travel. The number of young carers we support has increased exponentially.”
Another comment also reflected on 1:1 support, noting that ‘there are some advantages to having less interruptions and more focussed time for individual support than in a regular, open session’.
However, they also highlighted ‘lots of disadvantages’, including lower engagement and less young people seeking support, and fewer opportunities for the day-to-day, responsive conversations that can provide young people with ‘situational’ support. The picture is definitely mixed, and practitioners are reporting varied experiences with and for the young people that they are working with.
“Some young people like the online better than face to face in the centre in group work. but
this is only some of the group.”
“We have predominantly returned to face-to-face delivery as both the staff and the young
people seem to prefer this way of engaging, especially now that so much school provision
involves a screen!”
"The picture is really mixed and for some young people they have expressed how much they have enjoyed the small group creative online sessions, but that response is often connected to a broader feeling of disconnect and boredom - so I read it more as 'better than nothing' rather than 'better than face to face'. What has been working well is CPD - staff training across the different stakeholders in our collaborative projects. It is easier for people to access training online and especially for some of the support staff we work with they have been more confident in taking part than they may be in the room."
Six additional comments build on this second point - that moving online has increased opportunities and capacity for staff and volunteer training, and networking. Particular benefits have included: making training more accessible for volunteers ‘who are scattered all over the UK’; allowing for bigger numbers in sessions; online sessions being easier to organise; the creation of new networks; and reduced travel costs. One person shared an intention to continue offering online training for volunteers even when face-to-face provision resumes.
“Remote youth provision hasn't worked for us, but setting up a network of organisations working with young people in this rural area works really well remotely because there's no cost of travel, it's easier for people to fit a meeting in.”
“Certain approaches to creating new work and sharing ideas.”
“Training and networks for youth workers are working well online. I would not say 'better' but easier to organise logistically and with better attendance.”
"Face to face can not be replaced by online working when working with families. Only training, with no travelling needed, or expensive car parks has realistically been better."
There could be much more to explore here - for example, where has online support for youth practitioners and networks focused, in terms of specific skills or areas of work? Will this continue once lockdown restrictions lift, and what impact has this had or could this have on practice? What has this meant for youth practitioners’ experience of work during the pandemic?
Finally, there were a number of comments that highlighted some real benefits in terms of increasing access to provision for young people, noting that online activities had been reducing geographical and travel barriers, and barriers for young people with disabilities.
“Face to face is always better but online meetings have allowed young people with links to the area, but who don't live locally to join in some of the activity.”
“We have engaged with some young people who don't attend conventional provision due to rural remoteness or severe disability. Virtual will remain part of our offer going forwards.”
“Young parents being able to access groups / working across geographical areas is easier / some young people with disabilities / health conditions [are] more able to access projects as from home.”
As might be expected, this week’s survey shows a clear desire to return to face-to-face once restrictions are eased and it becomes safe to do so, but there are also indications that practitioners will be retaining specific parts of online delivery that have been working well, particularly for specific groups of young people.
Thinking about the research you’ve seen about the impact of the COVID-19 pandemic on young people, did it mainly:
- Completely contradict your experience or opinion
- Confirm what you already knew or felt to be true
- Largely confirm your existing view, but add more detail or insight; or
- Something else?
[12/02/21 - 18/02/21]
There have been a huge amount of surveys and studies undertaken throughout the COVID-19 pandemic, covering a wide range of themes that relate to the impact of COVID-19 on young people. For this week’s survey, we wanted to know how the research that has been conducted to date has corresponded (or not) with the experience of practitioners.
61 people shared an answer to this question. Participants could select one response each.
The vast majority of respondents felt that research about the impact of COVID-19 on young people largely confirmed their existing views but added more detail or insight (59%), or simply confirmed what they already knew or felt to be true (38%). A very small percentage felt the research completely contradicted their experience or opinion, or wanted to share a different perspective.
Several comments expanded on this, suggesting that the narratives presented through research were not always representative of their own experiences of working with young people - or, perhaps, of young people’s own experiences and voices:
“Useful but feel the angle the narratives are written about [is] a bit nice to those agencies and Govt dept they are writing about and not hard hitting enough as we see & hear in [the] community.”
“I think our collective response to Covid 19, like austerity, has shown us that as a country we do not value young people as we continually marginalise them and their views.”
We did not receive as many comments as we usually do this week, but those that we did were helpful in unpicking how ongoing research can be received and used by practitioners. One further comment shared a really helpful reflection on the ‘usability’ of existing research, the extent to which it can be perceived to be representative, and its potential to influence change at various levels:
"There are challenges keeping on top of it all and synthesising it. There are particular challenges in terms of spending the time to critically assess the methodologies used and statistical validity of the findings.
Some of the research clearly reflects the particular makeup of the young people that the organisation conducting it are in contact with. There can be variability in terms of how much this is recognised. This selection bias is not necessarily a bad thing (if properly acknowledged) as in some cases it can provide a focus on the particular groups of young people, such as the most vulnerable. However on many occasions this bias is ignored (or not explored) and the findings are extrapolated as if they apply to the whole population. This can be really frustrating, as there is usually not the evidence to support this.
There are so many pieces of research that you probably could find something that confirms and something that contradicts.
As most of it is at national level there can be limits to its usefulness for influencing at a local level because local leaders believe our area is so different from everywhere else that they won't accept that national research is relevant to our area.
There is a real opportunity to try and bring together the different pieces of research into one thing that everyone gets behind, so that it is big enough to have local data and then can test many hypotheses including ""our area is different”.”
These responses highlight some key questions to consider for ongoing research efforts:
- Where are the current gaps in research - for example, how representative are current findings?
- How/are organisations using existing research?
- How are we consolidating and sharing existing research with practitioners, young people, leaders, and other stakeholders? What platforms are we using to do this - and is it accessible?
- What are the key questions that we need and want to explore, in order to ‘get everyone behind’ the data?
Are the young people you work with experiencing ‘digital fatigue’?
[05/02/21 - 10/02/21]
This week we received responses from 66 people, who could each select one answer.
‘Zoom fatigue’ and ‘digital fatigue’ are terms that we have all become increasingly familiar with over the past year, as many feel the draining effects of staring at a screen and communicating via video calls for large parts of the day. With many young people now back to attending school online during the day, and with large parts of youth provision having been hosted online for coming up to a year, we wanted to know whether ‘digital fatigue’ is something that youth practitioners are currently seeing from the young people with which they work.
47% of respondents said ‘yes, they're less keen on joining online activities with us now.’ A number of comments highlighted that the return to online schooling has contributed to this:
“More formalised nature of school based online provision and more pressure on parents to work during lockdown has meant young people feel zoomed out and (at the younger ages where parental support is needed) just not got time to support anything beyond schoolwork.”
“Yes this is coming up more and more, sick of it with school and want to see faces in real life.”
“Some young people are getting a lot of online school work which means they are less likely to come on our session as well.”
Interestingly, one respondent did report the opposite - that online schooling meant the young people that they were working with were more likely to engage with digital youth provision:
“They seem to be more keen to work online when homeschooling. Engagement is definitely
better since January 2021.”
For others, the fatigue was felt more generally - “It's also restriction/lockdown fatigue in general not just digital”, particularly in areas where the restrictions have been more relentless over the past year:
“We are in an area that has been towards the higher end of any restrictions all the time (and
where local guidance and practice has, due to the persistently high rates, always been much more conservative than guidance allows). Compared to areas where there was significant easing for significant time periods, where colleagues and their young people felt a sense of release and break from mainly online, it has felt pretty relentless. Reasonable guess would be that we would be at higher level restrictions coming out of lockdown too. The impact on everyone on not having had much of a summer release cannot be underestimated but in national media this difference in experience seems to be downplayed.”
This comment, again, highlights the disparate way in which lockdown restrictions are realised across communities:
“Yes, some are, whilst others don't have devices to get online with.”
As would be expected, responding to and managing ongoing restrictions are clearly impacting adults and workers too:
"[...] and adults too ... and the two may not be disconnected :( “
“And it’s not just the young people but the staff as well, constantly looking for new & innovative digital engagement is draining....."
Online provision is requiring a whole new set of skills and competencies, not only in navigating digital tools but also in using them creatively and ‘innovatively’:
“I am having mixed messages from our staff team. I think it is a lot to do with the competency
of staff to use technology and their innovation to best effect.”
32% of respondents did not give a clear-cut ‘no’, instead reporting that ‘it depends on the type of remote provision they're engaged in.’ Many comments expanded on this response, noting that attendance and engagement varies between young people, and that it depends both on the specific activity (including nature and duration), and on how much time has already been spent online:
“Some groups are much harder to engage than others but most are still engaging.”
“Hard core still happy but the waverers have waived.”
“We've had the same group of young people attend every week online, they clearly aren't
experiencing digital fatigue. However there is a whole other group who never engaged online, wonder if they are experiencing digital fatigue or just don't want to do digital youth work.”
“Although it also depends on activity? e.g. their personal interest in the event; incentive to join in;
as well time; duration etc. But general sessions are now less well attended?”
Many are using screens all day so we have started to see some young people cancelling as
they've had enough, however it depends on the day, format of activities and previous amount of
screen time ( and varies depending on the type of support offered).”
Others are noticing digital fatigue presenting in different ways - for example, young people being less likely to speak up or to have their video on when joining sessions online. One respondent noted how some are ‘thriving online’, whereas for others “it is exhausting, exposing.” Zoom and other video platforms change the space and dynamics of provision, with young people (and adults) often having to reveal more about their personal environments than they would otherwise have to. This article takes a deeper look at how this is impacting young people attending digital school in the US.
15% of respondents said that the young people they are working with are “still keen to connect and engage with us online.” There is a recognition that “it wouldn’t be their preference”, but that whilst being tired of being online, they will still engage “where they feel it has value to them.”
"For some it's the only way to stay in touch with friends and our youth workers so they really look forward to it but in the end digital engagement is just not as fulfilling as human contact.”
For others, tuning in online is still essential:
“[...] other young people depend on our sessions as their place to go to have some fun in a safe environment."
If you’ve got more thoughts on this topic, we would love to hear from you. Please get in touch via firstname.lastname@example.org.
How do you feel about taking part in training and online support during the pandemic? (select up to three)
[29/01/21 - 04/02/21]
We have spent a lot of time reflecting on digital delivery for young people, in recent surveys, but this week we moved the focus onto digital provision for youth practitioners. With the majority of training and support needing to move online, have youth workers and teams been able to access more or less useful and relevant sessions?
This week we received 94 responses from 68 respondents, who could select up to three responses each.
A large proportion of respondents told us that they’ve been able to join more useful training and support than before the pandemic (46%), but that they only want to join if it’s really important or relevant to them (50%). This compares to 13% who said that they want to join in with all of the training and support opportunities that are available. On the other hand, 13% of respondents reported that they have been flooded with opportunities that they are not interested in, and 12% have had less time to join training and support opportunities than before.
Reducing geographical barriers
Several comments noted the benefits of online training reducing geographic barriers, both in terms of access, and in connecting with a wider range of colleagues:
"Not having to travel has meant I've been able to join more training events. With location not being an issue I've been able to connect with peers from other areas of the country.”
“Online training and networking has lessened some geographic barriers.”
More training, at scale
Others have been able to participate in (or host) a large number of events, with capacity for great numbers of participants:
“We have just held a learning and development week. 20 sessions which have been attended by around 700 participants! Been very well received.”
Lots of training and support on offer
A number of comments from this week’s survey also celebrated the wide variety of support and resources that have been made available over the past year, from a range of different providers:
“So much more access to fantastic resources, training and support online which is much more effective timewise than before!”
“The sector support and training has been great by NYA, UK Youth, IYW and London Youth.”
“Training has been promoted via many agencies. It has been valuable to the sector that we have not stood still. Online safety courses are regularly promoted.”
These benefits have not been universally felt, however. One person commented that there have been less relevant opportunities than they might have hoped for, given the circumstances. A number of comments also flagged more specific challenges with accessing online training and support:
“I've undertaken some training but there are also issues with doing training when on furlough and not allowed to undertake work.”
“I still think face to face training is preferable to online. It's easier to disengage when online.”
“[...] Some face to face stuff and informal support has dropped and not really been replaced or been harder to access, but some new stuff (or stuff that has become more accessible) has been really good.”
Fluctuating need, demand, and opportunities
One final comment highlighted how uncertainty and churn, magnified by the pandemic, can mean that need and demand for (and availability of) training can seem to be constantly changing:
“No simple answer [...] The fast changing nature of things during the pandemic has meant that the need for support, capacity to engage with support and availability of support can all alter considerably at very short notice and in some cases literally overnight. Almost all the above have applied for me at some point. Sometimes [we have] had extra capacity to engage, sometimes none. Sometimes [we] felt a real need to join, other times less so. Sometimes [it] felt like [there is] lots available, other times less. But the feeling of real need, the capacity to do so and the availability have frequently not all triangulated. Not really anyone's fault, [it is] just the nature of things.”
This is a real challenge when thinking about offering training and support, or connecting teams and colleagues with opportunities. It is clear that there have been some real positive experiences of online learning for practitioners during the pandemic, but it still proves a challenge to get the balance right in terms of delivery formats, timing, and content.
This is certainly something that we have found through various programmes of training and support that we offer at the Centre. One example of this is the Youth Programme Quality Intervention (YPQI), which supports practitioners to develop cycles of peer observations and improvement planning, to develop youth work practice that supports social and emotional learning for young people. With observations proving tricky given the context of lockdown and other challenges, we have moved our focus onto the ‘improve’ stage of the cycle, prioritising training in ‘coaching for quality’ and ‘youth work methods’ as a way to lay the groundwork for observations at a later stage. This is one way in which we have tried to respond, and we are looking forward to hearing back from organisations involved in the pilot about if and how this meets current needs and priorities (do get in touch email@example.com to find out more!)
If you’ve got any more thoughts on online training and support for practitioners over the pandemic, please get in touch firstname.lastname@example.org.
How confident are you that you can define what delivering your provision with 'high quality' looks like?
[22/01/20 - 28/01/20]
We received 57 responses to this week’s question.
Defining what ‘high quality’ delivery looks like is essential for supporting staff and volunteers to deliver a consistent, effective service for young people. Data about the quality of your work should also provide you with actionable insight that you can use to plan for, and make improvements. At a time where a significant amount of provision has had to be adapted, it is more important than ever - although not always easy - to be thinking about what quality ‘looks like’ and means for practice within the current context.
In response to this question, 27% of respondents said that they feel ‘very confident’ defining what ‘high quality’ delivery looks like for their provision. Overall, 79% of all respondents answered with a 4 or 5 - indicating a high level of confidence across the majority of practitioners taking part in this week’s survey.
Several people, however, also noted that whilst they felt confident, there is ‘always room for improvement’ and that they are ‘not closed to new or different approaches / ideas / aspects of this.’
Acting on quality
Building on this, a number of respondents noted specific tools, frameworks, mechanisms, or other resources that have found helpful in thinking about what ‘high quality’ looks like:
“There are lots of quality marks to use to ensure this as well as the National Occupational
Standards and peer observations.”
“The tools that have come out from the Centre for Youth Impact have been really useful and have helped us to be confident in this (e.g. the PQA tool for staff peer-led assessment), plus we use our youth voice group and user survey feedback to capture young peoples' views, and get external independent assessments from the local authority (and hopefully in the future our peers).”
These comments highlight an important point: that whilst defining quality is an important first step, it also needs to be applied through an ongoing process of reflection and improvement.
The PQA tool mentioned above refers to the Programme Quality Assessment, which sits at the heart of the Youth Programme Quality Intervention (YPQI). The PQA provides a way for practitioners to reflect and act on observations of staff practices and behaviours that will create safe, supportive, interactive, and engaging environments for young people. It is based on the premise that these environments - created as a result of intentional, high quality practice - will provide support for young people to build their social and emotional skills, such as teamwork or problem solving. The tool is used as part of an ongoing cycle of peer observations and improvement planning and implementation. Find out more and get involved here.
One further comments draws on these aspects of social and emotional learning when thinking about what quality looks like for their specific context for youth work:
“We're very happy with the quality of the work we deliver, outdoor education offers so much more opportunity to challenge and explore resilience than traditional recreational youth provision.”
It’s important that any definition of quality is bound up within a theory of change, specifically through the activities, and mechanisms of change that have been set out (mechanisms of change are the experiences that young people have in your provision that affect a change in outcomes, and are used as markers of high-quality delivery). One respondent is prioritising this in their organisation's theory of change development:
“Not me personally, but as an organisation I think we can, and we're currently going through a Theory of Change process where that's an element I want to be articulated.”
Last week, we spent some time reflecting on how many organisations currently have a theory of change for their work with young people (see the full commentary below). Thinking about defining quality is an important part of any theory of change development, and can be a really helpful, engaging, and important way to bring staff, young people, and other stakeholders into these conversations.
A small number of respondents indicated that they did not feel very confident at all in defining what delivering provision with ‘high quality’ would look like: 2% gave a ‘1’, and 7% gave a ‘2’. 13% felt that they were somewhere in the middle, giving a ‘3’. As discussed, it is not always straightforward or easy to define quality, and several comments noted some of the potential challenges that can be involved when both defining and monitoring quality on an ongoing basis. These included:
- Supporting new youth workers
- Practical challenges of doing observations, especially when delivery feels ‘inconsistent’
- Not being face-to-face and struggling to get ‘honest’ responses from young people - “when body language can’t be seen and you want to escape being asked things - easier then to say things are good/okay /fine when what you are delivering is felt to be rubbish by clients, but makes their life easier as they are not being challenged as much as normally would be the case”
- Another respondent felt that a reduction in funding and particular policy decisions over the past decade has directly led to a decline in quality.
Noting some of these challenges, one respondent added:
“However we can put in factors for assurance. If we have experienced, well-trained youth workers, clear aims, strong values, enthusiasm and commitment, an understanding of young people's needs/rights/strengths, consideration of equality and involvement of young people at all levels, then we can have a real sense that the work is of high quality.”
Another participant expanded on this, making the case for definitions of quality particularly during this turbulent time:
“High Quality provision leads to great outcomes! Understanding what 'high quality' looks like and monitoring that is as important as measuring outcomes. Other things could get in the way of great outcomes - you know life, pandemics! etc but for a young person receiving a high quality provision will be a benefit.”
The key to making quality improvement work for your organisation and provision is to create a culture that supports and expects improvement to take place. This involves leadership, values and systems which encourage and celebrate learning and improvement, and creates time for staff and volunteers to reflect on their practice. This can be much easier said than done, but it is also essential.
If you’re thinking about this, we’d love to hear from you - do drop us a line at email@example.com. We’re also running a number of sessions with the Regional Impact Networks over the coming months, so keep an eye out for one happening in your region soon. You can also find lots more information about defining quality on pages 14-18 of the Asking Good Questions guide.
Does your organisation have a theory of change for its work with young people?
[15/01/21 - 21/01/21]
This week we heard from 60 survey respondents, who could each select one answer.
Before we get started with this week’s analysis, we thought it might be helpful to clarify exactly what we’re talking about when we talk about a ‘theory of change.’ A theory of change is just that - a theory or set of ideas, developed by all those involved, about how provision creates change with and for young people. It’s both a process - of consultation, reflection and engagement - and an output - usually a diagram or image that captures:
- Aim: the long-term impact that young people and their communities achieve for themselves. This impact is usually experienced in the future, rather than ‘in the now’.
- Outcomes: the skills, knowledge, and assets that your provision supports young people to develop and that create the best conditions for young people and communities to experience the longer-term impact.
- Mechanisms of change: these are the experiences that young people have in your provision that affect a change in outcomes. They are often referred to as the ‘active ingredients’ in your work and are markers of high-quality delivery.
- Activities: what you do, how often, with whom, and in what setting.
As theory of change is not static! As you develop and refine yours, you may find that more questions arise, and gaps in your thinking and rationale may crop up. This is a very healthy sign that the process is working!
We’ve shared more thoughts on why we think a theory of change is important (and indeed essential) below.
How many organisations currently have a theory of change for their work with young people?
Just over half (52%) of respondents reported that they have some sort of theory of change - either an overarching one for their organisation (22%), or several different ones for specific areas of their work (30%), examples of which included a youth crime prevention project, and open access youth provision.
It is worth noting a potential bias: those engaging with the survey, and/or the Centre’s work more broadly, may be more engaged with evidence and evaluation, and therefore more likely to have a theory of change. However, the responses we received this week are similar to those we’ve seen on other projects where organisations might not be expected to be similarly engaged with evaluation, such as the Youth Endowment Fund Covid-19 Fund and the #iwill Fund Impact Accelerator.
A further quarter (27%) of respondents do not currently have a theory of change at their organisation, although a fair few comments noted that development work was in progress:
“We've had a couple of different iterations over the years, and the last set we developed two years ago weren't really adopted. However we're going through a ToC process at the moment, as part of our overall strategic review, and it has more senior buy-in, so I'm hoping the new ToCs (one organisation-wide ToC and some more specific ones for our offers) will be used more across the organisation.”
“We are planning to create an overarching Theory of Change this year as we've recognised the need for it. We want to bring our programmes under a unified approach, and define key concepts at an organisation level.”
One person also noted a particularly positive process of developing a theory of change with external support, involving young people, staff, trustees, and other key stakeholders.
A theory of change can help you to feel confident in knowing what is core to your work, and then adapting around those components as and when required - something that is particularly relevant right now. However, as highlighted by the first comment and noted above, writing a theory of change is only one step in this process. 15% of respondents this week do not know whether or not their organisation/provision has a theory of change, an indication that perhaps organisations aren’t embedding theories of change within their practice and processes, or sharing them widely with their staff and volunteers.
Of course, the coronavirus pandemic means that most youth organisations are working hard and fast to adapt their work. If you’re thinking about how your organisation’s theory of change relates to your practice in the current context, start by having a read of Research and Methods Lead Mary McKaskill’s reflections on how to engage with a theory of change in a way that supports intentional decision making and adaptations to delivery, specifically during the COVID-19 pandemic, originally shared back in June during the first lockdown. To support your thinking about how this extends to your evaluation planning and implementation, we’d also recommend BetterEvaluation’s three-part series on adapting evaluation in the time of COVID-19. NPC and Evaluation Support Scotland have also shared some really helpful reflections and recommendations on this topic.
Finally, we also received a few responses this week that indicated some people were both unfamiliar with the theory of change concept and sceptical about its potential value. Theories of change in youth work have never been uncontroversial, and - whilst they are becoming more widespread - there remain voices that challenge and question:
“What the f**k is that, we just do youth work”
“What's a "theory of change"? Sounds like nonsense to me”
In response to this, and building on the outline included at the start of this week’s reflection, we would add that a good theory of change should:
- Create time and space for you to reflect on your intentions and practice, alone and with young people and partners
- Help your team consolidate what you already know and uncover new questions to explore
- Explain why you do what you do, and what you do, clearly and succinctly
- Set out the ‘core’ and ‘flexible’ components of your work
- Help you design new projects and programmes with and for young people
- Provide a framework for learning and evaluation
Why is all of this important? There are lots of reasons, but it might include: building consensus on the ‘how’ and the ‘why’ of your work; communicating clearly to young people, partners and funders about your work; feeling confident defining what quality looks like - especially at times when activities need to be adapted; and providing opportunities for a wide range of stakeholders to be involved in shaping provision. There are plenty more, and we’d love to hear from you if you have any more reflections on how a theory of change has supported your work with young people.
If you are starting to think about a theory of change for your work and are looking for some guidance to support with this, check out this resource hub on our website, which includes a ‘theory of change toolbox’ that you can use for theory of change workshops within your organisation, as well as some background on how a theory of change will support you in asking key questions about your provision.
How prepared do you feel to continue offering provision for young people during the third lockdown?
[08/01/20 - 14/01/20]
We received 64 responses to this week’s survey. Each respondent could select one option.
52% of those who took part in the survey reported that they feel very prepared to continue offering provision for young people over the third lockdown, with delivery that they have already adapted. One respondent commented:
“Very prepared this time, good advice from Public Health and the NYA, good PPE, robust and reviewed risk assessments. Youth workers ready and willing. Shame about the weather.”
Another respondent also noted how they found the NYA alertness level guidance extremely helpful.
41% are feeling somewhat prepared, with plans in place but changes that still need to be made. A small number (3%) are feeling not at all prepared, and 5% told us something else.
Three main delivery challenges were flagged in this week’s comments. The first is a broad concern about limited resources, across a wide range of areas that include (a lack of) access to safe community building spaces, funding, and staff capacity. One participant commented that they were not currently receiving grants from major funders, and that they did not have the budget to qualify them for other grant streams, and another observed that staff in decision making roles within the youth organisation were ‘incredibly behind, focusing on other aspects of their work’ - perhaps emphasising the many demands on practitioners and youth organisations at this time.
Another practitioner highlighted a lack of capacity to meet young people’s individual needs:
“Every student we work with has such varied circumstances, it's so individualised and we don't have the capacity to be bespoke. I think our generic or blanket provision just won't work for everyone.”
Young people’s engagement and virtual fatigue also continue to make planning and delivering youth provision a challenge, particularly now that many young people are also now experiencing the school day from their homes and behind screens:
“We were really looking to meeting in person, as the groups were all zoomed out! This has affected some of our planned activities and some of our members are now not wanting to log on again, after a full day of on-line schooling!”
“We have the plans in place, however young people are increasingly not wanting to engage as much. They feed back they just want somewhere open access to relax with friends.”
“We can carry on, but it feels harder to keep the offer fresh and be innovative. Young people feel more demotivated and less enthusiastic about online delivery.”
Young people are not the only ones feeling this fatigue; the need to continually shift and adapt continues to take its toll on those who are offering the provision:
“Feel prepared for something and then the goal post gets shifted, so prepared for that and they move again. It's an ongoing battle that is feeling exhausting.”
On top of this, there is the emotional and moral weight of having to make decisions about safe delivery:
“We have a very dedicated team who are committed to delivering for young people but access to community buildings has become a challenge and also the responsibility to keep staff safe as well as deliver the services is a challenge.”
Some are still unsure about whether it is safe to be delivering permitted in-person activities:
“I think there has been good learning from Lockdown One - there remains a worry around the infection rate of the new strain so whilst we can continue to deliver we need to ask if we should?”
“Moral is the dilemma and proving to [the] community we are offering essential and much needed service to young people.”
This week’s announcement that youth workers currently holding or actively training for National Youth Agency (JNC)-recognised qualifications are classified as key workers is a positive step, and will hopefully provide support on this last point. However, there is no doubt that conditions for ongoing delivery are still very challenging.
A huge thank you to everyone who contributed a response this week. We’re shifting a gear with the next question and thinking about theories of change - does your organisation have one for its work with young people, either across the organisation as a whole or for individual activities? Let us know in the usual place, before 17.00 next Thursday 21 January.
Before Christmas, we asked ‘when do you think your provision for young people will return to how it was pre-pandemic?’
This survey was open between 11/12/20 and 07/01/21.
The situation across the UK has changed, furthermore, since we posed this question, as we now begin a third lockdown with restrictions much like those we saw back in March and April last year, and as vaccinations are rolled out at a larger scale. Whilst stricter measures for January were anticipated by many, would those who responded before Christmas respond in the same way today? As one participant commented, responding to this question “very much depends on government restrictions changing and vaccine roll-out being quick and effective. What are the chances?”
How did people respond?
A small percentage of respondents anticipated a return to their pre-pandemic delivery between January and March this year. This could be due to their unique context and type of provision (current National Youth Agency COVID-19 guidance does allow for small group or 1:1 sessions with vulnerable young people indoors.)
“We are piloting small youth group sessions within the building for the first time. In January we are hoping to extend that provision.”
In contrast, 13% do not expect to see a return to pre-pandemic provision happening in 2021 at all.
22% were hopeful for a return between April and June this year, the largest proportion of respondents anticipated this happening between July and September, and then 18% believed this would be most realistic between October and December 2021.
12% said something else. As ever, the additional comments shared in response to the survey highlighted the nuance and complexity that surrounds this topic.
Provision won't return to how it was pre-pandemic
Some practitioners anticipated resuming operations from spring or summer 2021, but expected that things will not be ‘back to normal’ until 2022, especially with some social distancing measures expected to stay in place for a while yet.
Others felt that provision will never look the same again. For some, there is the harsh reality of provision having to close permanently, or needing to invest significantly in rebuilding provision:
“Many of the voluntary clubs we support have lost volunteers and revenue which will force them to close permanently.”
“Hopefully some aspects should return sooner, but there will be much to rebuild across some locations/services that are suspended and some areas of work (e.g. youth volunteering).”
There is also a concern that the economic impact of the pandemic will lead to a further reduction in resources:
“I think even before the pandemic we knew that young people were disproportionately impacted by austerity, cuts, etc and my worry is that with Government talking about having to "pay it back" that young people, again, will be the losers and my big worry is that there will be more cuts on already devastated services.”
...and nor should it
Others noted that blended delivery is likely to be a more realistic way forward. For some, this is felt to be a good thing, especially where adapting delivery has led to new learning and positive changes to provision:
“I believe we will retain elements of online provision into the future, as it has been helpful to some of our young people and also offers an alternative method of contact?”
“We are aiming to return to pre-pandemic as soon as possible, maybe incorporating some changes from the things we've learned during the pandemic.”
“Never and nor should it. It’s changed how things are done and shouldn't go back.”
“We will continue to deliver remote and virtual programmes (something we had never done before March 2020).’
How long is a piece of string?
For some people, the situation still feels too unclear to estimate when provision might return to how it was before the pandemic, although the vaccine roll-out was felt to be a clearer indicator, especially for increasing the number of young people attending face-to-face group work.
“This very much depends on government restrictions changing and vaccine roll-out being quick and effective. What are the chances?”
“No clue, how long is a piece of string?”
“[...] I think face to face work will return to similar to pre-pandemic levels once the vaccine roll-out has been widened as most of our work is one to one or small groups.”
Another respondent detailed their hopeful plan for a phased return, continuing remote and virtual programme delivery in the immediate future, re-starting groups and clubs by March (at a smaller level and outdoors where possible), beginning camps and residentials by June, and resuming international work by August 2021.
They concluded by saying “we are hopeful, resilient and will do what we can.”
It is really difficult to plan and predict under the current circumstances, especially as contingency planning, increased risk mitigation, and responding to local and national contexts draws further on already stretched resources. The sector has responded and adapted, continuously, since this all began.
As well as demonstrating how tricky this is, this survey has also highlighted how practitioners are making the best of an incredibly challenging situation - drawing on learning gained from adapting provision - and maintaining hope that 2021 will see the return of some more provision for young people.
As we move onto the next survey, we are asking how prepared do you feel to continue offering provision for young people during the third lockdown? We recognise that this is about more than having a plan for delivery, with many other variables such as availability of staff and volunteers, local contexts, resourcing, and more.
We look forward to your responses and thank you, as always, for contributing to Just One Question.
Before we dig into this week’s results, here’s a reminder of some of the headlines from previous surveys that have focused on reaching and engaging young people with provision:
- In late May/early June, 58% of Just One Question respondents flagged ‘reaching young people’ as their biggest challenge in moving delivery online. From the most recent Data Standard in June and July, 80% of sampled organisations also reported that they are either already providing an online and digital offer, or that they were expecting to within the next six months.
- In July, 19% of respondents were seeking deeper insight into the young people in their local area who are facing the most risks, and 21% wanted a better understanding of how to reach them.
- In September, 43% of Just One Question respondents had found that moving provision online has not enabled them to reach or engage with new young people (or if it had, it was just a few).
- In October, 24% of respondents felt engagement from young people would be one of their biggest barriers to delivery in the weeks that followed. 27% also felt that evaluation should focus on whether we’re reaching the young people who need us most.
We are not reaching the young people who need our organisation's services the most
Now, in mid-November, 46% of survey respondents say that they are not currently reaching the young people who are most in need of their organisation’s services. Many additional comments described how current lockdown restrictions are influencing this - for example, one practitioner reports how not being able to run open access sessions means that there is no opportunity ‘for some young people to self-identify that they need additional support’. Another person identified the value of young people being able to ‘congregate’ as a group and provide support to one another - something else that cannot take place right now.
For those working in partnership with others, referrals and outreach work is also being limited:
'We aren't receiving as many referrals as usual because other services don't have the capacity to complete our referral form (they are focusing on crisis work) so there may be young people who aren't being identified / referred to us support due to the impact of COVID-19 on other services, e.g. schools and social care.'
A previous survey in early October highlighted some of these challenges, particularly when collaborating with schools (‘Are you/is your organisation collaborating more since the start of the pandemic?’ 02/10/20 - 09/10/20).
For those operating some kind of online provision, there are continued concerns about young people who are ‘digitally excluded’ being left behind, or about those who have found that digital engagement does not work for them, and are therefore ‘hiding away.’ One practitioner shared specific concerns about young people who are staying at home, despite it being a place where they do not feel safe.
'We are completely digital and I am confident that we are meeting the needs of young peope that we have established relations with but I have concerns that we are not meeting the needs of young people that we are trying to establish relations with.'
One person highlighted their frustration at the situation.
We are reaching them, but...
One practitioner noted that whilst they are reaching the young people who need their organisation’s services, they ‘don't feel that [they are] able to fully meet their needs due to the restrictions that [they] are currently working within.’ For example, existing members might only be engaging through social media or in different ways to how they might otherwise. Another person spoke about how whilst they are ‘seeing more young people than ever’ they ‘cannot be sure [they] are not missing people.’ Someone else added that whilst they are reaching these young people, there are still many more who need their organisation’s service.
Increased targeting means that we are reaching these young people - but not everyone
19% of this week’s survey respondents do feel that they are currently reaching the young people who need their organisation’s services the most. Several people described how their work has taken on a more refined focus, such as detached work, 1:1 provision, specifically supporting young carers, or focusing on ‘vulnerable group work.’ This does however mean that not all young people who might ‘normally’ be supported are able to access provision:
'But only in the sense that we are hitting the absolute minimum targets. There is no capacity for extras of any kind.'
One practitioner also noted that their provision is currently very targeted as they can only operate in smaller groups - and that this meant that they are reaching those who need it most. However, they also added ‘at least those that we know who need our support the most!”
How do we know?
Related to this point, over a quarter of this week’s respondents (27%) are not sure whether or not they are currently reaching those young people most in need of their provision. The question this week was deliberately totally subjective. Another question we might ask is ‘how are you currently determining which young people need your organisation’s services the most?'
The situation right now is frustrating, as one respondent shared this week, uncertain, and highly changeable - back in August, 87% of surveyed practitioners expected to adapt their offer in some ways over the following months. Whilst ‘knowing’ and objective knowledge is particularly difficult right now, it’s still an important place to aspire to - particularly where it helps us to respond, adapt and reach out.
As well as trying to meet need and demand over the coming months, it will be a challenge to determine quite what this looks like: understanding the need for your provision, checking whether you’re reaching the young people you are most focused on, and/or measuring overall impact.
If you’re currently focusing on these questions, we’d love to hear more from you about how you are approaching this.
We'd also recommend taking a look at questions one and two of the Asking Good Questions framework.
Are you currently reaching the young people who need your organisation's services the most?
[13/11/20 - 19/11/20]
This week we heard from 63 survey respondents, who could each select one answer.