Lessons from the Listening Fund (Part 3): Designing a diagnostic tool for organisational listening

Over the past two years, the Centre for Youth Impact has been working with the funders and youth organisations involved in the Listening Fund, a collaborative effort to support organisations to listen and respond to young people – and make this the norm. The Centre is evaluating both the England and Scotland Fund, and this week is reporting on the findings from England. This blog by Sarah Williams and Jo Hickman Dunne, who are part of the Centre for Youth Impact’s research team, reflects on the wider value of the self-assessment tool. ​

This is part of a series of blogs exploring the findings from the project. The rest of the series will appear here.

Developing a diagnostic tool
At its heart, the Listening Fund is designed to alter organisational behaviours, and this is no easy task. It is often assumed that organisations that work with and for young people are very good at listening and responding to youth voice across all aspects of their work in a structured and intentional way. The Listening Fund challenges this assumption by posing the question: are youth sector organisations always good at doing this?

The overarching aim of the Listening Fund evaluation was to understand the impact of dedicated funder support on organisational listening practices. In order to do this, we needed to understand where funded organisations (‘partners’) felt they were at in terms of their listening practices, and their (self) perceived progression across the life of the Fund. We chose to do this through a diagnostic tool – the partner self-assessment. The self-assessment was designed to explore different elements of how partners listen and respond to young people, and, as far as we are aware, this is the first of its kind. The self-assessment takes a broad conception of organisational listening, drawing on Jim Macnamara’s ‘Architecture of Listening’ framework, which adopts the following definition: 

“Organisational listening is comprised of the culture, policies, structure, processes, resources, skills, technologies and practices applied by an organisation to give recognition, acknowledgement, attention, interpretation, understanding, consideration, and response to its stakeholders and publics.” (Macnamara, 2015)

There are 27 questions, related to the above definition and adapted for the context of working with young people. It includes questions relating to the following domains: 

• Listening practice – e.g. what forms of listening does your organisation undertake with young people?
• Culture – e.g. does your organisation have an organisational listening policy?
• Skills – e.g. is listening explicitly included in the role description of any staff/volunteers?
• Resources – e.g. does your organisation use any types of technology to support its listening?
• Communication – e.g. to what extent do you communicate what you have heard back to young people?
• Acting on what is heard – e.g. to what extent does your organisation act on what it has heard? 

The tool was tested and refined before being used as part of the Listening Fund evaluation. The self-assessment is completed by partners at the beginning, mid-point and end of the Fund, and was designed both to aid the partners’ reflection on their listening, as well as enabling the evaluation team to identify change in the cohort’s self-perceptions and behaviour over time.

Using the self-assessment tool
Feedback from Listening Fund partners suggests that the tool has been of great value beyond the external evaluation. Some limitations have been highlighted, including the complex wording of the questions and the time taken to complete. The overall feedback from partners, however, is that the survey is meaningful as an assessment of organisational listening practice and useful as a reflective tool to plan improvements. Some felt that the self-assessment had given new insights: as one partner put it: 

“by completing this survey it has prompted some thinking around what we do not currently do.  And it has given us some ideas of what we need to do”. 

The mid-point self-assessment was viewed as a timely opportunity to recognise and celebrate progress, and look ahead to the next stages of individual projects.

The self-assessment tool and process of using it do not merely present an opportunity for affirmation, but challenges organisations to understand where they are doing well and where they may be less successful in their organisational listening practices. It indicates some standards of performance around how high-quality listening should be conducted within youth sector organisations, as well as allowing organisations to compare their progress over time and generate their own recommendations for areas that need improving. We are proud of this tool, and the progress that organisations made through using it. We believe that this tool and the process of using it can continue to add value to the youth sector beyond the Listening Fund. 

We would welcome the uptake of this diagnostic tool to support ongoing internal evaluation processes and the self-assessment is publicly available for any organisation across the youth sector to use to reflect on and analyse their own listening practice. It can be found on the Centre for Youth Impact website and the Listening Fund website, alongside other supporting resources.

More detail about the evaluation findings can be found in the full learning report, and in the accompanying case study report that takes an in-depth look at six partners’ projects.

[1]The listening Fund (England) is funded by The Blagrave Trust, Comic Relief, the National Lottery Community Fund and the Esmée Fairbairn Foundation.
[2] Macnamara, J. (2015). Creating an “architecture of listening” in organizations. UTS, Sydney.
[3] www.thelisteningfund.org/resources-for-partners