Last year we worked with the Mayor’s Office for Policing and Crime (MOPAC) and Transport for London to evaluate a Women’s Public Spaces Safety Programme. This involved two evaluations – an impact evaluation of the Women’s Night Safety Charter expansion programme, and process evaluation of a safety audit programme led by members of the community who were trained as researchers.

This would be our biggest project to date and on a topic we all really cared about, so we were thrilled to be working on it and excited to get started.

We carried out traditional methods including surveys and interviews as well as more creative approaches, and learnt a few things along the way. And as we’ve just been shortlisted as finalists for the MRS/ICG Award for Independent Consultants (not to brag or anything…), we thought we’d share a little about our experience!

1. The power of participants’ own voices

As women’s safety is clearly an emotive subject, we really wanted to do the evaluations justice and ensure we put participant’s voices front and centre.

For the process evaluation in particular, we wanted to empower community researchers to share their experiences in their own words. The safety audit programme itself used innovative methods including participatory mapping, multi-sensory walks and ‘on the go’ checklist-style assessments for community researchers to conduct audits in their local area, and we felt it was important to use similarly creative approaches in our evaluation to explore their effectiveness and capture learning.

We’ve now used digital ethnography platform Field Notes for a few other projects, and it provided the perfect way for community researchers to respond to questions and activities through sharing voice notes, images and video to illustrate their experiences of the process.

This approach gave participants control over their narratives and encouraged honest feedback and reflections. To amplify participants’ voices, we also embedded voice notes into our reporting. This meant stakeholders could directly hear the voices of community researchers, bringing findings to life in a powerful way.

The impact of including participants’ voices in this way went beyond what can be achieved through written quotes alone, and this is something we’ve since used in other projects and received positive reactions from clients, particularly when presenting ‘live’. Hearing comments like “I’ve never seen that done before!” definitely gave us a little buzz.

2. Conducting two evaluations gave us a unique insight into delivery

We were evaluating two distinct, separately run initiatives which both fed into MOPAC’s overarching women’s safety programme. Whilst the safety audit programme was community-led and focused on process and experience, the Women’s Night Safety Charter evaluation was more impact-focused and involved engaging with stakeholders and businesses across London.

This meant a lot of evaluation activity, and a LOT of insights including survey data from charter signatories and members of the public, crime statistics, stakeholder and participant interviews, reflections from observations, and digital ethnography outputs.

We produced two separate evaluation reports, but our work also put us in a unique position of being able to identify shared learning and opportunities across the two programmes.

We triangulated data and looked at the programme holistically to see where there might be common challenges and opportunities to strengthen process and impact. This helped to inform a unified picture of what was working and ‘ingredients for success’, which included local leadership, partnership working, and embedding lived experience into delivery and decision-making. We also identified opportunities for closer integration of the two initiatives, for example enabling signatories of the night safety charter to use audit-style tools to assess safety in their own venues.

3. There is strength in being a small team

There are undoubtedly times when the odds are stacked against the little guys, but for this project being a small team felt like a real advantage.

Building relationships was really key to engaging successfully both with stakeholders and community researchers. We were able to quickly embed ourselves into the projects, attending researcher training sessions and workshops to observe and engage, becoming familiar faces to those we needed to involve in the evaluation. This was particularly important with delivery partners, who can sometimes be a bit apprehensive about being evaluated.

We like to think our relaxed, personable approach helped to put stakeholders at ease and encourage them to feel comfortable opening up to us.

We were also able to pivot at a moment’s notice, which came in to play during fieldwork when the train I was on to meet a participant in London ground to a halt – cue panic! Maura was already in London doing another interview and was on route home to get back for a telephone interview, so we quickly did a switcheroo with Maura jumping on the tube to meet my participant, and me aborting mission and heading home to Essex to do the telephone call instead. Phew…crisis averted! And as both participants had met us at a previous observation day, they were happy with the last-minute change.

We’ve noticed that this kind of responsiveness and personal touch can be difficult to achieve in larger teams, and seemed to be a real strength in the delivery of this project.

4. Talking on stage is less scary when you know what you’re talking about

At the end of the project, we presented our evaluation findings in a Q&A style discussion with MOPAC at the annual Women’s Night Safety Charter Summit, an event bringing together signatories and partners for discussions, panel debates and partnership opportunities.

Whilst the thought of an audience of over 100 people made me want to run and hide, and waiting for our turn on stage had me anxiously jangling my leg up and down, in the end it went pretty well…

We were able to prep the questions in advance, and I knew deep down that we knew the findings and what we wanted to communicate inside out. So once the Q&A got started, I felt strangely calm and in control.

Our session was well received and we had some insightful chats with attendees afterwards on our stall, with people keen to hear more. It was great to put ourselves out there, and also gave us a good excuse to use our pull-up banner.

And now…

Our evaluations have helped shape improvements to future safety audits, and informed strategies to embed inclusive, community-led approaches to make London a safer place for women and gender-diverse people.

And now our work has been shortlisted for the MRS/ICG Award for Independent Consultants, we couldn’t be more chuffed! The MRS Awards (aka what we have been referring to friends and family outside of the industry as ‘the Oscars of the research world’…) will take place on 1st December, so watch this space!