For a few months now we’ve been working on digital tools to help decision-makers to identify which solutions they can use – and at what cost – to improve the lives of people who are experiencing (or at risk of) homelessness. A current project, provisionally called the Evidence Tool, will give users easy access to the best available evidence and give them an indication of which ones have the greatest impact, based on the evidence from our Evidence and Gap Map for homelessness. The format of these tools has its roots in the UK’s What Works Centres – a network of organisations focused on promoting the use of evidence in social policy and public services. Following that model, we expect our tools to be of use to local authorities, policy makers and third-sector organisations who want to have greater impact against homelessness.

The power of prototypes

At the moment these tools are in the prototyping stage – full of placeholder content and a little rough around the edges. But rather than working on these prototypes behind closed doors and releasing them when they’re finished, we’ve committed to sharing them with potential users as early as possible, even in their unrefined state. There’s several reasons for this approach. A tool needs to be used to be useful, so it’s important that we understand how our tool will be used in people’s day-to-day work. Bringing prototypes into the conversation makes it more concrete and helps people picture the tools more clearly in their own context. It’s also an opportunity to draw on the expertise of those working in homelessness. Our prototypes are a tangible expression of our current thinking – so sharing them with others and inviting expert feedback is a quick way to identify where we’re on the right track and where our own blind spots might be.

For our first official prototype feedback session, we held an open demo day in Edinburgh – a format borrowed from the tech sector where early-stage startups share demos of their products to get feedback. Over the course of the day, we invited small groups of decision-makers from government, health and third-sector organisations to give us feedback on our tool prototypes. After an introduction to our Centre for Homelessness impact and its ambitions, we opened the floor up to a discussion around how decisions are made in the attendee’s organisations. It will come as no surprise that these decision-making processes are rarely straightforward! Finally, we shared our latest prototypes of the tools – blown up to poster size so that we could capture feedback on specific parts of the tool on post-it notes.  

We asked whether the tools felt like they would be useful, whether they were meeting an unmet need and which parts felt most exciting. We also asked whether we were missing anything and what could be changed or improved. We received some fantastic input – ranging from advice on the high level framing of the tools, to new feature suggestions, to small but important phrasing changes. Now that we’ve had some time to let it all sink in, here are our top learnings from the day:

Beware the 'Tripadvisor Effect'

One of the most useful things about an Evidence Tool like ours is that it can give decision-makers a quick and easily digestible overview of their options. Most of the attendees agreed that this would be a valuable tool to have at their disposal. However, many also pointed out that we must be careful not to over-simplify an inherently complex issue like homelessness. While they provide a much needed overview, features like a simple 1-5 rating for impact can’t possibly capture the nuances of the interventions they represent (described by one attendee as ‘the Tripadvisor Effect’!). This is something we’ve also heard from the What Works centres through their own development processes.

Lesson

We’ll need to make sure the nuances of these interventions are fairly represented without getting too bogged down in the complexity of the challenge.

Known unknowns

The content of the Evidence Tool will evolve over time but the first version will be based on the studies found in our Evidence Gap Map. This means it will reflect what we know about homelessness through the high quality studies that have already been conducted around the world. Perhaps unsurprisingly, these studies mainly evaluate emergency interventions in housing and health. While understanding the impact of these interventions is critical to addressing homelessness, many of our attendees also pointed out the importance of preventative interventions and representing the structural and systemic factors that influence homelessness. Unfortunately, reliable evidence simply doesn’t exist for many of these interventions at the moment.

Lesson

These conversations highlighted the importance of showing what we don’t know, as well as the pressing need to fill the gaps in our knowledge.

Evidence isn’t a silver bullet

Evidence is a crucial part of understanding what works but it’s not the only tool at our disposal. Even though features like population or location filters allow users to find evidence that is as relevant to them as possible, it can’t possibly provide all the answers they need. An Evidence Tool like ours should help users to effectively narrow down their options and prioritise their resources, but any intervention will always need to be adjusted to suit local contexts, client needs and to fit the resources of the organisation delivering it. Our attendees understood this, but asked us to consider other avenues alongside this kind of evidence.

Lesson

Reflect on how evidence fits alongside other factors in their decision making, like local expertise and the voices of service users and beneficiaries.

Thanks to everyone who attended and gave their valuable input. This will be an ongoing part of our process so keep your eyes peeled for more events and opportunities to get involved!