We all know how difficult it is to launch something new. The journey from good idea to releasing a product or service into the wild can be long and tricky – even without the curveball of a global pandemic to reckon with…!
That’s exactly the situation that Bríd Brosnan, Innovation Officer at The British Red Cross, found herself in when starting her role in April 2021. The team had an excellent new initiative called Leaps & Grounds in development when the first lockdown hit the UK and presented an entirely unique set of constraints.
I was lucky enough to catch up with Bríd after coming across a blog post she had written about the project on the team’s Digital and Innovation blog. We chatted about how to navigate a pandemic and push forward with user testing, how to play validation bingo, and how sometimes constraints can also present opportunities.
Watch now
Watch the whole conversation here, or read the transcript and some highlights below.
Transcription
Nick
Hello, and welcome to Byte. I’m Nick Lockey. I’m a strategist here at 383 Project, and today I am joined by Bríd Brosnan, who is Innovation Officer at the British Red Cross.
We’re going to be talking about user testing and validation and overcoming all the challenges that anybody who works on a product has experienced when trying to figure out the right direction to get going in from the start.
If you work in digital products, you’ll know that assumptions often make an ass out of you and me, and the roadmap that you’ve got planned as well.
To get back on track, you need to understand what your users want and what they’re trying to do, and that means going out and testing some assumptions. Invariably that can take time, effort, and money – all the things that are the enemy of getting results fast, getting the results right, and building a real sense of confidence that you’re doing the right thing.
Today we’re talking to Bríd about how her team try to overcome some of those challenges at the Red Cross, about how they approach innovation, and particularly about one of the projects they’ve got on the go at the moment, a really fantastic initiative called ‘Leaps & Grounds’.
We’re also going to be learning how to play validation bingo, which sounds really fun… I’m not sure whether you win anything, like you do in an old folk’s home. Hopefully you win a really great solid roadmap, and an idea of where you’re going!
Bríd, thank you so much for joining us today. Shall we kick off by finding out a bit more about who you are and what you do?
Bríd
Yeah! I’m Bríd and I work as an innovation officer at the British Red Cross.
I’m part of our central innovation hub, which was set up about two and a half years ago, which is designed to help drive innovation across the Red Cross. We look at how we can lead the charge on some of our more disruptive projects, how we can connect innovation efforts happening across the organisation, and we also investigate what we call ‘big bets’, which are longer term projects that we think could be a step change for the Red Cross.
I joined in April 2021, and before that I spent a couple of years working in delivery for a tech startup in Vancouver called AvenueHQ.
Nick
What are the differences you’ve found between the tech start-up and the charity world, in terms of the approach to innovation and the receptiveness to what you’re trying to do?
Bríd
I think a lot of it is really similar, actually.
We’ve got a design approach that we use at the British Red Cross. You try and do a really good discovery and understand exactly what the problem is that you’re trying to solve, and define what success looks like. You come up with as many ideas as you possibly can. You try and validate those and choose the ones that are the most effective, and then prototype and test those in real life before starting to scale up.
I think the difference for me has been having so many different sets of users when you work on things at the British Red Cross.
Leaps & Grounds is a great example. We’re trying to design something that can support and provide better opportunities for people who have been through the refugee process, or the refugee experience, to move into employment. But we’re also then trying to build something that can be sustainable, that customers will want to support, so it has to work for them, too. And then, as a volunteer-led organisation, we’ve got a third set of users that we’re really trying to design something great for as well. That’s one big difference.
The other big difference for me is that, obviously, Leaps & Grounds is not a digital product. You’re trying to apply the same techniques of rapid prototyping, testing as often as you possibly can and learning and iterating quickly, but for something that is real and out there and in the world. The way that we looked at what we could test before we put something in the real world, before we actually put a coffee cart in our office, became a really important part of making sure that we were doing our due diligence and validating all of our assumptions.
"The way that we looked at what we could test before we put something in the real world became a really important part of making sure that we were doing our due diligence and validating all of our assumptions."
Bríd Brosnan, Innovation Officer, Red Cross
Nick
That’s a brilliant segue into Leaps & Grounds. You mentioned coffee carts – for people who might not be familiar with the project, can you give us a 101 on Leaps & Grounds and how it came about?
Bríd
It started in early 2020, before I joined the Red Cross, and it came from an ideas challenge that the innovation team ran across the whole organisation.
One of the winning ideas was based on the experience of one of our refugee service managers in Wales. The question that the innovation team took on to test and prototype in the course of twelve weeks was, how might we create a service that could support refugees into work?
From there, the team did a really brilliant discovery. They had to pause a couple of times and were re-deployed due to Covid, but they did a really great discovery and some really great co-creation with people with lived experience, with service managers who were supporting at the front line.
Together they formulated multiple different ideas, and the one we felt had the most potential was Leaps & Grounds, which is a social enterprise coffee cart.
The idea is that through flexible paid shifts on a coffee cart, refugee women can gain work experience, they can build up their confidence, and they can have a chance to practise their professional English language skills.
Equally, it’s an opportunity for us as the Red Cross to explore a new way to engage supporters. The idea is that it will be a self sustaining, or potentially even a revenue generating, service over time.
That’s where I stepped in. The project had been paused multiple times, but we had this really brilliant idea, and we had a huge amount of validation done as we’d developed the idea and selected it as the one that was worth prototyping.
I hadn’t been part of all of the discoveries and all of the conversations as we’d gone along, but there was this really great idea with loads of assumptions built in. My first challenge was to work out, is this as good as it looks? How do we start to understand what assumptions we’ve made? And how do we start to test if they’re true or not?
Nick
What’s interesting there is, you’ve come on to a project that already had some traction behind it, and then obviously there were a lot of challenges in terms of delivering something, especially real world, when the pandemic came along.
So, first of all, how do you onboard to the project? How hard was it for you to get your head around everything that had been done before and get up to speed with the research and the insights?
And then, what difference did the pandemic make to the plan in terms of testing this?
Bríd
Part of the digital data and technology directorate across the British Red Cross is that we work in the open. We have regular show and tells, we blog regularly, and we try to be as transparent and really embody that principle of working in the open as best as we can.
That was invaluable to me when I was onboarding. I think I spent the first two weeks just watching the show and tells for this project. It was particularly valuable because you could see how it evolved. You weren’t just looking at a finished piece. You were seeing, oh look, this is what the guys were doing in week two, and this is what they were doing in week six, and this is what they were doing in week eight, and this is how they went through these things.
You got to see what kind of questions people asked, which again is really beneficial. What kind of worries were coming up when they were presenting to different stakeholders? It gave me as nuanced an understanding, I think, of the project as I could get without being there.
I hadn’t thought about that element of the value of working in the open before. I’d never joined somewhere where you’re starting brand new. I think a lot of the time when you work in this way, as you guys do at 383 and as a lot of people do across innovation and digital product design, you expect to be part of every conversation and to be part of every piece of the journey, and it’s not always possible. I think that history of the project being public and being accessible meant that I could actually get as close to possible to the decisions that were made along the way.
"My first challenge was to work out, is this as good as it looks? How do we start to understand what assumptions we've made? And how do we start to test if they're true or not?"
Bríd Brosnan, Innovation Officer, Red Cross
In terms of how the pandemic changed the course of the project, originally this was designed to be a 12 week, really defined piece of work. We set out in early 2020 to investigate if we could answer this fantastic question of, can we design a service that can support refugees into employment?
The idea was that in 12 weeks, there would be three or four weeks of discovery, a couple of weeks of ideation, and then there would be time to do a full test – a couple of weeks pilot at a location. Obviously, we didn’t know what product we were going to come up with at that point, but, we would have time to fully test the idea within that 12 weeks.
We were a few weeks in when the first lockdown happened in March 2020. The whole team was redeployed to support the British Red Cross effort, providing relief to a completely changed society as a result of the pandemic.
A bit later in 2020, my colleagues came back together and did a second discovery on how the employment picture may have changed as a result of the huge change that we all experienced in this time. They refined the Leaps & Grounds idea, and they decided this was the one we wanted to test in real life. They were just ready to do a one week pilot when the October lockdown happened.
And so, again, different people left the team and moved on to other opportunities. We felt that it wasn’t the right time, obviously, in lockdown 2.0 to start to test something like this in real life. It wouldn’t have been a fair test. And also it was a time where the guidance was ‘work from home’, ‘stay at home’, so it wasn’t the time to open something that brought people together and had people in close proximity.
The project snoozed, then, until I joined in April 2021. It’s been snoozed multiple times since we originally hoped to pilot in June 2020, and then restrictions didn’t lift as quickly as we thought. But actually, that has proven to be a really interesting constraint, because if we hadn’t had those delays that were forced by circumstances, I’m not sure that we would have done the amount of pre-testing that we’ve been doing since then.
That’s where I was able to employ a lot of the things that we might have used in a technology context to test the assumptions that we had built into this model.
A great example of that is that when we looked at our ideal version of this product, we hoped that it would be something that, like everything else that we do at the British Red Cross, could be volunteer-led – with absolutely no idea whether that was something that volunteers would be interested in!
Ideally, we wanted to attract volunteers who had barista skills already because that would create a really great environment on the coffee cart, if you’ve got someone who’s an expert, and they’re able to support someone who is quite new to the skill. And also we knew that that would be really great for consumers, right? You’re guaranteed a really good cup of coffee. We had no idea if it was true or not. We just wanted it!
So, when I joined and after I’d tried to understand exactly where we were, we mapped all of these assumptions against what was feasible, what was viable, what was desirable, and what we assumed would be impactful.
Some of the things we knew a lot about already, because my colleagues had followed a user-led, evidence based process. We knew that this would solve problems, and we knew that it was something that was of interest to users, because we would have discounted this idea much, much earlier if those things haven’t been proven true.
It was more the nuances of the idea and the more detailed assumptions that we had much less evidence about, but that actually would be hugely impactful. For example, it’s hugely impactful, for what this idea is and how this would grow, if volunteers are interested in being part of it.
Because we couldn’t test in real life straight away ,we looked at how we could run tests really quickly, really cheaply, and validate our assumptions so that we could either pivot or move on.
For the volunteer element, I had had a lot of conversations with different people across the coffee industry to try and get up to speed on what it was like to work in the industry and upscale the understanding that my colleagues had already developed.
One of the things we found is that most people who are experienced baristas look for jobs on a site called Coffee Jobs Board. So, we ran an ad on Coffee Jobs Board for a month – I think it cost us £15. And we said, okay, we’re going to run it as a blind ad. We’re not going to say that it’s us as the Red Cross. We’re just going to run an ad for this project and put down some requirements. I think we asked if volunteers could commit maybe four hours a week and explained the idea.
We set ourselves up some confidence intervals. So if no one applied, we knew that we were on to a loser, and we’d need to change the model and think about if it was still a feasible or viable idea if volunteers weren’t interested.
If we got up to five applications, we agreed that the ideas was fairly solid, because, to be honest with you, the ad looked a bit dodgy! There was no information about the company or who was going to be delivering this volunteer role. We thought if five people were interested at this stage, we could probably bet that with better branding and more information – with an actual real ad that didn’t look dodgy – we could leverage that.
Obviously, if we got more applications than that, we were 100% sure, and actually the question would then be can we go further? Can we ask for more experience? Can we refine this even further?
We ran the ad for two weeks and we got five applicants. For us, that was enough. It felt like, yes, our number’s come up. We’ve defined the test. We’ve set ourselves some fair parameters for results. We’ve met that.
Now the question is, can we create a journey that works for this new type of volunteer? Can we make it as lightweight, as simple, as easy as possible? We did some work to get that set up, and we did some work to get the role approved internally, and now it’s live. We’re recruiting volunteers, and again we’re using Coffee Jobs Board, but we’ve also got a more formal and legitimate role live on our website. We can see if anyone applies for this one. And if they do, brilliant and if they don’t, we’ve learned.
The next test, I suppose, in this part of the journey is, when people have applied, do they do the training that they need to do? Do they sign up for any shifts? Do they actually turn up?
All of this is designed to learn. If we’re wrong, we’re wrong. At least we know before we’ve committed too much to the idea. We have time to change the plan. We have time to adapt and to learn and to develop something that does work.
Nick
It’s that classic lean startup approach – picking the assumptions that are the most critical to the product and then figuring out how you’re going to test them, how you’re going to have confidence in the results of those, and then moving on to the next big thing that you need to test.
We talked at the start about a technique you mentioned in your post about the project on the Red Cross digital and innovation blog called ‘validation bingo’. Can you talk a little bit about the specific tools and techniques you used to weigh up the most critical challenges?
Bríd
Yeah. So we used a couple of different matrices when we were getting started to map all of our assumptions.
As I mentioned before, first we mapped them against what was related to feasibility, what was related to viability, what was related to desirability, and what was related to impact. We took some of that modelling from the Testing Business Ideas book, which is really a good book. These were things that we would be doing anyway, but I’ve really found the canvases useful to force you to make sure you have every single assumption down.
From there, we looked at taking those assumptions and plotting them on an evidence versus impact graph. What were the things that were the most impactful that we had the least evidence for? Those became the first things that we tested.
The idea of playing validation bingo with these is that, much like a bingo card, you set out, these are the tests we need to do. These are the critical ones. And all of these things, or some of these things, need to come up for us to be able to get bingo and move on. If they don’t come up, then we have to accept that this model isn’t the one, and something needs to change.
Setting up all of those tests in advance forces you to be evidence led. I think all of us who work in this field, we want to be evidence led and we want to be user led. But it’s also really easy to get caught up in your own idea and to think, well, this is brilliant and it has to work, and so much work has gone into it, so it should work.
We use the test cards to agree that this proves it’s really true, this proves it’s only okay, this proves it’s absolutely rubbish. Setting up all of those and then seeing what happens with them and how the bingo numbers came up, allows you to force yourself to be evidence led and to feel like, actually, if it fails, it fails. It’s just the numbers didn’t come up.
Obviously you want everything just to succeed all the time, and so far, we’ve been pretty well validated in what we’ve explored. But if something doesn’t come up, you then have to assess whether the fact that the other things did come up means you can pivot, and it’s just a bit different than what you expected, or actually this is the central to the idea and without it, it can’t go on.
We’re gearing up now to try and run a real life pilot. The things that we’re testing in advance are determining how the pilot should look and what we should really be focusing on testing at that point, because there are things that you can’t test in advance. We can’t test if we can make you a cup of coffee until we have to make a cup of coffee. We can’t test how many people are going to come to the coffee cart until it’s there.
But what we can do is put out our designs for the coffee cart and see which are most attractive to consumers. We can run those smoke screen volunteering ads and see if people sign up for them. Testing as many of those things in advance gives us the confidence that we’re designing the right test for this four week pilot.
They also help us to, I suppose, feel more validated as we move through the project, and each one helps us to gain more momentum and really believe more that this is a good idea and something that we should do.
Nick
One of the things that struck me from your blog was around your outcomes.
We’re big fans of outcomes over outputs at 383 – what’s the result we want to get rather than the methods. The outcome that you wanted around volunteering is to have somebody with expertise of making a great cup of coffee on hand to support refugee women on the cart. But what you’re trying to figure out is, do we find baristas and make them volunteers or do we find volunteers and train them up to be baristas? It will get you to the same outcome, but both of those require thinking about and narrowing down a little bit, which I thought was a really interesting take on it.
Was there anything in particular that caught you out or required a learning curve in the transition to testing and designing a real world piece of service design, as opposed to software design?
Bríd
Actually at the moment, I’m going through a learning process. So, I like to work in sprints. On a Friday I’ll decide what I’m going to do on that Monday. I don’t love a big detailed project plan. I prefer to go where the evidence leads me and know what we could do next, but not necessarily say it has to happen in this order, or it has to happen at these times.
You do have say that if you want to put a coffee cart somewhere. You have to think about orders and timelines and deadlines, and at what point do I need to have made this decision so that all of the stuff can actually get there in time for us to open on this date?
We’ve got a really good team and that’s helped, because I think if I was doing this on my own, I might have let that run a bit longer, and not necessarily have paused and been like, actually, are we going to have enough time to make the flyers that we want to make? And how long will it take to print them? And how long will it take to do all of those really practical things?
I suppose, coming from a digital background, you can do anything you want in the time you have. You might have to work late. But, especially if it’s not a big build, you can almost always, for example, build something with a third party software that can happen instantly.
That’s not the case when you need to put cups on a table or get beans in a bean hopper, or find a coffee machine. You have to think a little bit more about the deadlines and the dates that something that needs to happen by.
So far, I think, in terms of our assumptions, we haven’t been too badly caught out. It might be coming, but not yet!
Nick
The Red Cross sound really forward thinking within the charity space. When it comes to that culture of innovation, I suppose the motivations might be different than just trying to sell something to an end user. There’s obviously all sorts of things to consider within the charity model.
To what extent are you dealing with cultural and mind shift change when you’re trying these experiments? Is it quite a receptive environment?
Bríd
I think it depends on what you’re trying to do to be honest.
You’re right, the charity sector is definitely a different space, and the appetite for risk can often be different because you’re in a position where you’re trying to deliver outcomes for people in crisis, right? Everything that we’re trying to do at the Red Cross is trying to deliver great outcomes for people in crisis. Our job in innovation is to push for something that could be a step change, and be really responsible in de-risking that before we put too much resource behind it.
I think what I’ve actually enjoyed most about working on this project is working with people across all of the different teams. I’m dealing with fantastic people in our property portfolio, who’ve really helped us to think about the best places to put the cart. Or I’m working really closely with health and safety, with people from our volunteering teams, with our branding teams. Everyone has been really receptive, actually, to this idea and helping it to become a reality.
As a wider team, our goal is to transform the way the organisation thinks about innovation and thinks about innovative ideas, so that we can become redundant. Our Head of Innovation, Jess Ferguson, wrote a really good blog on this recently, about how far we’ve gotten at this point, and where we think we are in terms of our maturity, and where we might want to go next.
We’ve definitely found that doing things together with different parts of the organisation to create something new is a really great approach to fostering this sense of innovation. Trying those things together with different parts of the organisation is definitely how we plan to move forwards.
Nick
There’s that classic bit of copy from the advertising industry – “Give a man a fish and he’ll eat for a day, teach a man to fish and he’ll eat for a lifetime.” I guess that’s kind of what you guys are trying to do? Rather than be the team that executes for everybody else, it’s about being supportive, being evangelists for the process, and getting everybody to think in that way.
I find it really interesting that the redundancy of your function as a department is almost an ideal!
Bríd
Yeah, literally! The strategy for the innovation hub when it was set up was to be redundant in ten years; to come in and be a central place while we needed one, but ultimately to not need one.
I think it’s something that is growing across the sector, as well. You see a lot more charities thinking about what innovation looks like for them and thinking about how to leverage the capabilities of digital on the whole.
There’s really interesting things happening across the Red Cross on this. Our digital and innovation blog area is interesting if anyone wants to keep up with the kinds of things that we are doing.
Nick
Fantastic. So, in terms of Leaps & Grounds, where next? What’s the next step with the project?
Bríd
We’re still aiming for a four week pilot, hopefully, around the end of September. A lot of the work that we’re doing now is gearing up to that.
In terms of the tests that we’re running, we’re in phase two of testing that volunteer journey to see if people who were interested on paper are interested in reality. Will they actually come along?
A big part of what we’re doing, also, is double checking that the ideal outcome in terms of the impact for our service users is going to be valid. That’s something that we really can’t be sure of until we give this experience on the cart for a few weeks and see what happens next.
We’re aiming to be open from our Red Cross UK office between late September and early October.
Nick
If you had to do things differently, based on what you’ve done now, what would be the biggest change that you would make to your approach?
Bríd
I think we’ve struggled because of restrictions. In hindsight, some of the things that we have tested, you could argue whether they were necessary to test this early on. We had the time to test in advance, so we did, and we couldn’t necessarily move to a real life version due to the pandemic.
I think if we were to have done things differently, we might consider moving ahead earlier and thinking about more domestic locations. Would it have been better to try and pop up in a neighbourhood if people were all working from home and validate some things sooner?
Or would it have been better to run more simple tests? Did we need to bring in the volunteering element at this point? I would argue yes, it’s really useful, but could we have skipped it and kept our idea simpler, and then maybe moved more quickly?
That’s another big thing, isn’t it, with this kind of way of working? As we learn things, we always think about things that we would have done, or could have done, differently. But all we have is what we have, and what we have done, and then what we do with that next.
"If we're wrong, we're wrong. At least we know before we've committed too much to the idea. We have time to change the plan. We have time to adapt and to learn and to develop something that does work."
Bríd Brosnan, Innovation Officer, Red Cross
Nick
As a product person and an innovation professional, what’s the best piece of advice that you could share with product teams who want to make a difference with the products they’re making?
Bríd
Being user led – not in the tools and the practise, but in what people are actually telling you and showing you.
A lot of the time it’s still the hardest thing to do, even though we all want to do it. One of the most difficult things is to say, well, I’ve designed this and it should work, but it doesn’t. And accepting that it doesn’t, and that I have to listen to why it doesn’t and what needs to change from it.
I always try to treat it like a game, as much as possible, because it makes it easier to say, oh, I didn’t get this one. But it’s fine, I can change it and do the next one. It makes it less difficult to be open to change.
And change is the only constant, right? As you develop any product, the only thing that you can be sure of is that you’re going to have to change something about it almost all of the time.
Try to find a way to let yourself be led by what people are actually telling you, rather than what you want them to tell you, is the best lesson that I’ve definitely learnt, and have been trying to implement.
Nick
Fantastic. Well, I’m off to design my own validation bingo card after this and have a bit of fun with that.
Thank you so much for today Bríd. That has been a fascinating chat, some really good insights, and what a fantastic initiative as well. We wish you all the best with it, and we’re looking forward to seeing how Leaps & Grounds turns out for you.
That’s it for today’s Byte session. We’ll have more sessions coming up in the future, so keep an eye on our website.
Thanks for your time. See you on the next episode.
