The easy and effective way to run a usability test

Usability testing is one of the most ignored steps of the design process, yet it is so crucial to the success of a product and the satisfaction of its users. There's not much out there on YouTube about this topic, so I thought, why not fix that? That's why in this video, together we'll conduct an unmoderated usability test for this swim coaching app. But before we get into the video, we need to play a quick game. All you need to do is, based on the prototypes shown, tell me which of these is more likely to convince a user to subscribe. If you think it's prototype A, just type prototype A in the comment section. And if you think it's Prototype B, type Prototype B in the comment section and explain why as well. By the end of the video, you'll know if you're right or wrong. If you're new here, I'm Samuel Fabayo and I'm a Senior Product Designer at Monzo, a leading digital bank in the UK. Before that, I was a Senior Product Designer at Deliveroo, a food delivery company that's well known in the UK and a few countries across Europe. These design-led companies use the same usability testing process I'll be walking you through in this video, so they're as real-world and practical as they can be. We'll cover everything from planning to setting up a test to launching a test and analyzing user feedback. I've included chapters for each of these steps so you can jump to any of the chapters if you want to jump ahead. The first thing you need to do when running usability tests is to plan your test. This usually involves writing 3 to 5 clear research objectives for your test, writing out the qualifying criteria for the participants you want to recruit, and writing your test details. The test details usually include a summary that explains what your test is about to participants, a welcome message, a set of instructions that you want the participants to follow, follow-up questions that you want to ask, And an optional thank you message. All of this will be needed when we get to the setting up phase of this video. You can do all of this in a Google Doc or a Notion page like I have done here. I like keeping the objectives to a minimum of 3 and a maximum of 5 so I'm not trying to learn too many things at the same time from a single test. For example, the 4 key objectives of this test is to identify any usability issues within the onboarding flow, evaluate ease of completing the onboarding flow, assess the clarity of the subscription page, and measure the task completion time. We'll test 2 variations of the app's onboarding flow. I've already designed and prototyped the app on Figma so we can jump straight into testing. The main difference between both variations is the subscription screen at the end of the flow, which I already showed you. And our main goal is to identify which subscription screen is clearer to users, and we'll be conducting the test with Lyssna. Lyssna is an all-in-one research platform where you can conduct user interviews, recruit participants, run usability tests, and even create surveys. This makes it the only platform you need to uncover insights about your users. Listener is used by design-led companies like Google, Figma, Miro, Asana, and many more companies. Thanks Listener for sponsoring this video. Now let's get into Listener to set up our test. So we're gonna head to listener.com, and once we get there, we just log in. So I'm in Listener now. First thing I'll do is to create a study. I want to create a test or survey, and we'll enter our test name. The name of our test is Swim Coaching App Test. And for the, for participation, where do we want our participants to take the test on? Do we want them to use all devices, mobile only, or desktop only? I would go with mobile only. Then we'll enable screener questions. Screener questions is one of the interesting things about Lyssna. You can set some questions that would help you to screen your participants. So in this particular case, I want participants that are involved in some kind of fitness or physical activity and people that have been using some kind of tracking app to track their physical activities. So I'm going to bring in some screener questions from the test plan that we already put together. First screener question is, do you engage in any regular sporting activity? And this is going to be a multiple choice question. The answer is going to be yes and no. For people that answer yes, I'm going to qualify them for the test. For people that answer no, I'm going to disqualify them. I can add another choice if I want to for the options, but this is just a yes or no question. I'll add another screener question, and that is going to be, which of these sporting activities do you engage in? I want to qualify people that engage in some kind of sporting activities where they need to track it, where they can use an app. So In that case, people that run use an app, people that cycle use an app to track that, people that hike use an app to track that, people that swim, of course, use an app to track that. The other ones are going to be disqualified. And then people that don't engage in any sporting activities at all are going to be disqualified as well. People that play tennis are going to be disqualified. So for the top screener question, it is, do you use any app to track your sporting activities? We want to actually do something interesting here. We are going to disqualify someone that already uses an app and we are going to qualify someone that doesn't, because now we want to get somebody that doesn't use an app to track their sporting activities. And we want to use that as a way to see how somebody that hasn't used— used an app to track their sporting activity would get to go through this app. And see what their feedback might be. People that have actually used an app to track their sporting activity might have some kind of bias and they might not be the best kind of participant that we want to get feedback from here. So we have the welcome screen, which is the next one. So we're going to customize the welcome screen. We already have the welcome screen message in the test plan as well, so we just copy that. We'll go to the next section, which is where we choose the type of test that we want to do. The type of test that I want to do is a prototype test. There are other types of tests here. You could do a 5-second test, you could do a design survey, you could do preference test where you just show participants 2 or 3 things and ask them which ones they prefer. What we want to do here is just go straight into a prototype test, and we are also going to tap the record here. Since it's mobile, we will hear their thoughts as they are speaking, and then we'll bring in the instructions that we have. The next thing is to bring in the Figma prototype. So I'm just gonna go and copy the link for the prototype, and I'll add the link. So now it's syncing. And the interesting thing about this is that it actually just syncs with Figma such that when you make any updates on your Figma file, it just updates here automatically. And now you also get to pick a goal screen. The goal screen is going to be the screen that confirms that the participant has been able to complete the test. So I'm just going to go through the prototype myself here so that I can pick the goal screen. So this is the exact flow that the participants are going to be going through. Which is what I'm going through myself right now as well. And this is going to be my goal screen. So I'm going to set this as the goal screen. So once that happens, once they get to the goal screen, a success message is going to be shown to them. Now we'll add our follow-up questions, which is, which is going to be the questions that will ask the participants after they are done with the test. We already have the follow-up questions in the test plan doc. You see why the test plan doc is such an essential doc, because You just go back there and you pick things and you just drop it here. So all of the follow-up questions have been added. These are the questions that will really help us to understand how we measure the research objectives that we've shared at the beginning. And then we'll go to the thank you screen as well. We already have our thank you message, so we're just gonna put that. Thanks for taking this test, and we'll paste in the thank you message. Now that we're done with filling in all the test details, what we want to do is we're going to go back, scroll back up here, and we're going to make a variation set from this test. And what this means is that we want to create a variant for this test so that we can have the second testing scenario. Remember that we want to test 2 different subscription flows. So we're really going to use the same test details. We're just going to change the Figma flow so that each participant that takes the test would go through those different flows. So we're going to click this now and make a variation set. We can call this Swim Coaching App Variants, and we create that variation set. Once we are done with that, we have the variation set now, and then we're going to copy the variation and we want to make it a variation in the Swim Coach Coaching ad variant. So we'll duplicate that. So we have the 2 variations now. I should have named it properly so I can differentiate it, but I think this is fine. What we want to do now is that we want to recruit. So like, as you can see here, it says this is waiting for recruitment and this is waiting for recruitment. So we'll go ahead to recruit. There are 2 options for recruitment. You can recruit from listeners research panel where they have lots of high quality panelists that can take the test. They have over 500,000 diverse people across different countries that will be able to take the test, and you can also filter those people by the kind of attributes or the kind of demographics that you're looking for. And you can also recruit with a listener link where you just set up a link and then you share that link with your external participants that you want to invite into the platform. So we go ahead now and we're going to reduce this to 6 participants because because we don't need more than 6 really for the first variant of the test. And then we're going to choose the demographic that I want to use. So I'm just going to breeze through this because it's a lot. You can choose through location, age, industry and occupation, financial products, shopping, technology, and just different kind of styles of the kind of people that you want to pick. So I've already recruited the participants that I want to, um, take this test. These are the demographics that I chose, just people that are from United Kingdom between ages 16 and 18 There's a lot more that you could choose here. I just chose like these 2 basic options first since we have the screener questions that is going to help us filter that down into people that are more likely to use our app. So it is currently recruiting, and what I want to do now is to go to the second variant and also recruit for that as well. So I'm just going to do the same thing that I did before. As you can see here now, it is recruiting participants for both tests. So we're going to wait for a couple of hours for them to take the test. So the responses are in now. We can now analyze the results of the test. Let's see the results for prototype A. So for prototype A, the average task completion rate is 2 minutes 20 seconds. It was filled by 6 participants, which is the same thing with second prototype. When asked the question, how easy was it to complete the onboarding flow? 6 of the participants all said it was very easy. When asked the question, how clear was the information on the subscription page? 4 of the participants think it is very clear, 1 of the participants thinks it is clear, and 1 of the participants is neutral. And that gives it a mean score of 4.5. When asked how confident they are that they understood the subscription options presented, 5 of the participants said they were very confident, and 1 of the participants said they were confident. And that has a mean score of 4.83. And then when asked how they would rate the overall experience of the onboarding process, 2 of the participants said they would rate it as excellent, and then 4 of the participants said they would rate it as very good. When asked the question, how likely are they to subscribe after going through this flow, 1 of the participants said they are very likely to subscribe, and then 4 of the participants said they are likely to subscribe. Then for prototype B, again, 6 participants. That had an average task completion rate of 2 minutes 32 seconds. Also, when asked the question how easy it was to complete the onboarding flow, 4 of the participants said it was very easy, 1 of the participants said it was easy, and 1 of the participants found it difficult. When asked how clear the information on the subscription page is, 3 of the participants found it very clear, 2 found it clear, and 1 Didn't understand it. And then when asked how confident they are that they understand the options presented for subscription, 4 of the participants are very confident, 1 is confident, and 1 is not confident. And, and then when asked how they rate the overall experience of the onboarding process, 3 of the participants rate it excellent, 2 rate it very good, and 1 of the participants rates it fair. And then when asked how they are likely to subscribe after going through the flow, half of the participants, that is 3, said they are very likely to subscribe. 1 of the participants said they are likely to subscribe, and 2 of the participants said they are not very likely to subscribe. So if you compare both prototypes on task completion rates, ease of use, how easy it is to understand the subscription options presented, how likely they are to subscribe, and every other thing that we asked them, prototype A is the winning prototype. So if we're going to ship this experience, prototype A will be the experience that will be shipped to users. I'd love to know in the comment section if you have any questions, and I'll be happy to respond to that. YouTube thinks this is the best video for you to watch next. Let's see if they're right. I'll see you in that video.