How to become a data-driven UX designer

Have you ever wondered why it feels like some of your favorite apps know you so well and they can preempt what your next action will be? Like how Spotify curates the perfect playlist for your Saturday morning run, or how Netflix recommends the perfect late night movie. Well, sometimes. The answer to that is data-driven design. Without data, we're at the risk of blindly assuming what our user needs are, and that could result in a frustrating experience for the user and a slow business growth. My name is Samuel Fabayo, and in today's video, going to be sharing a practical guide to developing a data-driven mindset as a UX designer, including how to run experiments and a free glossary of key user and product metrics that you should get familiar with. Let's get started. The first and most important question that you need to ask before you start solving a problem or before starting a new project is, how do we measure success? This is the first step to developing a data-driven mindset. Because you can't improve what you don't measure. The answer to this question needs to be clear before you start designing a solution in any problem space. More importantly, you need to have the answer before you ship that solution. So even if you are not clear on the answer before you start designing the solution to the problem, you need to have a clear answer of how you're going to measure success before you actually ship that solution to users. One thing that I'd like designers to do more of is to ask this question of their teams and of their product managers. So whenever you're presented with a product requirement document that has all the specifications around what you want to improve, around the problem that needs to be solved, around how that is going to impact users, and you see that one thing that is missing from that document is how to measure success, then you need to ask that question. That is going to immediately transform you from a designer who is just pushing pixels around to a designer who is focused on the outcome for the business and who is focused on how solving that problem can push the metric. For the business. So it is a very, very transformational question that you can ask. This could get really confusing because there are so many ways to measure success. There are so many metrics to keep track of. And that brings us to the next point, which is to understand the key user and product metrics. I wouldn't want anyone to start studying all the possible metrics that you can evaluate a product by like it's some certification exam. This is why I've put together a glossary of the key user and product metrics that you should get familiar with. You can grab that in the link that I've added in the description. Now, having understood these metrics, it is easier for you to know which metric to focus on depending on the user behaviors that you are seeing with your product or the kind of problem that you are trying to solve. While there are so many metrics that you can measure, most businesses usually have a North Star metric. A North Star metric is that one metric that captures the core value of a product to its customers. It keeps everybody aligned on the same goal and usually results in more revenue. for the business. For example, Spotify's North Star metric is the amount of time spent listening, the average amount of time spent listening. And Amazon's North Star metric is the amount of purchases per month per user. These are other examples of like North Star metrics across other companies as well. While companies might have their single North Star metric, each of the teams within those companies could have metrics that ladder up to the North Star metric to make sure that everybody's aligned on the same goal. Now, Whenever you're working on a new feature or on a new product, after you've understood how to measure success, the next thing that you need to ensure is to understand how that metric that helps you to measure success ladders up to the North Star metric. To be more data-driven as a designer, you need to understand the North Star metric that your product is focused on and also understand how your work impacts that North Star metric. That brings us to the next point, which is to learn how to run experiments. Experimenting helps to understand what works for you and what's great for the business so that you can double down on that. Referring to the Spotify example I gave at the beginning, when Spotify creates a playlist for you and the data showed them that you've listened to that playlist more than anything else in your library, then they would recommend more songs on that playlist for you and other people that have the same listening habits as you do. That way, they've been able to launch an experiment to understand what you enjoy, what you like, and they see that that has helped you to impact their own not that metric, then they ship that experience to you and other people that are in the same user group as you are. To run an experiment, the first thing you need to do is to develop a hypothesis, then develop a solution based on that hypothesis, and then test that solution to confirm if your hypothesis is true or false. Here's a video from Nate, a growth designer at Monzo, explaining what a hypothesis is.

So what is a hypothesis? Well, a hypothesis is a testable statement based on insight you already have. The insight could be assumption, an educated assumption, or it could be stuff that you've already got at hand, such as user research data or some market trend analysis.

And here's a simple structure that you can follow to develop a hypothesis. Number 1, start with what you know. For example, something that you've been able to identify from research. Then secondly, state what you want to change. And then thirdly, state the action that you believe that change will result in. For example, let's say you've been able to identify that some Spotify users have requested for audiobooks on the app, that they want to listen to audiobooks on the app, and you want to increase the number of freemium subscribers to paid subscribers. So what you can do to develop a hypothesis using this structure that I've provided would be something like, we know that 10% of our freemium users want to listen to audiobooks on the app. So if we enable freemium users to listen to 2 bestseller audiobooks for free and inform them that there are more audiobooks for paid subscribers, We'll see an 8% increase in subscription. Then you create a prototype to test this hypothesis and compare with some users that didn't have any change in experience in the app to see if your hypothesis is true or false. When setting up an experiment, it's also important to have some guardrail metrics in place. Guardrail metrics are critical business metrics that you want to monitor for a negative change while conducting your experiment. For example, using the same hypothesis that we've set, a guardrail metric that you want to put in place might be you want to ensure that this is not affecting the listening time for those subscribers that are converting. So while you're setting up an experiment to convert people from free subscribers to paid subscribers, you also want to be checking if the listening time of those people that are converting is not dropping. To set up a guardrail experiment and to make decisions, first of all, you start the experiment, then you start to monitor the experiment to see if there's any change. If the guardrail metric is triggered, that means the listening time is dropping significantly. Significantly. Then you try to understand why it is dropping. You could meet with your stakeholders to understand if you should continue with that experiment or pause the experiment. Depending on the outcome of that session, then you can decide to proceed with the experiment or pause the experiment. This is how experimenting helps you to meet user needs and achieve business goals. The next thing you can do to be more data-driven as a designer is to stay close to data. And what do I mean by staying close to data? I mean that you should not completely rely on a data analyst or a data engineer in your team to pull data for you. You should understand how you can self-serve data because this will help you to be more confident about presenting your design rationale using data-driven techniques, right? And when you have to rely on a data analyst or a data engineer for everything, you're wasting your own time and you're also wasting the time of that person. My whole point is that relying on a data analyst for simple things like checking the daily active users or checking the monthly active users or checking the churn rate is time wasting for you and for data analysts that you might have in your team. These are things that you need to be able to understand how to do for yourself by being more comfortable in a data environment. And how can you do this? All you need to do is to find sessions or one-to-ones with your data analyst or data engineer to help you understand the data environment that you are working with, and you can use that to immediately start to understand how you can self-serve data. For example, depending on the tools that you use to pull data in your company, it could be Looker, it could be Mixpanel, it could be Google Analytics, or whatever data tool it might be. There are beginner or entry-level knowledge that you can gain to be able to get simple things like weekly active users, churn rates, retention rates, engagement rates, and all these simple things. And then when you need to do more complex things or when you need more complex data, then you can introduce a data analyst or a data engineer to help you with that. This way you are closer to the data and you can immediately include data in your design rationale while presenting your design or while speaking in front of your stakeholders. And this makes you to appear more confident as a designer, which is going to help you grow really fast. Lastly, I understand that there are so many designers watching this video right now working in companies where data-driven design is not prioritized, and you're probably wondering, how do I even apply all of this when the company I work for doesn't care about data as much? Well, um, I see this as the opportunity for you to start advocating for data in your company. I got a comment on one of my previous videos about the same topic, and this was my response to the person, right? This is the opportunity for you to start advocating for data in your organization. You need to start looking for case studies that you can use to convince the company that data-driven design actually drives business outcomes. And when I say looking for case studies, you need to look for case studies that are relatable. I think the mistake that a lot of designers make is that they just look for case studies that are very ambiguous. For example, your company has 10,000 users and you're using an example of Airbnb that has, you know, 50 million users worldwide. You need to find more relatable examples. If the products that your company is involved in is selling cars online, for example, find other companies within that niche. Could be another car sales company, could be companies in your competitive landscape, or it could be a company that's in e-commerce. Find examples like that that your stakeholders can relate to. Also look for ways that you can Start with simple data analytics tools like Looker, Google Analytics, Mixpanel, and the likes. And if you still realize that the company is still not paying any attention to data-driven design, then that might be your cue to start looking for another job. If you stay in a company that doesn't value data-driven design, then you're not going to grow as a designer because the whole essence of design is to understand what user needs are by understanding and studying user behaviors and being able to profile solutions for that. And this is a proven way to meet user needs and also help businesses achieve their goals. If you've learned anything at all from this video, then you definitely find value in another video I made where I talk about how you can get ahead of 99% of UX designers starting today. I'll see you in that video, and if not, I'll see you in the next one.