Leveraging Data for a 360º View of Your Users

Hey everybody, this is Mike Belcedo, co-founder of Product Collective, and welcome to another industry interview. Good to have you all with us wherever you're coming from in the world. I am personally in Cleveland, Ohio, as always, and it's actually like sunny and 88 degrees here today. So I think it's like, this is like vacation land around here. At least that's— I could trick myself into thinking that. But anyway, I thank you for being a part of this industry interview today. If this is your first time, you know, I'll kind of start with a little bit of housekeeping on our end just to share what these are all about and, and give you some information on that. But first, this is one of the initiatives of Product Collective. So Product Collective is a community for product people, and we've been doing this for a few years. We're product people ourselves. Really started this whole thing because we felt like we were lost as product people and always wanted to learn. And I remember, you know, thinking back, taking my first product role, I was reading books and blogs and listening to podcasts and, you know, always asking myself like, am I even doing this right? And the short of it is we created Product Collective, you know, kind of for our own personal benefit so that we could answer that question. And what ended up happening in the next few years is something we couldn't have expected. This is now our product. These industry interviews, but also our newsletter that we put out every week, a podcast that we co-host, or that I co-host, Rocketship FM, and our conference, Industry: The Product Conference, which we put on twice a year in North America and Europe. Although this fall, we will— that's sort of good news, bad news. The bad news is we won't be in person in North America, in Cleveland, which is where we usually have it. But the good news is we are still having it. It's just the virtual edition of Industry. And actually, we announced earlier this week that one little special fun thing is we'll be having Ben Folds perform a virtual set for all of the attendees at Industry Virtual, which we're excited about that. But there, there are 20 other product leaders running sessions. There's video networking. There's, you know, you could visit the exhibits. It's It's the same kind of fun you'd have at a conference, it's just now you could do it right from home and it's a fraction of the cost too. So you could check all that out at industryconference.com/virtual or just go to industryconference.com, you'll find it. But these industry interviews, we've been running these, gosh, I should do the count. We've probably done close to 100 of these, maybe more. But these have been a lot of fun. Basically we have conversations with product leaders from all around the world and, you know, we focus on a different topic every single time. I am able to start off with questions of my own, but the— what I love about this is this gives you a chance to ask questions of our product leader, our guest, as well. So while I have some questions, and, and I'll be getting to my questions, sort of kick things off, I'd really love it if you would ask questions in the Q&A section. You'll, you'll see the, the chat and the Q&A section at the bottom. And, and the chat, everybody's using it. We're learning where people are tuning in from. I'm seeing Naperville, Illinois, Pittsburgh, Pennsylvania, not far from Cleveland. Atlanta, Georgia. Toronto, Canada. So, thank you for that. Yeah, keep sharing where you're from. It's always fun seeing where folks are tuning in from. You'll notice though, there's also a Q&A section and that's a perfect place to ask questions that you have for our guests. And you could start loading that up at any time, even right now. If there's a question that is on your mind that you want to make sure that I get to, I might take some questions kind of in between my own. I might take them at the end, but regardless, that's the place to sort of load up with questions, that Q&A section. I'll be monitoring that. But so yeah, with, with all of that, I'd love to sort of get into the topic of today. I'd love to introduce you to our guest as well, who is Ross Walker. And Ross is on the team at Mixpanel, and Ross is not in Cleveland, Ohio. Ross, welcome. Maybe you could share with everybody where you're tuning in from. Yeah.

Thanks, Mike. Yeah. So I'm tuning in from just outside London and we have a nice bit of sunshine in the breaks of rain that we have over the past couple of weeks. So yeah, it's great to be here.

Well, it's always nice. I know these are weird times these days, but anytime the sun is shining, at least kind of, kind of can forget about it for a little bit. At least, you know, when you walk outside your own home, you could look up and see the sun and it can feel warm. And at least that part's nice. But, uh, I'm glad the sun is shining for you, Ross. But you know, you're— I was excited to have this conversation with you because I know the topic that we're covering, which is, you know, getting this 360-degree view of our customers through analytics. It's one that I think interests people in that we all want to get a better view of our customers. And we all know that data is really important. But it could be true, at least I know from my own personal experiences, sometimes I'm not using it the right way, or I realize later on I'm not using it the right way, or I could be harnessing that data in a different way. So anyway, I thought that this was a great chance to sort of dig in with you, and maybe just to sort of start things off, um, you know, maybe you could just give your take on what is, what is the benefit of analytics as product people? Like, what is— we're all using them in some way, but like, what are we really supposed to be getting out of it? Maybe we could just kind of start there.

Yeah, sure. So I guess I'm lucky enough to work with a number of different industries, a number of different companies. So I get to take this kind of broad view of what are people leveraging analytics for. And I think that the number one reason tends to ultimately be that the ability to understand their user base, that user behavior at scale, and ultimately try and eliminate any of that bias that they may have. And I guess I put in bias because I guess ultimately we see kind of two types of data, both quantitative and qualitative. So I think that whether you're taking user actions, product data from a product analytics tool, or whether you're leveraging something like surveys and customer feedback, I think that ultimately it's important to rely on that data that is the most reliable to try and remove as much bias as possible. And ultimately understand, hey, what am I— what is not only what is a fraction of my user base doing, but how is that kind of scaling over time and using that information, I guess, as product people to take insights from that and make decisions that you, you know, are going to have an impact. Right. And when you release those features, when you make a marketing decision, how do you know that actually those decisions that you've made are moving the needle and moving the needle in the right direction? The right way, right? So being able to have a wealth of data that you can trust and a wealth of data that you rely on to inform those decisions and ultimately measure the outcome, I think is something that is super important to be able to get out of an analytics strategy.

I appreciate that. And I know like from my own experience in the past, you know, I've worked on products where we were just starting out. So there really wasn't a whole lot of data to access. But then I've also worked on products where, there were hundreds of thousands of, you know, users, so many data points. And I remember feeling, because I just jumped from a startup to that kind of role, and I remember feeling a little overwhelmed. You know, there's so much data that I could harness. That overwhelming feeling, I feel like I've heard that from other people too, other product people. Like, how do you sort of mitigate that? How can we make sure we're focusing on the right pieces of data when we are in a situation like that and there's so much data to pull?

Sure, and I think that this is something that's going to differ depending on your product, depending on your industry, but I think it's important and it all starts with setting out that analytics strategy. And we talk about, and there's a lot of frameworks out there, we spend a lot of time at Mixpanel talking about a product analytics framework that we leverage, but it's really around identifying a focus metric. And that focus metric is something that is not solely relied on, but it's a KPI, it's a performance indicator that is slightly slightly more important than everything else that you have. And it's something that should ultimately, if that number goes up or if that number goes down, should ultimately reflect the performance of your platform. So I think that kind of honing in on your focus metric, defining that, and then using that to build out an analytic strategy of kind of level 1 metrics that feed into that, level 2 metrics that feed into those, is something that is super important, but it's important to make sure that you're not, as you say, kind of drowning in data, right? Drowning in this knowledge and not quite sure where to turn. I think it's very important to have a very defined product analytics strategy that ultimately links into the key performance and the key KPIs of the organization, the KPIs of the platform, right? Otherwise, there's going to be a lot of information that ultimately you don't care about, that ultimately creates a lot of noise. In you trying to get insights out of, out of the data that you've got.

Yeah. So, and when it comes to setting that strategy in place, cause I, I like with anything, whether it's analytics or kind of anything we're doing as product people, I, I'm a big believer that we should have a strategy in place and follow that strategy. For people that find themselves in a place where maybe that, maybe there wasn't thought given to setting that analytics strategy. So they kind of have to set one or at least be a part of the process of setting one. What's most important to sort of setting an analytics strategy and where do you start? I mean, what's that? What do those beginning steps sort of look like?

Yeah, I think it's, first of all, I think it's a great question, right? And I think that like, first of all, it's, and I kind of come back to what I spoke about, it's kind of finding that one single metric I think that is going to help you kind of gain focus on what you want to get out of that. I think once you've got that, it's going to help you kind of help to define some of those other metrics that ultimately lead up into this focus metric. Typically, we'd say that that's something that's going to align with kind of usage, maybe it's weekly active users, kind of aligned to some form of value moment. When I look at kind of maybe some form of finance customers that we work with, we may look at transactions per week, transactions per month or something along those lines. But I think it's kind of defining that and that's going to help you understand, right, well, what else do we need to be tracking it is going to move that focus metric. But I also think to add on top of that, there is— we talk about this concept that you probably hear a lot around the democratization of data, right, and making sure that a lot of different people can start to leverage the data that you've got. And I think that when that plays a part, there is all these different areas within the business that ultimately want to have— want to leverage this analytics. So I think it's important to understand, right, what are your key performance indicators for all of your different business units? What other tools do you have in your tech stack that are kind of collecting a wealth of useful information? And how can all of that information come together to ultimately give you answers to the questions that you want, that are going to drive that analytics strategy, that they're going to drive innovation in the, in the market, right? I think that one of the big things we see, certainly at Mixpanel and the people that I speak to, is kind of that linking of product Analytics to attribution data to marketing data to data warehouses, right, to get that data into a BI team, for example. So I think these are all things that need to be thought about when you kind of sit down and when you start to define this strategy. And I think that, but first and foremost, I think it's important to start and hone in on that focus metric.

Yeah, I love that. And you know, there, you mentioned before, like there are other stakeholders that, you know, they need this data, they're using it in some ways too. And it does make me think about, you know, we're product people, we might, you know, set this strategy in place, but realizing that there are other stakeholders, like what does that process look like to involve them? Like who all should be involved in that process and how does that work? I mean, because there's sometimes that could be, I mean, I guess one could say like, gosh, almost everybody in the organization, you know, data is going to be important to them. So how do you, how do you go about that?

I think it's certainly an interesting point. Right. And I think that when I typically have these conversations with organizations, it tends to be at that product level. We tend to have marketing. Sometimes they have like dedicated data teams, but that tends to be the typical area that we would have these conversations with. But then more and more, I start to see these other areas of the business kind of start to to have a say in these conversations. And I think one of them typically that I've worked recently is media organizations where actually potentially editors have wanted to have a say, right? Because they care about the data that's being pushed into these platforms that allows them to get answers from. So as much, I don't think there is kind of a, unfortunately I don't think there's this definitive answer, right? These are the people that you should have in this conversation. But I do think that getting as much input from these different areas of the business as possible. It's something that's going to ultimately come together and give product people the ability to define an analytics strategy that takes everybody into account and ultimately enables everybody to be able to leverage the data that exists in the toolings that they have.

Well, I appreciate that. And so, okay, going back to thinking about how we can get started with setting this analytics strategy. Again, you're in a unique, unique role because you've worked in several different organizations, but Mixpanel, I mean, Mixpanel is like one of those companies that when people think of analytics, like Mixpanel comes up, comes to mind. I imagine that you've seen people go about it the wrong way. I imagine you've seen some mistakes made, and maybe you could talk about some of those mistakes. Maybe there's many, but I think it will help us sort of, you know, keep in mind what not to do. Maybe there are things that we could stay away from.

Sure.

Yeah.

And I think like it's kind of at the moment slightly kind of mixed panel, panel aside, I think one of the important things, right, is all around tooling. Right. And making sure that people are enabled and people have the right tooling to be able to get answers to the questions. Right. And it's one thing collecting this wealth of data, but I think not having the right tools, not having the right skill sets to be able to get access to the real-time data is something that kind of we see time and time again. And being a part of Mixpanel, we're able to come in there very easily and quickly and help kind of increase adoption of people leveraging data and getting the answer to their questions. But I think moving into how, when people have that tooling up and running, I think there are certain steps people can take to make sure that the data that exists there is certainly more usable by everybody in the organization. And I think it comes down to things like naming schemas and making sure that everybody has got a good understanding of what does this event do. Is there a description associated with that event? Is it named something that makes the most sense? So actually anybody that comes into the tools that they have available actually understand the question that they're asking or the event that they're looking at that users are triggering making sure that they're actually running that query on the right thing. Right. So I think kind of aside of the things that we've spoken about in kind of having, collecting a lot of data, but actually making sure that data is, is trustable, is reliable, and actually is usable in the tools that are given to product managers, to marketers, to the whole organization. Right. So, and I think that very much comes back into like defining of that analytics strategy, but actually it's making sure that you define a kind of a, productive naming convention when you do that.

Yeah, well, thank you. Just a reminder, everybody, if you have questions for Ross, I definitely want to make sure I can get to them. And I always love favoring audience questions over mine because they're usually better. So just load them up in the Q&A section. And if you don't want to sit back and listen, totally cool too, because I've got my share of questions for Ross. But just sort of as a reminder of that. So Ross, I mentioned, you know, kind of what mistakes you've seen, and you started to get into a little bit of like, okay, here, but the right way, you know, right way to do it is to, you know, have that one metric in place to involve the right stakeholders. What are other sort of best practices that maybe we, maybe they're not, maybe it's not conventional wisdom, maybe it's things that people don't often think about? I guess when we think about doing it the right way, what do you think it's important for product people to keep in mind?

Yeah, I think that like one of the things that I've kind of seen kind of around some of those people that like, I guess, live and breathe data, live and breathe kind of analytics, it's that they're always kind of thinking about tracking and analytics almost first, right? When they're building and they're shipping features, analytics and tracking is always going to be a part of that and it's not going to be an afterthought. I think that a lot of the time people are releasing features and maybe they, they build in that analytics later, right? They building those tracking later. So actually making sure that when you're doing those releases, that the tracking element of it is actually thought about almost as part of the feature, right? Because it's the way that ultimately as product people, you're going to be able to measure the impact on those releases and those changes that you do, right? If you release a feature and you don't have tracking embedded in that, How are you going to know if that ultimately is going to have an impact on those focus metrics or those key KPIs that we've kind of set out in that strategy? So I find that kind of you release this new feature, you want to get tracking built into that as early as possible so you can get a good understanding of like first and foremost, how those early adopters are starting to interact with your features. If it's moving the needle like you expect it to do and if you need to make any changes, you've got that data to rely on. To actually make a more informed decision.

Awesome. You know, one of the things that always comes to mind for me is like, and this is again, not necessarily analytics specific, but anytime we're setting a strategy in place, you have to review it. And sometimes there gets to be a point where you have to change course. I'm curious when it comes to setting an analytics strategy, like how, how do you kind of keep that continually monitored? And like, when do you know when you need to change course and maybe either change that strategy change the way that we're trying to accomplish that overall strategy, like changing the tactics? Yeah, what does that look like from your perspective?

Yeah, sure. And I think like it's, it's, it's something that is going to be ever ongoing, right? And I absolutely don't think it needs to be this static Mix panel, right? So that's the idea that you've kind of always got to be leveraging data, always got to be feeding that back around and back into. So if you release a new feature, you want to track the data around that, you want to feed that back into your analytics and you want to use that to make more informed decisions. So I think certainly the thing is to just make sure that as your key performance indicators are changing, as your metrics are changing, that that is absolutely reflected in, whether it's something as simple as a shared document, but you need to make sure that everybody has a view.

Ross, you are freezing up on me. So I'm going to just wait till you're— oh, here you are. I don't know if it was me or you or whatever, but now you're back with me now.

So I'm going to cut off there.

No, it was great. I mean, you started to talk about how, you know, again, just in terms of being able to like change course, having that sort of shared document, that shared vision with employees. I don't know if there's more that you wanted to add to that, but I thought that was a great start. And I think we might be frozen up again. Again, not knowing if it's me or on your end, Ross, but let me know when you can hear me okay, and I'm gonna— I have another question to feed to you if that's all right. Maybe in the chat, let me know if you can all see Ross or— oh, I think you're back.

Here we go. I apologize about this.

That's okay. No, totally okay.

At this point, totally okay. But I think hopefully what I was trying to get right is that all the changes that you make, make sure that that's ultimately being fed back into the analytics strategy and kind of going around in these circles so you can always use the changes that you've made as more informed data to make changes going forward. Going forward.

Got it. Love that. You know, one, one hot topic that's kind of come up, and again, not, not that this is new necessarily, but it does feel like this last year or two it's just becoming more and more prevalent, is this role and function of product operations. And I know oftentimes product operations, you know, when it comes to the data, like, that's the team that sort of ends up owning it. I'm just curious your take on Product operations, we've done sessions, we've done reports on it, but I'm just curious, you know, when you think of product ops and product people, like how, how, how are you seeing that sort of role evolve and come into more importance?

Yeah, I think that like time and time again what I see is that like, I guess we call it product analytics, but I think it's becoming more and more cross-functional, right? So like more and more there's more data that everybody can leverage. And I think that kind of when I think about product operations, when I think about, I guess, who is leveraging this data, I think something I spoke about earlier is kind of how is your product analytics tool fitting into your overall tech stack, right? And I think that that's something that's been massively important for, especially as at Mixpanel, we have a very kind of large product— sorry, partnerships team that works on integrations with Mixpanel in the tech stack, but actually making sure that there is this overarching view of actually how is product analytics informing marketing, how is marketing informing product analytics, how are we feeding this into our BI teams is something that I've kind of started to see happen a lot more in the coming months. And actually a product analytics tool like, I'm talking about Mixpanel a lot because I work for Mixpanel, but it's how product analytics can actually help not only inform product, but how it can link into your A/B testing and to ultimately help have that single holistic view across the product landscape and across the user landscape. And that tends to be managed at that kind of like operations level at some point.

Yeah, awesome. I'm seeing a couple audience questions, so I want to make sure I can get to these. Thank you for sending them my way. The one question, I'm going to abbreviate this a bit, but basically this person They're on a startup, it's in a closed beta, there's 50 to 100 people, and they started a sort of a tracking plan, you know, with data points they want to keep tracking, but they're up to like 60 data points, and this person's a little bit worried that maybe they're tracking too many things, especially with this few people. What's your take on when you're setting that initial sort of strategy in place, like how much data is too much data to track? Like what, what advice might you give to this person?

Sure, yeah, and I guess I think that there is not this kind of magic number of saying, hey, you need to track this many events and if you go over this, it's wrong for you and it's wrong for your organization. But what I will say is to ultimately make sure you are tracking metrics that matter, right? And I think that it's very easy to come in here and say, well, everything that my user does matters, right? And I think that you kind of can go back and forward in that a little bit. I think the one thing that I will probably point you in the direction of and something that's available on the Mixpanel website is a report that we produce called A Guide to Product Metrics. And actually, A Guide to Product Metrics is very good at taking you through that process of actually how do I define my events, how do I define my metrics. And it's great at helping you understand actually is this metric that I'm tracking here is that something that ultimately feeds up into my KPIs, into my metrics that we care about as an organization? And if it doesn't, then ultimately I think there's an argument to be saying, actually, should we be tracking that? So I think that, yeah, to kind of answer your question a little bit, what I'm going to do is I'm going to point you at this report that we produced. And I think that will be a good kind of basis for helping you to understand, is what I'm tracking actually meaningful and actually does it matter to me as an organization.

Yeah, that's great. And you know what? After this session later today, I'll be posting the video recording to the Product Collective YouTube channel and I'll be emailing everybody a link to that video. But maybe I could go and try to find that guide to product metrics and I can include that right in there as well. It sounds like that'd be helpful to a lot of people for sure. Another audience question. Somebody had asked about, okay, when setting a product analytics strategy, Like what are the prerequisites? Like are there, are there things that sort of have to be in place for you to set that strategy? Maybe. Yeah, I don't know what that might be, but I figured I'd ask you the question here.

Yeah, it's good. And I guess thinking about it, I don't think we have this kind of checklist, right, of like here's everything you need to do before you're kind of ready for an analytics strategy. I think that I'd say personally that kind of that's something that as soon as you can get that defined, as soon as you can set that up, the more information it's going to give you to helping you make decisions going forward. The quicker you can get some of these metrics in that matter, the quicker you can start to measure at least high-level metrics about your user behavior, it's going to help you understand, well, actually, maybe we're missing this little bit of information. Maybe I wanted to get an answer to this, but actually I don't have this property or this event about my users. So I think that as much as there's not really any prerequisites, I would say the quicker you can start to gather this data about your user base, about your features, the more you're going to be able to kind of come back and feed back into that, right? And start to grow your analytics strategy, start to grow the data that you've got, start to grow the answers to the questions that matter to you, and then ultimately grow your platform.

Well, that's awesome. I know time's kind of flying by. We're towards the end of the conversation here. So maybe I just kind of to finish things up, Ross, just kind of ask you, like for people that, you know, they're tuning in and they're different stages. Some we've heard from startups, some might large organizations, what's the one thing you'd want to make sure that somebody took out of this conversation? And maybe also to add to that, if you could, I know Mixpanel has a whole host of resources that might be great for people, maybe you could reference those too.

Sure, yeah, I think like just looking at the Q&A, I can see a question around like an example template and things along those lines. I think just to touch on that quickly and say that again, on the Mixpanel website, we have some great examples of analytics strategy documents that can really help you start to define that. And again, maybe I can share them with you, Mike, and we can, we can share those out to the audience. But I think the one thing that kind of to take away from this, I think it's that making sure that you are tracking the metrics that matter the most to you. Right. And I think that that can happen in a number of ways. That can happen directly from your customers, from tracking user behavior. It can happen through integrations with other tools in the tech stack. It can happen through qualitative data as well, but ultimately being able to understand what those metrics that matter the most to you and then build your analytics strategy around that so you can get the answers that matter the most to the core metrics that you have in a quick time and you're not kind of digging around trying to find these, these, these metrics. They're there, they're in front of you and they're usable by the whole organization.

Yeah. That's great. Well, I think that's a, that's a great final takeaway. And yeah, what I'll do is when I email everybody the recording, I'm going to include some of those resources, definitely that guide to product metrics, and hopefully that'll be helpful to everybody. Well, Ross, I just want to thank you for, for being a part of this. This is great conversation for sure. And yeah, so thanks a lot for being a part of this.

Perfect. And thank you, Mike, and thank you guys for your time.

Yeah. And thank everybody for being here. I appreciate you all taking the time. I know especially now that the sun is out there. Probably better to— some of us, you know, we could go on a walk and enjoy maybe a quick walk around the block, or we could keep learning about products. So I appreciate that y'all have, uh, chosen to keep learning about product. Um, we do plenty more of these coming up. We actually do these twice a month, so I'll point to some of the other ones coming up, and hopefully I'll see y'all again soon. So thanks, Ross, and thanks everybody. Bye now.

Thank you, guys.