Just How Well Do Product Managers Know Their Customers? w/ Laura Klein, Product Leader and Author

Hey everybody, this is Mike Del Sito, co-founder of Product Collective, and we're back for another industry interview. Thank you very much for joining us. Hope you are having an awesome day. I am having an awesome day. The sun's shining in Cleveland, Ohio for once, so that's a good thing. Actually, it shines a lot in the summertime, so we're just getting to that point now though. Anyway, I, I'm glad that you're here with us. I'm glad that we're going to dig in on the topic that we have at hand today. We have an awesome guest lined up. For those of you that are tuning in for the first time, though, I just thought I'd give you a little lay of the land of what an industry interview is, what it's all about, and also a little bit about Product Collective. So these industry interviews, these are live video Q&A chats where you'll get a chance to interact with the guests that we have, and we have guests from usually all around the world, just product leaders of all types. Today we have Laura Klein as our guest, and I'm really excited to get to chat with Laura. But when you have questions, and really at any time, if you have a question that you have for Laura right now or just as we're talking, use the Ask a Question section. You'll see it. You should see it at the very bottom of the screen. All you do is open up that section and you could pop in your question there. We'll be monitoring that throughout and I have a lot of questions for Laura, but I'd much rather ask your questions. They're probably going to be better. So definitely do that at any time, including right now. And then the chat on the side, that's usually an awesome place to introduce yourself, share where you're from, maybe what products you're working on, that sort of thing. So feel free to do that. Actually, it'll, it'll help give me confidence that you can see and hear me if at least you say something in that chat right now. So just keep an eye on the chat as well. And then before we— well, I should mention at Product Collective, we have all sorts of other things too. We have a newsletter, goes out to about 20,000 people every week. We have a pretty active Slack community. There's about 8,000 product people in the Slack channel, and you could take part in all those things if you go to productcollective.com and you just sign up. It's free to do that. And then we'll send you an invite to the Slack channel. You'll start getting the newsletter invites to these chats. And all that. And then we do meet in person twice a year at Industry, the product conference. And I am wearing my Industry shirt. This was from Industry 2018. But in fact, I think I'm going to make this— this is now my normal Industry Interview outfit, just an Industry shirt from the past, all different ones. So we have our global event coming up in September, should have about 1,200 product people with us. Including Common, Jason Fried of Basecamp, all sorts of— April Dunford, she actually just released a book yesterday or today. So there's about 20 people that will be presenting all sorts of things. You could go to industryconference.com and learn a little bit more about that. I also want to, before we get started, thank Logi Analytics. Logi Analytics is the sponsor for this particular industry interview today. And they've been a longtime sponsor at our industry events, but you might see them in our newsletter. You might see them, you know, again, here are these chats. So thank you to Logi. You actually probably see a green button with a link that'll lead you to some case studies that Logi's put together in working with some of their clients. So they've worked with, at this point, hundreds of organizations, mostly, you know, large companies that think analytics is a big problem for them. But when they start working with Logi, it makes everything a little bit easier. Check out Logi Analytics. And thanks again to Logi for sponsoring this. But with all of that, I definitely want to get into the conversation because this is one where I'm definitely no expert at it, but I want to get better at this. And we're going to talk about, you know, just how well do we know our customers and how can we get to know our customers even better? And there's probably not a better guest for that than Laura Klein. Laura, how are you?

I'm doing well, thank you. How are you?

I'm doing great. I'm doing great. And I appreciate you joining us here.

And but this is—

I know this is a topic that I believe like you've written about. I know you've talked about it. So anyway, yeah, I'm excited to sort of, sort of get right in if that's cool.

Yeah, let's do it.

Well, one of the things I was curious about, I know, you know, you've been a product leader for some time. You have in the past done consulting and worked with companies and you've seen their problems firsthand. I'm curious, how well do companies generally think they know their customers versus how well do they actually know their customers? I'm curious if there's any sort of gap there or not.

No, no, never. No, they always, yeah, yeah, pretty much all the time. So both of those vary wildly. I have talked to clients who have known exactly who their customers were and have known that they did. What I will say is that they no matter how well they actually know their customer, I think they always overestimate how well they know their customer, if that makes sense. So there's always a gap. Generally what happens is, and this depends really a lot on whether you're talking about, for example, who are my current customers versus who might my customers be. If you're talking about in the sort of the company lifecycle, if you're talking about a brand new startup and, you know, oh, who, who is our customer going to be versus, you know, we've actually been around for several years and we have real customers that we talk to all the time. So the, the issues there differ quite a bit depending on all the factors, right? How, how long the company's been around, how long the product's been around, how much actual research they do, what kind of interaction they have just on a regular basis with their, their customers, whether they have, you know, a, a really solid specific core group of customers versus, you know, we're for everybody. So and, you know, just how much time they put into it.

When you, for those that you've talked with where there might be a significant gap, like what do you think the reason is? Like, is there, is there usually something common among them? Does it, does it just sort of vary widely?

So, I mean, the common thing is that the people who don't know anything about their users and, but who think they do are the ones who, they're also the ones who will spend a lot of time telling you that, oh, customers don't know what they want, so we don't have to talk to them. Tends to be the same people. Just to be clear, often customers don't know what they want. You still have to talk to them because you have to understand what their problems are and what they're doing. And honestly, people are pretty good at knowing what they want sort of generally. They're just not that great at designing products, which is fine. That's your job. So yeah, I think that the issue is really that people, they will, they will say things like, oh, we don't have time to do user research or Oh, you know, we don't talk to customers because we already know, or, oh, we're building the vision and they won't understand it. Okay, then they probably won't buy it either. All sorts of, all sorts of those, basically just excuses for not talking to people. The ones who do know, and I don't want to just rag on companies, the ones who actually do tend to know their customers well tend to be the ones who either go out and proactively do good user research, right? So that is one way of doing it. Also, customers, also places that have really good customer success teams that are proactively reaching out. This tends to be more on sort of B2B and enterprise. They're proactively reaching out and working with key customers. And, you know, places that have actual account managers that are going out and talking to people all the time. Even then, you do tend to, I would say overfocus again on existing customers, which isn't always who you want. I mean, you might know a ton about your existing customers and you might not know a thing about this other group of customers that you could also have if you just did a little bit of research.

So, no, that I, that's great. And well, I guess it makes me wonder like what is good user research? Cause I imagine there's some companies where they're like, you know what? We get that we don't know our customers maybe as well as we should. We want to get there though. Like we want to be better at this. Like what, how do they start? Like what are the things that they could do to sort of start to at least get into the mindset and practice of good user research?

Right. So this is, it's so tricky, 'cause this is a little bit like asking, you know, like what's good medicine? You know, it depends on what you got and what you need and what you'd like to change. So, and it, might vary again wildly. There are a ton of great books on how to do good user research, mine included, I think, but lots of them by actual full-time user researchers who are fantastic. And so I always sort of figure out what you need to know and what you don't know. Here are some general guidelines. Under— try to figure out what you need to learn about. So, for example, if you have a product and, you know, that's already out in the market and people are using it, a really good place to start is often with just usability tests if you're not doing them. And if you're not doing them, do them. Okay, I can't stress this enough. If you want to do them, try maybe a book by Steve Krug, you know, like Don't Make Me Think. There's lots of other good ones, but that's a great place to start. I always call usability tests sort of the gateway drug of user research because it's just very, I mean, I'm not gonna say it's easy. Nothing's easy. I don't wanna denigrate anybody's actual job. It's not easy to do well, but it's actually pretty easy to get started and just watching people use your product is going to help so much. Just start there, right? If you haven't done that and you have a product, start there. After that, or if you already do that, congratulations, well done. Or if you don't have a product that you can actually watch people use, that's fine. Then you're going to need to go into sort of a more what we call user research mode. There's usability testing where you're testing your product and there's user research where you are researching your users and you are trying to understand who they are, what their needs are, you know, what they do, what, what their sort of general problems and behaviors are. And that's a lot wider and bigger in terms of understanding your customer. So definitely I would say in those cases you might actually want to bring somebody in as a consultant to help you understand what are you trying to learn, what are the, what are the big gaps in your knowledge that you're not seeing. It's very hard to see those yourself, so I often recommend that people bring in a guide or consultant or a coach to kind of help them with that. Also, I just want to say, and I— this is always fun because this is always the part where I get lots of user researchers arguing with me, don't ignore quantitative data. You know, if you have— again, if you have a product and you have quantitative data, take it into consideration. It's super helpful. It will tell you exactly what is going on with your product. It will not tell you a damn thing about why that is going on. So you still need to do the research, but it'll Great thing to use to, you know, to bolster your research abilities. It's actually a great, it's kind of a fun way to figure out what you don't know. Because a lot of times I was working with a client fairly recently who were like, yeah, this thing happened, this, you know, we, we all of a sudden our conversion dropped off, dropped off a cliff. Oh, why'd that happen? We have no idea. Okay, well, that's a really good thing to design a research study to try to figure out. And that research study might involve things like going in and looking at more quantitative data and also talking to users and also watching people use your system. And, you know, so a lot of, a lot of different things there. Like I said, it's— there are a lot of ways to do this. You have to figure out what you need to know and then you need to design the research study around that.

Yeah, that's, that's great. That makes a lot of sense. I, I am curious, you know, I know on this video chat we probably have— it's probably the majority product managers. That's a, it's a lot of times that's who's tuning in. It's who we serve at Product Collective. But when I consider somebody as a product person, it's more than just the product manager. It might be, you know, to me, somebody that's in UX is absolutely a product person. Even, you know, as a designer, developer, they're product people. Who should, of product people, like, or even beyond product people, who should be involved in that research process? Like, is it, I have to imagine it's not just, it shouldn't be relegated to one role, but who, like, who should that all include?

So again, this is, and it's terrible 'cause I'm such a UX designer. Every question, every answer's going to start with, well, it depends. Let me tell you what it depends on. So first of all, there are, again, there's a bunch of pieces to research, right? There's the figuring out what you want to learn. There's the planning of the actual study. There's the running of the sessions. Again, if you're doing both quantitative and qualitative, there might be, you know, different kinds of work to be done there. There's the analysis and the synthesis of the data. And then there's the sort of coming up with the next steps and figuring out, okay, what does this mean for our roadmap? What does this mean for prioritization of what we're gonna fix? Different people are involved in different steps. Everybody is involved, should be involved at some point. I, I feel like a lot of companies divide it up very strictly. And if I say, oh no, the product manager should be running the sessions, then everybody who's got a user research team is going to go, well, that's not going to work here. And so here's, here's what you need to do. Somebody who knows how to do this well, hopefully, if you have someone, should be guiding the process and making sure it happens. Product managers, as we know, you know, you say like, oh, that's the product. What is the product management role? It's like 700 different things. All of which are different at every company. So, you know, your mileage may vary on any of this advice, but everybody is involved. That doesn't mean that everybody does every single thing together. I get a little annoyed at these companies that are like, oh, now we're all going to sit down and do the project plan together with all of the engineers. And I'm like, no, they, they don't want to do that most of the time and they have other things to do. Leave them alone. but involve them in reviewing the plan or involve the ones who want to be involved in reviewing the plan. Ask them for their help in, you know, what they need to learn. Ask them for their help in, you know, getting the data that you need. UX designers, if they're not also doing the research, sometimes they're, you know, just UX designers, you know, they're going to be really heavily involved with, you know, Maybe observing sessions so that they get some firsthand information. Definitely in the synthesis stage where they're trying, where you're trying to make sense of all of the information that you just got and 100% in the sort of what are the actual changes that we could make to the product that would help us fix these problems that we saw or serve these needs that we saw. And of course, the product manager is involved throughout the whole thing. Again, Maybe not doing every single bit, but definitely making sure that they're involved, they know what's happening, they're cool with what we're learning. And that, remember, at the end of it, they're the ones who have to generally have to make the decisions about, okay, which of these things do we do? And if you don't trust the process coming up to that, if you don't, if you haven't signed up, you don't, if you didn't watch the you know, the user research sessions or at least the videos, if you didn't get that feedback personally, how are you gonna figure out what, how to prioritize this stuff? I don't know is the answer. So yeah, everybody at different points that makes sense. And so that we all sort of understand what comes out of it and can say, great, this is what we learned. This is how we're changing our behavior based on it. This is how we're changing our product. These are our priorities. Let's go do that.

Yeah. Well, and I like how, you know, cause it's, yes, we, it probably is common sense to a lot of people that, well, hey, as many people should be involved in the process as possible, but I like that, you know, if we think about it, it doesn't mean that everybody's doing the same thing or going through the exact same process. Yeah.

Everybody doesn't need the same level of involvement by any means. I do think it's great if everybody on the team can watch some of the user research, whether it's actual whole sessions or, you know, if you have somebody who can cut it up into, you know, if you've done video of it, if you can cut it up into the important bits, you know, I mean, it's all important, you know, but, but also there are important bits.

Yeah, for sure.

Yeah.

I do want— I know we see some audience questions coming in here, so I want to, I want to go to a couple of them. And but by the way, If you haven't submitted one yet, plenty of time to just go to the Ask a Question section, and I'd love to, love to address your questions for Laura too. One of the questions I actually had down in my notebook is pretty similar to one that just came in from an audience member, which was, well, I'll read their question and I'll maybe add to it a little bit too. Is there any tips for getting leadership to buy into the importance of spending the time and money to really get to know our customers? Which I'm glad they asked because I've seen— I've been a part of those organizations in the past where it's like, well, we, we know, we already know our customers well enough, but maybe we don't. How do we— how can we as the product people that maybe really want this process, how can we help sort of sell it up?

Yeah, this is— I, there are a few questions that every time I talk about user research, I always get. This is one of those. And the really sad part of it is that I think my answer changes every time because every time I'm like, oh, did you try this? And they're like, yes, it didn't work. So here's what I'm going to say. The thing you're feeling where you feel like nobody believes in user research, you are not alone. I, I feel for you. I am in no way making light of this. It is a serious problem. And it is, I think it is less of a problem now than it was, but it is still a really serious problem. I would say that there are a few answers to this. Often what you have to do is you have to do some user research and then show the benefits of it in some way to the people who make the decisions. So you are almost never going to be able to just argue somebody around to your way of thinking. Believe me, I have tried. Does not work. Uh, you need to demonstrate the value in some way. Like I said, doing things like usability testing is a phenomenal— that, I mean, one of the reasons it is a great gateway drug for user research is because if you do it and then you take again those sort of, you know, do, do your work, do the videos, clip, put some clips together. It doesn't really take that long, you know, cut it down, find some, like, find 3 really bad problems and then show them to everybody that you possibly can, right? And say, hey, remember how we weren't doing user research? This is what we're actually doing to our customers. Right? Go out, get those quotes from people, get actual feedback on what's going on and what their needs are, and then show that information to people. That will often make people at least understand what you mean by value. And it's a little easier, I think, to understand than sort of general, you know, oh, this is our user and these are personas and things like that. Like those can be a little bit more, those can be a little harder to convey the value of. Which is unfortunate because they're extremely valuable, but you know, people, people, people react really strongly to, oh my God, is that what we're inflicting on people? Oh, that's terrible. Like, what, what are we doing? We're monsters. Um, I've had engineers like fix bugs in user research sessions because they couldn't stand to see people struggling. So, um, that's a really, I think, good first step is to just kind of do it and You know, show them the value. Also, if sometimes they'll say, oh, we don't need to, we know that what you need to do is I do this thing called hypothesis tracking sometimes, and it's a little punitive and I'll warn you right now, people don't love it, but it's helpful. But other people might not take this well. When somebody's saying something like, oh no, no, we absolutely have to do this, I know that it's The important thing, write it down. Just write it down. Okay. Get, get the following information. Okay. Why do you think we need to do it? What do you think it's going to do? What metrics is it going to improve? You should be doing that anyway. You're product managers, that's your job, right? Like somebody comes to you with a feature. Why? Like what, what will that do for us other than giving us a new feature, which is useless? Okay, great. So why do we need it? Write it down. Have them sign it, right? When will we know this by? Then build the feature, because if they're your boss, there's not a lot you can do about that sometimes. Then check back, like actually check back and look at what did we predict would happen and did it. I know it's revolutionary, but so many companies don't ever do this. Then do a postmortem on it, right? At the point, like maybe they're right. Great, fantastic. Why did you think they were wrong? Maybe they're wrong, or maybe you were wrong. That's fine too. Do it to yourself. Absolutely do it to yourself. Okay, what could we have learned ahead of time that would have prevented this disaster? You can also do these kind of postmortems on things that you didn't write down ahead of time, but it's easier if you do because sometimes you kind of can retrofit the past. To fit what actually happened. Oh no, we knew that that was gonna decrease acquisition. That was intended all the time. No, it wasn't. So really like do postmortems on features that you've built and then ask yourself the question, did this do what we thought it would do? Why not? What could we have learned ahead of time that would have better informed us about what would happen here? And how do we not do that again in the future? So many people leave out that really important step of figuring out how their, you know, behavior actually turned out.

Well, and I imagine it's like then you have data and a lot of times leadership loves data, right? Like, so that, that can be, that can be one helpful way. And so actually, That kind of leads me to another question. How do you best share that data with the rest of the team? Because I know that, you know, marketers can help market our product better if they had better insights and engineers can. So what's— do you have tips for people that want to, again, share this insight outside of the sort of product team?

I do. I do. I have a generic tip for this one, which is fun. Which is treat it like a product. Treat it like a product with different users. This is, again, so many people think that when you're working internally with your teams, that that's somehow different than working with customers, and it isn't. Anytime you have a thing that you want to communicate to a human, you should figure out what that human wants to get out of your product or the thing you're providing them and how they best consume data. Maybe it's PowerPoint slides, in which case, I'm sorry, make a PowerPoint. Maybe it's a dashboard, which fantastic. Those are fun and easy and you can build them and then just let them deal with it themselves, which is great. You know, maybe it's an Excel spreadsheet. I don't know. Maybe it's a conversation. I— you need to figure out what you're trying to convey to the people, what's the best way to convey it, and what they'll actually take away from it. But your goal, remember, every time you're giving anybody anything, a feature, a product, a piece of data, your goal is to help them change their future behavior in a way that's good for them and good for you, hopefully. And you know, I'm just going to throw this out there, good for the rest of the world. I think we should also think about that. So that's, that's, that's what we do, right? So apply that internally.

Yeah, it makes, makes a ton of sense. It's like, hey, these are, these are real people too that we're dealing with. So how do we, everybody's different. How do we, how do we cater to each one of those groups differently? I like that.

Stakeholders are users. They're just a user of a different product than your end user.

Yeah, like if we were thinking about it like a product, it might, might seem pretty natural to us.

Yeah, it all becomes much easier. And also you can then pull in your UX designers and your marketers and have them help.

I like that. Well, time's flying by. We kind of only have a few minutes left. Maybe to sort of wrap up, I'm curious if, especially for people that are sort of getting on this journey of, you know, wanting to research their customers more, but they're just getting started. I imagine that there's oftentimes like common mistakes and traps that people fall into. Are there— knowing some of those might be helpful for us so we can avoid them. Do any come to mind for you?

Yeah, I mean, just doing the wrong kind of— well, so there's the doing that wrong kind of research. You know, I said that you have to do the right kind of research. In my book, Build Better Products, my most recent book, I have a whole sort of exercise that you can go through to figure out like, do you need to be doing qualitative or quantitative? Do you need to do sort of research or usability testing? Testing, like what, and it'll give you sort of a rundown of how to figure out what kind of methodology you wanna use depending on the question you wanna answer. That's a huge one that I see. They just, they always default to the thing that they know how to do. And I mean, while that you're still getting some data, it's not necessarily what you need. The other thing is synthesis. Synthesis is incredibly important. Going out and talking to people is not research, it's shedding. Um, the— you have to synthesize the responses and turn it into next steps, or it was a giant waste of time. Honestly, if you're— I mean, yes, you can do sort of foundational human understanding research, but most research that product managers are going to do, even if it's like understanding users and, you know, real, really ethnographic and like understanding context, still should lead you to, again, to make different decisions, right? To change your behavior. The research you should be doing should change your behavior. To do that, you need to do good synthesis of the information and try to turn it into insights and, you know, calls to action, really. Like, okay, what are you gonna do differently having done all of this work to figure this out? Those are big mistakes that people make. They just have like all this data and they're like, we have data here. We did 300 user interviews. And I'm like, oh, God, why? Just no, stop. Don't know what.

It's just as good as zero if you don't do anything with that, right?

Like, oh, that's fantastic. With those, that sure will fill up your server. Well done.

Well, I, in wrapping up, because really at the end of our time, I'm just, you know, for sure there's probably questions I didn't get to. How can people keep tabs on you, the work you're doing, your books? Could you just let people know how to how to keep up, keep tabs on all that?

Yeah. So a couple of things.

I am on—

I am very online. I am very on Twitter @LauraKlein. That is my handle. Also, I have a website which is usersknow.com. That's K-N-O-W. So usersknow.com. I have a podcast called What Is Wrong with UX, which is what you'd expect. It is very much a podcast and there is a lot of yelling. And so I think that's probably the The best way is check out the website. And if you go to usersnote.com, you can find the books, you can find the podcast, the occasional blog post that I write once in a great while.

Awesome. Awesome. Well, I'm going to put that in the chat too. And yeah, I appreciate you making the time for this, Laura. Thank you for that. And thanks to everybody for joining us today. We have a couple more planned and then very soon we'll have— so the next two interviews for June are already up, and then July will be very soon. So keep an eye out for that. And, uh, yeah, thanks again, everybody. And Laura, thanks again for, for being here with us.

Thanks so much for having me. This was fun.

All right, I'll see you later. Bye.