Rapid Research, w/ Sarah Doody - UX Designer and Consultant

Hey everybody, this is Mike Bellocito, co-founder of Product Collective, and we're back for another Industry interview. Um, I am excited but also a little nerve-wracking because for me, I've got about a month before our big conference, Industry, the product conference. Um, but our guest today is actually going to be at Industry too, so I'll talk a little bit more about that. But if you're If this is your first time joining us, I just wanted to share a little bit about what an industry interview is all about and what Product Collective is all about. And hopefully you could join us for some more sessions too. But these industry interviews, these are 30-minute Q&A chats. We like to think of them as sort of like virtual fireside chats with product leaders from all over on topics that are relevant to product people. So our guest today, Sarah Doody, I'm really excited to have the conversation with her. About what it means to be involved with research when you're a product person. And we're going to get into that topic in just a bit. But at Product Collective, we're really a community for product people. So these industry interviews, this is one thing that we like to do to sort of help the product community, if we will. And, but there's other things too. And you could go to productcollective.com and check that out. We have a weekly newsletter. A very active Slack channel, about 10,000 folks in the Slack channel. And of course, we meet in person in the fall in Cleveland, Ohio at Industry, the product conference. And so that's coming up in about a month. We'll have about 1,000 product people with us. I just did the count today. It's, we have people from, I think it's 38 states and 10 countries coming this year. So people really come from all over. We also do a European edition of Industry in the springtime. So If you are one of our European friends, you could visit us in Dublin, Ireland, and visit us in Cleveland, Ohio. It really doesn't matter where you're from. We would love to have you. So you could check out the details. There should be like a button that will share more about Industry, or you could just go to industryconference.com. And I want to— a couple of things about these chats. We're going to get into a discussion. I have questions for our guests, but I would really love if you had questions for our guests. In fact, I'd much favor your questions over mine. There's a section called Ask a Question. Anytime you have a question, and whether you want to address, maybe it's a topic that just came up, maybe it's one that we haven't even talked about yet, feel free at any time, including right now, to put it in the Ask a Question section. I'll be taking a look at that throughout, and I hopefully can get to all audience questions. There is also a chat. You probably see that as well. That You could put questions there, but I'd love it if you put in the ask a question section. The chat's really for introducing yourself. In fact, even right now you could let us know, you know, where you're from, maybe what kind of product you're working on. It's just sort of fun to see where everybody tunes in from. So with all of that out of the way, I want to get right into the conversation and unofficially welcome you, Sarah. Thanks so much for joining.

Thanks for having me.

Well, I'm excited to have you at Industry too. I know you'll be giving a talk and also putting on a workshop really on sort of this very topic on what it means to research when you're— maybe when research isn't in your title, maybe, but when you're a product manager or some other product role. But I want to kind of sort of maybe tee that up. Maybe we could talk a little bit about that and maybe to start. Maybe you could just talk about, first of all, like, how, how important is it that product people, especially if they don't have research in their title, like, how important is it for a product person to still be involved with that research process?

Yeah, so this topic of research really is near to me because when I was much earlier in my career, I was working a couple of different startups. And we did not invest in research. And those companies are not around, and they wasted a lot of time, a lot of money, and made a lot of mistakes because we didn't do really any research. So if, you know, you are at a company where maybe you don't have budget for research or time, or you get a lot of pushback, I would really just encourage you to think about kind of the side effects of not doing research. Because ultimately, in my example, you know, we wasted weeks, probably months, building features that no one wanted, or raising money and then ultimately having the company, you know, not exist anymore. I kind of think not doing research— I was thinking about this this morning and trying to come up with a good analogy, but I think not doing customer research would kind of be like being a doctor and having someone like come into the office and maybe you just look at them and then make some decisions about what to do with that person rather than like looking at their lab results or asking them questions and things like that. But I think as soon as you can flip the switch from not doing research to doing research, you'll realize like the treasure trove of information that you're getting from your customers with honestly not as much time, effort, money as you think. And I'd also say, I think based on all the research projects I've done, Every time I do a research project, um, the client is always surprised because something that they thought or had assumed is completely proved opposite or shot down from the research. It happens over and over and over.

By the way, I think I have had a couple doctor's appointments where I'm pretty sure that's exactly what they did. At least that's how it felt, and it didn't feel good. So analogy. I am curious though, when we think of research, you know, people might be thinking of different things. And obviously there's sort of— there can be a spectrum, I imagine, from lighter weight things to things that are sort of full-fledged, like this takes up all of my time. Maybe could you take a step back and sort of help us understand what are— when we say research, what do we mean? What are those things that sort of make up research activities for product people.

Right. So back to that kind of analogy of these companies from earlier in my career when I hadn't even done research, to be honest with you. I remember one of the companies did try to do research and it was very high budget, very labor intensive, and that was hiring a recruiting company to find people to talk to, renting a big kind of research room, lab, if you will, with, you know, glass windows or mirrors and ordering cheap Chinese food and hiring a moderator and all this. And that, I don't know the budget, but it was a lot. There were many sessions conducted, so it must have been a crazy line item on the budget. But I think that's what people assume when they hear about research. And you can do expensive research, but concerning the spectrum of, you know, low budget, fast, etc., all the way to these more formal projects, some of them could be like just going and talking to your customer support team. That is free, that's in your building or on Slack. And those people, even though they don't have research in their title, just off the top of their head, they probably have a ton of insights about people's problems, suggestions, objections to like, why aren't they purchasing, et cetera. Another one I really love is kind of doing social media listening. So if I'm doing a research project for a company, one of the things that I'll do is just go to Twitter and like type in the company name, or if it's more of a topic like, planning a family reunion or running a 5K or something. I might just type that into your variations of that into Twitter, into Facebook, into YouTube, and see what topics, questions, etc., come up. Because that's going to give you clues, especially if you're more on the exploratory side of things versus like doing usability testing for a product and market. But you could also use that same strategy of listening to kind of content that's already out there by maybe going and reading Amazon reviews or iTunes reviews or App Store reviews. Maybe just Google that product name to see if there's been any blog posts written about it or reviews or articles, or do this for your competitors as well. Again, super cheap, super quick. You could have someone else on your team do this. And then, you know, you could discuss it together after. Let's see, another one that I really, really like right now is creating communities for either just people interested in your kind of general topic or your actual customer base. But I've seen a couple of companies doing this, either having their own Slack group, Facebook groups. And the cool thing about that is it's an amazing, just kind of place to collect feature ideas, problems, etc., but it can also help reduce your customer support because your customers, naturally, there will be a couple of people that bubble up and start like kind of being advocates and they'll just hop in and answer questions. And then of course we get into the more, a little bit formal things like a survey or one-on-one interviews, usability testing, etc. But even if you didn't have tons of budget, you could automatically like today and tomorrow, go talk to your customer support team, do some Googling, searching on Twitter, YouTube, etc. And maybe create a community or go hang out where your people might be hanging out elsewhere online.

I love some of these ideas. I'm always a person that's in favor of things that I can do right now. Like in some of those things you just mentioned, there's nothing stopping anybody right after, do it after the video chats, we still have lots to talk about. Right after this video chat, I mean, we could be rolling up our sleeves and doing some of these things. So I love that. I can also attest to the fact that having your own community can be an awesome resource, especially when it comes to doing research. For us, you know, Product Collective, we're really fortunate to have, again, the Slack channel, where when I want feedback on something, I can type it in. And this goes for other members of the community too, but we can type that in and then we have other product people that are just like us responding. So I love that too. So I think that's great. Those are great ways that we can all start. And that helps also sort of maybe take the weight off of thinking of research as something that's like, has to be this big heavy thing that might take a lot of time and effort. Some of these things we could get started right away. So thank you for that.

Well, can I add one more thing about the community? It's, it's interesting because a lot of people think, oh, I'll have a community just for my paying customers. And I definitely agree with that. I have a couple of communities for my own products. I have some products related to kind of UX careers and UX portfolios and job interviews and things. So I have one community that's only for people who have paid for some of those things. But I have another community that is free to join. You don't have to pay for anything. And that's really useful because it's kind of a petri dish for me to be testing blog post ideas or, um, Facebook Live topics, or just also seeing what content people are referring to each other in the group to give me ideas of what's working, what topics are hot, etc. And maybe someday, someday those people convert to customers. But if not, they're still getting a ton of value for me. And that's going to pay me back in the future in some way.

Yeah, I think it's a great point for sure. I'm glad you brought that up. By the way, I have— there's one question that came in and I'm going to get to that in a little bit. But just as another reminder to everybody, if you have questions for Sarah, I would love to ask those questions. So use that Ask a Question section. And, and yeah, I will definitely get to it. So please do that. But I do have a question for you, Sarah. So there are probably some people— I know a lot of our attendees that join us for these chats, some people are in startups, so they might be in organizations where everybody's wearing so many hats. A lot of people, though, are also in very large organizations, and some of those organizations, they might have colleagues that do have research in their title, even still, do you still think it's important if maybe my colleague has research in their title, I'm still a product person, should I still be taking on some of that research work myself too, though?

So I think it's a great question and I think it's largely dependent on the organization you're in. But I think if we take a step back and think, can, who can benefit from developing the skills of someone that has researcher in their title? And the answer to that, in my opinion, is everyone. Because even if you are not talking to, you know, customers or potential customers, or conducting usability testing sessions and things like that, if you can learn to be a better listener, learn to kind of see problems below the surface, be able to ask really probing questions, start learning how to connect the dots and spot insights and things like that, that's going to help you in your day-to-day role because maybe you're not doing research or applying research techniques to users and things. You can still apply them to clients, colleagues, other teammates, etc. So if you don't have research in your title, I would say, you know, this afternoon, if you're in a meeting, try and start becoming a better listener so that you can ask more pointed questions to kind of start peeling back the layers of whatever topics are kind of coming up, if that makes sense.

Yeah, it makes total sense. And it's like, yeah, even if we have a team member that is doing research and we don't have it in our title, it'll still help us do our jobs better if we had more information. Right. So I love that suggestion. One other question, and I guess it's sort of on the opposite end of things. So I guess if an organization had people that did have research in their title, they, they maybe, you know, value research, right? They're hiring people to actually focus on that, I would hope.

Yeah.

There are probably situations where people are tuning in here and maybe their companies they think that research isn't— it's not valued like it is in those other organizations. And it might be something they need to sort of sell up. And I'm curious, your advice to people, product people that want more research to be involved in the organization, but maybe they feel like they need to sell it to their leadership. Like, what, what sort of feedback would you give to them?

Yeah. So this is probably one of the top 3 questions I am asked. How do I get buy-in for user research? And there's a couple of things to consider. First of all, I think a lot of stakeholders, CEOs, founders, et cetera, have this immediate aversion to it because they think it's gonna be 6 months and $40,000 and, you know, too much to commit. So first of all, I would say don't propose doing a massive research project. Think of what is the smallest amount of research you could do to have the maximum impact. And I wrote this article, it's on Medium somewhere, just Google minimum viable research. And that'll walk you through some ideas of how to do just this, how to get that buy-in. But one thing to think about is try and highlight the symptoms of not doing research. So can you quantify or come up with examples that you can take back to those people that are being a little stubborn about research and say, hey, last quarter we spent this amount of time doing this feature, these 3 features, and we launched them, and here are the metrics. No one used them, or whatever the metric is. And then if you can also say, and then we spent, you know, X sprints redoing it over and over and over. If you can kind of go back and retroactively highlight some of those highlights, then that could help kind of get that buy-in. But also, I would say, when you're deciding which research to do as your kind of minimum viable research project, think to yourself also, what could be the most important metric to that founder or stakeholder CEO. So if you're working for maybe an e-commerce company, don't say you're going to do a research project about, like, the entire site. If you know that checkout drop-off has been on their mind and it's really important to them, then use that as your kind of guinea pig research project to prove the value of the research. And then after that, That'll be a baby step to get them to say, okay, now we can go research product detail pages or search or whatever else is like the next big part of the site or experience.

I love that. That's, that's great advice. And I think I found— and I'll look for it later on too. I think I found at least one article that you had on the topic.

So yeah, I forgot Hacker Noon moved over to their own site. So that's it for sure.

Okay, cool. I do want to go to a couple of audience questions because they're coming in and some of these are good. And this was actually one that I was planning on asking you too. When do you feel like, or when do you know when you have enough information in your research to actually start putting it to, you know, taking action, putting it to use, maybe building out requirements for something to build? What's your take on that?

Yeah. So I can't give you a specific, you know, number. I know there's numbers out there that float around. Jared Spool wrote an article years ago, and I think it said like 5 people is the best number of people to ask. And now he's saying like, stop circulating this article, like my opinion has changed.

I didn't know there would be a magic, magic number too.

In the article, there's all kinds of math that I can't follow, but The point is that it's subjective and you kind of have to like apply or apply your right brain a little bit and think to yourself, when does it feel like we have enough patterns that are so standout that we feel confident that we should move ahead with this? So if you only— well, if your product, you know, is already in market and you have 30,000 users or something, only talking to 5 of them might not be enough of a sample. If, if a pattern emerges from 2 out of 5, like, I don't think that's enough. So you kind of have to balance it with what's your user base, um, and how strong are the insights that you're developing. But I would say one great tip that I always recommend to my clients is do a survey before you start doing any one-on-one interviews and things like that, because a survey is a great way and fairly inexpensive to cast a, a really wide net to see some of those patterns start to emerge. And then you can identify some potential good people to contact for, you know, follow-up one-on-one interviews or something like that. But that survey will be a great opportunity to kind of start to identify some of the major topics or areas that you might want to go deep in user interviews with. Great.

Another audience question that came in, which I think is a good one, it's, how can we, how can we do something to validate our own biases in actually, like, leading our research? So, You know, knowing everybody has their own biases, even if we don't want them to be showing through in our research, is there anything that we could do to protect against that?

Well, I think part of it is how you phrase your research questions. So one— and I think this is going to answer this question, but let me know— one thing I've been starting to do is asking questions kind of in first person so that the person also does not talk in kind of hypotheticals and maybe their own biases versus like what they think they would do versus what they actually did. So an example would be if you are researching travel and it was related to, I do not know, holidays with kids or something. If one of the first questions in the survey was like, if you were planning a family reunion, like, what types of destinations would you be interested in? That's a little bit hypothetical because it's like, what if, versus if you ask, um, the last time you planned a family reunion, you know, what were the— what types of places did you look at or something? The point is like If you can phrase the question with the last time you, the last time you bought a vacuum, the last time you booked a flight, the last time you did this, that's going to take the person back to retelling you that exact story of how they did that versus, well, what I might do. You know what I mean?

I love that. So I teach a product management class, an undergraduate one at Case Western Reserve University in Cleveland, and And I kind of did that as an experiment in my class. I said, okay, I want to ask everybody, like, how many people think, you know, would it be a good date, you know, to take somebody to the Corner Alley, which is a local kind of like bowling alley bar? It's this place. And so many people raise their hand. I'm like, okay, awesome. In the last 2 years, how many people have gone on a date to Corner Alley? And like, nobody raised their hands. And so it was, it was only meant to be sort of like an eye-opening kind of thing for people to understand. Hey, sometimes what we say we would do and what we actually will do, it doesn't jive all the time, right? So I, I really like that suggestion. I think that's a great one.

And like, from my experience, of course depending on the topic, but when people are recounting something that they did, they're probably going to go into great details. So when I've done this for my own products, I get like paragraph-long answers back, and when I said earlier, like, use a survey to cast a wide net, that's what I mean. Because based on this survey, especially when you have open-ended questions, that's going to indicate to you who might be a better candidate to do a one-on-one interview with. You know, if someone only writes like one sentence, then versus someone that writes three paragraphs, I want to talk to the three-paragraph person.

Yeah, yeah, for sure. Well, there's another audience question that came in and it was around competitive analysis as one type of research. And the question is just how valuable is it? And, you know, it sounds like in their situation they're worried that leadership might, might view competitive analysis as a part of research as just sort of like chasing features. So I'm curious your take on that.

Yes, I have definitely seen feature chasing happen as a result of competitive analysis where the stakeholders and the rest of the team, like, are leeches and latch on to something and then must have that in the product, you know, come hell or high water. And I think it's important if you do that competitive research to make sure you're doing it with enough competitors that you can say, like, these 10 companies all did this, or 3 of these 7 didn't. And you can start to kind of see patterns. I think if you only look at one, then that's going to be probably a disaster. But I always remind people to reiterate that, like, you might be looking at something on a competitor's product, but you might be looking at an A/B test. You might be looking at some feature they haven't had to— they haven't had a chance to do anything with that they know is broken, but it's just like on the back burner. So if you do have those situations where, say, stakeholders, and etc., are wanting to, you know, when Pinterest came out, the Pinterest grid layout was like running rampant all over the internet. But the problem was like, was that right for their product? A news site, I think it was like the Today Show or something like that. All of a sudden they went to their homepage once and it was the Pinterest layout, but it was like horrible because news images are not like beautiful, like interior blogger images. So remind people, you know, this is a great feature, but is it right for us? And then justify that, is it right for us?

That's great. I— we're coming towards the end of our session and it's gone really fast, but I want to make sure I can at least ask one or two more questions. One of them being, so when we go through and we do this research, I am curious, like, what how, like, what happens next? Somebody did ask about, like, what artifacts should come out of it. I'm also curious, like, should we be sharing this with our teams? How do we share it with our teams? Could you just maybe share a little bit about that?

Yeah, so it really depends on the type of research you are doing and also who the user of the research is, right? If it's like your fellow teammates, maybe they don't need like a wonderful PowerPoint or Keynote or something going through everything, like reiterating how many people you talked to and why you chose this type of research, et cetera. Whereas if you're doing research back up the ladder, so to speak, you probably need to kind of do something a little bit more formal. So first of all, consider the user of the research. And consider, like, how familiar they were with the lead-up to that research. If they weren't, then they are going to need a lot more of that backstory to kind of buy into it. But the big must-do when it comes to presenting the research is no matter what you create, you have to think like you are a lawyer presenting evidence for a legal case. So for every statement or finding that you have, it's not enough just to say, you know, the checkout process was too long or something like that. You need to provide evidence. And evidence could be analytics. It could be quotes from survey or from user research interviews. It can include heat maps. It could include video clips. It could include, um, you know, anything else that's going to back up that claim. Because without that, then that stubborn stakeholder is just going to argue with you all day long. But if you can have like a quote or a video clip that shows someone in a usability test like not clicking the big green button on the, you know, top of the page or something, then it's gonna be a little harder for a stakeholder to dispute that because they might honestly look a little silly if this video is clearly indicating like no one can see this button. So think like a lawyer is really the best advice I can give rather than to kind of go into the nitty-gritty of what should actually be in a keynote presentation for research or something.

Yeah, I like that. I like that very much. Think like a lawyer.

All right.

Well, I am looking at the clock. We're almost at 1:30. So I don't— we can't get to all the questions, but I know you're active on Twitter. Maybe you could just share how folks can keep tabs on what you're doing and maybe how they can, you know, follow your work.

Yeah. So Twitter is where I hang out the most. It's kind of a good central hub. Also, I have a lot of content over on my YouTube, which is just YouTube.com/SarahDuty. There's actually a research playlist over there as well that you can check out. And then that's about it, really.

Well, and I'll also mention again, we have Industry coming up in a month. Sarah will be one of our speakers. We do have a few slots open in her morning workshop too. So there's a— it's a half-day workshop. The afternoon workshop actually sold out pretty quickly. But if you're interested in going deeper with Sarah on this. She will be at Industry, so you can learn more at industryconference.com. And, and Sarah, I'm just, I'm really appreciative that you made time for this. This was a great conversation, so thank you so much.

Thanks for having me. I hope to see some people there for that morning session as well.

That would be awesome. And thank, thank you to everybody else for joining, and hopefully I will see you all at a future Industry Interview. But until then, I will see y'all later. Bye now.