The Importance of Challenging Your Assumptions w/ Cindy Alvarez, Director of UX for Yammer

All right, uh, hello everybody, this is Mike Belcedo of Product Collective here for another product launch session. I know we got started a couple minutes late, so thank you for staying with us on that, but I'm really excited to be here because I have Cindy Alvarez with me. And how are you, Cindy?

Hello, I'm good, thank you.

Good, good. Well, I'm psyched that you're here with us. I want to dig in and talk about You know, challenging assumptions. I know that that's, that's the topic for today. It's also going to be the topic that you're focusing on when you provide a half-day working session at industry later this fall. So I'm pumped about that. Before we get into it, just a couple quick reminders, or at least sort of notes for those that are joining us for the first time. This is a live Q&A chat. I will be asking Cindy some questions, but If you all have questions, definitely feel free to feed them to me. There's a section in here called Q&A, and you can add questions there. I've already noted that there was a few of those, so I will do my best to continue to, you know, ask as many questions from all of you as I can. So there's that. Also, I just mentioned, you know, Cindy will be with us at Industry. Cindy's actually giving one of three working sessions, so one thing I did beforehand is I created a promo code. So for those that want to come and purchase an all-access pass and take part in Cindy's session, we, Product Collective, will give you $100 discount. We're keeping it open for a week, but I'm not like publicly promoting it, just those that are with us here today for Product Launch. So little, little secret discount for everybody.

All right.

But yeah, so I'm excited. And by the way, congratulations are in order for you, Cindy, because you have a new role at Microsoft, don't you?

I do. Yes, so previously I had been running the research team for Yammer, which was acquired by Microsoft a few years ago. And I don't know if you can hear me because now I don't see a, now I don't see Mike. Ah, okay, good. So the rest of you can hear me, so if Mike's not, if Mike can't, then that's still fine. Yeah, so I actually had been doing customer development with the rest of Microsoft various teams, and I actually pitched my own role to where I am now, which is the Cloud and Enterprise Division. They make developer tools for Microsoft, have been doing a ton of trying to become better at listening to their customers. They've all read the book, they've been doing workshops, but of course, as in any big company, they have a ton of divisions that don't do the best job of working across each other. And so I said, you should just have someone who does that. So I am now tackling some of the most weird ambiguous problems where people are like, I don't even know where to start. So that, that is my job now, figuring out where to start and getting people to say, did you know that this person over here is doing something really similar? Let's get you two in a room and figure out what we should be doing.

That's awesome. Well, it'll be, I imagine, a lot different than from some of your startup days, right?

Yes, yes, it is definitely putting the testament to the fact that these things work in a very large company too. It's not just startups. You can do this in a huge division, it will be slower and sometimes it's frustrating. But it actually, it's really interesting, it gets deeper into the soft skills of the negotiation and the convincing and the challenging your assumptions. Because the more people you have, the more assumptions you have and they kind of build on each other.

Yeah, well, you know, speaking of challenging assumptions, you know, I wanted to start off by asking you, you had mentioned before that, hey, if we don't challenge our assumptions, we all know product people especially, we make a ton of assumptions, but if we don't challenge them, we miss out on opportunities. What do you mean by that exactly?

So I find that a lot of times we are partially right. You know, we're smart, we know our business, we know our industry, and so we will have an answer that on the surface, you know, smells right. And so, for example, you know, probably the thing I see the most often is around pricing. So you might hear from customers that something's too expensive, and there's two ways for things to be too expensive. One is, you know, it costs $6 and I have $5, so there's just— there's no getting around that. I can't afford it. It's a non-starter. But what I think happens much more often is that something costs $5, but all I have is $20, and I'm thinking, is it worth spending a fourth of what I have to get this thing? No, there's not quite enough value. For me to spend $5 on this. And so that doesn't mean that your product is too expensive. It means that it's not providing enough value for me. And so you could try lowering the price, but you'd be much better off increasing the value. And so a lot of people stop at the, it's too expensive, and they don't dig in. And there are a lot of things that kind of fall into this line of the customers give you an answer and it seems very reasonable. And yet what we give is that first answer is often not really what would drive us to change our behavior.

Got it. Let me ask you this. Are there, are there certain assumptions that we make? I mean, you mentioned, you know, when it relates to pricing, you know, that's, that's how customers are purchasing our product. Are there, whether it's customer assumptions or product assumptions, there are certain assumptions that we make that are more important for us to make sure that we're challenging constantly versus others, or how should we best think about that?

Sure, so I think what it's— most assumptions fall into the category of, I assume Mike wants this, right? And again, you might have indicated something through your past purchases or behaviors that this is the thing you want, and yet it might not be the thing you want. And you'll even see this in internal meetings. So one of the exercises that I will have teams do sometimes is to have each person take a sticky and write down the goal or write down what's the core value of our product. And these are people who've worked together sometimes for months across the table from each other, and they'll write down their stickies and they're never exactly alike. And sometimes they are really not alike. And you have this moment of, but, but Mike, we've been working together for 6 months. How could you not understand that this is the core value of our product? And you're like, I think it's this. And the thing is that the customer thinks it's something else. So those, those kinds of things, that's like the root of all potential assumptions is that like, I assumed you wanted this and I had a good reason for assuming it, but I never actually tried to invalidate it by asking, what do you want? Like we often ask, you know, would you like X? To which I might say yes. But if you said, what do you want most? I might say something completely different.

Yeah, got it. So I guess let me ask you this. What about when, when those assumptions are being made about, I guess rather the assumptions coming from, say, internal stakeholders like upper management versus assumptions that, that they're your own, you're making about the specific customer. Is there a different way that you handle each? Are there techniques that you would make to use to challenge those that might differ because you're trying to challenge an assumption that somebody else is making, but you still need to validate as a product person?

Sure. So, I mean, you obviously have to be more diplomatic when there are other people's assumptions. And I think one of the reasons why we don't is because it's very easy to have challenging assumptions sound like challenging that person's authority. And there's plenty of like, well, why are we doing that? I don't understand why we're doing this. Those sound very negative. They sound like, you know, as a manager, it would be very easy to inadvertently just kind of say, look, just do it. Like, I don't have the time to explain this to you. Just do it. Because when you push back sort of generally, the person you're pushing back to isn't actually sure how much they would need to explain to get you on board. And they think about, do I have to have an hour-long conversation? Like, I just don't have time for that. And so I try and give them like a very specific question to answer, which is like, okay, so you're saying that we need to build this. When we build that, when we're building it, what is the problem we're trying to solve? Or if I think I know it, to guess at it and say, okay, you're asking me to build this. Correct me if I'm wrong. It seems like we're trying to build this in order to solve problem X. Is problem X the problem we're trying to solve? Can you just, you know, can you confirm that? And you say it in a nice way, very conversational, and the person says, "Yes, problem X is what we're trying to solve." Or they say, "Well, no, that's not quite it." I'm like, "Well, I wanna make really sure I'm clear on the problem I'm trying to solve." And you don't even say, "And I'm not gonna build this yet." You can just start with, "Before I even get into the feature, I wanna make sure I'm really clear on the product, the problem." And so you ask for that, the person tells you the problem, and then you kind of think, does that make sense? Does building this solve that problem? And if it doesn't, you can kind of innocently act dumb. It's like, I'm not sure how that's going to solve the problem, though. So is that the core problem? And you just keep kind of doing that. And you can do it from the other side, too, which is, OK, if I build this to solve this problem, how are we going to measure that? How will we know if it's worked? And you put this— you couch this in the guise of not, I don't want to do what you ask, but I want to make sure I do a good job of what you ask. So if it's your own assumption, you should just probably ask yourself, why do I believe this? You know, how should I, how should I make sure that I'm right or not? I should at least try and devil's advocate myself. When it's someone else, you go through that kind of rigor more.

You don't, don't need to be as diplomatic when it's your own assumption.

Exactly. You're like, is this dumb? Maybe this is just dumb.

Yeah. By the way, I just want a quick aside. I think most people are hearing and seeing us okay because some are participating in the chat, but Somebody had asked if the webinar had started yet. So if you could in the chat, just to make sure you're seeing and hearing Cindy and me okay, maybe just mention it in the chat. That'll give me some reassurance that, yes, that you see, that you see us.

I know at least one person heard me earlier, but all right.

Yeah, it looks like, looks like, uh, looks like people can see and hear us, uh, great. Oh, and by the way, I think I forgot to mention before, that discount code that people can use, it's Challenging Assumptions. So if you, if you did want to see Cindy at industry, you could use the code challenging assumptions. I'll get that out of the way. OK, so let's, let's say it is my own assumption. I buy into the fact that, no, I need to challenge this. I need to always be thinking of ways to challenge it. And this is a question that came from the audience, actually. Are there specific tools and processes that we should keep in mind that seem to work really well when it comes to challenging assumptions? What would you suggest to, you know, me as a product person when I'm at the stage where I buy into it, but now how do I, how do I do it? Or how do I get started?

So I think the first thing is literally just reminding yourself to do that, you know, and sometimes like a physical reminder of like, you know, double-check yourself. And when, once you've done that, it's, that makes it easier. It's also a lot of times if you're working on a team, to sometimes there's one person who enjoys just— this is the role that they love to play is to be the devil's advocate. So if you have that person, great, then that's the person you run it past. I remember early on doing some internal workshops and I kind of had an idea of what I wanted and I didn't go to someone who had already said, I want to do your workshop for feedback. I went to someone who I knew would be sort of skeptical and said, look, I'm going to do this workshop. Here's an outline of what I'm going to cover. You know, do you think this would be valuable? Or, you know, where would— where could it be better, even better? As you ask them in that sort of like, I know this is bad, like, how could it be better? And had that person say, well, I don't believe this, like, this will never work. Actually, that could be interesting. And, and then that taught me something about when producing this kind of things that might be new ideas, I generally frame it in that way, which is Like, I want to hear what— I want to hear why this won't work. I'm going to give you the little spiel about customer development. We're going to talk about that. Now you tell me why it's not going to work on your team and we'll go from there. Because you need to meet people at that point of where they are. And some people say, we're all bought in, but we're not trained, or we're all bought in, but our boss doesn't believe it, or some of us want to do this, but we don't have time. And, and those are different obstacles that have different solutions for them. And, and sometimes the people themselves can come up with that solution. You just need to put engage them in that. So having a devil's advocate, if you don't have a natural one, sometimes that's a role that you can deliberately assign to people. Say, like, Mike, you know, for this project, you're going to be the devil's advocate. So like, even if you agree with Cindy, you need to kind of challenge politely about how could we be sure that that's true.

Yeah, and I bet people, maybe not even in the organization, but somewhere have that person. I know for me, I have a friend Robert, super smart person, and he's somebody I go to often with ideas. But he, you have to impress Robert for him to really be positive about something. And so I remember there are certain times I'm like, God, even Robert thinks we're going in the right direction here. So I bet everybody has a Robert. So you had mentioned that there are a few things that people point to when they kind of get stuck. I was going to ask you like what typically prevents people from challenging assumptions? Is it those things that you mentioned? Are there other things that come to mind that are sort of like the common barriers, if you will?

Sure. So one of them is just that, I mean, someone playing devil's advocate, you can also, you can actually dismiss someone as that, like, ah, you're just playing devil's advocate. You don't really know. So it's a useful, it's a useful tool, but it can also be something that folks can dismiss. So you need to think about Are we doing this just for the fun of arguing, or are we doing this productively? And so some teams just really like to argue. And if you've ever been on one of them, you're like, ah, this is, this is a waste of time. I had a perfectly good idea. Now we're all spending 40 minutes arguing over it. That doesn't get us any closer to the truth. So people often don't know how to challenge an assumption in a way that is more structured than just arguing around the table. Especially product people, engineers really love arguing around the table. So there's sort of some templates of like, well, what would we do for this to be not true? And I find that just offhand, really good people who are really good at this are like data analysts because that's what they do is logic. So if you're like, this is how I would disprove this, they're like, no, no, this is how you would disprove that. So let's say you've got a product idea and you think it's good and everyone thinks it's good and it's got some roots in customer feedback. Customers complain about this, about this, so we're gonna build this. We all think it's great. Maybe we even showed it to a couple customers and said, hey, what do you think? And they're all like, it's great. So they think, well, how would we test if it was actually not great? And so part of that is just taking that flipped narrative of not what do you think, but what's bad about this. So maybe you decide to show it to some other customers and say, look, if tomorrow you came into the office and you had this feature, instead of the one you're using today, what potential problems would you run into? And so now the only answer that you've suggested to them is negative, so they're gonna give you negatives. And some people might say, oh, there's nothing, it's great, we want it right now. And to which you might kind of gently prod like, well, walk me through how you'd use it. What would be different than what you have today? And they'll kind of say, oh, you know, I really like the way this looks, but I'm not sure if this thing would still work. I'm not actually sure if my boss would be okay with this. Is this gonna cost more? So you want to give permission for people to give those issues. You might also want to look for just things that are logically inconsistent. And so a lot of times we can find analogs in other behaviors. So for example, people saying, I would pay X. For this tool? Again, pricing is such a numerical thing that these things become obvious. Oh, I would buy, I would buy that tool for, I'd pay $5,000 a year for that tool. Like, okay, I'm just curious, what other software have you bought for, you know, for your company in the past year? And so they name things off, and if nothing even comes close to $5,000, that calls into question that yes. Now if they've spent, you know, $15,000 for this and $20,000 for this, maybe you should raise your prices. But I've had people tell me very sincerely that they would pay thousands of dollars a year for software, and when I asked what else they were paying for, nothing. Literally nothing. I'm using free SurveyMonkey, I'm using free Google Analytics, I'm using, you know, every freemium tool out there. Yeah, you're not gonna pay me.

Yeah, it's like what people actually did versus what they say they will do can be two totally different things.

Absolutely.

Yeah. One question popped up, by the way, and it's kind of in line with what we're, what we're really talking about, which is, okay, this, I buy into it, this makes sense. And when we have a lot of time, you know, we can build in the challenging into the process. But what about if we're on a tight timeline? What if we only have, you know, whatever kind of project it might be, it's just not enough time, right? Not enough time that you'd want to be able to work with. Are there, are there things that you can do when you don't have a lot of time but you can still be actually challenging your assumptions, and how might you do that?

Sure. So, you know, not statistically significant, not marketing reliable, but, you know, the fastest thing you can do is give people one contrary question to ask and a tip of who to ask it to and use your, use your internal work network. So my research team on multiple occasions has had like a question where if we get interesting, contradicting advice today, we might change our mind, and if not, we're just gonna run with it and say, okay, let's quick huddle. What do we think? What kind of question could we ask that might reveal that we're wrong? Who should we ask it to? Okay, so we wanna talk to people who work not in front of a computer. So like teachers, hospitality, ooh, you're in the dark.

I guess I'm sitting too still.

So yeah, we gave an example. We want to ask this kind of person this kind of question. And then we literally went around the office and we're forwarding Yammer messages to people saying, right now, get on Facebook. Any of your friends who meet these criteria, like not engineers, not product managers, people who are teachers or work in hospitality or work at a hotel or they're a field scientist, Whoever it is that meets that, like, ask them this question right now. And we got back a handful of responses that were interesting enough to make people go, huh, that's, wait, maybe we should dig more into this. So you can do that kind of, you know, really, you know, really scrappy, just who's on, who's got a green light on Facebook right now? You know, go into LinkedIn, post something in your status right now, who can respond to this? And the key there is really figuring out that person and giving the hints so that it's very easy. If I want you, Mike, to ask your friends who are teachers this question, I should like write everything out so all you have to do is copy and paste. And if I do that for you, you will do that for me. You will copy and paste into some messages and then you'll copy and paste what you get back and you'll send it to me and say, all right, here's the responses I got. And it's pretty easy for you and it's pretty fast for me because depending on the size of company you work in, you might be able to, you know, leverage 30 to 40 to 100 people doing this. And if every person gets one response back, that's actually a lot of research you could get in a day.

Well, and I love the idea of something is better than nothing. I mean, it's easy to get caught up with how we're under the gun, but yeah, you can likely get some sort of, you know, feedback, some sort of time to challenge those. So I like that. I like that idea a lot.

And that something might buy you more time. So if you got 20 responses and 10 of them were really contradicting what you assumed, you can go to whoever's the timekeeper and say, look, if we proceed, we proceed, but I'm just gonna tell you that I have these things that make me a little uneasy. So do you want to give us another day to get more feedback or another week to get more feedback? Because here is the cost of if we make this wrong decision, what will happen? But you need to lay that out for people is this is how much more time we would need. This is what could potentially go wrong if we don't do this. And now it's your decision. Like, it's usually not my decision to go/no-go. But yeah. Yeah.

Now, that's great. Somebody else had asked the question, is there, are there good assumptions versus bad assumptions? Or, you know, and I'll add this part, is it, is the idea it's more important to challenge the assumptions no matter what they are? But I'll, I'll ask for your take on that.

So there are assumptions that have big or small consequences. right? So it's like, if you are wrong, and this is a good other thing to remind, it's like, if you were wrong, what would the cost be? And there's a lot of times where if you were wrong, it's not that big a deal. We assume that this color scheme is great. If it's not, you fix the CSS, you push it out tomorrow, no big deal. If you send this email and people get kind of annoyed, but you only sent it to 5% of your, so maybe you send out another apology email being like, sorry, we tried this new thing and kind of pissed you off and we didn't mean to and we won't do it again. That's a really small price to pay. And as product managers, you're never going to be 100% certain. Like, you're lucky if you get 70% confidence on something. So that 30% of, you know, of challenging assumptions, of the things you need to worry about, are the things where if you get this wrong, bad things will happen. And I find there's like a, just a kind of a useful binary of important. If I get this wrong, will bad things happen? No? Okay, then run with it.

Yeah, those, those assumptions don't need as much time to be tested, it seems like, right?

Yeah, because you're even testing them, a lot of times you're going to be wrong. So it's fine. You show it to some people that say, yeah, we like it. You A/B test it, you're like, oh, they didn't like it, or they did like it, but they made them decrease their usage. So not good.

Is it— one thing I did want to ask, but especially before our time ends, is just, could you even outline some examples of what you've seen work well in the past, whether it's teams you've worked with and coached or, you know, at Microsoft in Yammer, it might help set some context for everybody else on the concept.

In terms of, sorry, in terms of challenging assumptions?

You know, assumptions that had to be made that were challenged and, you know, and the better outcome happened as a result. Or if nothing comes to mind, maybe we'll hit you up on Twitter because I'm sure we could keep the conversation going afterwards. Yeah, yeah, absolutely.

You know, I think it's just, let's see, I don't know. We've seen things around, you know, so for example, on Yammer, a lot of times people have asked for things like additional formatting for Yammer posts. So it would be a reasonable assumption based on looking at, say, our UserVoice that this is really important, that this is something that could drive a lot of usage. And we said, like, well, what would it, what would it mean that that could drive usage? Like, what would be indicators that this really is important to lots of people versus the people who just bothered to come to a UserVoice? And one of the things we looked at was how long are the messages that people are posting? 'Cause you can draw kind of a reasonable, again, assumption that things like formatting and bullets and the more text you have, the more those become necessary, 'cause that's where they really are important for legibility. And we looked at message length and it was like the length of a tweet basically. So it's like, for that, should we build this thing or should people just put on caps lock occasionally? And for a long time it's been just put on caps lock. So that was something where we had this one piece of evidence saying this is really important. We looked at actual usage, which suggested probably not that important. Now it could be wrong. It could be that if we added this, then magically people would start changing their behavior and posting these long, you know, long rants, but we don't think so. And so making that decision and being very crisp on we're making it and here's why allowed us to do other things instead. I've seen similar things around people saying, well, we assume that customers really care about X. For example, a classic Microsoft thing is investing a lot in things like guaranteed, customer service response, like someone from customer service will get back to you within X hours. There are lots of legacy customers who, for whom that is a checkbox that's incredibly important. But there is a new set of people for whom they wouldn't even occur to that. They don't even, they don't put support tickets in. They Google, how do I do this thing? And there better be something that comes up in the search results. So, How many people do that has been— I've seen that particular assumption change. As you know, initially it was, well, no one does that, like everyone just goes to support because they already paid for it. And saying, well, let's watch what people do. Oh, they're Googling this. Oh, they're sending a Skype chat to their friend asking how this is fixed. It's just the evidence changes over time.

Well, I know we're coming close to the end of our time. One thing I wanted to ask is if there was, you know, knowing that the session will end, just if there's one thing that you wanted to leave people with and one thing you wanted people to take away, whether it's sort of something actionable that they could do after the chat, what that might be. And yeah, I'll ask you that and maybe one other question afterwards, but maybe you could give us your take on that.

Okay, so I think there's a really interesting framing if you have something that you want to do, which is to say, here is a problem that exists. Not, this is a feature we need to build, but here's a problem that customers have. Here is the evidence that they have this problem. Here is what bad thing could happen if we don't act on it. And so if you kind of start everything from those first principles, you can keep returning to them and it helps a lot of communications to say, like, we have this problem. Here's the evidence that we have this problem. Here's the bad thing that will happen if we don't fix it. Because then those are 3 very objective things that people can then argue with any one piece of it. So they say, I don't think that's a good idea. You've laid that framework so you can say, OK, Mike, you know, but I want to know what part of it is not a good idea. Is it that this problem doesn't exist? Is it that this evidence is not reliable evidence? Or is it that this consequence won't happen? Because a lot of times people just sort of disagree and they don't specify. And if I don't specify, if you don't specify what the problem is, my next solution is not going to be any better. Like, I am not going to iterate internally and be better unless I know which part of that is flawed. But it requires a sort of very clear communication where you're like, this is what I'm starting for. These are my building blocks. And now if you have a problem, which one of the building blocks do I need to go fix? And that can make conversations go really fast. It can also prevent the situation where there's a coworker or a manager who gets lost in the details because they love details. They're very good. That's why they got to where they were. But then you end up having a conversation about why that text label line wraps, which is not productive. And then you can say, you're right, Mike. It does line wrap. We'll fix that. But, What I really need to know is which part of these assumptions is flawed. Got it.

That, that's— I like, I like that way of thinking. And I— this is definitely going to help me think differently too. So I appreciate that. Cindy, I'm excited to spend some time with you this September at Industry. I know we only have, you know, a minute or two left, but any idea of what we might be able to expect in that working session or, you know, anytime somebody might work with you for a workshop on challenging assumptions, just what might go into that. I'm just curious for those that might be interested.

Sure. So I talk a lot about cognitive biases because I find that the best way to overcome them is to know them. So there are all these shortcuts that our brains make to help us deal with the world and the million pieces of knowledge that are thrown at us every second. And a lot of them are not super logical, like they kept us from being eaten by the woolly mammoth, but they're not good for building product. So I tend to use it as a foundation because it's a way of grounding things that sound kind of fluffy or soft skillsy, which sometimes gets dismissed, in like, no, this is actually how your brain chemistry works. Like, these are things that you, that happen to you, they happen to your customers, they happen to your boss. So know this, accept it, and then here's how to work around it.

That's awesome. That's awesome. Well, Cindy, I appreciate you taking the time to be with us here today. If folks wanted to continue the conversation with you, I know Twitter, it's just @CindyAlvarez, right?

Yep.

Awesome. And yeah, so I thank you for spending the time with us. Thank you everybody for joining us for this chat with Cindy. This will be immediately available to stream afterwards. So if you didn't catch the beginning or anything like that, you could catch it then. And again, Cindy, excited to have you at Industry. I know we'll be talking a lot more before then. Yep. Awesome. Well, thanks everybody. We have a couple more product launch sessions that we've already listed coming up in May. So check those out, ones with Wade Shearer, and the other is with Peldi, the founder of Balsamiq. So getting into design a little bit in May, so I'm excited for that. And we have plenty more coming the rest of the year. So thanks everybody. As always, if you have any ideas or thoughts or feedback, feel free to email me at mike@productcollective.com. But until the next product launch, we'll, we'll see everybody later. Thanks again.