“The Playbook for Building Great Products” with CPOs of Slack, Rubrik and Grammarly

Anika Gupta, CPO at Rubrik. Noam Lovinsky, CPO at Grammarly. And Tamar Yahoshua, partner at IVP and former CPO at Slack.

Hello everyone. I feel so special sitting here at this desk with my two esteemed colleagues.

I think you are kind of far away.

This is gonna be really informal. So we're here to talk to you about how do you build great products, and I think it's wonderful that we have people who've had a lot of experience in the trenches, and our goal here is to give as much practical advice as we can from, like, the actual experiences and war stories that we have, and we are going to talk for about 30 minutes and then open it up to the floor for questions for 15 minutes. So that's the idea. So first, I would like to introduce Anika Gupta, who is the CPO at Rubrik. She's been there about 2 years, previously was at LiveRamp, and Rubrik is a cloud and data security company, for those who don't know Rubrik. And at LiveRamp, she was the president and head of product and platforms. So thank you, Anika.

Thanks for having me.

And Noam Levinsky, who is the CPO at Grammarly, which is an AI-powered powered writing platform. And he's done a lot of things before Grammarly, been involved in Zero to One projects at Thumbtack, new initiatives at Meta, early at YouTube. And, um, welcome, Noel.

Thank you.

And I'm Tamara Yoshua. I'm a venture partner at IVP, and I was previously the CPO at Slack, and before that at Google and Amazon working on scaled products. Uh, And I'm very excited to have them. And both Rubrik and Grammarly are our IVP portfolio companies, which we're very excited about. Um, so first I want to just get an idea of who's in the room. How many people work in a product org, either as a CPO or a product person? Awesome. And how many people are either CEOs or founders? Wow. So we had about a split here. So hopefully we'll have something for all of you, of both on the CEO founder side and on the product and CPO side. So the, um, I would like to start with the most important thing for a CPO. A CPO has to build trust with the CEO and founder. I know this is obvious and a lot of the guidance points I'm gonna give here are gonna be like pretty obvious. The question is how you do this. So I'm gonna ask both of them, but I'm gonna start with Noam. How do you build trust with the CEO or founder?

Well, the thing to do is you get in there and start making all the decisions and setting all the strategy from the first minute, as if you know everything and you've already done everything. That's typically the best way to warm yourself to a CEO or founder. But in all seriousness, I, you know, it's funny, I give this speech to PMs joining a new team, and I think the speech is true for PMs and product leaders at all levels. And that's that your first stage is you need to come in and be the team note-taker. And I mean that both figuratively and literally. Carry the water, take the notes during the meetings, help the team on their process, make them a better executing, operating team in the way that they want to be, and find ways where they can start operating more efficiently, better. Start there. Then the next phase is you move to the team evangelist. Make sure that everyone to the side of you and above you knows what your team is doing, knows what your goal is, knows what your strategy is, so that they can kind of work together and complement one another. And then finally, stage 3 is, okay, start setting and creating strategy together with the leadership team on the team. And I think that a similar arc is true even in the CPO role. Like, you should, you should start kind of in listening mode. You should start in not-making-decision mode. You should start in accelerating and, you know, helping the team do what they already do best. And as you learn and as you kind of start carrying the water, then eventually you get to the point of like, all right, now let's set some goals, let's set some strategy together and actually build together. So that's kind of my view of how the warm-up happens.

Yeah, I mean, I completely agree with Noam. What I'll add to this is, you know, Tamar and I were actually just backstage and we ran into a tech CEO of a very successful company. And he was like, what do you do? Because, you know, us as CEOs, we really want to be involved in the product. And my answer to that was, you just accept that. And I think a really, really big foundational part about when starting at a company as a chief product officer and building that relationship with the founders and CEOs is asking the question really directly to the CEOs and to the other founders. How do you want to be involved in the product? What role do you want to play? What problems are you struggling to solve? And figuring out what are, like, how do they want to get involved? Where are the gaps? How can you start to add value in small ways and build that trust? And then figure out like how that working relationship is gonna go. And not trying to push your own agenda so much into that working relationship, but really, as Noam was talking about, sitting back and listening and understanding their role and understanding how to be integrated with that and pushing the company forward. And it all comes back to like asking great questions and being highly emotionally aware aware of where people want to play and also what their capacity is so that you can figure out where to fit in into the puzzle. Because it really has to be a puzzle in order for the company to be successful and the executive team to be integrated.

So let me summarize that by saying, come in softly, listen and add value quickly. And note-taking is a way of adding value, so make sure you're doing something that adds value. And I will add on from my experience, On top of that is just spend a lot of time together.

Yeah.

And if you're new coming into the company as a CPO and there's a founder CEO who's been there for a while, it is a challenging relationship and you have to get it right. And the only way you get it right is spending a lot of time together.

So, um, can I add one thing which I feel like maybe is really relevant for CEOs and CPOs in the room, that almost everyone shortcuts is the onboarding, the onboarding piece. You know, several times in my career this happened at Grammarly, this happened at Thumbtack. You come in, I'm like, listen, I've got a strong onboarding plan, here's my 6-week plan, here's what I'm going to do, here are the one-on-ones I'm going to have, here's where I'm going to listen, I'm not going to make decisions at this point, and you get aligned with your CEO and your founder on that, and everyone's like, yes, yes, we're going to do that. And sure enough, at Thumbtack, like week 2, it's like, all right, well, we got to run company planning for next year, so can you do that please? Or, and you You know, it's inevitable in these fast-moving, hard-charging places. You want to get to, like, let's start getting things done. But I have to tell you that that upfront onboarding boot-up process, if you really try to respect that and really try to invest in that, it just pays dividends in month 6, in month 9, in month 12. And I've seen so many failure cases happen because, to your point, you didn't spend that time.

You didn't spend the time.

And then you run into an issue later that it's too late to kind of— yeah.

Very good point. So for those founders and early CEOs in the audience, Annika, when should you hire your first CPO?

So I get asked this a lot by founders and CEOs at different stages of the company. And I do think that you really have to evaluate when do you need your first head of product. And head of product may not be a chief product officer. Chief product officer is Usually a very senior role.

You—

I have seen a lot of companies that have made mistakes trying to bring in a really senior person too early. And the failure mode there is then you get someone who's really high-powered, who wants to take the reins, who wants to make all these changes. And either A, your company doesn't have like product-market fit, so they don't have enough to do, or B, it's like they come in and, and they really want to do the role that the CEO actually wants to do. So my advice is to like really think about what you need, really think about what is the role that you want to play. Understand that if you're pre-product market fit, you may want to be at the reins. Like, you or your co-founder should be at the reins driving the product. And only when you get to the point where you're like, wow, I'm breaking, I can't spend the time, I have too many customers, I need to start prioritizing the feedback, then you may want to start bringing in a product manager. And maybe the first product manager hire is more of like a director of product management. And then as you scale and as you get to a point where you really need someone to not only prioritize your roadmap but really build the org, build the team, and build more structure around how you're gonna strategize around your second, third, and fourth product, maybe that's when you bring in a more high-powered, um, uh, product leader for your organization.

Great answer. As, and as of, uh, as you said, it's amazing how, how often we get this question. Uh, so now you've started to build some trust. Um, Noam, how do you align on the vision and strategy in the early days when you've just arrived with, uh, with the founder to make sure there is this CPO-founder fit?

I think that the aligning on the vision strategy is really about figuring out what problem you're obsessed with. Like, what is the problem and how well do you understand it and how, I would say, stable and enduring is that problem? We all get very solution-focused too quickly and we all get— solutions are shiny objects. It's easy to get excited about them. Oh, we could build this, we could build that. But actually identifying real problems, problems that are real, you know, need real painkillers for businesses or for consumer products. Most of the time people don't know the problem that you're even trying to solve for them. If they did, someone's already solved it. If it's that obvious. So I think that spending time together on that question and make sure you really internalize the founder's understanding of the problem that you're trying to solve and make sure you've actually identified a problem worth solving. And then the, like, how do we solve that problem, I think becomes much easier because that's likely going to shift and move and, you know, as you learn and as you iterate, you're going to update that and adjust that. But if you realize that, like, oh shoot, we thought we were trying to solve this problem, but really We should be solving this problem. That's a much more monumental shift and a change in alignment and strategy than we were still solving this problem for this customer. We tried this way, but it didn't work. Here's what we learned. Let's try this way. That's a much easier thing to manage, I think.

Any tricks for how to keep these conversations focused on the problem?

Yeah, I mean, I think just, you know, keep— you have— you can— there's certain conversational tricks like the 5— keep asking why, right? The 5 whys. Why, why, why?

Oh, like the annoying child?

Yeah, like the annoying child. Exactly. And also just kind of like being very rooted in the sort of product and design thing that is about the customer outcome. Like, what do we want to be true about the world if we succeed? What do we want to be true for this particular type of person, knowledge worker, marketer, whatever, if we succeed? Like, just very much try to ground it in that empathy of what problem are we actually solving for our customer? And how do we know if that's a problem we're solving? And how do we look at that on the regular and make sure that we have conviction on that on the regular?

I like that. Anika, just is, what do you think when you're not aligned? Like when you've gotten to a point and you're very clearly not aligned, is that the time when you quit or is there a better thing to do?

I think there's a better thing to do before you quit. So I experienced this many times at LiveRamp. I was at LiveRamp before Rubrik for 11 years and I worked for 3 different CEOs during that time. And so there were certainly times in that journey, especially later in the journey when we had CEOs who were no longer the founders of the company, where there was more of a struggle amongst the executive team around which direction should we go in. And it manifested as, hey, should we make this new product investment? Should we make this new acquisition? And it was all coming from a place of we need to find new avenues for growth for the company. Which new avenue for growth should we focus on? And I think the approach that I took in trying to shepherd these conversations, and I really felt myself like I had to be a shepherd in this of really getting everyone together and figuring out what path we were going to align on and go with, was one, was to really take the emotion out of the conversation. There were people that had a really favorite pet project or something, and as Noam was talking about, focus on the customer problem. So what's an immediate, what's an urgent and important customer problem that we need to solve? What evidence do we have? How do we look at different options side by side, both in terms of the strength of the problem, the strength of our ability to solve that problem, and the amount of overall investment that it's going to take to solve that problem? And also, it's like, are there other like leaps of faith that we have to make that that are gonna potentially have to be true if we're gonna build a very successful business around that. And when I started laying that out on paper and we started having conversations, it became a lot easier to say, okay, these are the reasons why. And again, ask why, why, why when someone says something or has an objection, but really trying to get to a place where we could say, okay, this feels like the highest probability path for moving forward. And you know, a lot of times it didn't— your job as a chief product officer is not to make everyone happy. It's to, get to a decision and hopefully a decision that's gonna be a right decision for the company. And so sometimes like I had to let go of something that I wanted to do and just disagree and commit because there was enough alignment around a certain idea that we should go pursue that and move that forward.

[Speaker:JASON M. LEMKIN] So it really is being empathetic with the customer, taking the emotion out, focusing on the problem. And focusing on the customer's problem solves a lot of issues. I've always found when you go back to, we used to even say at Slack, sometimes Stuart, the CEO, would say, how do you think our customers would feel if they knew you were working on this today? Whatever that was. And it's very, uh, it grounds you. Uh, so you've created trust with your founder CEO. You have a strategy and you have a vision. You're focusing on the customer's problems. Noam, how do you now deal with the reality of the roadmap? And because once you start building the roadmap, things change. How do you stay focused on the customer problem? And I'm looking for like tactical, what are your processes? How do you make sure that that roadmap is clear?

Yeah, great question. Um, one, I think first thing is I think depending on the stage that you're in and depending on the maturity of the business and How far can you realistically forecast? And how do you want to set your process up around— are you— like, often, for example, Grammarly. Grammarly is a very large business, almost a 1,000-person team in an incredibly dynamic and ambiguous space. First thing I did is, we're not going to be planning yearly. We can't do that. I can't tell you, you know, forecast with confidence, like, what is going to be happening at the end of the year. When I first joined Grammarly, it's like, oh, there's these foundational LLMs. We're going to be doing everything on top of them. 2 months later, it's like, oh, there's these open source LLMs that are just as great and we can do everything on top of it. It's just like, it is so evolving. So I do think that just being realistic of You can set longer-term goals. You can have an idea of where you want to be and what kind of a strategy might be to kind of get you there and differentiate yourself in the space, achieve those goals. But then tactically, you know, how far ahead can you forecast? Do you say, okay, we're gonna plan ahead, you know, set goals, big goals, 6 months ahead, and then we're going to do kind of like month plus 1 roadmapping, and we're going to do a rolling roadmap. And yeah, we're a big team, but like, we can do that, and we can, like, we can build the process to do that. And maybe we plan for— since we're planning for shorter horizons, we can use shorter planning windows to do that. And, uh, and so those, those are the sorts of things that I would—

I like that. Break the mold. You don't have to do what everyone says in the product management books.

Exactly.

Yeah, you can do what's right for your company at that time.

My starting point often is like, I, I've I've had the opportunity to work at things from zero employees stages all the way up to thousands of employees stages. And so often when I kind of go back to first principles of how we should operate here, I actually go back to like, how did we do it when we were 10 people in a room? How did you kind of get to the answer then? What was the process used there? And adapt, start that as your principle and adapt that to the size. You know, there's certain things that won't work in that size. You have to communicate differently, etc., etc. But there are many of those kind of principles that will still carry over, I think, through those, through those stages. So anyway, that's, that's it.

And most people say, oh, we've gotten so big and we're slowing down. You say, well, then go back to the smaller. I like that.

Yeah.

Annika, what do you, what kind of metrics and outcomes do you think matter most when you're putting your roadmap together, and what do you look at?

So what's interesting is, uh, you know, Rubrik is a fairly late-stage company, and we have a multi-product portfolio, and we have different products products that are in different stages of their lifecycle. We have our very mature products that are heavily adopted with thousands of customers, and we have newer products that have hundreds of customers where we just launched our first version 6 months ago. And so when I'm looking at the metrics and, and what we're trying to look for for success in each of these kinds of products, I'm looking at different things. But ultimately what I'm looking at is how are these driving the business forward. And, and for the larger products, we'll be looking at What's the total ARR this is driving for the business? And what are our goals around how much new ACV do we want to be getting from this product? And then how does the, how does our roadmap align to that, align to those goals? Where's the growth going to really come from? How do we make sure we enable high CSAT? Whereas for earlier stage project products, it's really about getting initial customer traction and getting really strong feedback and really understanding like, Do we actually have strong product market fit here? And if we don't, what is it going to take to, to get us there? And, and so we, like, I run quarterly business reviews for all of the different parts of our product portfolio. But the way that I look at and the way we evaluate these different areas truly changes, even though like the templates and all look the same. Um, we have to think about them in a different way because the measures of success are going to be quite different.

So going also a little bit further on the road mapping, prioritization is really hard. And I think this is a CPO's hardest, hardest issue is how do I prioritize my resources? And first it's about also do I build or buy? And some of you are at smaller companies where it's not a question yet, but as you get bigger, uh, do you supercharge your innovation through buying? Rubrik just bought Laminar. Anika just came back from a visit to, uh, in Israel to visit the team for the first time, so she's a little jet-lagged, forgive her. Uh, but I— what, what was that decision-making process like? How did you decide that this was an area you wanted to buy instead of build?

So I would say the journey to acquire, uh, Laminar, which is a cloud data security company, started about a year ago for us. And what we realized, um, at Rubrik is that, um, we had we were not making— like, we service customers that have data that sits on-premise, in the cloud, and across SaaS apps. But a lot of our DNA has come from the on-prem world. And so we looked at ourselves a year ago and we said, hey, if we truly want to achieve this vision of being able to secure data across all of these, all of the areas where data lives, what's standing in the way of us accelerating in the cloud? And we saw this opportunity to say, hey, can we bring in, can we acquire a company that's going to bring in the right DNA that's gonna bring in complementary capabilities and, and actually then accelerate that path. And then we went down this path of really exploring all those companies. We did have a focused set of capabilities that we were looking for, and we knew there was— there were a bunch of startups in an emerging market that were starting to solve these problems. So we actually went and talked to pretty much every single company in the space. We talked to our VC partners like IVP, who had done research into this space. And we got really familiar with what was, what were, who were the companies, what was out there, what were the technical capabilities, how would this fit into our portfolio. And we developed a really strong point of view. And from there we went and actually met teams and found a target where the architecture was going to be complementary, where the culture was really strong, where we thought the product was really strong and had good initial traction or had good bones that we could build off of. And the biggest gap that they had was actually a lack of go-to-market skill. And that was something that we knew could be complementary in, in that we could bring that, that we were really good at incubating new products within our portfolio and that we could bring that expertise and accelerate our joint vision together.

Very impressively methodical.

It was very methodical. Yeah, because it was the first big acquisition that Rubrik had done. I think we wanted to be very thoughtful about it. At LiveRamp, we did a lot of acquisitions. We were probably not as methodical about it. Uh, but I think, uh, you don't necessarily always have to go through this level, like this amount of methodology, but at least kind of going through some of those steps and thinking before you, you acquire, because you want ideally an acquisition is drastically accelerating your roadmap and your ability to achieve your vision. However, most acquisitions fail and most acquisitions fail because you aren't able, you have this idea of what the, the, where the synergies are going to come from, where the acceleration is going to come from, but in the integration, you're not able to manifest those things. And so the work that we did up front was really about how do we prepare for the— how do we know what we're getting into and then how do we prepare for the integration in a way that—

I look forward to talking to you in a year, seeing how the integration went and what you did that made— because I have confidence it will be successful because you are so thorough in thinking about it. So the build versus buy is one thing. And the other thing that if anybody's worked in product, they've always had a salesperson come to them and say, you need to implement this right now or we're going to lose this deal. And if you haven't had that, you're not working with a sales team. So Noam, how do you deal with that?

Well, I mean, I think like a good PM, I think the first, first step is like, you need to be empathetic. Uh, you need to like walk in that person's shoes and understand like what they're, what they're running into. I think oftentimes we're so analytical, we like fall back to like, quantify that for me. You know, what, what portion of pipeline is, is at stake? Okay. Then I can prioritize. Prioritize it relative to that. But you know what? That's not how a lot of sales works, right? A lot of it is they're out there pitching our wares, they're having these conversations, and there's holes in the conversation that stops the conversation and ends the deal. And I think you have to be empathetic to that. You have to be the salesperson yourself. Oftentimes in a product role, to do a good job with a product role, you kind of have to do a little of the other roles too. You wanted good product marketing? Go in there and do good. Go be a product marketer. You want good sales? Go in there and do some of the sales job. And so I think you have to empathize and understand it in that way and then think about your strategy, how you would have that sales conversation, how you do that, and try to do the kind of the art and science of balancing some of those requests versus the work that you have more high conviction that's going to lead to this sort of outcomes. That you're looking for because the numbers are telling you that or something like that. And I think the other thing that I'd say is if you're telling no to your sales team and that's really causing a problem, there's usually something underneath there that actually is a lack of confidence in your overall strategy. Because if you can say, hey, we shouldn't build this right now because we're building this and this is going to be so incredible and we're going to be able to deliver this on this date and you'll be able to tell this story and they're like, eh, then that's actually the underlying issue and you should go figure that out. You're just, you're, you're dealing with a symptom, not the disease.

100%. They, they don't have confidence in the, uh, they don't have confidence in the strategy if they don't understand it. And one of the things I want to really emphasize that Noam said is PMs at all levels need to be meeting with customers. And if your PMs are not meeting with customers and they can't be empathetic and they're not in front of them having to say, no, we're not building this. I just think that's so important, and it changes how people work. So we can't have a conversation without talking about AI in this semester. So there sometimes are transformational technologies that come along. This does happen a lot. This one happens to be really transformational. You got your roadmap. You're balancing your build and buy. You're balancing what the salespeople are saying. And then all of a sudden, this thing comes in. And blows up your whole roadmap. Noam, how do you stay focused on what you were going to do and deal with this, or do you not stay focused and you just pivot?

Yeah, I think like most answers, I think that the honest answer is depends. Um, but I, I, what, what, what I'd say in this, and like the founders and CEOs in the room like know this, you know, when you're in zero to one product development mode, the thing you should be optimizing for is not necessarily even outcomes. You should be optimizing for learning. The faster you learn, the faster you get to confidence in the things that you learn, the sooner you're going to win. And I think oftentimes in more scaled product, we're not optimizing for learning. We're optimizing for the metric. We're optimizing for the outcome. We want to land this thing that we committed to. But then when you have a plan and an LLM comes and punches you in the face, you no longer have a plan and you got to really quickly start the learning, the zero-to-one discovery process to figure out how can you use this new technology, this new platform to accomplish your goal. Does that actually change something about your roadmap? Do you need to completely rewrite the roadmap? And I think you need to move some portion of the team into this mode where you're not necessarily optimizing for the win. If you've learned what the win might be, amazing. Capture the learning. Don't worry so much about when you're going to capture the win. But you gotta have some portion of the team that's oriented that way and is, is doing that sort of product development.

Fast iterations, fast learning.

Exactly. Yeah.

Anika, one of the things that you and I talked about earlier is that sometimes you have to put a vision out there even when you don't know if it's the right vision. So how did you do that?

Yeah, I, so one of the things that I did like early on is, uh, in when I was connecting with our CEO and other founders around where do we want to take the company? Is really starting to understand from them, from our customers, and from other people on the team, like, what are the— where do we want to be as a company? And then what are the existential risks that we see in the market and in anything around us? What's gonna stop us from achieving that vision? And I think in asking those questions, it really creates clarity around, okay, there's a lot of, you know, there's always this pie-in-the-sky vision of like, oh, we want to be the number one cybersecurity company. Okay, that's awesome. We absolutely want to do that. But then thinking about like vision from like also how are we going to get there? What are the things that we have to do and where do we want to, want to take this? It really comes down to like being really looking at where we are today, where are our customers going, what are the existential risks for the business? And then like keeping that North Star pie-in-the-sky vision in mind and then trying to bring that all together. And the way that I bring it all together is, is ultimately is coming up with like, hey, here are the 5 things that we need to do, or here are the 3 things. It can't be more than 5, but here are the 3 to 5 things that we have to do over the next 4 years that are going to get us to where we wanna be. And getting alignment on those 3 to 5 things and then just repeating it, repeating it, repeating it, repeating it in every forum. I do it every time I have a product all hands. I do it with customers. I do it with our sales teams. I do it with our engineering teams so that everyone is super, super aligned on like, Not just like, okay, this is, we have this, this goal out here, but this is really what it's going to take to get there.

Repetition, repetition until you're bored of saying something because, uh, people don't, people don't get it, which is actually my final point here is about communicating. When you, how do you make sure that you've communicated to your team, to your peers, to your customers? Noam, some like super tactical, like What are the brass tacks of things that you have to do to Annika's repetition?

I find that one of the most effective things is just paint a story. I think one of your main jobs as CPO is chief storyteller. I think you actually have to visualize and manifest the journey and make sure that everyone has that movie in their head. Brian Chesky at Airbnb, there's the famous thing where they did the Cinderella thing on the wall where they basically illustrated with Cinderella the Airbnb experience, that journey. And so when everyone walked in, they walked past that journey of what is the Cinderella Airbnb experience. I don't think you can do something that metaphorical, but I think that's the idea. You should— that vision that you have, that destination that you have, whether it's a year away, it's 2 years away, manifest it through something as sort of high fidelity and as easy to consume as possible and distribute that widely Talk to it, repeat it, present it, and then update it at some regular cadence as you're learning and as you're going. So everyone kind of has that story in their head week to week, quarter to quarter. So all their little individual decisions, you know, not little, but little, I mean day-to-day decisions, are aligned towards that overall vision.

People remember stories. That's what resonates with people. Storytelling is important. And the other thing I think is really important is different modes of communication. Some people are visual learners. Some people want text. Some people want to hear you. So you have to cover all the mediums so that you cover everyone. With that, this is just a summary of the things that we talked about. Hopefully you remember the stories, but I'm going to open it up for questions. There are mics in each area if people have any questions. Feel free to come up to the mics. If not, I'm happy to keep asking questions.

Happy to kick it off. Thanks, folks. Um, so I've got a question, uh, for you, Noam. So Grammarly obviously is something that a lot of people know as consumers, um, in the Chrome sort of extension store. That's great. You've also got obviously a lot of business, um, users as well. When you've got PMs in your team, or sort of perhaps previously, there's some obvious things that need to change in how you do product work between the two. What's something that catches people out when sort of working on stuff for consumers versus businesses? I'm sorry, it's very, very hard to hear. Can you repeat that a little bit louder? Long story short, how do you do work between consumer products and business product, and what catches people out beyond the obvious? Got it. The question I would ask back is, is Microsoft Word a consumer product or a business product? And obviously it's a product for everyone. But when people at Microsoft are thinking of what they're building, they're building for business people. That doesn't mean that you can't use it to, you know, create a flyer for the high school dance or you can't use it to write a nice note, a thank you note to someone. But the people that are building it are thinking about a business use case. That's Grammarly. We are a B2B product. Half of our users use it at work. The other half use it at school. Those two are actually a lot more similar than you think. You know, they're writing you care about for an outcome that you really care about to achieve a goal that you really care about. We really have a hard time finding things that don't generalize between those two personas. And so there's no such thing as someone at Grammarly building for consumers. That's not really a a word we use. We're all building for professionals, but the thing is we're building a mass market product that every professional on the planet can use. And turns out that means pretty much everyone on the planet should be able to use it. That's why I use the Microsoft Word analogy. Does that make sense? Yeah, really helpful. Thanks. Thank you. Hi, my name is Joris. What advice would you give to hire product managers. They need to think strategically and operationally, emphasize with the customer, work with development. What advice would you give them?

Uh, I would say the number one thing I look for in product managers are people that are curious and can learn quickly because you can always hire PMs that have domain experience. You can hire, but your needs, especially if you're at a startup, your needs are gonna dynamically change. And I feel like I can teach a lot of the craft of product management, but I can't teach people to be curious, and I can't teach people to have the intellectual rigor to learn quickly. So that is the number one thing that I look for.

Yeah, there's many dimensions to what is a PM for different projects, for different roles, but I would kind of think about it this way. First, completely agree with what Annika said. I think there's just some fundamentals that you must have. You must be a strong communicator. There's no way around it. You must be a good systems structured thinker. You have to have some analytical chops as well for, I think, modern-day product management, the way that it's done. And then I think there are certain spikes, like are you more of a growthy PM where you're much more numbers-driven and much more experiment-driven? Are you more of a designy PM? More into like the consumer experiences, like the end-user experience, and is that the spike you want? I think that the one thing that is also I think very hard to coach and teach for that you should really try to find ways to evaluate for is what I just called product instinct. And maybe the way I would explain that is there's one method of doing product management which is you can run as many experiments as you possibly can to go from here to there. But a good product manager knows the right number of experiments that you might need to run to go from here to there, and that requires some instinct. And how does that instinct develop? It's experience. It's being obsessed with other products. It's by seeing patterns in other products, understanding user and customer behavior, and developing an intuition that's kind of backed by some experience that allows you to kind of navigate that path of, from, you know, question to answer in as few steps as you can rather than by brute force A/B experimentation from here to there. Does that make sense?

The one thing I'll add to that is I also believe in a strong diversity in the team, diversity of skills.

Definitely.

In that product managers, there's a huge range of people who are more technical or more, as Nom said, designy. But having a mix of those skills in a team makes a much richer team. Totally. I'm gonna say something controversial, but I find it very hard to really evaluate PMs. And so if they're not working, change it out fast.

Yeah, thanks. Really good point. I also would say like, it's not only just diversity of skill set, but like diversity of like gender, race. Like when you're building a product for billions of people or millions of people, if your team doesn't represent your customers, you're going to run into troubles. I'm happy to talk about experience of that at Google and YouTube and Facebook. Like it is very important to a diverse team from both skill set and background.

Absolutely.

Yeah.

Hello. Firstly, thank you so much for such an interesting talk. So my question for you, Noam. So over the past years, we have seen so many products trying to compete with Grammarly, right? Saying that, hey, we are better in this, we are better than that. But you guys have still managed to stay ahead of the curve, like especially for the product front, how you have been able to kind of continuously grow and stay ahead? Yeah, so I've been at Grammarly for 6 months, so I'm not going to take credit for a lot of that, but let me just say this. What's very obvious, and actually it's obvious, Tamara was even complaining to me about this backstage, it's an example of how good and obvious it is. When you're dealing with a use case like communication, when it's something that people do every day, all the time, small differences in quality matter a lot. Um, Tamara was just telling me that she's noticing a decrease in quality. Like, to have a product that you use often enough, that's in your face often enough, where you would actually notice that says a lot. And while, um, you know, everyone was like, oh, all you need is spelling and grammar, and that's in Google, that's in Google Docs, and that's, that's in, that's in Office. Grammarly's No, it's not just about spelling and grammar actually. If you want to own and really enable effective communication, you have to go much deeper. It's about tone and clarity and fluency and all of these things. And small differences in quality of those things, even a point here or there, because we see it in the numbers, it makes a really big difference. So I think it's just one of those use cases that you need to own deeply and where the quality really matters. And frankly, that's why You know, something like ChatGPT can create words for you, but how often do you use those words in your day-to-day work on a regular basis? It's a huge gap from "I can create words for you" to "These words would be really good for this moment to represent you in this interaction with a colleague." There's a huge gap, right? And so I think that's what we're going to watch play out, I think, over the next couple years. Perfect. Thank you. By the way, I love your weekly insights. So thank you. Thank you.

Yeah. Hi.

So when hiring my first PM, how many experiences, how many years of experience and how senior he should be for the first PM at a startup or company?

Ooh, that's a good question. I think that really, it really depends. It really depends on what you want that PM to be doing and again, what role you want to be playing. In, in product. I think you heard, I think, a great summary of the things to look for when hiring a PM. If it's truly like your first PM, you really just want them to help. You're getting a lot of feedback from customers. You're getting that.

You're—

you have the vision, but then you want your PM to really help with the execution. You don't need someone that's super experienced, but you do need someone with good judgment. You do need someone that's gonna learn quickly, and someone that's gonna be able to work very effectively with the rest of your team and make that happen.

Even for a pre-product market fit area?

Yes.

Yeah. You, I would say the CEO founder is usually setting the strategy and a big mistake people make is I need somebody really strong in strategy as my first PM and then they just clash. So instead it's like you need somebody on the ground meeting with customers, understanding if there is product market fit, giving you the feedback and being like, really the scrappy person making it happen. So they complement the skill set of the people who are there.

Yeah. And I would just say, I think that for pre-product market fit especially, like, you, you know, you're, you're searching for very faint signal, very, very faint signal in a lot of noise. And the more layers of indirection you put between yourself and the ability to observe and hear that single signal, excuse me, the more likely you are to fail. And so like, I think this is true for research. I think this is true for product. I think I'm not saying you shouldn't have these. If you, if you need these functions, you shouldn't, you shouldn't have them. But I would make sure that there's zero layers of indirection between the founder and that signal when you're pre-product market fit and you're trying to discover, discover that because it's just the game of telephone is very quickly going to create so much signal loss that you'll, you'll lose what what little signal there may be there.

True.

So not too senior, like a couple of years of experience, who can do the operational work, get data. I would say min bar is you should not hire a PM who you'd have to tell how to do the PM job when they went back to their desk. You should be able to say, hey, these are the goals, like, these are the things we want to build, etc., etc., and they should know how to kind of go back to their desk and do that with the engineering team without you having to say Okay, step 1 is this, step 2— that's, I would say, the min bar at that stage. If you're kind of training someone on the job in that way, I wonder how much kind of like leverage you would get of that. Perfect. Thank you so much.

Cool.

How's it going? I'm curious for like companies that are pre-product market fit, like how much should they be iterating and when and why should they be iterating and at what point is the decision made that, hey, this market's too crowded, it's not for us, we should move on or quit, you know? Who wants to—

You want to take that one, Noel?

Sure. That's like the million-dollar question, right? Or I guess the billion-dollar question. I would just say that there's— this is art and science, right? I guess what I would say is as much iteration as possible to kind of get to that get to that signal and feel the pull. I don't know how to describe it other than that, but when you've felt like— when you've hit it, everything starts to move a little differently. Your customer is pulling the solution from you. They're willing to work through all the janky UI and everything else just to kind of get to the answer. And if you're looking for it in retention, or you're looking for it in these kind of down-funnel metrics, and you're trying to like— it's a lot of times it's hard to see it in that sort of signal early on. And this is why I think you have to like sit with people, watch them use it, like get their actual reaction, like put it out in the wild in certain ways, in good kind of good testing environments, and see the pickup and see what happens. And I think you should keep going until you run out of money or don't have any ideas of what else to do. That would be kind of my answer.

One of the things I think is really important is when you're looking at your product is, are people complaining about it? The more people complain, that means they care. If there's silence, that's terrible. But if they're like, you need to add this, and they come angry, that means that they like— it's important to them, and it's important enough that they're gonna complain to you about it. And people sometimes mistake that to all our customers are unhappy. No, your customers care because it's important.

Yeah.

And that's something that I, you even find internally within teams. If you're a PM and your engineers are constantly coming to you and saying, this isn't working, this isn't working, that means that like something's moving. 'Cause if they're not doing anything, then nothing will be broken. So it's kind of that same theme. But you have to, you yourself also have to have conviction. You have to believe that this is a problem worth solving. And that's when you give up is when you're not sure if the problem is worth solving. To Noam's first point earlier, it's not about the solution, it's about the problem. And if this problem isn't important to anyone, then you should pivot.

Yeah.

We're exactly at zero. My God. Perfect timing. Thank you all for coming.

Thank you.

Nice job.