“The Playbook for Building Great Products” with CPOs of Slack, Rubrik and Grammarly

Annika gupta, cpo at rubrik. Noam levinsky, cpo at Grammarly, and tamar yeosua, partner at ibp and former cpo at slack.

Hello, everyone. I feel so special sitting here at this desk with my two esteemed colleagues. Yeah, I think you are kind of far away. This is going to be really informal. So we're here to talk to you about how do you build great products. And I think it's wonderful that we have people who've had a lot of experience in the trenches and our goal here is to give as much practical advice as we can from like the actual experiences and war stories that we have. And we are going to Talk for about 30 minutes and then open it up to the floor for questions for 15 minutes. So that's the idea. So first I would like to introduce Anika Gupta, who is the CPO at Rubrik. She's been there about two years, previously was at Liveramp and Rubrik is a cloud and data security company for those who don't know Rubrik. And at Liveramp she was a president and head of product and platforms. So thank you, Annika.

Thanks for having me.

And Noam Levinsky, who is the CPO at Grammarly, which is an AI powered writing platform. And he's done a lot of things before Grammarly, been involved in Zero to One projects at Thumbtack, new initiatives at Meta, early at YouTube and welcome, Noam.

Thank you.

And I'm Tamari Oshu. I'm a venture partner at ivp and I was previously the CPO at Slack and before that at Google and Amazon, working on scaled products. And I'm very excited to have them. And both Rubrik and Grammarly are IVP portfolio companies, which we're very excited about. So first I want to just get an idea of who's in the room. How many people work in a product org, either as a CPO or a product person. Awesome. And how many people are either CEOs or founders? Well, so we have about a split here, so hopefully we'll have something for all of you both on the CEO founder side and on the product and CBO side. So I would like to start with the most important thing for a cpo. A CPO has to build trust with the CEO and founder. I know this is obvious and a lot of the guidance points I'm going to give here are going to be like, pretty obvious. The question is, how do you do this? So I'm going to ask both of them, but I'm going to start with noam how do you build trust with a CEO or founder?

Well, the thing to do is you get in there and start making all the decisions and setting all the strategy the first minute as if you know everything and you've already done everything. That's typically the best way to warm yourself to a CEO founder. But in all seriousness, I, you know, it's funny, I give this speech to PMs joining a new team, and I think the speech is true for PMs and product leaders at all levels. And that's that your first stage is you need to come in and be the team notetaker. And I mean that like both kind of figuratively and literally. Like, you know, carry the water, take the notes during, during the meetings, like, help the team on their process, make them a better executing, operating team in the way that they want to be, and find ways where they can start operating more efficiently. Better start there. Then the next phase is you move to the team evangelist. Make sure that everyone to the side of you and above you knows what your team is doing, knows what your goal is, knows what your strategy is. So that can kind of work together and compliment one another. And then finally stage three is, okay, start setting and creating strategy together with the leadership team on the team. And I think that a similar arc is true even in the CPO role. Like, you should, you should start kind of in listening mode, you should start in not making decision mode. You should start in accelerating and helping the team do what they already do best. And as you learn and as you kind of start carrying the water, then eventually you get to the point of like, all right, now let's set some goals, let's set some strategy together and actually build together. So that's kind of my view of how the warm up happens.

Yeah, I mean, I completely agree with Noam. What I'll add to this is Tamar and I were actually just backstage and we ran into a tech CEO, a very successful company. And he was like, well, what do you do? Because us as CEOs, we really want to be involved in the product. And my answer to that was, you just accept that. And I think a really, really big foundational part about when starting at a company as a Chief Product officer and building that relationship with the founders and CEOs is asking the question really directly to the CEOs and to the other founders, how do you want to be involved in the product? What role do you want to play? What problems are you struggling to solve? And figuring out what, what are? Like, how do they want to get involved, where are the gaps? How can you start to add value in small ways and build that trust and then figure out like how that working relationship is going to go and not trying to push your own agenda so much into that working relationship, but really as Noam was talking about, sitting back and listening and understanding their role and understanding how to be integrated with that and pushing the company forward. And it all comes back to like asking great questions and being highly emotionally aware of, of where people want to play and also what their capacity is so that you can figure out where to fit in, into the puzzle. Because it really has to be a puzzle in order for the company to be successful and the executive team to be integrated.

So let me summarize that by saying, come in softly, listen and add value quickly. And note taking is a way of adding value. So make sure you're doing something that adds value. And I will add on from my experience, on top of that is just spend a lot of time together. If you're new coming into the company as a CPO and there's a founder, CEO who's been there for a while, it is a challenging relationship and you have to get it right. And the only way you get it right is spending a lot of time together. So can I add one thing which.

I feel like maybe is really relevant for CEOs and CPO's in the room that almost everyone shortcuts is the onboarding. The onboarding piece. You know, several times in my career this happened at Grammarly, this happened at Thumbtack. You come in and I'm like, listen, I've got a strong onboarding plan. Here's my six week plan. Here's what I'm going to do. Here are the one ones I'm gonna have. Here's where I'm gonna listen. I'm not gonna make a decision at this point and you get aligned with your, with your CEO and your founder on that and everyone's like, yes, yes, we're gonna do that. And sure enough, at Thumbtack, like week two, it's like, all right, well we gotta run company planning for next year, so can you do that please or. And you, you know, it's inevitable in these, like fast moving, you know, hard charging places. You want to like, get to like, let's start getting things done. But I have to tell you that upfront onboarding Buddha process, if you really try to respect that and really try to invest that, it just like it pays dividends in month six, in month nine and month 12, and I've seen so many failure cases happen because to your point, you didn't spend that time.

You didn't spend the time and then.

You run into an issue later that it's too late to kind of.

Yeah, very good point. So for those founders and early CEOs in the audience, Anika, when should you hire your first cpo?

So I get asked this a lot by founders and CEOs at different stages of the company and I do think that you really have to evaluate when do you need your first head of product. And head of product may not be a chief product officer. A chief product officer is usually a very senior role. I have seen a lot of companies that have made mistakes trying to bring in a really senior person too early and the failure mode there is, then you get someone who's really high powered, who wants to take the reins, who wants to make all these changes and either A, your company doesn't have product market fit so they don't have enough to do, or B, it's like they come in and they really want to do the role that the CEO actually wants to do. So my advice is to really think about what you need, really think about what is the role that you want to play. Understand that if you're pre product market fit, you may want to be at the reins. Like you or your co founder should be at the reins driving the product. And only when you get to the point where you're like, wow, I'm breaking, I can't spend the time, I have too many customers, I need to start prioritizing the feedback, then you may want to start bringing in a product manager. And maybe the first product manager hires more of like a director of product management. And then as you scale and as you get to a point where you really need someone to not only prioritize your roadmap, but really build the org, build the team and build more structure around how you're going to strategize around your second, third and fourth product. Maybe that's when you bring in a more high powered product leader for your organization.

Great answer. And as you said, it's amazing how often we get this question. So now you've started to build some trust. Noam, how do you align on the vision and strategy in the early days when you've just arrived with the founder to make sure there is this CPO founder fit?

I think that the aligning on the vision strategy is really about figuring out what problem you're obsessed with. Like what is the problem and how well do you Understand it and how I would say stable and enduring is that problem. We all get very solution focused too quickly and we all get. Solutions are shiny objects. It's easy to get excited about them. Oh, we could build this, we could build that. But actually identifying real problems, problems that are real, you know, need real painkillers for businesses or for consumer products. Most of the time, people don't know the problem that you're even trying to solve for them. If they did, someone's already solved it, if it's that obvious. So I think that spending time together on that question and make sure you really internalize the founder's understanding of the problem that you're trying to solve and make sure you actually identified a problem worth solving. And then the like, how do we solve that problem? I think becomes much easier because that's likely going to shift and move. And as you learn and as you iterate, you're going to update that and adjust that. But if you realize that, like, oh, shoot, we thought we were trying to solve this problem, but really we should be solving this problem, that's a much more monumental shift and a change in alignment and strategy than we were still solving this problem for this customer. We tried this way, but it didn't work. Here's what we learned. Let's try this way. That's a much easier thing to manage, I think.

Any tricks for how to keep these conversations focused on the problem?

Yeah, I mean, I think just keep. There's certain conversational tricks, like the five keep asking why the five whys. Why? Why? Why?

Oh, like the annoying child.

Yeah, the annoying child, exactly. And also just kind of like being very rooted in this sort of product and design thing that is about the customer outcome. Like, what do we want to be true about the world if we succeed? What do we want to be true for this particular type of person? Knowledge worker, marketer, whatever. If we succeed, like, just very much try to ground it in that empathy of what problem are we actually solving for our customer? And how do we know if that's a problem we're solving? And how do we look at that on the regular and make sure that we have conviction on that on the regular.

I like that. Annika, just is, what do you think when you're not aligned, like, when you've gotten to a point and you're very clearly not aligned, is that the time when you quit or is there a better thing to do?

There's a better thing to do before you quit. So I experienced this many times at Liveramp. I was at Liveramp before Rubrik for 11 years. And I worked for three different CEOs during that time. And so there were certainly times in that journey, especially later in the journey, when we had CEOs who were no longer the founders of the company, where there was more of a struggle amongst the executive team around which direction should we go in? And it manifested as, hey, should we make this new product investment? Should we make this new acquisition? And it was all coming from a place of, we need to find new avenues for growth for the company. Which new avenue for growth should we focus on? And I think the approach that I took in trying to shepherd these conversations, and I really felt myself like I had to be a shepherd in this of really getting everyone together and figuring out what. What path we were going to align on and go with was one was to really take the emotion out of the conversation. There were people that had, like, a really favorite pet project or something. And as Noam was talking about focus on the customer problem. So what's an immediate. What's an urgent and important customer problem that we need to solve? What evidence do we have? How do we look at different options side by side, both in terms of the. The strength of the problem, the strength of our ability to solve that problem, and the amount of overall investment that it's going to take to solve that problem? And also, are there other leaps of faith that we have to make that are going to potentially have to be true if we're going to build a very successful business around that? When I started laying that out on paper and we started having conversations, it became a lot easier to. To say, okay, these are the reasons why, and again, ask why, why, why? When someone says something or has an objection, but really trying to get to a place where we could say, okay, this feels like the highest probability path for moving forward. And, you know, a lot of times your job as a Chief Product officer is not to make everyone happy. It's to get to a decision, and hopefully a decision that's going to be a right decision for the company. And so sometimes, like, I had to let go of something that I wanted to do and just disagree and commit because there was enough alignment around a certain idea that we should go pursue that and move that forward.

So it really is being empathetic with the customer, taking the emotion out, focusing on the problem and focusing on the customer's problem solves a lot of issues. I've always found. When you go back to. We used to even say at Flack, sometimes Stuart, the CEO would Say, how do you think our customers would feel if they knew you were working on this today, whatever that was? And it's very, it grounds you. So you've created trust with your founder, CEO. You have a strategy and you have a vision. You're focusing on the customer's problems. Noam, how do you now deal with the reality of the roadmap? And because once you start building the roadmap, things change. How do you stay focused on the customer problem? And I'm looking for like tactical. What are your processes? How do you make sure that that roadmap is clear?

Yeah, great question One I think first thing is, I think depending on the stage that you're in and depending on the maturity of the business and how far can you like realistically forecast and how do you want to set your process up around are you like often? Like for example at Grammarly. Grammarly is, you know, a very large business, almost 1000 person team in an incredibly dynamic and ambiguous space. First thing I did is we're not going to be planning yearly. We can't do that. I can't tell you forecast with confidence, like what is going to be happening at the end of the year. When I first joined Grammarly, it's like, oh, there's these foundational LLMs. We're going to be doing everything on top of them. Two months later it's like, oh, there's these open source LLMs that are just as great and we can do everything on top of it. It's just like it is so evolving. So I do think that just being realistic of you can set longer term goals. You can have an idea of where you want to be and what kind of a strategy might be to kind of get you there and differentiate yourself, differentiate yourselves, space, achieve those goals. But then tactically, you know, how, how far ahead can you forecast? Do you say, okay, we're going to, we're going to plan ahead, you know, six set goals, big goals, six months ahead. And then we're going to do kind of like month plus one roadmapping and we're going to do a rolling roadmap. And yeah, we're a big team but like we can do that and we can, we can build the process to do that and maybe we plan for, since we're planning for shorter horizons, we can use shorter planning windows to do that. And so those are the sorts of.

Things that I would, I like that break the mold. You don't have to do what everyone says in the product management books.

Exactly. Yeah.

You can do what's right for your company at that time?

My starting point often is like I've had the opportunity to work at things, things from zero employees stages all the way up to like, you know, thousands of employee stages. And so often when I kind of go back to first principles of how we should operate here, I actually go back to like how did we do it when we were 10 people in a room? Like how did you kind of get to the answer then? What was the process used there? And adapt, start that as your principle and adapt that to the size. There's certain things that won't work in that size. You have to communicate differently, et cetera, et cetera. But there are many of those kind of principles that will still carry over I think through those stages. So anyway that's.

And most people say, oh, we've gotten so big and we're slowing down. You say, well then go back to the smaller. I like that. Anika, what kind of metrics and outcomes do you think matter most when you're putting your roadmap together and what do you look at?

So what's interesting is Rubrik is a fairly late stage company and we have a multi product portfolio and we have different products that are in different stages of their life cycle. We have our very mature products that are heavily adopted with thousands of customers and we have newer products that have hundreds of customers where we just launched our first version six months ago. And so when I'm looking at the metrics and what we're trying to look for for success at each of these kinds of products, I'm looking at different things but ultimately what I'm looking at is how are these driving the business forward and for the larger products we'll be looking at what's the total ARR this is driving for the business and what are our goals around how much new ACV do we want to be getting from this product and then how does our roadmap align to those goals? Where is the growth going to really come from? How do we make sure we enable high csat? Whereas for our earlier stage products it's really about getting initial customer traction and getting really strong feedback and really understanding do we actually have strong product market fit here and if we don't, what is it going to take to get us there? And so I run quarterly business reviews for all of the different parts of our product portfolio. But the way that I look at and the way we evaluate these different areas truly changes. Even though the templates and all look the same, we have to think about them in a different way because the measures of success are going to be quite different.

So going also a little bit further on the road, mapping prioritization is really hard. And I think this is a CPO's hardest issue is how do I prioritize my resources? And first it's about also do I build or buy. And some of you are at smaller companies where it's not a question yet, but as you get bigger, do you supercharge your innovation through buying? Rubrik just bought Laminar. Annika just came back from a visit in Israel to visit the team for the first time. So if she's a little jet lagged, forgive her. But what was that decision making process like? How did you decide that this was an area you wanted to buy instead of build?

So I would say the journey to acquire Laminar, which is a cloud data security company, so started about a year ago for us. And what we realized at Rubrik is that we were not making like we service customers that have data that sits on premise in the cloud and across SaaS apps. But a lot of our DNA has come from the on prem world. And so we looked at ourselves a year ago and we said, hey, if we truly want to achieve this vision of being able to secure data across all, all of the areas where data lives, what's standing in the way of us accelerating in the cloud? And we saw this opportunity to say, hey, can we bring in, can we acquire a company that's going to bring in the right DNA, that's going to bring in complementary capabilities and actually then accelerate that path? And then we went down this path of really exploring all those companies and we did have a focused set of capabilities that we were looking for. And we knew there was, there were a bunch of startups in an emerging market that were starting to solve these problems. So we actually went and talked to pretty much every single company in the space. We talked to our VC partners like IVP who had done research into this space and we got really familiar with who were the companies, what was out there, what were the technical capabilities, how would this fit into our portfolio? And we developed a really strong point of view and from there we went and actually met teams and found a target where the architecture was going to be complimentary, where the culture was really strong, where we thought the product was really strong and had good initial traction or had good bones that we could build off of. And the biggest gap that they had was actually a lack of go to market skill. And that was something that we knew could Be complementary, that we could bring that. That we were really good at incubating new products within our portfolio, and that we could bring that expertise and experience, accelerate our joint vision together.

Very impressively. Methodical.

It was very methodical, yeah. Because it was the first big acquisition that Rubrik had done. I think we wanted to be very thoughtful about it. At Liveramp, we did a lot of acquisitions. We were probably not as methodical about it, but I think you don't necessarily always have to go through this level, like this amount of methodology, but at least kind of going through some of those steps and thinking before you acquire because you want. Ideally, an acquisition is drastically accelerating your roadmap and your ability to achieve your vision. However, most acquisitions fail, and most acquisitions fail because you aren't able. You have this idea of where the synergies are going to come from, where the acceleration is going to come from, but in the integration, you're not able to manifest those things. And so the work that we did up front was really about how do we prepare for the. How do we know what we're getting into, and then how do we prepare for the integration? In a way that I look forward.

To talking to you in a year, seeing how the integration and what you did that made. Because I have confidence it will be successful, because you are so thorough in thinking about it. So the build versus buy is one thing. And the other thing that if anybody's worked in product, they've always had a salesperson come to them and say, you need to implement this right now or we're gonna lose this deal. And if you haven't had that, you're not working with a sales team. So, Noam, how do you deal with that?

Well, I mean, I think like a good pm, I think the first. First step is, like, you need to be empathetic. You need to, like, walk in that person's shoes and understand, like, what they're. What they're running into. I think oftentimes we're so analytical, we, like, fall back to, like, quantify that. For me, you know, what. What portion of pipeline is at stake? Okay. Then I can prioritize it relative to that. But you know what? That's not how a lot of sales works. Right. A lot of it is they're out there pitching our wares. They're having these conversations, and there's holes in the conversation that stops the conversation and ends the deal. And I. I think you have to be empathetic to that. You have to be the salesperson yourself. Oftentimes in A product role. To do a good job with a product role, you kind of have to do a little of the other roles too. You wanted good product marketing, go in there and do good, go be a product marketer. You want good sales, go in there and do some of the, some of the sales job. And so I think you have to, you know, empathize and understand it in that way and then think about your strategy, how you would have that sales conversation, how you do that and try to do the kind of the art and science of balancing some of those requests versus the work that you have more high conviction that's going to lead to the sort of outcomes that you're looking for because the numbers are telling you that or something like that. And I think the other thing that I'd say is if you're telling no to your sales team and that's really causing a problem, there's usually something underneath there that actually is a lack of confidence in your overall strategy. Because if you can say, hey, we shouldn't build this right now because we're building this and this is going to be so incredible and we're going to be able to deliver this on the state and you'll be able to tell this story and they're like, eh, then that's actually the underlying issue and you should go figure that out. You're, you're dealing with a symptom, not the disease 100%.

They don't have confidence in the, they don't have confidence in the strategy if they don't understand it. And one of the things I want to really emphasize that NOM said is PMs at all levels need to be meeting with customers. And if your PMs are not meeting with customers and they can't be empathetic and they're not in front of them having to say, no, we're not building this. I just think that's so important and it changes how people work. So we can't have a conversation without talking about AI in this sester. So, you know, there sometimes are transformational technologies that come along. This does happen a lot. This one happens to be really transformational. You got your roadmap, you're balancing your build and buy, you're balancing what the salespeople are saying and then all of a sudden this thing comes in and blows up your whole roadmap. Noam, how do you stay focused on what you were going to do and deal with this? Or do you not stay focused and you just pivot?

Yeah, I think like most answers I think the honest answer depends. But what I would say in this. And like the founders and CEOs in the room, like, know this, you know, when you're in zero to one product development mode, the thing you should be optimizing for is not necessarily even outcomes. You should be optimizing for learning. The faster you learn, the faster you get to like, confidence in the things that you learn, the sooner you're going to win. And I think oftentimes in more scaled product, we're not optimizing for learning, we're optimizing for the metric, we're optimizing for the outcome. We want to land this thing that we committed to. But then when you have a plan and an LLM comes and punches you in the face, you no longer have a plan. And you got to really quickly start learning the zero to one discovery process to figure out how can you use this new technology, this new platform to accomplish your goal? Does that actually change something about your roadmap? Do you need to completely rewrite the roadmap? And I think you need to move some portion of the team into this mode where you're not necessarily optimizing for the win. If you've learned what the win might be. Amazing. Capture the learning. Don't worry so much about when you're going to capture the win. But you've got to have some portion of the team that's oriented that way and is doing that sort of product development.

Fast iterations, fast learning.

Exactly.

Anika, one of the things that you and I talked about earlier is that sometimes you have to put a vision out there, even when you don't know if it's the right vision. So how did you do that?

Yeah, so one of the things that I did, like early on is when I was connecting with our CEO and other founders around where do we want to take the company is really starting to understand from them, from our customers, and from other people on the team, like, what are the. Where do we want to be as a company? And then what are the existential risks that we see in the market and in anything around us? What's going to stop us from achieving that vision? And I think in asking those questions, it really creates clarity around, okay, there's a lot of, you know, there's always this like, this pie in the sky vision of like, oh, we want to be like the number one cybersecurity company. Okay, that's awesome. We absolutely want to do that. But then thinking about like, vision from like, also how are we going to get There, what are the things that we have to do and where do we want to. Want to take this? It really comes down to, like, being really looking at where we are today, where are our customers going? What are the existential risks for the business? And then, like, keeping that North Star pie in the sky vision in mind and then trying to bring that all together. And the way that I bring it all together is ultimately is coming up with like, hey, here are the five things that we need to do, or here are the three things. It can't be more than five, but here are the three to five things that we have to do over the next four years that are going to get us to where we want to be and getting alignment on those three to five things and then just repeating it, repeating it, repeating it, repeating it. In every forum, I do it every time I have a product all hands. I do it with customers, I do it with our sales teams, I do it with our engineering teams so that everyone is super, super aligned on, like, not just like, okay, this is. We have this goal out here, but this is really what it's going to take to get there.

Repetition, repetition, until you're bored of saying something. Because people don't. People don't get it. Which is actually my final point here is about communicating. When you. How do you make sure that you've communicated to your team, to your peers, to your customers? Know some, like, super tactical, like, literally, like, what are the brass tacks of things that you have to do to Annika's like, repetition?

I find that one of the most effective things is, like, just paint a story. Like, I think. I think one of your main jobs as CPO is like, chief storyteller. And I think you actually have to, like, visualize and manifest the journey and make sure that everyone has that movie in their head. You know, Brian Chesky at Airbnb, there's the famous thing where they did the. I think he did the Cinderella thing thing on the wall where they basically illustrated with Cinderella, like, the Airbnb experience, that journey. And so when everyone walked in, they, like, walked past that journey of what is the Cinderella Airbnb experience? I don't think, like, you can do something that metaphorical, but I think that's the idea. You should. That vision that you have, that destination that you have, whether it's a year away, it's two years away, manifest it through something as sort of high fidelity and as easy to consume as possible and. And distribute that widely. Talk to it, repeat it, present it, and then update it at some regular cadence as you're learning and as you're going. So everyone kind of has that story in their head week to week, quarter to quarter. So all their little individual decisions, not little, but little, I mean day to day decisions, are aligned towards that overall vision.

People remember stories. That's what resonates with people. Storytelling is important. And the other thing I think is really important is different modes of communication. Some people are visual learners, some people want tech, some people want to hear you. So you have to cover all the mediums so that you cover all, everyone with that. This is just a summary of the things that we talked about. Hopefully you remember the stories. But I'm going to open it up for questions. There are mics in the, in each area. If people have any questions, feel free to come up to the mics. If not, I'm happy to keep asking questions.

Happy to kick it off. Thanks folks. So I've got a question for you, Noam. So Grammarly, obviously is something that a lot of people know as consumers in the Chrome sort of extension store. That's great. You've also got obviously a lot of business users as well. When you've got PMs in your team or sort of perhaps previously, there's some obvious things that need to change and how you do product work between the two, what's something that catches people out when sort of working on stuff for consumers versus businesses? I'm sorry, it's very, very hard to hear. Can you repeat that a little bit louder? Long story short, how do you do work between consumer products and business product and what catches people out beyond the obvious? Got it. The question I would ask back is, is Microsoft Word a consumer product or a business product? And obviously it's a product for everyone. But when people at Microsoft are thinking of what they're building, they're building for business people. That doesn't mean that you can't use it to, you know, create a flyer for the high school dance or you can't use it to write a nice note, a thank you note to someone, but the people that are building it are thinking about a business use case. That's Grammarly, we are a B2B product. Half of our users use it at work, the other half use it at school. Those two are actually a lot more similar than you think. They're writing you care about for an outcome that you really care about to achieve a goal that you really care about. We really have a hard time finding things that don't generalize between those two. Personas and so there's no such thing as someone that Grammarly building for consumers. That's not really a word we use. We're all building for professionals. But the thing is we're building a mass market product that every professional on the planet can use. And turns out that means pretty much everyone on the planet should be able to use it. That's why I use the Microsoft Word analogy. Does that make sense? Yeah. Really helpful. Yeah. Thank you. Hi, my name is Joris. What advice would you give to hire product managers? They need to think strategically and operationally emphasize with the customer, work with development. What advice would you give them?

I would say the number one thing I look for in product managers are people that are curious and can learn quickly because you can always hire PMs that have domain experience. You can hire but your needs, especially if you're at a startup, your needs are going to dynamically change. And I feel like I can teach a lot of the craft of product, of product management, but I can't teach people to be curious and I can't teach people to have the intellectual rigor to learn quickly. So that is the number one thing that I look for.

Yeah, there's many dimensions to like what is a PM for different projects for different roles. But I would kind of think about it this way first completely agree with what Annika said. I think there's just some fundamentals that you must have, you must be a strong communicator, there's no way around it. You must be a good systems structured thinker. You have to have some like analytical like chops as well for I think modern day product management the way that it's done, you know, and then I think then there are certain spikes like are you more of a growthy PM where you're much more numbers driven and much more experiment driven, you know, are you more of a designy pm? Are you more into like the consumer experiences, like the end user experience? And is that the spike you want? I think that the one thing that is also I think very hard to coach and teach for that you should really try to find ways to evaluate for is what I just called product instinct. And maybe the way I would explain that is there's one method of doing product management which is you can run as many experiments as you possibly can to go from here to there. But a good product manager knows the right number of experiments that you might need to run to go from here there. And that requires some instinct. And how does that instinct develop it's experience? It's being obsessed with other products. It's by seeing patterns in other products, understanding user and customer behavior, and developing an intuition that's kind of backed by some experience that allows you to kind of navigate that path of from question to answer in as few steps as you can, rather than by brute force AP experimentation from here to there. Does that make sense?

The one thing I'll add to that is I also believe in a strong diversity in the team, diversity skills in that product managers, there's a huge range of people who are more technical or more as a designee. But having a mix of those skills in a team makes a much richer team. I'm going to say something controversial, but I find it very hard to really evaluate PMs and so if they're not working, change it out fast.

Yeah, thanks. Really good point. I also would say it's not only just diversity of skill set, but diversity of gender, race. When you're building a product for billions of people or millions of people, if your team doesn't represent your customers, you're going to run into troubles. I'm happy to talk about experience of that at Google and YouTube and Facebook. It is very important to have a diverse team from both skill set and background.

Absolutely.

Hello. Firstly, thank you so much for such an interesting talk. So my question for you, Noam. So over the past years we have seen so many products trying to compete with Grammarly, right. Saying that, hey, we are better in this, we are better than that, but you guys have still managed to stay ahead of the curve, like especially for the product front. How you have been able to kind of continuously grow and stay ahead. Yeah. So I've been at Grammarly for six months, so I'm not going to take credit for a lot of that. But let me just say this, what's very obvious and actually it's obvious Tamara was even complaining to me about this backstage. It's an example of how good and obvious it is when you're dealing with a use case like communication, when it's something that people do every day, all the time. Small differences in quality matter a lot. Tamar was just telling me that she's noticing a decrease in quality. Like to have a product that you use often enough, that's in your face often enough, where you would actually notice that says a lot. And while, you know, everyone was like, oh, all you need is spelling and grammar and that's in Google, that's in Google Docs and that's in office. Grammarly is like, no, it's not just about Spelling and grammar. Actually, if you want to own and really be like, enable effective communication, you have to go much deeper. It's about tone and clarity and fluency and all of these things and small differences in quality of those things. Even a point here or there, we see it in the numbers, it makes a really big difference. So I think it's just one of those use cases that you need to own deeply and where the quality really matters. And frankly, that's why something like ChatGPT can create words for you. But how often do you use those words in your day to day work on a regular basis? It's a huge gap from I can create words for you to these words would be really good for this moment to represent you in this interaction with a colleague. There's a huge gap. Right. And so I think that's what we're going to watch play out, I think over the next couple years. Thank you. By the way, I love your weekly insights. So thank you. Thank you. Yeah. Hi. So when hiring my first pm, how many experiences, how many years of experience and how senior he should be for.

The first PM at a startup or company? I think that really, it really depends. It really depends on what you want that PM to be doing and again, what role you want to be playing in product. I think you heard, I think a great summary of the things to look for when hiring a pm. If it's truly like your first pm, you really just want them to help. You're getting a lot of feedback from customers, you're getting that you have the vision, but then you want your PM to really help with the execution. You don't need someone that's super experienced, but you do need someone with good judgment. You do need someone that's going to learn quickly and someone that's going to be able to work with very effectively with the rest of your team and make that happen.

Even for a pre product market fit area.

Yes. Yeah, I would say the CEO founder is usually setting the strategy and a big mistake people make is I need somebody really strong in strategy as my first pm and then they just clash. So instead it's like you need somebody on the ground meeting with customers, understanding if there is product market fit, giving you the feedback and being like really the scrappy person making it happen. So they complement the skill set of the people who are there.

Yeah. And I was just saying, I think that for pre product market fit especially, you know, you're searching for very faint signal, very, very faint signal in a lot of noise and the more layers of indirection you put between yourself and the ability to observe and hear that single signal. Excuse me, the more likely you are to fail. And so like I think this is true for research, I think this is true for product. I think. I'm not saying you shouldn't have these. If you need these functions, you shouldn't have them. But I would make sure that there's zero layers of indirection between the founder and that signal. When you're pre product market fit and you're trying to discover that because it's just the game of telephone is very quickly going to create so much signal loss that you'll lose what, what little signal there may be there.

True.

So not too senior, like couple of years of experience who can do the operational work, get data. I would say minbar is you should not hire a PM who you'd have to tell how to do the PM job. When they went back to their desk, you should be able to say, hey, these are the goals, like these are the things we want to build, et cetera, et cetera. And they should know how to kind of go back to the desk and do that with the engineering team without you having to say, okay, step one is this. Step two, that's, I would say the MIN bar at that stage. If you're kind of training someone on the job in that way, I wonder how much kind of like leverage you would get of that. Perfect. Thank you so much.

Cool.

How's it going? I'm curious for like companies that are pre product market fit, like how much should they be iterating and when and why should they be iterating and at what point is the decision made that hey, this market's too crowded, it's not for us, we should move on or quit. You know, who wants to.

You want to take that one now?

Sure. That's like the million dollar question, right? Or I guess the billion dollar question. I would just say that there's. This is, this is art and science. Right. I guess what I would say is as much iteration as possible to kind of get to that, get to that signal and feel the pull. I don't know how to describe it other than that, but when you've felt like when you've hit it, everything starts to move a little differently. Your customer is pulling the solution from you. They're willing to work through all the janky UI and everything else just to kind of get to the answer. And if you're looking for it in retention or you're looking for it in these kind of down funnel metrics. And you're trying to, like, it's. It's a lot of times it's hard to see it in that sort of signal early on. And this is why I think you have to, like, sit with people, watch them use it, like, get their actual reaction, like put it, put it out in the wild in certain ways and good kind of good testing environments and see that. See the pickup and see. And see what happens. And I think you should keep going until you run out of money or don't have any ideas of what else to do. That would be kind of my answer.

One of the things I think is really important is when you're looking at your product is, are people complaining about it. The more people complain, that means they care. If there's silence, that's terrible. But if they're like, you need to add this, and they come angry, that means that they, like, it's important to them and it's important enough to. That they're going to complain to you about it. And people sometimes mistake that to all our customers are unhappy. No, your customers care because it's important. That's something that you even find internally within teams. If you're a PM and your engineers are constantly coming to you and saying, this isn't working, this isn't working, that means that something's moving because if they're not doing anything, then nothing will be broken. So it's kind of that same theme. But you have to. You yourself also have to have conviction. You have to believe that this is a problem worth solving. And that's when you give up, is when you're not sure if the problem is worth solving. To Noam's first point earlier, it's not about the solution, it's about the problem. And if this problem isn't important to anyone, then you should pivot. We're exactly at zero. My God, perfect timing. Thank you all for coming.

Thank you.