Responsible AI Innovation [Entire Talk]

Who you are defines how you build.

Welcome Stanford and YouTube community to the Entrepreneurial Thought Leader Seminar at Stanford University brought to you by basis the Business association of Stanford Entrepreneurial Students and stvp, the entrepreneurship center in the School of Engineering at Stanford. I am Ravi Balani, a lecturer in the management science and engineering department at Stanford and and the director of Alchemist and accelerator for enterprise startups. Today we are thrilled to have Rahul Roy Choudhary, CEO of Grammarly here with us. How many people have used Grammarly? Oh wow. Ok, so it almost needs no introduction. How many people have used Chrome? Chrome, yeah. Google Chrome. Ok well if you like either of those products and many more, the man behind them is our guest speaker tonight, Rahul. So Rahul. Rahul is the current CEO of Grammarly and used to be the global head of product of Grammarly. He's coming to the CEO position from a deep product background and Grammarly, as you guys all know, and it needs no introduction, is an AI and NLP driven writing enhancement tool whose mission is to make all lives better by improving communication across people. Rahul formerly spent 14 years at Google where he really honed his chops and in product management and he rose up the ranks to become ultimately the Vice President of product at Google. And on the way he was overseeing products like Chrome and Chromebooks and many, many others. He's a grad of Stanford's from the business school. He got his MBA from the Stanford Business School and before that he was also going to get a PhD in AI. So he's been immersed in computer science and AI before AI was cool when sometimes people called it AAI, artificial artificial intelligence way back when and now it's become this reality that we see. But Rahul was originally getting a PhD from Columbia and he dropped out with a Master's because he was called to have impact in the world and to leave the safe realms of academia to actually create product. Before that he also got a bachelor's degree in Math from Hamilton College. And so today we're going to have a discussion about all things, but centered also around AI and this powerful wave of AI that's ushering into all of you. And then we're going to have an interactive conversation. So prepare, think about all of your questions, everything that's fair game. And please join me in welcoming Rahul to etl.

Thank you. Thanks Ravi, appreciate it. Hey everyone. Great to be back on campus. Wow. This is so awesome. I want to talk to you about the future of AI, but before I do that, let Me ask you all a really, really important question. What do you call an AI that has learned how to take a picture of itself? Anyone? You call it selfie, aware. Get it? I'm famous. Or I should say I'm infamous in Grammarly for sharing terrible jokes. I just had to get one in. And you know, the future of AI, that's a big topic and so it's almost preposterous. Who am I, why am I talking about this? What does this mean? And what I'd like to share with you all today is we at Grammarly have been building and using AI to help our users for a long time. Many of you are users. Thank you. And we have developed tools and technologies and an understanding of what it takes to build an AI system at scale. And I want to share what we've learned in the hopes that you can get benefit from it. Some of this can be broadly applied, it's not specific to Grammarly, but really what I want is, I want this to be a conversation. I want the end of my session to be the beginning of a conversation, an important conversation. A conversation about how AI can be used to help us, not harm us. How AI can be used to increase our capabilities and our potential, not reduce it. How AI can be used to augment our skills, to help us communicate in a new and better way, not rob us of our voices. So that's a conversation I'd love to have with you all today. So to tee up the conversation, let's kind of just do a situational assessment. Where are we now? The reality is AI is the most powerful technological force that is shaping our world today. No question in my mind at least it is already not future, already having a massive impact on almost any industry that operates at scale. Think of manufacturing. AI is being used to create better components. AI is being used to optimize global supply chains. Think of health care. AI is being used to help radiologists detect cancers earlier, faster, more accurately. Think of medicine. AI is being used to help in the drug discovery process, to analyze billions of compounds, to pinpoint the most promising areas to focus research on. Think of software. AI is enabling new and personalized tools to help us be more productive in our professional and personal lives. And we're just getting started in our lifetimes. AI is going to help us solve the most important problems, things like global warming. And with the rise of generative AI, which I'm sure is on everyone's mind, AI is going to help us with creative expression as well, to take a high level idea and turn it into something concrete, a work of art, a piece of writing. And that is awesome. I am, at heart, I've been working in technology for a long time and at heart I'm a technology optimist. I believe that technology is going to help us make our lives better, improve the world. And, and I've seen this happen over and over again in my career. I remember we talked about Chrome in the early days. I remember when I worked on Chrome I saw the impact that access to the web and low priced smartphones, bringing access to information to billions of people around the world. And I saw the impact that had. That was technology helping people at scale. So AI is fantastic, it's awesome. But with great power comes great responsibility. And AI can also cause real harms, either by design or just as an accidental byproduct. Not intentional, but it just happened accidentally as a consequence of the technology being very powerful. So you can think of things like AI based recommendation engines. They've caused real harm and they've led to societal problems. Things like radicalization or addiction or just getting so immersed in this technology that you lose interest or don't spend time on real world interpersonal relationships. I mean, our brains are no match for these precisely engineered hits of dopamine that these systems can deliver to us. And as a parent of young kids, I worry about this. I mean, I want my kids to get access to all the great things that AI has to offer, but I want to protect them from these harms. If you think about AI making decisions for consequential things in our lives, things like screening job applications or making decisions about who gets approved for a loan and who doesn't, those systems have bias in the underlying data and those biases cause real harm. And the fact is people are getting better, systems are getting better, things are improving. So it's not like this is a gloom and doom story, but the harm is real. If you, I don't know if any of you watched John Oliver's Last Week Tonight talk show. It's really funny, it's awesome, it's a great show. And a couple of months ago his show was, he did a show on AI and it's well worth watching, especially given this class and this talk. But one of the things he highlights is AI being used to screen job applications. And they go through a lot of issues and bias and how the system has a lot of challenges. But the one thing that really struck me from that John Oliver segment was one of these leaders of these companies providing these services. His advice to job seekers Is, hey, don't stand out. Your job is not to be unique or special or differentiate yourself. Your job is to get past the AI gatekeeper so you can actually get to a real human. And that's awful. That's so disempowering. That's so dystopian. That's not a future I want for myself. I'm sure that's not a future you want for yourselves. And so the harm is real. So the question is, what do we do? There's real benefit. This is a very powerful technology, but there's real harm. And so what I see is people pick a couple of lanes. One lane people pick is they say, oh my goodness, stop, let's stop. It's too dangerous, too difficult. We can't move forward. That's not great because the benefits are very real and we should get those benefits. I've seen another lane people pick where they say, let's just go full steam ahead. Innovation at all costs. We'll hope for the best. And that's not great either because the harms are real and hope is not a strategy. So we don't want to stop, we don't want to just heedlessly charge ahead. What should we do? Well, I think we should pick an intentional path forward. And the thing I really want to get across to all of you is we decide, all of us in this room, folks on Livestream, we get to decide how this technology gets developed and, and deployed. Don't let the technology be done to us. Let us decide how this technology is deployed and how it affects us in our daily lives. We have the power. Don't cede that power. And that's a really important thing for you all to remember now as you go into the job market, as you embark on your careers. Just remember that we have the power and the control to decide how this technology is shaped and what shape it takes in our lives. And that's a conversation that really today is about. So at Grammarly we are working on frameworks and technologies and tools to try and bring AI to people at scale. And we have the basic belief that AI is about augmenting human potential. We call it augmented intelligence. And you can think of augmented intelligence as, as the belief that AI is successful when it augments us, when it makes us better, when it enhances our capabilities. And that should be the goal of these systems. In fact, I really wish we could have a do over and rename AI to stand for augmented intelligence instead of artificial intelligence, because I actually think that that is really what we want out of these systems, we want them to help us. Artificial intelligence leads you down these conversations about is it conscious, is it sentient? Really, we want these systems to be utilitarian and help us. And so we want to build systems and we want to deploy systems to our users that remain true to this vision of augmented intelligence. To do that, we use something we call the true framework. The T is trust, which is a commitment to world best in class privacy and security. The R is responsibility, which is a focus on fairness and user safety. The U is user control, which is really putting users in control and making sure users have agency through the entire experience. And the E is empathy, which is making sure that we're actually solving, we understand first of all what our users need and we're solving a real user problem. So that's a true framework. What I'd like to do is give you a case study of how the true framework gets applied. Just real practical advice based on building systems at scale. So to do that, let me take a step back and let me explain to you what Grammarly does, because that'll help you understand the case study. So Grammarly is in the business of helping people communicate better. There we go. And the thing you should know about communication is it is just really hard to get right. We all have different styles, we have different contexts, we communicate using lots of different tools. And the chances of someone being understood exactly the way they intended, using the right tone, using the right information, is not that high. And in fact, the number of tools we're using to communicate has gone up because the tools landscape is fragmented. And the way we work has changed dramatically in the last couple of years. There's much more remote work, distributed teams, global teams trying to work together with different contexts. So communication has always been hard to get right. It's even harder now than it was before. And the cost of not communicating very well is very high. In fact, we did a study that said that the cost of ineffective communication just in the US in a single year is $1.2 trillion. And this number, this is about a year ago, this number is not going down, it's going up because the volume of communication is going up and the effectiveness of communication is going down. So this is a hard problem to get right. But when we do get it right, it's fantastic. We are able to get our point across. We feel like we understand each other. We feel like in a company we can collaborate better, get things done. And companies report this, they see better outcomes, they see teams are in flow State they move faster, they achieve better outcomes. So that's a communication problem. That's a problem we are solving at Grammarly. Now everyone knows generative AI is the thing. Everyone's working on generative AI. We've been working on generative AI at Grammarly and about a month ago we launched our take on generative AI, a product we call Grammarly Go. And so to bring Grammarly Go to our users, we have millions of users around the world, we use the true framework. So let me walk you through it in detail. How do we think about this framework when we thought about this product that we just recently launched to our users? So let's start with Trust. Trust is about making sure that you understand what users expectations are around their data. These are questions like, well what data are you collecting? These are questions like well what are you doing with the data? How long are you keeping it for? And really at the root of all of these questions I think is an underlying question of hey, what are your incentives? And are your incentives and my incentives, are they aligned or not? And so at Grammarly we have built a business and a company around this idea of making sure that our incentives and our users incentives are always in 100% alignment. And so it's a very simple decision making process. A new business model comes our way and we think about it and we say, okay, there's revenue potential, that's interesting, but does it cause us to lose alignment with our users interests in the way we are thinking about user data? And the answer to that is yes, we're not gonna do it. So user alignment is critical. That's really the root of privacy, is getting understanding alignment. If privacy is about alignment and what are you gonna do with user data? Security is about keeping that data safe from harm, from misuse. And at Grammarly we've invested a lot in security over the years. We have the whole Alphabet soup of certifications, SOC2, HIPAA, CCPA, you name it, we've got it. And these things don't happen quickly. They take a long time and take a lot of investment to do and it's hard to bolt on after the fact. And now with new technologies like large language models, new threat vectors emerge and the full landscape of threat models for something like a large language model isn't fully understood. And so the best way to approach that uncertainty is to really have a multi layered security story. You're not just relying on one thing to work well, you're relying on multiple things working together, having this portfolio approach, things like a bug bounty program or things like having a red team. The concept of a red team is you have a team. We have this team at Grammarly, and their job is to proactively find vulnerabilities in your systems so that we find them and fix them before an outside hacker does. And so really investing in this portfolio of security approaches is the right way to deal with a new piece of technology. The thing I just want to kind of give you as a takeaway for this whole trust idea is it is very hard to earn, it's very easy to lose, and it's very hard to bolt on after the fact. So as you embark on your careers and you think about all the cool apps and businesses and technologies you're going to build, keep trust in mind because trust needs to be the foundation of everything else you do. Because we need to earn and strengthen that trust as the foundation. It's going to be very hard to scale a successful company without that basic foundation in place. So that's trust. So with Grammarly Go, we applied this framework, these technologies, all, all of our investments, they helped us launch Grammarly Go, keeping our trust posture intact with our users. Responsibility is about making sure that we are focused on fairness, we are focused on user safety. And there's a lot of technical work to be done here. Things like evaluating and making sure you have very high quality training data for your ML models, making sure that data is free from bias, making sure you can do things like track and find sensitive text like hate speech. And so, for example, one of the things we've invested in Grammarly is a sensitive text classifier. And so you can think of a large language model generating text. And when our classifier catches that text and says, hey, that's sensitive text, we can then take some appropriate action. For example, we can say, well, if a user sees this text, it's not going to be a great experience for them, so let's just not show them this text so we can actually keep our user safe from the underlying harm that these models can cause. And so we invest in these technologies and I would highly encourage everyone thinking about building AI at scale to really think deeply about how to do this. Well, quality evaluation, sensitive text classifiers to detect these things, these are important things to get right. But it's not just about the tech. Because no matter what you do, no matter what you build, no matter how good your classifiers are, there's always going to be some unexpected thing that you're going to run into when your users encounter your product. And what are you going to do then. And that's a question you've got to think through ahead of time and not react after the fact. So at Grammarly, for example, the way we approach this is we give users a lot of different ways to send us feedback in the product, email, a customer service team. And it's not just giving us feedback. We have a process so that when we get that feedback, we can take action. Someone is looking at that and saying, oh yeah, this is a problem, we gotta fix this. Who's gonna fix it? Who's on call? Let's fix it. Let's roll out the fix. Let's keep our users safe. So a key part of responsibility is not just the technical innovation that's happening, which is important and great, but also making sure that you have some plan for the inevitable, unexpected thing that's gonna happen when your real users use your product in their real lives. And so what's your answer then? And it's important to have an answer. It may not be exactly the Grammarly answer, but we need some answer. So that's responsibility. User control is really the idea that users should feel like they have control over the entire experience. So they don't feel like the technology is being done to them. They feel like they are in control of the experience, they are driving the process and they are driving the outcomes. So with Grammarly Go, for example, we are generating text on behalf of our users. So number one, we want to make sure users feel like the text generated by Grammarly Go sounds like them, represents their voice. So we built a feature called My Voice where users can control how they sound. So it's not just boilerplate, generic text, it reflects how you want to sound. And then the entire experience allows users to see what's happening. We provide nudges, we provide prompts, but ultimately we always defer to the user to make the decisions. And that's not an accident, that's not just a thing that happened. That is a deliberate design choice. As you're designing and building these features and products, you're going to be faced with these choices. Do I bring the user in? Do I not? My suggestion and my hard won experience tells me, bring the user in, make sure that they feel they have control over the experience. That's the user control piece of true. And then the final piece is empathy. And empathy is the idea that we are trying to solve a real user problem. And it sounds like an obvious thing, like, of course that's what you would do, why wouldn't you do that? But in fact, what I have seen over and over again in my career is that it's very easy to lose sight of that because there's so many distractions, you're busy working on your product and feature. You started out with the best of intentions. Here's the user problem, I'm going to solve it. And then stuff happens like, oh man, my competitor over here did this other thing and so I got to respond, or here's this cool new tech, I got to get it out there, let's see what happens. And so it's very easy to get sidetracked. And you're kind of looking at a bunch of different things. And all the time you spend doing those things is time you spend not thinking about the user. And when that happens, usually your product is going astray and actually helping solve a real user need. And so you've really got to will yourself, sometimes unnaturally, to come back and ask yourself the question of what does the user need? What is the problem we are solving for them? And how am I confident that my solution is actually a solution to the problem they actually have, not just something I've came up with because it's pretty cool and I'm just going to roll it out there and see what happens. So empathy is something that is really important to keep bringing back to the conversation as you build and deploy these systems. So that's a true framework. Now, we use it at Grammarly, but there's nothing specific to Grammarly about this. So my hope is that you all can see some value in this and apply this in your jobs, in your future jobs, in your careers. As you think about building applications using this framework, imagine a world in which magically the true framework became the de facto way of doing things. Imagine a world in which all apps, all industries, everyone was using this true framework. What would that world look like? Let's just kind of try to go through this visualization exercise. Well, that's a world in which people are going to be able to feel that they are empowered to do better, to achieve better outcomes, that they're better understood. That's a world in which people will feel a deeper sense of connection with each other, a deeper sense of belonging. They'll work better, businesses will be more productive and will achieve better outcomes. And that's a world in which we are deploying AI at scale to help people achieve better outcomes, help businesses achieve better outcomes. And that sounds like an awesome world. That's the world I want. That's how we get the promise of AI that we talked about in the beginning without all of the issues that come along with it. If you don't think about it methodically and intentionally. So remember, we have the power, we have the control. We decide how this technology gets deployed and built. You all are leaders in tech or future leaders in tech. So my wish and my hope is that all of you think deeply about how you can use that power. And my request is please commit to using that power to build AI that augments human potential, that makes humans do better work, to improve our potential, our capabilities, to give us superpowers. I mean, if we don't make that commitment, who is going to? So let us make that commitment because we all want to build a better future using AI. So let's make that true. Thank you.

Let me ask one or two questions before we go to the students. First of all, thank you, Rahul, for providing that framework. It's fantastic to actually have something actionable when thinking about generative AI.

Excellent.

Not everybody is as empathetic as Grammarly is. Do you think that having frameworks themselves are sufficient or what's your position on government regulation? Do you think government regulation, do you invite that or just.

I think my take on government regulation is it's an important conversation. As I said in the beginning, it's an important conversation. So I'm very glad that the conversation is happening first and foremost because it's critically important. I think government regulation for a technology this powerful is needed and welcome. But really I would say that all of us as leaders in the technology field, as future leaders in the technology field, we should be taking an approach to responsibility that is forward looking. So if I'm doing everything right, when there is regulation and there's clear guidance, I'm already doing those things and more because I've actually thought deeply about how I deploy these systems at scale, responsibly, not because of regulation, because it's the right thing to do for our users.

And do you think the right thing will also win in the marketplace? Or do you think if there's a ruthless competitor that has no morals, will they win the generative AI space over a ethical counterpart?

I believe so, because I've always seen that technology creates dilemmas, but technology always finds a way out of those dilemmas. And technology always creates good outcomes, ultimately becomes a force for good. And I've got to believe that that's going to be true for AI, but it's not a law. But the thing I want to make sure we all just understand this is not an a priori thing. This is Not a law of nature. This is us. Our behavior and our actions bring about these outcomes. So the question of like, you know, do we make sure the good guys win or not. It's not that there's a law of physics that will enable that outcome. It's us. It's our collective actions that'll make that true or not. And so this is why I'm actually excited to have this conversation with all of you. Because I just want to make sure you understand the power you do have to shape that outcome.

I'm going to ask one more question, then we'll open it up. But is it pernicious? Because we had OpenAI's Chief Science Officer Ilya here a couple weeks ago. If you talk to the OpenAI guys about what strategic decisions they're going to make in the future, this is mainly Sam, his co founder. He'd oftentimes answer by saying, well, we're going to ask the AI what to do. The AI is so integrated now into the processes themselves that this line of where I as a human end and the AI begins can get very blurred. Just your example of the feed influencing even your senses of alignment and incentives can be affected by the generative AI itself. Can we stay distinct from the generative AI if it's so deeply integrated into the product development process?

I think we can. I think we should. I think we absolutely should. This is about user control. This is a good example of let's use the AI to help us. Maybe AI can help us create great ideas and figure out what do I do next? What's my product roadmap look like. It'll be great if the AI helps me figure out Grammarly's company strategy. But ultimately it has to be us. We have to be in control and we should not give up that control. That's important to build into our systems.

Okay.

Hey Rahul, awesome talk. I'm actually a PM at Google working on Generate AI. No kidding. Bards love it. One thing that kind of bothering me, not bothering me, kind of keep me up the night, is as a user, I see the potential of AI. I love to using AI to create different things. Like let's say I write a song just like Beyonce. Yep. But a lot of the industry right now is kind of panicked. The news industry, the music industry. So how do we balance kind of the AI capability from the assistant right now to become the actual content creator? And how do we give the power back to different industry? Yeah, yeah. I think there is a lot of uncertainty and a lot of panic. I think that's true of every big tech innovation. If you go back in time. In the 70s was the first time that ATMs were deployed at scale in the US and bank tellers panicked because they're like, oh my God, ATMs, our jobs are done. And actually there was vandalism. Vandalism happened. ATM machines were vandalized because people were just fearful. What ended up happening is that because of the widespread use of ATMs it was much cheaper to open a bank branch because you didn't need as many people to staff it. And so bank tellers could move on to do more meaningful, more complicated, higher order tasks. And the number of bank tellers employed in the US pre ATM was lower than the number of bank tellers employed in the US post. ATM technology always finds a way to help us, but we need to make sure that we make that true. So like you're a PM at Google, the stuff you build really makes a difference to billions of people around the world. So bring these principles to mind. Think about you're building this thing that is very consequential. Don't throw it over the fence. I'm not saying you're doing it, but I'm just saying be intentional, put the user first, make sure that you're solving user problem, have empathy for the user task, make sure user control is front and center. So we shape this future based on the actions that we collectively take. Each individual action. As we added up over hundreds and thousands of engineers and PMs working on really cool stuff, that is what makes the future. Thank you.

So my question would be, you are working at Grammarly within LLM, I guess. And so I would be curious of how you trained the model. Did you select high quality data or did you work with web crawl services? Or how did you select the data?

Yeah, we with the launch of Grammarly Go, this is a very fast evolving space. With the launch of Grammarly go, we use GPT 3.5 turbo running on Azure. And so what we did was we applied all of the other deployments we used, such as the sensitive text classifier as a post processing step. So you can look at the output generated and say, well this is not great, change it. Things like my voice, think of that as prompt engineering into that underlying model. And then in the future the space of these foundation models are changing very rapidly. So in the future there may be some other model that's better. Maybe it's an open source model that's better. Maybe it's a local model running in A device that's better. And so we've invested a lot in just quality evaluation. So if a new model comes along, we have a team of experts, human experts in place that can evaluate models for quality, for bias, for fairness, for safety. And so we're ready to make that change as these models evolve. I have a question about. It's actually a follow up to what Josh was asking about LLMs. Right. You had touched Rahul on trust and safety, et cetera. As you know, there's been a lot of talk about the inputs that go into LLMs and the vastness of the data and the transparency of the data. As an individual user, could you comment on how me or anybody else can get a sense of what goes in and how as a user I get to have either a say or some sort of visibility into how these LLMs are being trained and how I can get a sense of sort of control or visibility? I think that's really what you're asking is really an artifact of the design of the system as opposed to how, how the model was trained. As a user, it's very hard to understand what the training data was that went into generating a specific output. As you said, the space is very large. So as an example of what you can do is with Grammarly go, you can create your own like you can create your voice. You can say this is how I want to sound, this is my voice. And so the product experience enables you to control what goes in, what comes out. So my feeling is the way that you solve this user conundrum of there's so much complexity. How do I explain to a user what this complexity is is really through very thoughtful design where the user that goes back to user control and that it's really important that users feel like they understand and have a voice in shaping that experience. My question is like there is some extent that AI can do certain things, but there's some like it's right now at least we're still figuring out how can it make the AI to be as close to human coach as possible to give people like real time feedback and coaching in the context, I'm curious from your own experience and research, what can the AI do in terms of improving communication? What are the things they cannot do yet and where do you see how it's going to evolve over the years? Yeah, good question. I use AI, I use Grammarly go. In my day to day life, one of the things that I find very useful is there's a feature called quick reply. So I get an email I get a ton of email every day and maybe there's an email from someone saying hey, I'd like to meet to discuss some topic. And so what Grammarly go does is understands the email, understands the intent of the email and gives me some choices on how to respond like yes, accept, suggest sometimes or politely decline. And it generates an email for me that's kind of a pro forma email, but it does it with full context in my voice and it generates an email that I actually feel happy and proud to send as opposed to like just a generic piece of boilerplate text that sounds robotic. And I use that a lot. And I just, it's a huge productivity hack for me and it just helps me in my day to day work. There are many things that LLMs can't do yet, but honestly the space is changing so quickly. Building a model of the world, reasoning skills, they all seem to be emergent properties as these models get bigger. And it seems somewhat. I'm not a researcher in the field, but it does seem a bit of a mystery at what properties emerge from each next generation of model. And I don't know if anyone has really understood that. So we'll learn. And the last six months have been insane. I mean, just think about six months ago, maybe you folks are an exception, but most people had no idea what an LLM was. Suddenly it just took over the world in six months. It's crazy. It's happened so quickly. So how this proceeds over the next six months, 12 months, 24 months, I'm not quite sure. I think we're just going to have to see.

And if you're a Stanford student, just passionate about generative AI, with all the time in the world, where would you focus your time if you wanted to do something entrepreneurial?

Ooh, hard to. That's interesting. You know, I think here's what I would say. I would say that focus on a user problem. It's not a thing to say. I'm going to do generative AI. That's my view on it. Other people may disagree. It is a thing to say. I'm going to solve a user problem and I can do it really well using generative AI. So, you know, one of the things that I've learned in my career is it's really good if you fall in love with problems but don't fall in love with solutions because the solutions will change over time and often what ends up happening is we end up falling in love with the solution and not the problem. You don't even realize it's happening and then when there's a better solution out there, you're like, no, but my thing, that's just so awesome. It's actually, you know, it's better not to get too attached to any one solution. So I would say really the best advice I can offer you is find those problems. Find problems worth solving. Start there. And generative AI will be a great way to solve those problems.

Awesome. Can I ask a question? Do you view Grammarly as a product or a platform?

It is. It's a product.

It's a product.

It's a product experience. But it works across all of your other applications. So it's not a bespoke experience you can think of as. It's a product that layers onto every other product that you use.

But even as you're releasing your own model, you're not conceiving of that as this ecosystem that's going to be built on top of Grammarly.

Yeah, I mean, I think in the fullness of time we'll move from today. We have multiple products, we have an extension, we have mobile products, a native application. And at some point we'll move from sort of a suite of products to a platform. But right now we're very focused on we have these entry points to our service and that service works everywhere that people write. That's what we want to do.

Terrific. I think we have a question here at the front row.

So my question is about like every time that we have a technological shift, we get like new ways of communicating. What do you think are going to be the ways that are going to be enabled through this revolution of generative AI? And like, what do you think about this meme of like my AI want to be talking to your AI or something like that? I hate that meme. Let's make sure that meme doesn't come true. Because look, I mean, okay, so let me give you some, let me give you some real, real data points. I talk about the $1.2 trillion loss thing. Here's the reality. The average knowledge worker in the us this is a global problem with just us numbers. Average knowledge worker in the US spends more than half their work week. I mean, I should say we, I mean, I'm one of them and it's true for me. I spend more than half my workweek engaged in written communication. That number has gone up by 18% in the last 12 months. So that's going up. We're writing more. The effectiveness of that written communication during that same 12 month period has gone down by 12%. So the volume is going up, the quality is going down. So we can't outrun this problem. We can't solve this problem by saying, let's just do more. If I can only just run a little bit faster on the treadmill, Salvation is at hand. On the other side of that, actually, the more you do, the worse it gets. So really the answer has to be we've got to communicate better. Not more, but better, because that's what the data says. I'm not making this up. This is what the data says. You've got to figure out how to communicate better. So if you think of, if you think about generative AI, we're already communicating so much. Right. We're already lowering the quality of communication. And then you think about the cost of communication or cost of content going down. Just now, easy to create reams of blog posts and what have you. With generative AI, you can do that. The technology enables you to make that true. But what's that going to help? You're just going to increase that 18% to like 50%, 60%. You're going to drive that 12% down to like minus 50%. That's not going to help anything. So that's why you really go back to user problems. And if you look at the user problem, you're going to realize very quickly that it's not more, but better. And then when you think about what business do I start with generative AI or how do I use it? That's your mantra and that'll guide you to find the right solution.

How do you prioritize, given that everything's changing so rapidly and your mission is so broad?

Yes.

Communication applies to everything. Yeah. And you've been a master of building these amazing products over the last few decades. How do you, within Grammarly, actually decide or prioritize your roadmaps?

There's some technology, some techniques we use, but at the end of the day, it's. It's an art, not a science. So there's some things you could figure out, like, what are the user problems you're solving? Why are these the most important user problems? What data do we have to back it up? We talk to users, we conduct interviews. So there's a lot of kind of input coming in to help us understand the problem space. Then there's things like, well, what's the privacy story? Are we aligned? Are we not aligned? So you can kind of very clearly start using filters there, and then you can do some sort of an impact assessment. Well, okay, this thing would be Great to do, but the impact might be marginal. This other thing would take the same amount of time to build, but actually would have massive impact to our users. So let's focus on maximizing impact. So there's a set of these things that you kind of put in place as guide rails and guardrails and filters. But at the end of the day, there's judgment. And teams spend many hours inside Grammarly, inside Google, inside every company really debating and discussing and arguing about the roadmaps and priorities. And, you know, maybe at some point AI will just tell us what to do, but right now it's really something we have to really engage in a lot of debate and discussion on. I'm curious about when you were speaking about responsibility and this sensitive text feature, I guess, kind of to play devil's advocate. Where for you is the line between filtering out, like potentially triggering speech and censorship and these AIs sort of taking limiting access to certain ideas? Yeah, it's a great question. And it's a question we ask ourselves every single day. I don't know that there's any perfect line etched in stone or in paint or whatever the metaphor is. We have a team of linguists. They are tracking and studying authoritative sources of what constitutes different kinds of bias, hate speech. We try to assemble from a set of these sources sort of rules of the road. What is the thinking around this? What's the research, what's the scholarship? And how can you apply that in a meaningful way in the product? And that changes, right? So there's not like you do it and you say, okay, check the box, I'm done. This is a constant evaluation. When Covid became prevalent, one of the things that was new was lots of biased ways to refer to Covid. And that became a thing. And so Grammarly's sensitive text classifier, CAT caught those things and was able to suggest to users. I mean, at the end of the day, we can't write for you, but we will suggest to you that, hey, the thing you're writing is actually considered pretty biased as a way to refer to this virus. And so maybe you should consider this, and here's why, and you should consider this other thing. So this is a thing that keeps changing and adapting and we are trying to be as close to current as we can. Yeah.

Thank you. Thank you, Rahul. Thank you so much. Thank you. This is awesome.

Thanks for the time. I really appreciate it.

Next week we're going to be closing out the year with Josh Wolfe, who's the managing director of Lux Capital So we'll have a venture capitalist in for the final talk. And as you guys all know, you can find this talk and others on the E. Corner YouTube channel or this and other talks are also at Ecorner Stanford. So thank you guys for coming to this week's etl.

Awesome.