SUMMIT STAGE

Chew!

Please welcome CEO and co-founder Aloy Simon Kubica.

Hey, how's it going everyone? Sorry about the wait. Uh, we got some really exciting stuff to show off today, uh, for vibe coding and specifically vibe coding for existing products. Um, I think usually when we hear about vibe coding, we're thinking about, you know, some of these amazing app builder tools that have come up over the last or two, uh, that help you create new things from scratch. Uh, but if you're a product leader or you're running an existing organization, chances are you probably have a code base that is now capable of being used to design new ideas and enable your whole company to build. So we're gonna look at exactly how you can do that. Uh, and Alloy's been around for, uh, just coming up on a year now. I previously used to work at Atlassian as a PM. My co-founder was at Canva, uh, and we've done what we can to sort of blend the, the spirit, um, of those companies and tools together. Just give you something really easy to pick up and adopt in the, in the AI age to move faster. So let's see how we're going with our clicker. All right, so the recent history. Of building features. This should feel familiar, uh, if anyone was around in SaaS in 2024. Uh, and then we've sort of graduated to something that looked a little bit like this sort of new app building generation in 2025. Still had the ability to just prompt your way to success, but very much sort of gradient Purple gradient heaven, doing things from scratch for the first time. Great for landing pages, great for new apps, but didn't really look like your existing product, which was a little bit of a challenge. And then something really crazy happened. So I'm from Sydney. We have the New Year's Eve fireworks, which— has anyone been to those? Highly recommend. There we go. One guy. Around about the time that these were popping off, we were getting this really, really crazy change in AI capabilities. And that was Opus 4.6, came out right about the turn of the year. And we really saw when this came out that we were going to go from legacy design, drawing rectangles, being a professional design expert, graduating to consumer app building in 2025, what I was showing you a little bit earlier, which is still amazing if you're starting from scratch. Um, but now we have something that's a lot more like PMs being empowered as full-scale builders working in professional code bases without having to be design or, or engineer experts. So if we take a look at the evidence of this, We've seen 4.6, now 4.7, I really can't stress this enough, go vertical on just how capable it is and how long it's able to run for to think through your business context, think through the code base, and actually make changes almost like a professional engineer would just on the basis of ideas that previously you might express in a Slack channel to a human, and now you can expressed via AI inside of one of your app builder tools. And so, in terms of adoption of Alloy, we also saw a giant uptick around this time. Toby, if anyone knows him from, I'll make sure I get it correct, Shopify, not Spotify, has completely embraced this new workflow. And Ramp has actually built an internal background agent. We call it a cloud agent. Purpose-built for empowering everyone in the company, um, to start working in this new way. And so, uh, you can grab tools like Alloy off the shelf. You go build your own as, as Ramp has done. Um, vibe coding is definitely the new product management, that's for sure. And so we think 2026 is gonna be the year of cloud product agents for PMs, designers, for engineers. We're all gonna meld into builders, and I'm just gonna show you exactly how we're able to do that, uh, because this comes with a whole host of capabilities. It's a new superpower, uh, and there's two ways to adopt it that we can just jump into and take a quick look at. So let's get rolling. So use case one that we're gonna be walking through is instantly exploring product ideas. And we're gonna do this with your existing product. Um, no need to connect to design system, get admin approval. Uh, you can just take any screen and start building with it. So very much going from this old way of waiting for review, getting design resources, hoping that, uh, the visual that has has been mocked up in a design tool is sort of to the idea that you had in your mind to this new way that just takes a couple minutes of prompting your change, effectively being done. So why don't we show you just how quick that actually is by jumping into this demo. Awesome. You can see that up there. Uh, so I'm gonna use a product that I really love, Linear, uh, as an example of a tool that I might be a product manager for, um, or a CEO or head of sales. Uh, so let's imagine that I'm just working here, and, uh, I maybe received, uh, an idea from a teammate that I want to go ahead and implement. What you could do in the old way is maybe, you know, take a screenshot of this, paste it into an App Builder, or get IT admin approval to connect your code base. But it's actually much simpler than that in 2026. You can just pop open what we call a Capture Extension. And we'd love if you used our way. There's other— tools available that you can use for this. Uh, the, the main thing to understand is that you don't need permission, um, from, uh, you know, a long stretched enterprise approval processes. Uh, you don't need to put your company's security at risk. Uh, as long as you can access your application, you're able to now just capture it, feed it to, um, an AI app building system, and start to use it as context for the feature idea that you're about to build. So I've just hit the capture button here and I've captured the exact screen that we were looking at and you can see it's pixel perfect all the way down to the icons and, and the hover states. Uh, and I'm able to just start, you know, vibing my way to the feature idea that I've been thinking about. So what I might do here is give it an instruction along the lines of, let's change this list of issues into a whiteboard canvas where I can drag cards around. So I'll submit that prompt. It's gonna go ahead and, and take a few minutes. And while we're waiting for that, I'll just fast forward here to a version that I completed a moment ago to show you what the end state looks like. Speed us up a little bit here. So here's my original. I've captured the page and you can use this on any website, marketing asset, internal application of yours, anything that you want to improve. I've got that into Alloy, fed it my prompt, and then a couple minutes seconds later, I'm gonna get back my result here, uh, which you'll see is, uh, a whiteboard canvas that maybe is a top feature request from a stakeholder, my manager, even a customer. And I can get a feeling for what this would look like. Um, I can demo it inside of a live stakeholder call, get a feel for dragging these around. And if I choose, I can even decide that I maybe wanna make make some visual edits. So what would this look like if we got rid of the issue key? What would this look like if, uh, we made the card borders a little bit rounder? So I can go into a sort of Figma style, um, canvas style design mode, and then I can share this with someone instantly. So in just a couple of minutes, I've gone from screen that I was looking at to the exact idea that I'd envisioned in my head. Zero setup or, or permissioning. I can now just vibe my way to exploring my ideas. Now, there's an interesting limitation to what I just showed you, uh, which makes this great for starting, but then there's a way to go even further. So the limitation that I'm talking about is because we've captured a page, um, the only context that we fed Alloy for this sort of level 1 of, of vibe coding is the page that I was looking at. So I can't necessarily click on any of these pages, um, or states. I can't interact really, um, with what I've captured and, and put inside of the application. And so I'm somewhat limited here in what I'm able to, to vibe. Now, I can always capture multiple pages, start a, what we call a capture recording, where I go, maybe I want my issues page and some of my views pages in here, and let's do, you know, the inbox as well. And I can finish my capture and get 3 states into the product. Actually an even more powerful way that you can push the Alloy tool, just get the entire context of your application into this cloud-based web sandbox so that you don't have to capture anything at all. So now that we're done with level 1, uh, of this, uh, whiteboard idea that we were working on, let's take a look at how I can just prompt with my codebase. Directly. So I'll go back to the homepage of my Alloy app here. And by the way, you can do this from Slack, so your chat tool, you can just @mention, uh, the Alloy bot, uh, or you can do it from the alloy.app homepage. Uh, and instead of selecting one of these web pages that I've captured here, what I'm gonna do is I'm gonna select my codebase. and so if you wanna go further with your app builder system and put all the context of your application into the cloud so that anyone in the company is empowered to just vibe with it, you can connect GitHub or GitLab, and then come in here and start to give out some more advanced requirements. And so let's come in here and say, I'd like to, to add a section above the prompt box on the Alloy homepage called Ideas. It should include some sample prompts that I can click on to populate the box to give our users a sense for how they should get get started? That seems like a fair prompt. I'm gonna go ahead and send this off to my product agent. And what you'll notice is because I selected codebase, uh, if we're lucky with the demo gods here, I'm gonna get all of my codebase, which in this case is Alloy. So it's a little bit meta. I'm gonna get all of Alloy inside, or this would be your product, inside of my sandbox here. So without having to capture anything, I've actually got my entire product environment available and ready for me to use. So finally, I can click on these dropdowns. I can switch to all the states of the application. Depending on how advanced your product is, may have tons of screens to play with, but This is almost running, you know, what my development team is using locally on their machines, but available to anyone in my organization. Just go to alloy.app. They've got their product now in a sandbox, or your product in a sandbox, and you can just vibe with it and start to explore your, your product changes. So similarly to what we did last time, let's fast forward a little bit and, um, go to the completed version of this. This that I was just working on, on the way. So we'll have a hunt for this ideas section. Gave it the exact same prompt. Wait for this to load up over here. And yeah, we'll see that the agent has gone ahead and not just used the capture from a screen, but gone into the exact file in the code base started to build out this feature and built it in a way that's interactive. So we've got a couple of ideas for how I might want to prompt this. Adding a dashboard sounds like a cool idea. So again, a little bit meta because we've got Alloy inside of Alloy, but it's going to be your product in this sandbox over here. And you can see this feature feature works great. I can now get a feeling for prompting users with ideas. This sounds useful. We've always wanted a notifications panel. So I'm just gonna test my example here. And you can see it's, it's even taken me into the next stage of my app because it has context of every screen, every flow. I have basically uninhibited prompting and vibing. Almost like I was using one of those app builder tools, but I gave it my real enterprise code base and it's gone ahead and, um, built this ideas feature. So let's say I'm happy with how this looks now. Uh, I've maybe presented it to a customer, gotten sign-off that we'd want to add this feature to our product. Uh, the next phase of development is going to be the engineering handoff phase. In the old world. Uh, but in the new world, you actually have all of your context inside of your sandbox already. And so I can just have my developer or myself flick over to this changes column. I can see exactly the code that my product agent is proposing that we add to the homepage of my product. Uh, developer is able to review this. Maybe prompt further changes if they have some code nits or feedback on how to, um, you know, make this match the patterns of our codebase. Uh, but assuming that we're happy, what I can now do, because the codebase is connected, is I can just push this up and, um, move it to GitHub so that we can roll this out as a real product feature. And it's gonna hit this, this push code button, going to allow this popup on the new computer. Uh, and then of course I'm not logged into my GitHub account, so I'm gonna get this screen. But, um, if you have your authentication set up here, you're just gonna get straight to your GitHub pull request, your GitLab pull request. Engineers can review that inside of their workflow, and then without having to run your application locally, go through any of the developer environment setup, rigmarole. Uh, I basically just come up on this stage, logged into my Alloy account, uh, and I've built a real feature that, uh, might even be, uh, available for our customers this evening, uh, once engineering goes ahead and reviews it in GitHub. So just as a quick recap, you have the capture feature available to you now in this new AI world. You can get any context into, uh, a prototyping system to explore visual directions without losing context of what your product looks like today. That has a ton of really nice advantages. Uh, you can share it instantly. Of course, you can prompt to your heart's content, make those visual edits. Um, but you're never going to be able to interact with every screen of the application. Application because the agent doesn't have that context yet. But if you want to take it to that next level, actually not just be a prototyper, but build production-grade features and empower your whole organization to do so, then you're able to do that by going ahead and— where was our feature? Over here. You're able to do that by connecting your code base, starting a prompt from there, going ahead and reviewing your changes and then pushing that straight up to GitHub or, you know, again, just sharing it like it's a Loom link or a Canva link and getting feedback from one of your stakeholders or customers without having to give them a long set of steps on how to run this developer environment or sending them video that they can't interact with or, or play with. So I will just head back to slides over here, and then we'll quickly wrap and cover, you know, what does this look like in an enterprise context if you want to go from insights that you've had in your mind, like maybe building a whiteboard or building an ideas into your, uh, into your prompt box, uh, whatever your, your product is or your feature idea is. So how do you get the most out of this and, and actually deploy it practically in the enterprise? We get this question a ton at the booth. Uh, so number one, I would say the CAPTCHA extension that I showed you is permissionless and you can just start. So we see tons of, enterprise deployment there. Um, and it's a, it's a really, really easy way to just get going. Then you also are gonna want to, as you see broader adoption of these tools, ensure that it's not just people trying to hack together design systems inside of the AI-native app builders, but you're connecting your code base, you're unlocking that maximum power, making every state of your application available in the cloud, making the code context available to the agent so that it can do its best work on not just visually exploring your idea, but actually building it in a way that you can roll out to customers. Uh, and then thirdly, if you really want to take this to 2026 grade, your AI native workflow tier, uh, adding MCPs is where this really gets wild. So if you're connecting up Snowflake or you're connecting up Jira, you can just ask inside of your prompt box to have it, you know, pick off one of the longstanding features that you haven't been able to build from the Jira backlog and have it get to work on that, view it visually directly next to your agent context. Or you can give it a data warehouse and ask it to look for drop-off in your onboarding flow if you're connecting that data. Visually explore some solutions on how you might be able to, uh, capture people throughout more steps of the funnel rather than having drop-off. It's really this MCP unlock of making sure that your product agent has access to all of the systems that a human has access to inside of your company, uh, lets it really operate on that further tier of power. And so I would say just do it. Especially if you can just capture and start. There's really not much to adopting this workflow other than plugging a few things in and just having the awareness of, of how you can take advantage. And so the future is really here today. Uh, it's available in Alloy. It's available in a bunch of, you know, innovating products, um, that are pushing forward AI design and prototyping and bringing the development workflow not into a, a waterfall style handoff world, but actually into AI native world where you have all your context available in the one place. So thank you so much for making the time. I'll just be down out the back if you have any questions, and I really hope that this was a useful demo today. Thank you.

Hey! Hey!

Hey!

Hey—

hey-hey-hey-hayy? Huh?! Uh...

As a courtesy to our speakers and fellow attendees, please take a moment and silence your devices. Give a warm welcome to the founder and general partner of SuccessVP, John Gleason.

Let's get this party started here. How's everybody doing today? All right, right on. Welcome to the 2026 Forward Deployed/CCO Summit. My name is John Gleason. I'm the founder and managing partner at SuccessVP. That's my day job, but my evenings and weekends job is running the Customer Success Meetup with my friends Monica Perez and Jenann Pang, who couldn't be here today. How many have been to one of our meetups before? Right on. We got a good crew here. All right, so we've been running these things since 2015, and it started as a passion project. We like to tell the story, the first meetup we ever hosted, we had more pizzas than there were people. People, but we kind of hit our stride, and it grew, and it grew, and it grew, and now we host huge events. I'm super honored to be here with SaaStr, and we also host our own events in New York City, San Francisco, and more recently Boston as well. We're really proud of the customer success community and what we've been help— or what we've helped to build, and I think the reason people always keep coming back is because of our values. All right, so if you've never been to a Customer Success Meetup before, the magic is actually when we get together here in person. We shake each other's hands, we say hi to each other. There's lots of great things online, forums, but we think the real magic comes when we get in person together, so we're here today. The second piece is, Jenann, Monica, and myself, we're just the people who put on the content, content, with the help of Sastre here today, of course. But it's the people behind it, the volunteers who help us, that really make it a special thing. Communities only thrive when you put back into them, and there's a lot of people who put into this community. So we're super grateful for that. So that's value number 2. Value number 3, and you're gonna see this in spades today, we really believe in content by practitioners. Practitioners. The thing that we've always strived to do is put people on stage at our events who are scaling through hypergrowth, people who are kind of at the frontier of building. We think those are the people who know the secrets to the future. So you're going to hear from a lot of those today. So content by practitioners for practitioners. And finally, we want to be as inclusive as possible. Anybody who needs to come to our events should be able to be there. And, you know, through the generosity of Saster, we were able to give out a lot of scholarship tickets to today's event as well. So we really honor that, be as inclusive as possible. And so if you want to join the community, take a picture of that QR code. That'll give you everything you need to join the community. We're very welcoming. So if you're not a part of it already, welcome to the team. And let's talk about what's going to go down here today. So we all know customer success is entering this new era. All right. There's a lot of confusion right now. We're all figuring it out. But like I said in the values, it's the practitioners, it's the people building at the forefront who are going to figure it out first. So we've got a lot of AI leaders here today. They're the ones who are already solving tomorrow's problems, and that's who we're going to hear from. We've got an incredible roster, so let me tell you what we've got planned. Up first, we're going to have my friend Monica Perez. Monica is the head of customer success at Lovable, one of the fastest-growing companies in the world, and she's going to share some insights into scaling a team through incredible hypergrowth. Next, we're going to have my friend Tom Ronan from Harvey, and he's going to tell you that maybe your agents don't need as much onboarding as you think. But your customers certainly do. Followed by Tom, we're gonna have Ryan Seams from Assembly AI, and he's gonna tell you a little bit more about the modern motion and why you need less CSMs and more an engineering mindset. And finally, we're gonna talk about everybody's favorite topic, which is revenue. We've got a star-studded panel, and our friend Bobby Cooper Jason M. Lemkin, uh, is going to be hosting that one with, uh, a roster of incredible people like Ashvin Baidyanathan from LinkedIn. We've got Ursula from Content Squared, and then we also have, um, Daniel Silverman from Carta as well. And then finally, and hopefully you stick around to the very end, I'm gonna be sharing some of the metrics that you ought to be measuring if you're operating in an applied AI company. So stick around to the end. We're going to have a great afternoon. Thank you all for coming. Let's get down to business. Ready? Let's go. Here we go. All right. And it is my pleasure to welcome our first speaker to the stage, my very good friend, Monica Perez, Head of Customer Success at Lovable.

We're good.

All right.

Can you guys hear me? Yes. Mic is on. Mic's working. Hey everyone. I'm excited to kick off our FDE Summit slash CCO Summit here at SaaStr. And I'm I'm gonna talk about how the game has changed. So if you haven't realized it, we are standing in a once in a generation moment. AI is allowing us to reshape not just how our tech works, but also how net new value gets brought into the world. And so today I'm here to share 5 moves, 5 counterintuitive moves that I believe will help you seize this moment. These are 5 moves that we discovered at Lovable, but I think it will benefit every single person in this room, whether you are a founder or a tenured CS professional, as we kind of think about navigating this this moment together. All right. So as John introduced, I'm Monica Perez. I lead customer success at Lovable. And for those of you that don't know, Lovable is an AI-powered software application building platform where you can build software in plain English. So, um, we're also the fastest growing company in the world to 400 million, and we're just getting started. But the first thing that you should know about me is that I've spent my entire career in tech, and I'm not technical. I don't code. I never have. But I've always believed that technology is there to enable creative people to do extraordinary things, even if, and especially if, you're not technical. And that belief has shaped my entire career. I've always been drawn to products that make technology accessible to people like me. At Notion, I saw what happened when software becomes kind of modular and DIY. It was this low-code experience that made me feel like a builder for the first time, and it felt like magic. It worked how my brain worked. And Lovable is like that, but on steroids. You can build real software in plain English. AI handles everything under the hood, the infrastructure, the code, all the barriers that used to stop people like me from bringing my ideas to life. And that is exactly where the value is getting created today. It's the net new possibilities that AI enables, the possib— AI getting bigger for everyone. And that's why I care so deeply about this moment in history. And it's also why I care so deeply about customer success. Because customer success is where promise meets reality. And by the way, when I say customer success, I don't just mean CSMs. I mean the way our entire company thinks about making customers successful. So depending on your product, that might mean forward-deployed engineers, that might mean product managers, consultants, or domain experts like lawyers, accountants, recruiters, specialists who speak your customer's language, all of it. Every person whose job it is to close that gap between what you sold and what the customer actually experiences. So what I'm gonna talk about today is a challenge to think about that function differently because we all sit at the intersection of expectation and outcome. And that intersection, especially now, is much more consequential than ever before. It's no longer, just about adoption. It's about transformation. It's no longer just about helping our customers use our software. It's about rethinking how work gets done at all. So with this kind of once in a generation moment and opportunity that I've been describing, we need a new playbook. And the reality is that the playbook that all of us are running or that you expect to run is outdated and it may not survive. So as I said, I'm gonna walk you through the 5 moves that we made at Lovable. 5 moves that felt counterintuitive. They kind of initially felt wrong, but they're all working. So I'll share today not as a as a perfect formula for you to follow, but as a lens to think about how we should evolve when AI is changing the rules. So let's get into it. The first one is going to sound very controversial from somebody who works at an AI company, but I don't care. We stopped talking about AI as the headline. Every pitch deck in this room starts with AI-powered, including this one. Sastre told us to do so. Every keynote you've been to has AI in the title. Every team I talk to is told to evangelize AI. But what, what does that mean? We cannot talk about AI for the sake of AI. Why? Because it shows that you're, you're not thinking 10 steps ahead. We know that in the very near future, even now, AI will no longer be the novelty. It will be the baseline. It's the ambient reality that powers everything behind the scenes. So why not just live in that future now? Nobody walks into a meeting and says, hey Steve, have you tried the internet today?

Right?

So AI is headed in that exact same place. And at Lovable, we're already there, so we act like it. I tell my team, if we're the fish, AI is the water. It's not the story, it's the setting. So our onboarding conversations don't start with, here's what AI can do. They start with, here's what you'll be able to unlock knowing that AI AI is already everywhere. When you stop focusing on AI for the sake of AI, you free yourself to focus on what actually matters, which is the value being created and the transformation happening inside of your customer's organization. And then what happens? What happens when you start actually focusing on the transformation? You stop— AI-ifying existing processes and you start thinking about how work should work entirely. If you're in customer support, for example, and you're answering tickets, the instinct is, okay, now AI can handle the tickets. But what if the concept of tickets went away entirely? What if customers could just tell you what they need, voice, text, omnichannel, omnipresent, wherever. And the resolution just happened. That's what we get the opportunity to rethink. We're not bolting AI onto old processes. We are designing new ones completely from scratch. And I know that this is already the case because our customers are asking for this. Our customers come to us and they don't say, hey, we're talking to Lovable because we want to use AI. They say, I have an amazing idea for a product that I want to launch. I want to drive this transformation on my team. That's what they're looking for. AI isn't the conversation, it's the canvas. The moment you stop saying AI in every conversation, is the moment you actually become an AI-native company. So stop talking about AI and start talking about what you're actually building. Number 2. Number 2 is all about pace. We rewired the clock. Traditional customer success operates on quarterly cycles. 90-day plans, QBRs, annual renewals. That whole operating model was built for stable products that had couple releases a year and then renewed and repeat. That world is gone. In today's modern companies, code ships every single day, sometimes multiple times a day. Definitely the case at Lovable. So founders are thinking of build cycles in days, not in quarters. The value that we ship into our products is increasing exponentially, and we expect every function to match that pace, including and especially customer success. So what did we do? We redesigned customer success to move at the pace of our products. Product, because when go-to-market and product move together, nobody gets left behind. The customer doesn't get left behind. The business doesn't leave value on the table. Our goal should always be to ensure that our customers are realizing as much value from our products as our teams are building into it. There's no gap in time to value. Zero. Instant time to value. Value. The companies that are winning right now aren't winning because they have better AI. They're winning because every single person at their company, every single person in the building, including post-sales, is moving at the speed of the product. Oh, sorry. Um, so this is the point. Your go-to-market team cannot be operating on a different clock than your engineering team. If it is, you have a huge liability. That's just the expectation now. If a CEO hires their first post-sales leader and the leader comes back and says, okay, I need 90 days to figure out my plan, that does not fly. The world is completely different in 90 days. You would never accept that from an engineer. You would expect them to ship code on their first week. That same mentality has to apply to us. I experienced this firsthand during my interview process at Lovable. They didn't just bring me in and ask me questions. They brought me in for a 1-week work trial, and I onboarded a customer that week. A real customer. Customer, not a mock customer. I onboarded a Fortune 100 customer in that interview week. And my first 30 days at Lovable, when I joined, I met every single customer in those 30 days. There was no 90-day plan. It was ship on day 1, or day -1 in my case. All right. The most important part of rewiring the clock is actually what it does for the customer and from their point of view. Because in the old world, the customer waits for you to schedule quarterly business review. You show up with a deck full of slides that show that you kind of proved to be valuable. But in the new model, we should be continuously connected. When our customers need us, we're there. We move at the same speed they do. The relationship is alive all the time. And so you're thinking, Monica, how is, how is this possible? What do I need? Remember move 1? AI is everywhere. There's no excuses anymore. So use it to your advantage to really rethink the entire customer journey. Customer experience. Support responses that happen in real time, not in a queue. A product that anticipates what the customer needs before they need it. Marketing campaigns that think in terms of individual personalization, not in terms of personas. This customer, this need, this moment. At Lovable Every customer gets a Slack channel, and they can ask any questions into that Slack AI bot, except the AI bot is not answering their questions based off of some generic knowledge base. It's actually scoped to that customer-specific instance, their Lovable Workspace, their projects, their users, their history, even their commercial contract terms. It's not a Slack bot, it's actually hundreds of individualized Slack bots in channels, and it wasn't harder than building just one. That's the power of AI. So just ship, do it fast. All right, move number 3. We designed customer success around experiences, not expertise. Activities. For a long time, I'm sad to say, customer success has been organized around activities. Who runs the onboarding call? Who owns the renewal? Who handles the implementation? In other words, we shipped our org chart to our customer. Our customers do not want an org chart. They want an experience. Let me give you an analogy. Think about the last time that you signed up for a gym. Why did you sign up for the gym? Because you probably wanted to feel healthier, build muscle, have more energy, maybe lose weight. That was your goal. But what did the gym— what did the gym measure? The sign-up. They don't They don't care if you never walk through their doors again. They checked their box, they got the membership, they got the signup, that's their gold standard. Unfortunately, that's how a lot of customer success teams still operate today. Did they log in? Did they open this email? Activity, activity, activity. But nobody asked, did the customer actually get the outcome that they came in for? Now imagine a gym that took that perception and that into their operating model. Imagine a gym that operated where they don't measure themselves on memberships, they only measure whether you achieved your goal or not. What would that gym do? They would probably have a leaderboard and see how many times you walked in. They would probably set you up with a hyper personalized nutritionist to help you with your plan. They would write a custom plan based off of your goals. They wouldn't hand you a generic welcome packet. That's a different experience. And that's how we're thinking about it at Loveable. We asked ourselves a very simple question. If AI handles all the admin, all of the data entry, all of the note-taking, the CRM updates, Then what's left? Everything that actually matters. The human stuff, the experiences, the things that your customer actually needs and remembers 6 months later. If your CSMs are spending their day updating a CRM instead of standing in the room with their customers, you've automated the wrong thing. Stop measuring activity and start measuring transformation. Not did they use the product, but did they get the outcome they came for? We're building our CS org around that today because our customers don't come to Lovable because they want to push buttons. They come because, as I mentioned, they want to launch products faster. They want to re-engineer entire workflows, and they want to bring their technical and non-technical teams closer than ever to ship real business solutions. So our job isn't to help them use Lovable. Our job is to help them get closer to that outcome. But how do we actually do this? Monica, this sounds great. Tell me about the how. Well, for one, we have a team at Lovable whose full-time job is putting on hackathons at our customers' offices. Why? Because our customers want to enable their employees to ship, ship, ship. And what better way to do that than a a single focused day to build a safe space and the Lovable experts by your side. So we show up and we run a full day hackathon at their offices with their executives. We recently did this with T-Mobile. Their teams were vibe coding on Lovable all day. They thought through the internal productivity tools they wanted, and by the end of the day, we had over 100 over 1,500 working applications shipped. Another thing we do is that we provide product management and engineering as a service. So we tell our customers, what do you wanna build? Don't worry about it, just dream it up, tell us your dream, and we will product manage it and build it alongside your team with our FDEs and our deployment strategists. We literally become an embedded part of their team. Okay, what about for our users who want to found companies and build businesses? We have founder mastermind communities running 24/7 on Discord where they can learn from each other and access the latest and greatest. We are creating a direct line to the outcome that our customers are are looking for, not an activity that we need to complete or a vanity metric that we need to hit. AI handles the admin. The humans deliver the magic. And by the way, this changes completely who you hire for. The people in the old model, the rigorous, process followers, playbook runners, activity completers. Those are the ones, unfortunately, that struggle most in this environment. We hire for builder mentality, non-traditional backgrounds, people who can vibe code an internal tool on a Sunday, people who see a broken process and immediately fix it, they don't file a ticket for someone else to fix it. When we interview, we say, show me your projects, show me your portfolio. What have you built? Show, don't tell. A founder recently asked me, hey, I'm hiring for my first post-sales leader, post-sales person. Should it be a CSM? And my answer was, It depends what you mean by a CSM. If you mean someone who fills out Gainsight, no. But if you mean someone who can deploy your product, run a workshop, build an internal tool, and tell your customer their ROI story in the same week, yes. That's the new profile. It's not a CSM, it's a builder who happens to sit in customer success. All right, let's talk about move number 4. Move number 4 is about controlling your destiny by controlling your stack. I'll start with a history lesson about tools. Think about the word tool. Go all the way back, the original definition. For example, we built a hammer. A hammer let humans do what our hands cannot do, right? It existed to amplify the human. And that was true for a very long time. Tools made us faster, more capable, uh, able to produce more. And the majority of revolutions in human history, the agricultural and industrial revolutions were built through tools that amplified what we could do. And then something really weird happened. The SaaS explosion of the last 20 years flipped this upside down. Tools went from amplifying the human to burying the human. We stopped using tools and we started managing them. 15 logins, 12 dashboards, entire full-time jobs were created to make sure that our tools were talking to each other. And even then, we, we didn't get the insights that we needed. When did we ask to be managed by tools? That was never the point. And now the problem is even worse because those rigid tools operate on the same 12-month cycles that we just spoke against. If it doesn't work for you, you have to wait until the renewal to change it. It doesn't evolve with you, and it's often the very thing preventing you from moving faster. If your tools move slower than your team, the tool is the constraint. So here's the move. Build what you need. Rebuild it when you don't need it anymore. Your tools should amplify you. They should not manage you. At Lovable, we rebuilt our customer success platform ourselves, literally, on Lovable. We rebuilt Gainsight. Our custom tool is highly adaptable. This is all anonymized, by the way. This is not the real thing. Our custom tool is highly adaptable to our workflows, right? We built our own command center, and so it tracks our customer portfolio in real time. It surfaces risks before they become fires. It surfaces expansion opportunities. It recommends best next steps for every account. Account, it tees all follow-ups up automatically so that nothing falls through the cracks. And the— my favorite part of it is it generates custom customer hubs that are individualized for every single customer that we share with them. It's a living, breathing hub for every single customer as our relationship evolves. So we didn't buy a CS platform. We built one. And when it stops working for us, guess what? We version it every week, every month if we have to. We're not waiting for a vendor to hopefully ship our feature request. We fully control our own destiny. And that's my love story. I'm not technical. You don't need to be technical. You don't need to code. Go back to what I said in the I said in the beginning, the power should not only belong to engineers. Anyone anywhere can now build exactly what they need. So take control of the stack and don't let it control you. Last move. We made value and revenue the same thing. What do you mean, Monica? Our revenue, your revenue, the customer's customer's revenue? Yes, all of it. It's all the same thing. This is why. In the AI world, something big is happening. Most AI products, Lovable included, are moving to credit-based or consumption-based billing. You use credits, you burn tokens, tokens, you only pay based off of how much you use. Do you see what that does? It, it completely flips the incentive structure. The more a customer uses the product, the more they pay us. Revenue is driven by adoption, and adoption is driven by everything that we've talked about— the product, the onboarding, the hackathons, all of it. It's one value flywheel. I know we've all seen the SaaS flywheel for the last 20 years, right? You have awareness and then you go to interest and then you buy and then you renew and then you advocate. If we're honest with ourselves, that was not a flywheel. It was pretty much a straight line that hopefully came back around at some point, but we're actually able to have a flywheel Now for the first time, the customer success motion and the business motion are completely one-to-one. The more I help you build, the more I help you succeed, the more value you get and the more revenue we earn. There's no tension. There's no awkward upsell conversations. There's just, let me help you do more of what's working. Perfect alignment, value creation, and revenue generation on both sides of the table. And guess what? Customer success leaders, all of us, we get to play a huge role here. We lay the foundation and we get to operationalize and design our motions, our talent, our programs in a way that they directly drive drive all of the leading indicators that produce the lagging indicator of revenue. We are closer to revenue impact than we have ever been in the history of this function. But there is also a hard truth that comes with this and the reality that comes with this. Now that AI is the default, AI makes everything possible. Possible. So guess what? Possibility is no longer the impressive thing. Proof is impressive. That's what's impressive. The proof. The customer who walks into their board meeting and says, this is what it did for us. And our job is to arm our customer with that sentence. Adoption kind of stops being— stops being impressive. Outcomes are the only currency left, and our customers are asking the harder ROI questions more now than ever before. Efficiency gained, time saved, it's not going to fly for very long. They want to see top line and bottom line impact, and the customer success team that can prove that value wins the renewal, wins the expansion, expansion. Everybody else is just crossing their fingers and hoping. So here's an outcomes story. So we have a customer called eXp Realty. They are a publicly traded international real estate brokerage with over 10,000 realtors, and we directly help them realize over $1 million in savings. and 3 very simple steps. Step 1, they ran a company-wide hackathon with their C-suite, and in 3 hours, they built dozens of prototypes of ideas to replace internal expensive software and to streamline their operations. From those ideas came 27 real products, including a pretty robust transaction management system and a microsite generator that allowed them to create websites localized in all languages. Those two alone replaced two major tools that they were using internally, saving over $1 million. Then they scaled this culture of building by launching launching vibes.exp Realty.com, which is an internal app store where anybody in the company can build and ship apps that will drive savings across the business. They ideated at scale, they built real products, and then they empowered everyone else to keep going. That's the proof. In this AI world, value value precedes revenue generation. And that's always the case. Revenue generation for you, your customer, your customer's customer, and everything in between on this flywheel. So your job is to focus on the value. So as you can see in these 5 kind of counterintuitive moves. There is really great news behind all of this. The good news, the great news is that customer success is not being disrupted. It's being promoted. The scope for us is bigger. The stakes are higher and the seat at the table is real, but we need to show up ready to claim it. So these moves ended up being our highest reward moves. As a reminder, number one, stop talking about AI, just swim in it. And when you do, you're starting from the right point. Second, rewire the clock. Align your customer success motion with the speed of the product so that nobody gets left behind. Your customer your team, engineering org, everybody moving together, everybody arriving together. Third, design for the experiences, the outcomes, not the activities. AI will handle the checklist. You deliver the magic. And whatever your product is, whatever your vertical is, there is an experience there. Waiting to be designed. Stop shipping your org chart. Fourth, control your destiny by controlling your stack, right? We talked about this. Build tools that amplify you, and when you need them, build tools that amplify you when you need them. But if you need them to change, you change them. Rebuild them when the world changes, because it will, and your tools should change with you, not hold you back. Finally, drive revenue through value creation. In a consumption world, value generation is revenue creation. That's the metric, that's the whole game, and we are the ones who get to drive it. As I said at the beginning, I truly believe that this is a once in a generation moment. I don't want us to lose this one. We're standing in it. So let's go. And I'll leave you with this. The companies that will win in the next 5 years, and I don't even think it's 5 years, I think it's the next 1 to 2 years, are not the ones with the best AI. It's gonna be the companies that, whose customers know that they have the best AI. Why? Because somebody in customer success proved it to them in person, in their office on a Tuesday, and they will say, "How can we do more?" The customer will end up the hero in this story, and they will tell the world about this. Thank you.

Welcome everyone, and thank you for joining us today.

Our program will begin shortly.

Please make your way to your seats. Give a warm welcome to the VP of Customer Success with Harvey, Tom Ronan.

Do we have the mic on? Can you guys hear me okay? Awesome. Hi everybody. I'm Tom Ronen, VP of Customer Success at Harvey. Excited to be here. Prior to Harvey, I spent 9 years with Monday.com, presumably one of the dead SaaS companies. And now I get the chance to kind of rebuild with AI. I'm extremely excited to be here today. I have to say, that's pretty humbling entering the SaaS world from doing something completely different over a decade ago. I leveraged Saastr's content quite a bit. It taught me a lot. So thank you, Jason and team. While we were on the topic of, uh, heartfelt confessions, I'll start with a confession of my own about my time at Harvey. We run a pretty traditional customer success motion at Harvey, executive business reviews, tons of, uh, onsite meetings and so on. Uh, a lot of change management frameworks that kind of look like they were printed back in 2001 by a Big Four consultant. And when a lot of CS leaders approach me to get some advice on how to run an AI, uh, native SaaS playbook, I tell them this and they look at me sideways and I go, yes, you know, that's, that's exactly right for us. Over the next 30 minutes, we'll try to make it quicker. Uh, I'm gonna walk you through why that's exactly right for a lot of you as well. A bit about Harvey. We are the leading AI platform for the legal profession. We work both with in-house legal teams as well as large law firms that wanna completely revolutionize the way they work through AI. Quick show of hands, do we have any lawyers in the crowd? Okay, I thought so. Not surprised. Overall, in January, we hit— we crossed the mark of $190 million in ARR. We are leveraged by over 100,000 attorneys in over 60 countries, in over 1,500 different organizations. All of these great numbers got us to a valuation of $11 billion, which we're very proud of. But you know what? These numbers usually will make the headlines. The numbers that will not make the headlines is those on the right here, which is just over the last 30 days we've seen something around 9 major releases from our product team. The product velocity is massive. These are not UI fixes, these are not some bugs that our EPD team managed to crush, these are major new surface areas added to the products. But I have to say that the product velocity, as much as it's is challenging, it is not the biggest challenge that we're facing. When a law firm of 200 person, let's, let's say buys Harvey, their product working itself is really not enough. We need to completely help this customer change the way that they're working. Think about it. Partners that have been practicing law for 30 years doing relatively the same type of work now need to completely give up their work product to a new AI solution. Workflows that have been built and fine-tuned decades over decades now have to completely be rebuilt. And in my opinion, that's categorically different than anything that what we call today's modern CS teams are actually set out, uh, to achieve. And so in this talk, we're gonna talk about more why it's super relevant for all of you running CS teams today. A bit about the, the, the crazy motion that we're all in with AI, how much the market itself is moving. The numbers behind me are taken from a Thomson Reuters research, research surveying over 1,700 legal professionals. And I think it was done in midway through 2025, so probably right now it's pretty outdated. But some of the numbers here, 26% of legal teams out there today are leveraging GenAI solutions. That doubled from 2024, and I think by now it has probably doubled again. 78% of legal professionals believe that AI will become central to their work, um, in the coming months. That's not a nice to have, that's central to their work. And the number that I think stands out the most for CS teams is this 4x numbers. Firms with a visible AI strategy are 4 times more likely to CROI than those with ad hoc adoption. The big sales opportunity here is that the deal will get made. Legal professionals are going to buy some sort of an AI, uh, tool. It's not about the initial land. It's really about what we do after that initial land. How much are we setting our customers up for success? Or are we really sitting in the sideways, letting our customers tinker around with a product, hoping that they'll be able to, uh, transform themselves. In a lot of these SaaS stages, what we're talking about here in this conference, agents, agents, agents, automatic QBRs, uh, super fancy health scores and so on. Humans working with agents in harmony together. If you're listening only to podcasts, that's probably what you're thinking your AI motion is going to be, to be once you're stepping into, uh, an AI-native company. That's the canonical AI-native CS playbook. I understand why it exists for many products. That's completely wrong for us. Our customers want to meet in person. They want a consultant-grade engagement. They don't want a Slack channel or a knowledge base. They want to be working with a person that understands their business deeply and is willing to go the extra mile for them. And here's the thing. That's not really a failure of our product. It's not that our product is not self-serve enough. It's great. Trust me. It really is a core principle that when you're selling, we are not selling software. We're selling a fundamental change in how the profession does its work. And so no automated QBR or a fancy health score can actually help you achieve that. So when you're selling an AI as the main core functionality functionality of your product, 3 things happen all at once. Something go— some things go, and we'll talk about the busywork. We'll talk about the assumption that adoption equals success. Very relevant if you've been here for Monica's talk right before me. We'll talk about what stays. That's the human layer. I'm actually the last person to be an AI doomer for the account management or customer success profession. I really do believe we are here to stay, and that's purely based on my experience from Harvey in the last 9 months and what evolves. We're gonna talk about change management quite a bit. If you didn't pick that up until now, I'm gonna say the word change management quite a bit.

All right, what goes?

The busy work, the administrative layer has to go out the window. Don't get me wrong, we use AI at Harvey quite a bit. We leverage Glean, we leverage Claude, we dogfood our own product to make sure that all the administrative busy work that CS SAMs need to get to goes out the door. Preparing for an EBR, making sure that you understand the next steps across a book of business of around 40, 50 customers, all of that is automated. And if you're running a CS team today and that's not table stakes for you, you're falling behind. The other thing that goes away is this assumption that adoption equals success. And this is where kind of my experience with Monday.com and, and, uh, more kind of traditional SaaS comes to play. This may be a, a controversial take, uh, that I hear often from Jason M. Lemkin, but I, I do agree with, at the end of the day, for a lot of SaaS products, if you have decent adoption, if the seat utilization is actually in place and you're talking to the same buyers, same decision makers and so on, uh, there's a good chance that CS did its job and we're good. We're gonna sign the renewal and all will be well. But with AI adoption, it's actually just a starting line. A firm can be logging into Harvey even quite often without actually changing the way it's operating. And so what we are actually looking for is to drive organizational change that shows up in business outcomes. For us, it's dramatically reducing non-billable work. It's speeding up the time it takes to get through a matter. It's helping firms win more business because their, their AI strategy strategy is actually so much more forward-leaning than their competitors. To get there, you need to understand and track the right metrics. So behind me, this is a, a page taken from our CS playbook. These, uh, this is the framework in which we help CSMs and, uh, our customers measure the ROI that they're seeing out of Harvey. We have these 5 ROI pillars and we ask our CSMs to continuously track value stories and tag those against these value pillars so that we can understand that we're not just tapping into, uh, monthly or seat utilization and so on. We're actually moving towards outcomes.

All right.

The thing that stays is the human layer. I can talk about it quite a bit, but I think I'll leverage a, a real story that I've gotten the chance to experience, uh, in my first, uh, couple of months at Harvey. So when I joined in as a CS leader, I was like, yes, I'm gonna join a sales cycle to see what people are actually selling that my team will actually need to be implementing later on. I set myself up for quite an involved, uh, engagement for us. We run very involved pilots. These are long engagements that, um, start with some sort of a business case where the firm thinks that they have a notion of what AI can do for them and then in a quite different place. So in this specific story, um, I was, uh, we were meeting with an AMLOF50 firm. It's a, it's a very big and prestigious firm. And we came in with quite, uh, difficult headwinds. Just securing that first meeting to kick off the pilot was not easy at all. Their complete senior leadership, all the partnership layer was very AI skeptics. Their CIO, the chief innovation officer, which is a role you'll find in many law firms these days, were actively benchmarking the application layer companies like against what the foundational models can do. And so it was difficult. We went on-site often. We built relationships one step after the other. And in one of the sessions that we had there on-site, we identified a senior funds partner that we were told is quite influential. We managed to secure a lunch with this person and we sat down for lunch. And after talking about the shitty weather in New York and the recent sporting events, our legal engineer, which is our forward deployed expert, we'll cover more on that in a second, ask a pretty direct question, hey, what's slowing you down today? And the conversation kind of paused for a second. The partner kept thinking. The output of that question was we were back in his office working on real live matters that actually involve the cases that this person is working on today, not demo, of a generic platform, not demo documents to upload it to the system to showcase some sort of an output. The result of that was a transferee DDQ review agent for the funds practice, a debt review summary tool for the debt finance group, an LPA comment consolidation workflow, all tied to real live bottlenecks that somebody in the firm actually had. Now for you, maybe I just spoke Chinese, but for legal professionals, these are the exact things that they're dealing with every day. These are major bottlenecks that will help them move the needle. We build these agents, one thing led to the other, and we found ourselves in a partners retreat. For those of you that don't know what this is, law firm partners love to work over the weekends, and they even love to do it more so in Florida, or if you're in a European firm, that's actually more scenic locations. But we get invited there to also get the chance to work over the weekends with our customers. And, um, we present about the landscape of legal AI and, and about Harvey and so on. And so we went there, our legal engineers were presenting about Harvey, um, quite standard. When our legal engineer gets off the stage, that same partner that we've built these workflows for grabbed his laptop, got off the stage, connected to the screen, and actually started to demo the workflows that we have built for him and his team. You can just start to imagine the type of flywheel that that generated internally. We got flooded with requests for custom builds, real live matters that people were working on through that weekend. We were there helping them solve in person. The CIO, after we secured the first land, yay, that account expanded 16x. That happened like 3 months ago. So good for us. Between that initial time that we just joined the office and had to deal with all these skeptics to that partners retreat. The model didn't change. The platform evolved, but not in a groundbreaking way. What changed was actually the human connection, the relationships that we've built, our access to information that we couldn't get over Zoom. And that's why the human layer stays and is probably just gonna evolve more and more. All right, the research validates base everything I'm saying. I don't wanna get too academic, but two researchers that I'm putting in front of you here today. One was run by Harvard, uh, where they took 15, 515 startups, divided them into two groups. One, uh, each group received the same AI tools, the same training. One group got somewhat of an additional help with change management. They received a document that's saying how teams like theirs have leveraged these same tools. in the past successfully. That one documentation, that little bit of help with change management resulted in 2x more revenue for, uh, the experiment group. And they were also 18% more likely to acquire paying customers. The second research here is actually from Microsoft surveying 500, uh, decision makers in massive enterprises making over a billion dollar in revenue. Um, it's quite a long long and involved survey. I, I looked into it. There are two components there after like a lot of technical details and so on. Uh, one of them is your, how strong is your AI vision? And the other component is, um, how, how much do you have confidence in your ability to execute, execute and run through change? And so what happened, the results were that was what was most correlated to success of the company and for their ability to deploy AI programs was the vision and the change management. All the other technical things were somewhere correlated, but the human element was the one that really drives revenue, and the research keeps coming up with it. For more of a SaaS audience, behind me, a quote from Aaron Levie, the CEO of Box. The winners in AI will be those that can deliver the change management that actually drives the change in workflow for the customer. Not the best model, not the best UI, the change management. All right, so let's double down on change management. I'll give 3 pillars that, that we operate by in, in Harvey and talk to you a bit about kind of how we actually do these things, uh, internally. Everything here was earned through blood, sweat, and tears, trial and error, and a lot of time spent live in person with our customers. All right, the first concept is build trust progressively. It's rooted in the ADKAR change methodology. Methodology. It's a quite common change management methodology that probably a lot of us know. One of the biggest things it says is that humans do not skip steps. You can't build desire without awareness. You can't build knowledge. You can't build ability without knowledge. And so for us, taking that into practice, one of the first things that we do as we deploy with customers, before they even log on to Harvey and actually click around and start leveraging the tool, we host a session that demystifies what LLMs do. We teach them about prompt engineering best practices and give them a framework to think about it. Now, I know a lot of lawyers' jokes by now. One of them is that lawyers like to follow rules. Haha, hilarious, I know. But it's true. When you give folks a framework to think about how to leverage AI, especially when there's so much at stake in the legal profession, the walls go down and they're able able to kind of soak in a lot of that, increase the desire and soak in more of that knowledge. The second thing we do very intentionally is map the culture of the customer we're about to work with. Who are the AI-leaning folks that like to play around with all the recent technology? We should be turning them into our champions. So we identify them and we give them special treatment. Who are the folks that will probably be the skeptics? We put all of those in a room. And we give them a personalized training. Sometimes we build together with them one-on-one. Lastly, we think about the business outcomes the customer is looking to see, and we think about how we can tie those into a successful change process. So for us, in a lot of these law firms, you're going to have the crown jewel practice group, that specific business unit that is either very prestigious or drives the most revenue. And we do enough discovery to understand and we try to put points on the board with that specific practice group so that everybody sees that the M&A department at so-and-so are doing amazing things with Harvey and will want to follow. The second bit is all about domain expertise. I get a lot of questions about how Harvey run an account team. So I'm gonna break that down for you here. But the main thing to take out of this slide is the importance of domain expertise. The first persona that we have within the account team is the legal engineer. They are not real engineers. Yes, we do have forward deployed engineers, but these are legal engineers. They are ex-practicing lawyers, uh, around anywhere from like 6 to 10 years that have practiced in big, large, prestigious law firms. They speak the language of our customers. They understand the operational nature of rolling out change within a law firm, and they're also problem solvers. Product experts. They sit down and go deep with our customers and open up time to value that no generalist CSM can do. We have CSMs on each one of our accounts. The CSM is that engagement manager, that conductor, the person that does the extensive discovery as we're going into our customers. They track adoption patterns, they get the feedback, and they understand, should we pivot? Are we on track? And work work side by side with our customers very closely. We hire specifically for folks that have a track record of driving change management within highly regulated environments. Folks that work with financial institutions, folks that work in the legal tech industry for sure. But there, I'm sure there's a lot of CSMs out here that are selling into that audience. These are the folks that we need. It's different than selling into a marketing agency. In terms of domain expertise, one of the things that we do that we found quite effective is segmenting the CSM's book of business deliberately and into kind of the smallest unit economic as we can. We don't just segment by SMB, mid-market, and enterprise. We'd split up a book of business by whoever is specializing in working with law firms versus in-house teams. They operate quite differently. Even within the in-house, if you're working with asset managers, if you're working with large telecom companies and we'll try to even give it the regional flavor. We have somebody working with telecom companies within the DACH region, for example. It's very hard operationally, but especially when deploying AI, that domain expertise is actually worth it. After the CSM, we have the AE. It's your classic value seller. I think what's specialized our AEs than other sales professionals that maybe you've been working with or working side by side with is how much they are truly value sellers. They run these very involved pilots that at the premise start with a business proposition of the transformation that they were able to drive. They work alongside that pilot and they basically setting the ground for that change management process. It starts pre-sale, not when CS just joins the account. Our AE, our AEs stay with the account post-sale. And make sure that we are seeing through to that business case. That creates a great deal of trust. It makes sure that the change process is seamless and no, no gaps in between. That fourth component that you see here, the Transformation Office, is a service center that we've assembled internally filled with ex-consultants that help us build these change frameworks and help us run more bespoke sessions with our customers, as well as ex-leaders legal innovation practitioners. So these are folks well respected within the legal innovation community that have rolled out AI programs within their firms or large in-house departments in a large enterprise. The extent that these folks can open doors for us, I cannot overestimate. When they sit down with a managing partner, the level of credibility that does— that buys us is way more than any big splashy type of product demo. Overall, around 40% of Harvey's go-to-market teams are with somewhat of a legal background. That's a number that we're proud of and we're trying to push that higher up and up. That's not a cultural decision cuz we have a lot of lawyers. It's a structural one because we believe domain expertise will accelerate go-to, go-to-market. Domain expertise doesn't actually flow one way from us to the customer. In every firm, in every large organization that we work with, there are these change management experts, folks that know the customer's organization cold, and we partner with them closely. We put them at the front of the change process. We learn from them, we don't preach, and we work together to deploy Harvey in a way that's customized for that, um, customer. Some examples that I've put here, you can see all of that. If you're interested in one of our blogs. So Alex Mason from Lewis Silkin distributed AI ownership within the different practice groups using what they're calling knowledge and innovation lawyers. Esther Bauer from Honigman designed a training program that's built around how lawyers like to work. They don't have time to sit in a classroom and listen to a boring presentation about how to use Harvey and what happens if you click here and there. So she designed a training module that's all about light bulb moments, these specific micro lessons that you can receive as a lawyer that will actually unlock new values and new use cases for you. And Hillary Goodyear at Ashurst did both dissemination of expertise within different practice groups, but they've also built an internal law firm that does everything in AI that's quite cutting edge, especially for a firm at this size. McKinsey validates kind of that domain expertise, saying that the most effective change agents are people who share professional context of the people being asked to change. All right, the last bit is all about going deep and not wide. And again, going back to that point I made about adoption is not all, for us we measure both outcomes as well as adoption. And when we measure adoption, we want to see daily use. What makes Harvey irreplaceable is when a customer uploads their golden precedents, their playbooks that they've been fine-tuning time after time again. When we take a workflow that's here in some senior partner and encode that into Harvey, this is when we are extremely sticky. This is how the CS motion turns a piece of software into infrastructure, and you can't really rip out infrastructure. And that's the power of going deeper with your customers, especially with AI tools that can do so much more once they have the right context. Okay. We're approaching the end, folks. Uh, 3 things that I want you all thinking about maybe doing differently after this presentation. The first thing is audit your CS motion against a real change management framework. You probably are all doing kickoff, you're probably doing some level of discovery and you're probably doing quarterly business reviews. That's great. That's probably not enough. The level of transformation that we need to drive, the level of discovery and depth that we need to go to does not lend itself well to a monthly meeting or even a, a quarterly business review. Learn a bit more about change management. Think about how you're applying that internally and you'll see that If you don't do that, the next wave of renewals can expose the gap of how deep you're going with your customers. The second bit is investing in domain expertise. I mentioned I come from monday.com. That's a horizontal platform, and I understand how hard it is to bring these domain experts where maybe your product is not a classic vertical SaaS. In Harvey, we have it somewhat easy. We bring a lot of lawyers. Boom, we're covered with domain expertise. But I urge you to think about how how you can verticalize your books of businesses, how you can bring in maybe not necessarily somebody who can actively talk to a customer, but infuse some level of domain expertise internally within your go-to-market teams, it'll go a long way to accelerate time to value for your customers. Number 3 is redefining stickiness. Think about what you're measuring. And Monica was here before me, uh, preaching about that as well. Monthly active usage is just not enough. We want to see daily use as a baseline and consider what are these outcomes that you're trying to drive and what is a systematic way that you can measure that.

All right.

As a closure here for all of us folks, I think everything I just spoke about, they're not new novel concepts. They're pretty antiquated, but they are— when they're applied to the new context in which we live here, in which we live today, they're quite powerful. The CS lead The leaders who win the next decade won't be the ones who automated the most. And I know we're all in this conference to learn about how the heck we can automate more, but that's not really what does the trick when you're applying AI in the real world. They'll be the ones who figure out that change management wasn't the cost of selling AI, it's actually the moat. Thank you everybody.

Welcome everyone, and thank you for joining us today.

Our program will begin shortly.

Please make your way to your seats. As a courtesy to our speakers and fellow attendees, please take a moment and silence your devices. Please welcome the Vice President of Customer Solutions with AssemblyAI, Ryan Seams.

Okay. Wow. Everybody can hear me? Yeah. All right. Awesome. Let's do this. It is what, 2 PM? A little bit after lunch. I'm going to try to keep this fun and interactive as we go along. So my name is Ryan Seambs. I head up the Customer Solutions team at AssemblyAI. If you have not heard of AssemblyAI, AI before. Uh, we build voice AI infrastructure for developers. So if you use a note-taker like Granola, Fireflies, you've actually probably experienced our product, but we're selling it at the infrastructure layer and selling models so that other companies can build amazing products and experiences. In terms of our team at AssemblyAI, we cover the entire customer experience and we're often working with, uh, founders, developers, VP Eng, VP Product, really this technical persona to figure out how our AI models can work with inside their products. I'm here to talk to you today about why your customers actually don't need a CSM though, and they actually need an engineer. So to kick things off, let's start with a little game to keep this interactive. I'm kind of curious, outside of the fact that I made this AI-generated image and putting it on on the screen. What do people see here? Smile. Okay. Anyone else? What? Laptop? Well, a smile and laptop, those two go together, right? That is a happy customer, right? This is somebody who is like excited to talk to you. They're happy to be here. Their body language is strong. Let's look at the same image again. What do people see here? Sorry, what? Yeah? A bug in the system. Okay. Frustration maybe, right? Anyone else? Maybe confused, befuddled. I don't know. Potentially upset. All those things are probably true in ways that you could describe the person on the left versus the person on the right. But the reality is this is actually what my customers looked like What was it like when I introduced myself as the head of customer success at AssemblyAI? When I started there 2 years ago, I went on a listening tour to go and talk to our customers to understand how do they use our models? What types of problems do they have? How have they been engaging with our team and how's the experience been? And the moment that I said, I'm the head of customer success, they actually had a visceral reaction in their body language language changed. They almost became defensive with the wall, and the way that the rest of the call went was not warm and fuzzy and getting strong feedback, but instead them thinking that I was trying to sell them or do some sort of voodoo magic on that call. And what's interesting, right, about that is it's not the person that comes to the call and does an introduction, it's actually the stigma that the title customer success manager carries around now in the industry, especially with the folks that we work with who are technical, right? And they're actually going to go and implement our product. And so when I reflected on that moment, to me, I think the core problem is that CSMs have really become seen as these commercially-minded message relayers. They're there to get the renewal and the upsell. They're there to make sure they do the QBR, to do the renewal process with you, to make sure the next task, right, is checked off the list, and send over all the work to the product team, the engineering team, the operations team, right? But what your customers actually need, right, they need people who are builders, people who have a knack for technology, and people who can actually get shit done and make things happen for them. And when you introduce yourself as a customer success manager, you're actually actively working against the role that you're trying to get done with that customer. So that's happening right now. Let's talk about what's happening with AI. You heard a little bit about this from Monica earlier at Lovable, right? Where she was talking about, you know, the 90-day onboarding compressed to 1 day. The reason this is happening is because every single company that's building with AI is racing to become the biggest, fastest company to grow in, in revenue in history. And the reason I pulled this specific chart was not because Harvey went right before me and it shows Lagora, but because it was literally from last month with another new company announcing that they are the fastest to $100 million ARR. ARR in history. The way that these companies are getting to $100 million ARR, right? It's not by putting your customers' needs on a roadmap and delivering them over the next 4 quarters. It's by literally sending an engineer to go work with these customers hands-on and build the feature set that they need today to help their own customers. If you're not the company that's going to go and do you don't do that with your customers, your competitor is going to do that. And ultimately they're gonna grow faster than you. They're gonna be the next company to $100 million ARR and they're gonna leave you behind. And so what we're seeing with AI and companies that are growing at rates that we've never seen before is that it's really broken what it means to be fast, right? Your customers, They no longer want to be on the roadmap waiting for you to actually go through your process and go out there and develop the feature, put it through the alpha and beta, finally get them to use it in production. They want these products built with them. I probably was polite here by putting weeks, but the reality in some cases is like days, and they wanted them to be built by engineers. I'll give you an example from our own team of this actually happening in reality. We, like I said, build voice AI infrastructure. We supply an API that allows you to send in an audio file like me talking right now, and you would get text out, right? Transcription on the other side. We had a customer who told us, hey, I actually want a bunch of additional metadata about the audio file to be what could be returned from that API? Things like, what was the amount of noise to the amount of speech? How noisy was the audio file? Can you score it for me? What type of accent did the person have that was actually speaking on that call? And they basically told us, "Well, if you can't do that, right, I'm gonna go build it myself or buy one of your competitors who can." And so one of our 4 deployed engineers actually sat down with them them, gathered their requirements, integrated 3 different open source models, deployed a custom API endpoint on Railway, and delivered that to the customer as an actual product that they can go and use for their voice AI infrastructure. All of that happened literally over the course of a weekend. And so instead of that customer going somewhere else, buying from one of our competitors and building it themselves. Now they're working with us. They're excited to work with us because they had a great experience, and we're going to help them grow their business. And what's great about that, right, is all of that insight and all the things that we build can become part of our product in the future and almost serve as like betas and POCs for our own engineering and technical teams. So where am I going with this? Well, what this has culminated in is this forward deployed engineer title. And if you've seen this in the industry, this thing is on fire right now. I think John has some real stats around this later, but I actually want to give some background on this title and why I think it's so important that you're aware of what it is and why you should be, uh, looking for it in the market. And so if you think back to the creation of forward deployed engineers, Palantir is often credited as the company who created the forward deployed engineer. But at that time, forward deployed engineers were probably very different than what you're thinking about forward deployed engineers today. These were folks who literally were going on site with the customer, coding in the customer's environment, deploying PRs, and actually building the product with that end customer, right? And so that model used to be like pretty expensive and hard. You would need to hire like engineers who were trained. They'd probably cost you, I don't know, $300K, probably more a year. Uh, you'd take months to onboard them and get them trained up so they could go in the customer's environment. Now that basically anyone can vibe code and has the ability to go and write code using natural English, almost anyone could actually be a forward deployed engineer. And so you've seen companies like Anthropic OpenAI, right? They're hiring for forward deployed engineers and not only hiring, they're literally creating consulting arms of their companies that are specifically designed around folks like forward deployed engineers going into their customers and driving business outcomes by performing technical work. You have other companies like Cursor, lovable, right? They're a little bit more at the application layer, but still, right, even if Monica maybe calls that role customer success manager. If you look at what that person's doing, right, hackathons, actually showing up at the customer's office, making custom applications for them. Sounds a lot to me like what a forward deployed engineer would do. And when we think about this and we look at what we had at AssemblyAI, I just wanna talk to you a little bit about the evolution that we actually went through through on this particular role and why we ultimately landed on calling our team Forward Deployed Engineers and what it means to our customers. So, we used to have a customer success manager role. We had the introduction, right, of what our customers looked like when I was introducing that to you. And so, it was about, geez, I think it was maybe 5 months ago, we were like, cool, we gotta go rename this role. It's clearly not the right role for what we're looking for. We went and did this big survey of the market, and me personally, knowing the origin of forward deployed engineer, I probably had a little bit too much bravado. I was like, well, these people aren't going to the customer, they're not actually coding in the customer's code base, they're not shipping PRs on behalf of the customer. I think it's disingenuous to call them forward deployed engineers, right? And so we actually went ahead and moved to the title TAM, which which many of you might know as technical account manager. That title seemed at the time like it was the right thing. These folks are highly technical, they do own the account, they're gonna be there to go answer the customer's question. But it turns out TAM also has its own stigma in the marketplace. A technical account manager is kind of seen as like a glorified support person who handles escalations, and they just do like the level 3 or level 4 tickets, and all they're really focused on is like, yeah, they just like do the tickets and, and that's it, right? They're not there to take really ambiguous problems and go find custom solutions for them. And to us, that's where we ultimately led from moving to TAM to forward deployed engineer. We actually tried to recruit for this TAM role for like the first 2 2.5 months of this year, and it was a disaster. I think we got like literally 2 good candidates in 2.5 months. We changed it to Forward Deployed Engineer, right? And we have an entire pipeline of candidates and all the right people. But the thing is, like, the actual role that we're asking them to do didn't really change. It's really about the branding and the stigma. Forward Deployed Engineer is like the exciting new version of customer success. and it's carrying that stigma around in the recruiting process, in how you work with your customers, and you should lean into it versus leaning out of it. So I wanna talk a little bit about this, like, CSM title and how it is a tax on your customer experience. So I mentioned earlier the body language from folks when they heard, hey, I'm the head of customer success, right? We actually literally had engineers who would, like, look at your Slack profile and see that you were a CSM, and they would, like, shut down and stop talking. It's kind of the same thing that happens when an engineer knows that someone is, like, truly just a salesperson at the end of the day. They're like, "Oh, I don't wanna deal with this. Like, I'm here to build. I'm not here to be sold to," right? And so we actually had a huge problem that, like, our own customers wouldn't really talk to us because our team was listed as customer success managers. Like I said, we moved to TAM, but this really decoded to our customers as, like, glorified ticket handler. They were thinking, "Ah, I could, like, escalate things to this person, like, they'll help me, but I don't think this person's, like, actually gonna, like, build things for me at the end of the day." FDE, however, right, this attracted real engineers, and it's made customers lean in. I took a screenshot here of an actual customer quote from a call. If any of you all all use Granola in the room. This is actually how AssemblyAI works. We take the speech, we turn it into text. You can see it on the screen. And I'll read this. It says, "If you guys have any sort of maybe, let's say, forward deployed engineer who might be able to help us early on, especially, you know, given the scale of, you know, hopefully our partnership together, it'd be great to just kind of get off the ground as quickly as possible." This customer's like literally begging us for a forward deployed engineer. And what's funny is the person who was working with them had literally updated their Slack tile to forward deployed engineer lead, like, a day before and was like, "No, no, I am your forward deployed engineer. That's what I'm here to help you with." And the problem here, right, is like, the customers want it and we have it, but like, why are we not just uniting those two things together? And to give you maybe some more anecdotal evidence to this, before I went on stage, I was actually talking to somebody that I know who's also at the conference, I was saying, oh yeah, you know, hey, my talk's actually gonna be about how like, you know, forward deployed engineers are really in right now and customer success managers are not. He literally said to me over there as we were chatting, yeah, I don't actually buy software anymore unless it comes with a forward deployed engineer. That's like the first question I ask. And so you may be actually disqualifying yourself from a number of potential customers with the titles that you ultimately have. For us at AssemblyAI, this is really like our new go-to-market org chart. Instead of having all of these different roles with all these specializations, forward deployed engineers are super technical. They can be generalists and they can handle the customer, everything from that initial land all the way through cross-product adoption growth. New deployments, et cetera. And so you'll see how simplified this org chart becomes. You have somebody in sales, right? Somebody's gotta go generate demand. I don't know if account executive is the right title. We might be here in like 2 years talking about how account executive became evaluation specialist or something like that and changed as well. But like, somebody's gotta go generate that demand and get, get the customer excited and start them through the process. But after that process, Good point. We have our forward deployed engineers take over with the customer and help them run their evals and benchmarks, roll out their initial deployment and get our actual models into production within their products. And then of course we're an entire infrastructure platform. You might imagine we have lots of different models that that customer could use, lots of different features they could use as well. And so they're focused on making sure that that customer has all of those things rightly enabled for their use use case and what they're building with voice. You'll see that there's this other category, which is like everyone else, and that's really where agents fit into the picture. And what's great about this setup, right, is your 4 deployed engineers, as they spend their time working with customers, figuring out the best practices, learning what works, what doesn't, writing custom code, they can take all those best practices and turn them into documentation. That documentation, those GitHub code examples, they serve as the basis for building amazing agents. You need context to go and build these agents. And so the 4 deployed engineers not only get to build for your customers one-to-one, but you get to use agents to scale their impact to all of your, your customers. If you compare this to a prior org chart, I didn't even put one on the screen, but I mean, I've seen them, you've probably all seen them, right? There's no CSM, there's no QBR owner, nobody's focused on activities, right? We're laser-focused on how do we make sure that the customer is getting the most value out of our technology and has the models integrated in the best way possible. So maybe let's talk about some examples about how this actually plays out in reality and the types of outcomes you could have with your potential customers. So first, I'll talk about what we affectionately internally call Club— Cub Claw, excuse me. A cub is effectively a baby bear, and so Cub Claw is our baby version of Open Claw. When you go and build a customer support agent, you'll probably go on the market and you'll go look at a bunch of different vendors. You'll pick one, you'll go roll it out, and you'll start to set it up so that it starts to answer customer questions, right? And you, uh, ideally increase your ticket deflection rate. Many people will go and talk about how their customer support agent has increased deflection rate, and they'll go and they'll say stuff like, "Oh my gosh, our team's amazing. We've got an 88% deflection rate." Then you ask them the key follow-up question, which is, "Okay, and how many of those customer issues does your agent pick up? Are they handling 100% of the issues?" And normally they'll be like, oh no, it's at like 20%. That's not really an honest metric of how your agents are performing in production and how you're able to scale your team's impact. The real impact here comes from actually taking that pickup rate and multiplying it by the end-to-end resolution rate. And so here you actually get what is the agent's performance end-to-end on helping customers. Customers with their problems. And with us, we had a vendor for this. We tried to customize it and tweak it and fit, you know, our specific workflows and FDE best practices into that vendor. And ultimately, like, the best we could do is like 10 to 15% on this metric. We decided, well, our team's technical, we could build this. We know how to work with customers. We know how to build agents. We ripped that out and we went from 10 to 75% in 45 days. And so now our customers on demand, 3 out of 4 times can get the best practice answer that our 4 deployed engineers would be giving at any time. This is pretty cool and awesome to see. This of course though doesn't deal with that other 25%, which is like, What's happening there, right? What kind of custom work's happening? Where are people spending their time? What are some of the unique things happening there? And so, the reason that this agent is named Cubclaw, which is baby bear, is because internally we have Bearclaw, which is our fully functioning internal for-deployed engineer agent. We've taken what our team is doing day in and day out and turned those tasks into the skills that are built into our agent. And so a common use case that our forward deployed engineers would be doing is someone's actually going and testing our models. They're gonna try us for the first time. And so they're gonna have a whole set of configuration on the API of features they have enabled, what languages they're supporting, all that fun stuff. In a prior world, our forward deployed engineers would like go to Datadog, query BigQuery, maybe look at some dashboard in Salesforce or customer success platform, right? And try to figure out exactly what's going on. When you go agent native, you could teach this agent to go and actually review every single customer's configuration live in real time as it's happening and provide best practice advice on how we should actually improve that customer experience. And so instead of this being a case where where like it probably takes somebody, I don't know, the whole day to go and look up some of these configurations and maybe they get to 5 customers. You can scale what those 4 deployed engineers are doing to all your customers at once and make sure that everybody who gets the wrong API configuration gets a nice reach out and message that says, "Hey, you should probably do this differently. You know, your use case is X and you're using our product in this way. You should actually go and change how you're doing that." And this doesn't just apply to APIs. AI configurations. If you think of something that's like classical, which is like a support ticket with a bug, it can actually take the bug and go submit a PR to our code base and send that to our engineering team so that they can actually review it and go merge it into production. You could do things like running health checks on your accounts, right? And so it can do things like query your calls and meetings, your support tickets, your emails, your Slack messages, and actually figure out, hey, what's happening for real with this account? Like what should I be aware of? What are some of my blind spots? For us in particular, the key thing we were able to do with this is we were able to scale our customer support 3x year over year, customer count, with the same number of folks internally on the team. And so we were able to have extreme efficiency and create this on-demand customer experience. So this is kind of funny, right, though? Like, this probably sounds a little funny, familiar to everybody in the audience. It is actually the old CSM job, but it's done a very different way. Your FDEs are going to be able to build this experience. They can make your customer experience a product that you can scale on demand and look at metrics like something like, you know, agent success rate and really improve it across the board. CSMs, unfortunately, don't typically have the skill set to go and do that. And so there's a big difference in who you're hiring for, not just how they're gonna work with customers, but how they're gonna be able to scale your processes and your teams internally. And so, if you take anything away from my talk today, I would say hire FDEs, not CSMs. There's a lot of evidence that customers actually want engineers years now to build together and fast. If you want to be one of those fastest growing companies going to $100 million ARR, right? Even if you don't, your competitor does. And so you need to be fast to be able to give your customers the experience that's going to be differentiated. Second, if you can, you should try to retool your CSMs into being forward forward-deployed engineers. Now, there is a thing here where you should beware, right? You can't just turn someone who's non-technical into a forward-deployed engineer. You will have to retool, train them, coach them. But if you really lean into it, anyone can learn to code using Cloud Code, Lovable, and all these tools out there these days. And so retooling is also something you could pursue because that CSM title is actually a tax on your customer experience. And then last but not least, you should really treat your customer experience like a product. Your forward deployed engineers are the people who are building that product. They're the product managers, the engineers, the designers. You should really think about how those FDEs can build that customer experience as a product and scale that through their work with agents. So my name is Ryan Sims. I'm the head of forward deployed engineering at AssemblyAI.

We'll see.

We'll see.

Yeah, I'd say, I'd say there were less sad faces. You know, there were definitely some smiles and I appreciate the claps. But the thing that I wanted to note about this, right, is like the body language definitely hits different. And you could try it. You could even ask your customers, what resources do you want? What type of profile, right? And maybe it's not exactly FDE, but I bet you'll find they're looking for something that has a lot of the similar traits and characteristics that are in this presentation where you can actually build with them and solve problems. So I'll close with this. You should start making the transition from CSM to FDE now because your customers, they already expect it. Thank you very much. Appreciate it a ton. Hi, how are you?

Thank you.

Welcome everyone, and thank you for joining us today.

Our program will begin shortly.

Please make your way to your seats. As a courtesy to our speakers and fellow attendees, please take a moment and silence your devices. Please welcome CEO and founder, Retention Intelligence, Bobby Cooper.

CCO, LinkedIn, Ashwin.

Hey, yo.

CCO, LinkedIn, Ashwin Vedyanathan.

SVP, Customer Success and Support at Content Square, formerly at Meta, Ursula Jabrez. And Global VP of Customer Success and Head of Business at Carta, Daniel Silverstein.

All right.

Are we on?

We are.

We're on.

All right. I guess we can do a quick little introduction here. My name is Bobby Cooper, founder and CEO of a company called Retention Intelligence. Before that, or early-stage startup, before that, I've been in the customer success world for about 14 years, leading CS at 4 different organizations. And yeah, love the CS world. Um, let's introduce the, uh, the panel today. Uh, Ursula, tell us about yourself.

Thank you. Uh, really nice to see a lot of familiar faces. Ursula de Bresse. I'm currently at Content Square leading the post-sales organization. Have been in the customer success post-sales world for a long time, starting with when Salesforce first started the, the practice. So I date myself. That way. Really excited to have this conversation and look forward to connecting with many of you afterwards as well.

Awesome. Ashwin.

Ashwin Vedanathan. I lead our post-sales teams at LinkedIn, was at Gainsight prior to that and several other companies, and have two young kids who are the toughest customers I work with in my life. Daniel.

Yeah, I'm Daniel Silverstein. I look after the post-sales organization at Carta. I'm also responsible for the broader cap table business end to end. So I've been at Carta for 7.5 years, built it sort of, yes, from ground up. Excited to have this conversation with you guys.

Yeah. You know, if you caught Jason M. Lemkin's keynote earlier today, he, he actually talked a lot about like this reboot that's going on in this new world of AI. And I think it was, the way he phrased it just really like hit it home. And what we're gonna talk about today is really just kind of exposing a lot of this broken strategy over the last decade. But I think Jason really summed it up as this, the need for this reboot. So yeah, we're gonna have a really casual conversation about this today. One of my observations is that our counterparts on the acquisition side, on the sales end, over the last decades, like, it's become very data-driven, very process-oriented, very predictable. And what that's actually led to is that AI tools and AI implementation will be acquisition side actually seem to be easier and more repeatable. And that's why you see probably 90% of the AI companies that are in the expo hall are mostly acquisition tools. But one thing that we have seen over the years in customer success is that CS sometimes can be less repeatable, can be very reactive, can be very fragmented. And a lot of A lot of that is seen in a lot of the resources that come into post-sale on the RevOps team. What that's turned into is now that we're going into this new world of AI, implementing AI-first CS strategies, this reboot, it doesn't seem to be as straightforward and methodical. We want to talk a little bit about how to go into this new world with success in mind. So let's dive in.

So first question.

So what's your take on companies that are trying to reboot their customer success experience, their customer experience in the new AI world? What are your findings, your observations, your learnings?

You want to start? Oh, I can go.

You go first.

One, and I'll give you a little bit of context on what's the foundation for this observation. In the last year and a half, I spent as an operating advisor to a private equity firm. So I was really exposed to a lot of leaders in different companies at different stages trying to implement AI strategies. And now as an operator back in the operator seat, observing in terms of how how my company is thinking through AI. And my observations are that we're coming from the SaaS world and we're looking at bringing AI like you would a SaaS deployment, where I think we need to start looking at AI as you're bringing new people into your organization in the sense that is what are the tasks that AI can do like a human would, for example, right? In terms of are they practitioners, are they generalists? Instead of thinking about what is the practice, ERP or CRM or my SDR process, for example, right? So one of the observations that I see is that many companies have started to create agents to solve for tasks, activities, and then they're trying to stitch together together a process. And I think good or bad, it's creating a lot of efficiencies, but there's also a lot of counter-effects as a result of that change. And so observations is I don't think there's mistakes being done yet. There's a lot of learnings and experimentations. And I think that the idea of this silver bullet in terms of cost reduction is actually a little, it's becoming expensive right now because we are learning. We're trying to figure out how AI plays a role and are we applying AI to existing broken processes, right? So my philosophy now, my observations is sometimes we need to go back to the fundamentals and then think how AI can solve problems like you would bring an individual with, particular expertise to help you solve those problems. So those are some of my early observations, but I'd love to hear from—

Yeah, I could— I'll add to that. I think your original question, Bobby, was maybe if we're applying some of the stuff on a broken systems from the past and the way I've been thinking about this a lot at Carto, we deployed Claude to all 2,000 employees about 2 months ago. So the data AI is all democratized. All the, the walls that you would've had to get to the answers that you needed to before, they don't exist. And so the way I've been applying this to all my post-sales teams, CSM, support implementations, is to start with the outcome. This is a customer problem for sure. We've lost track of that over the years, the customer outcome being the focus, but starting with the outcome, even with, with AI, Claude doesn't think linearly the way we do. And certainly CS practices especially have been very linear, step by step by step. I do this in this order because I do it. We built our own playbooks, the concept of a playbook we created. So I'm really encouraging as we internally build our own, we build on our own internal CLI, we're building skills, we're wrapping plugins. We are, this is a real life thing that we're doing every day now to stop thinking about steps 1 through 10 and tell Claude the outcome and and let Claude do the thinking for you on process. You have to iterate. You have the business knowledge, you have the context to know when it's wrong. But to stop thinking so linearly about every process that we've had in a customer journey, because I think we've gotten lost over the years on the customer outcome. And so, you know, when we're thinking about how we apply AI to it, maybe the first rule is like, don't follow the rule you followed before. Let, let AI do the thinking for you. And the second one is, Is this something I should do or something I can do? People get lost in the world of all the possibilities that AI can offer them and they lose track of the outcome they're trying to get to.

So a huge plus one to the idea of going back to the basics around outcomes. Sometimes, like in the world of post-sales, we tend to overcomplicate things as well. And like, I'm equally guilty of it. I think sometimes it's very helpful to go back to the basics. I think the biggest thing that I'm seeing is this relentless focus back to outcomes. And the reason the customer success and the post-sales teams in SaaS came to be was because of the consumption gap. There was more being produced in the product than what was being consumed. And that was the original basis for the CS function amongst many other things. It's almost like in this new AI world, are back to that same— like, we've kind of lost our way and now we have to come back to driving that adoption. And change management is single-handedly the biggest thing we can do. So I find it helpful to go back to the basics. Now, a few things that have changed with AI in our workflows, etc., is playbooks and other means to achieve those outcomes are not static anymore, and we don't have to just lead with, this is the only way for you to do your job. There are many paths that we can take, and the ability to have the human plus the machine change tax and work along the way is something that is like available to us now that maybe wasn't available in the same way in the past.

Awesome. I, I think like a takeaway just from hearing you, and, and I would question like as you guys are thinking of the reboot in your organizations is ask yourself, like, am I implementing AI as a cost efficiency measure? Am I training my AI to just do the same things my CSM is doing right now? Or am I actually looking at a fundamental reboot and reorg to do things different? Because we have an opportunity to actually reorg things differently. So here's a question, the provocative question here. Does AI actually improve a customer experience or does it just help us save money?

I think the jury is still out there.

TBD.

Yeah, TBD.

So I will say the efficiency is something that is very short-term. Is very perceivable. So many companies are starting to mine that efficiency. Whether it actually results in improved customer experience, I think the strong hypothesis is that it can. The thing that we haven't nailed yet is how the human and the machine journeys interact with each other. Like, people beat up on Comcast or whatever, right? You call Comcast, call them and something is broken, etc. How quickly do you connect from that AI world into a human world and then maybe back to an AI-driven world? All of those still need to be figured out. And I— but my strong hypothesis is that it will improve customer experience. Too early to tell, at least from the use cases I've seen.

Yeah, I would agree with that. I think the, the, some of the, you know, common topics around human in the loop. There's things that we need to figure out there. There's also customer friction that we created that I think AI will eliminate. Just all of us as consumers of both software, as B2B software and B2C experiences, the friction points are all disappearing one by one. And some of the things I think about with standard CS practice over the years, even our support teams that we have, sort of this concept of like trying to force a call, try to, you know, make your calendar the blocker to getting a customer outcome solved. A lot of those things I think will be done faster and more efficiently over time. And so, you know, it is— I think eventually will be both. Yeah. Is it a cost saving thing? It's hard to say. If you cut too far to the bone, you churn customers, then it has a different problem. But I think in the end, if we— if you think about it more of a— from a customer from a customer experience perspective, am I actually just removing the number of times I'm the blocker to them getting an answer? And not just ticket deflection and things of that nature, but actually just sharing knowledge faster, using other means of surfacing information and serving information to customers. I think it can have lots of impact there, and trying to meet a customer where they are instead of forcing them to come to where you are.

Yeah.

On our end, what I'm observing is we, Content Square, you know, we are in the, in the business of providing insights and data intelligence on the customer experience, right? So all the products that we provide customers are there to help them see around the corner and how they can serve their customers better. And we launched an AI Sense Analyst, an agent that allows them to do a lot of the work that a lot of their members do. So what's interesting is, is that this tool, this AI agent, is adding value in particular to those customers that went through layoffs and are reducing their headcount in those marketing teams. The challenge is it is starting to do a lot of the functions that these people were doing. The part is how do the people that are still doing the function adapting to do more strategic work and let the AI agent do this analysis, reporting, evaluation portion. So it's part of us learning how do you evolve your customer's role in terms of using that agent that you're presenting. And secondary is the level of AI proficiency across our customer base and even within our teams varies greatly, right? There's people people who are at the top of the spear, right, in terms of innovating, building their agents, really driving a lot of efficiencies in their own daily workflows. And there's people who are not there yet. They're a little bit fearful. They're using, you know, the GPTs as search engines still. And so that's the same thing that we observe internally, we observe externally. And so to say that there— it's adding value. I don't think we have a tangible data point yet, but I do see that change management, if we can help our customers evolve their own roles with the help of the technology that we're providing, as well as us evolving our own concept of what post-sales and CSM does while allowing our peers, the AI agents to help do the functions we used to do, that's when we're going to start to see some of that impact.

Awesome. Love it. Love it. So Ursula, a second ago you talked about like the fundamentals, the fundamentals of value. So what are— what fundamentals can we not forget when we rebuild our teams in this new AI world?

We were, we were having this conversation. It's very It's easy to get seduced, I call it, by technology and all the problems that we need to solve for. And for example, one of the fundamental elements to me is the customer at the center. Oftentimes when we're trying to solve a process or a problem, we forget to literally include the customer in that equation. We put them at the very end. We'll use the customer where I think we need to start with the customer first. Oftentimes your customers articulate the value that you can bring better, can actually tell us what we can do easier, what are the new expectations on how they feel that we should support them better. So I always say start with the customer first, always at the center of what you're solving for. That's a fundamental thing. And the other element, I always say it's less is more, even in the way you craft your KPIs and the way you're really narrowing and focusing on the problems you're trying to solve. So I always say one of the elements that we should never forget is customer first as one of the elements. I'm sure you guys have other key elements.

Yeah, yeah. So I'll dovetail a little bit into one of the other things we were talking about earlier today. Part of this to me is you mentioned like losing sight of the customer and I alluded to this customer outcome problem that we've lost sight of. Some of this is also other parts of your company and organization that also have lost sight of that concept, this shared metric. We talked about engagement as a thing that product cares about versus other things that CS cares about, uh, the customer outcome and the customer input into what you're doing. Uh, we, we've done, and this is another sort of good AI use case, uh, when you start to rip apart all of the data layers that keep you from making decisions as a business leader and as an individual contributor, uh, and that really should be customer feedback, the sources of all those things are all easily digestible and easily sorted through. Now we have a, a very well-established voice of the voice of the customer program that we've had for years. It would take, used to take weeks and weeks and weeks to compile the report, to get in front of leadership team and align on what we're gonna build and what we're gonna fix. These things are almost instantaneous now. So no matter what you're using in your stack today, you should be able to digest that stuff and get real customer input very, very quickly and before something is built. And we were revamping our voice of the customer customer program to, one, outcome-based, of course, but two, to make sure that we're actually hearing this stuff directly from customers versus validating what we did at the end of a roadmap, which is something that I think we've all kind of lost sight of in that roadmap concept. Get two-thirds of the way through the build before you get a customer to validate it, and you almost present it to a customer like you want them to pat you on the back for what you did versus starting the other way around.

Awesome. So shifting gears a little bit, I wanna talk about the CSM. So at Retention Intelligence, we have a predictive analytic platform, data science platform, where we dive deep into data to understand the root causes of retention. And one of the things that we find frequently with our customers is sometimes over 50% of the activities the CSMs are doing actually has like no correlation with retention or the goals that are on their plate. So as we go through these organizational shifts, how do we make sure, um, what our teams are doing stay focused on the outcomes that we're trying to do? Because I do think over time CSMs tend to be a jack of all trades and they keep getting work piled on, which brings up bloated teams. How do we ensure the CSMs that are this valuable human element we have on our team stays focused on getting the most important work done?

Yeah, we've— maybe I started off with this, but We've gone back to the basics of being extra precise about what a CSM's job is. And it's not just the CSM's job. We've kind of made this the services team's job, the support team's job, etc., which is driving product adoption, eventually leading to outcomes, is single-handedly your job. The retention, etc., will follow if we can consistently consistently and predictably deliver on this. Again, this isn't rocket science. This is where CS started many, many years back. We've kind of gone back to that core thing. And so then we've started mapping activities to saying, does this activity truly change the product outcome for the customer? And if it doesn't, then it has no place in our playbook going forward. And we've started correlating, if not causating, those activities to things, and this is not just a human activity, it could be an email campaign activity, it could be an agent interaction activity, any activity that the person or tech is doing, is that resulting in meaningful improvement in product outcome within the timeline that we intend for it to do? And if it doesn't, then it has no place.

On my end, Stefania, and I would love to get your thoughts on this. In the past, you know, and CSMs always ask for more prescription in their role. Like, what am I supposed to do? What am I not supposed to do? I agree. Like, the role of the CSM is to deliver on the expectations set at the very beginning of the journey, right? Customer success is really delivering on the expectations expectations set when we sold the product and making sure that we are realizing the value that the product is supposed to bring to the business in the context of what the customer is expecting. And so ensuring that we can capture those expectations, what are those value moments? The CSM has to have that ability to have moments in that journey lifecycle to deliver upon those moments that matter, those value engagements, I call it. But in terms of being super prescriptive, it depends on one, what your problem, what you're trying to solve for, right? So in our case, retention is a lagging indicator. So we gotta go back in terms of where are the broken parts in the process and what are the things that we need to solve process-wise or human-wise, right? The very first moment is relationships. I do think that CSMs, not machines, are the ones that build the relationship and we cannot lose sight of that. That's number one priority for every person in my team is make sure that you're multithreading the relationship, that you're capturing, that we have the right data mapping to be able to leverage those relationships for different things, right? So number one. Is understand your business, understand the data in your customers. Each CSM may have a different book of business with different profile companies, different profile use cases, and I think instead of being super prescriptive is more providing guidelines. This is, they want more prescription, I wanna provide them guidelines in terms of look at your job as your CEO of this universe of accounts, right? And the outcome is to ensure that they are, that the product is delivering on the expectations of value and that as an outcome of that, we increase retention, we have the conditions for growth, right? In essence, what is that? In order to do that, what are the things that we need to do? May slightly vary. I mean, there's a playbook, but how they apply the playbook I think especially for the enterprise and above, not the scaled, varies. But even for the scaled, now with AI, we can be ultra personal. We can really personalize those journeys at scale. And I think there's an opportunity to be less prescriptive and more framework driven in terms of what do I think success looks like and use the humans to really focus on building the relationships and understanding and capturing that customer insight and feedback.

So, and I'll maybe build on Ursula's point on multithreading and relationship building. So at LinkedIn, one of the products we sell is into the recruiters at companies who are not sophisticated technologically to do their jobs, right? maybe code development, maybe on the front end of using technology. And so in this world, like, the two biggest activities that we've seen a CSM can contribute to is one, like, we've always had strong relationship building with, like, the decision makers or buyers and maybe the administrators. But who is actually working with end users a lot more and where where can the CSM help multi-thread with managers, like first-line managers, second-line managers of these recruiters. Those are the ones we are seeing as like makes or breaks Agentik adoption for the products that we've launched. And so we've had to develop new playbooks for those types of personas and use cases. So that was one. And then the second thing that we saw was for these Agentik products to be successful with users, actually requires product workflow and product re-engineering at the customer. But in addition, also change management, because we need those users to train these models on an ongoing basis. And if you fear for your job being lost to that agent, you're not going to go all in on those— on training those agents. And so how do you get people over the hump in working with them, using some of the champions to help convince the detractors. These are all playbooks that we've kind of lost over the last few years that we've had to reinvent and reinvigorate with the CSMs.

Yeah.

The only thing I'll add, and this is not meant to be a controversial comment, I don't think it is at this point in time. One of the things that we've always been responsible for, I say responsible now accountable for, is revenue. Maybe another sort of lost thing in the midst of all the years of CS practice is like, you know, some CS teams have had no revenue responsibility, have not been maybe to your point, been forced to maybe think as business owners. So the business acumen and the business skills required to be revenue responsible and/or revenue accountable. We were revenue responsible for many years. We held quota, we upsold, we built expansion motions. We've moved now into to revenue accountable, and rather than just hiding behind sort of your blanket NRR, GRR metric around that, there is a revenue mindset that comes from engagement, and in terms of the prescriptive concept that you're talking about, everyone wants to know exactly how to do that. It's actually pretty simple. It is just finding the right moments in your journey and getting there before the customer gets there, and if you see that the right way and you know what those moments are, are you understand the customer journey, not the way you mapped it that became static over the years, but the way they actually move through your product over time. Revenue results from that, and it does matter to the business deeply. And that is something also that keeps that CS part of your organization to some degree more shielded from AI becoming a cost-saving metric. If you can prove that you're both revenue responsive responsible and maybe revenue accountable also, that's a way to think, is this activity going in that direction or not? If it's not, then there's a pretty good chance it's not the right thing.

Yeah, awesome. Love it. So you said CSM's job is to deliver on what was the expectation at the beginning, right? So when I think about that, we're talking about a lot of alignment. Alignment, right? There needs to be a big alignment. We need to make sure we're maintaining alignment between the acquisition and retention teams. So I want to get really like tactical here. If there was like one KPI that we could literally measure and compensate every single employee from sales, marketing, customer success, support, like the entire go-to-market team to drive drive better customer-centric alignment, what would that KPI be? Like one KPI across the whole org to bring alignment together.

To bring alignment together. So I answered that question earlier. I had one KPI, but now I thought of another one.

Okay.

So the one that I just thought of that would bring alignment is, And we don't measure it, I don't think, handover success. Because handovers is where a lot of the fail points happen and a lot of the data's missed and a lot of the problems happen from one team to the other. So I would love to have a standard way to measure handover success that correlates to impact on time to value, which is my right now, my first priority metric. Why time to value? Because the shorten— what is value first? It's articulating how do you measure value, not from your own product's perspective, but coupling that with meeting that first set of expectations, right? Shorten that time to value. The rest, obviously, gross revenue retention, net revenue retention's our kind of outcome metrics, metrics as well. But those would be my— time to value is my primary right now priority to improve. But I would love to add handover success as well because that really reflects on that ability to cross-collaborate across all the different moving parts in that customer journey.

I love it. And I agree. I think time to value, time to customer, That tangible value, in my opinion, is the most important in my view. I'll give you an example when we implemented this one time. I was working for this company called Weave. I joined when they were pretty small, about $8 million ARR. We grew over the course of a few years. I grew from about $8 million to $200 million ARR. We IPO'd. When we were right there around like $10, $12 million ARR, we made a fundamental shift on when we could mark a deal closed one.

Mm-hmm.

And closed one was not marked by sales or rev ops. Closed one was marked in the implementation phase when a customer met certain criteria and implementation that we deemed had a really high likelihood of success. So what happened was, is it actually solved for the handover problem.

Mm-hmm.

Meaning if a deal was put pen to paper, but they never kicked off, it was that failure to launch, it was never actually counted as a booking. Deals were counted as bookings when it crossed a certain threshold in implementation. So what happened was, is the sales team was super driven to make sure that they were selling customers that crossed this point of success. and it actually accelerated, one, it drove a ton of alignment. Culturally, it was great, and it drove, we took churn from 4% per month down to like about 0.5% per month while scaling there. So, love time to value. I think that's a great answer. I'd love to hear from you guys.

I'll go, like, we've recently launched some Agentik products, so this is very fresh 'cause we are, We're in the throes of this. And what we've realized is this notion of product adoption as measured by percent of customers who are successful and percent of seats that are successful has been a single unifying factor for us across all of these teams. Because what we realized was if sales didn't sell to the right team and we hold sales accountable to those adoption metrics, They stopped, like they started qualifying deals much better. And we also started like putting some clawbacks into the compensation models, etc., that if you don't hit a certain percent of successful seats or users, then things are clawed back from your compensation structure. Again, all tied to the idea of adoption and percent successful seats, successful customers has been game game-changing for all of us to row in the right direction. The other team that has now been more invested than ever before has been our product team because they are also measured on that same metric. And so now they're also in it to say, how can I build more in-product nudges so that I can bump this number up? Otherwise, like, it used to happen based on, like, collaboration between those two teams, but now we are using metrics. Tricks as a way, way to force collaborate and like grow in the same direction.

Love it. Yeah, I, I like that. Agree with both, your perspectives on this. And, if I could just maybe on the time to value thing, make it the same. I feel the same way you both do about it. Time to value realization, just swapping out the time to go live or time to activation. Yeah. Because the value realization to your point, Bobby, is like, hit the customer doing the milestone that shows that they're getting what they paid for, to your point, which generally is a leading indicator to success in renewal and commercial outcomes. So, you know, we focused relentlessly on time to value and onboarding-related churn from customers who didn't engage during onboarding, maybe weren't sold the right thing, maybe they sold to the wrong person at the company. And, you know, we had a massive onboarding churn problem that was years in the making and a backlog backlog of churn, of onboardings that had not gone through as well. So the concept of value, time to value as a metric to throw up is, was really just for us. The value realization is for them, and knowing what those first steps are, which are, to be fair, segment-specific, product-specific, depending on what they bought from you. There could be a land and expand where the expand part of it is the success metric that you need them to get to, depends on what was sold to them, but really focusing on the value realization part of that.

And I would love to have alignment to the point where I could get clawbacks.

Not that I'm going to be the clawback police, but the point we've tried this concept before, same, which is like, you know, closed bounty or some version of that with a RevOps like opportunity record stage where you get paid in full when first renewal happens or some version of like first milestone. Those concepts I think would be, again, if you're asking a wish list, what could I, if I snap my fingers, what would I do? That would be probably the thing I would do is align on like what that segment-specific, product-specific value realization moment is and pay in full when that booking comes to fruition because that's the best indicator they're gonna renew and succeed.

Yeah, no, absolutely love that. All right, so staying on the topic of KPIs, what KPI should we just kill?

I guess like, Yes.

Like what KPI handstand should we get out of our CS language and no longer report on?

I think unanimously we agreed NPS.

NPS. Why?

Um, I have lots of reasons. You guys go ahead.

I, I'll tell you an example. Okay. We have a great NPS score that does not reflect on the, on the GRR scores. There's a big divide between NPS, which is showing like, oh, our customer loves us, but they're leaving. They're not loving us with their wallets. So I'm like, where's that? Why is that gap? And so it's a vanity. I call it a little bit of a vanity metric. The way it's executed, it doesn't necessarily provide an easy ability to action. It requires a big translation engine, let's put it that way, into like, what does this mean? How do I action this in a tangible way to actually address it? And if I do the— if I follow the outcomes of this NPS, is it actually— is it pointing me in the right direction? I don't think it does necessarily. So I think you guys have other opinions about it.

I do.

You go ahead, because I was going to steal something you said earlier and credit you for it.

NPS, definitely. Like, I also haven't seen correlation between that to any business outcome metrics, but I'll maybe tack on one more, which is oftentimes post-sale leaders tend to hover in the input metrics bucket quite a bit. Like, how many calls have you made? How many QBRs have you done? How many— like, if they're are email campaigns, etc., how many clicks do you have, etc. I don't think those have any space anymore. Obviously if you're early in your maturity as a team, perhaps you should start there just to make sure you're dialing things in, but I wouldn't like— we are guilty as a community of being in that space for far too long.

Yeah, I was gonna— we were talking about this earlier. NPS, I have— I'll give you 2 or 3 reasons, not all the reasons why I think it's terrible. One was that, like, the white space there, like, who's not responding to NPS? The response rates are generally very low as a percent of your total user base, and the ones who are not responding are the ones that are churning, which is, you know, I'm not saying that's a one-for-one correlation, but there's a lot in who's not responding to NPS. A lot of NPS scores are administered through a number of different methods. It's in-app, it's email, it's like there are different delivery methods. But most of them are time-based. So you get people a moment in time. And even the person who, in our use case, where they don't log in often, they might log in because of the survey for the first time in 4 months and remind themselves that they aren't getting what they wanted for whatever reason. And that triggers a negative reaction, or they It happens to hit them every 3 months or 6 months, right after a major thing happened during busy seasons of the year, gets them their lower scores. I think they're all influenced by the wrong things. And to your point on inputs, I don't think any of the inputs really show up on NPS in a way that's meaningful. I can't kill it. The board cares about it. People understand what it means. It's a universally agreed upon thing that people look at. But what I do is supplement it with CSAT and CES and other things and aggregate them all as a score, as opposed to just using one single input. Because CES and CSAT, for implementations and support teams, are moment-in-time transaction-based. How hard was this problem to solve with me right now? And we plant those things in product as well, so it's not just a CS problem, but it's a, did product build the right thing? Was this easy to do or not? So I think it's, if I can't kill it, which I would love to, at least I can try to supplement it and make it more well-rounded.

Yeah. I would add my own other metric. I talked about the metric I love of time to value. I think one we should kill is time to live, because most companies I've worked with, consulted with, time to live is a very arbitrary metric that for the most part, it's more of a— sometimes a finance metric. Time to live is when we bill them or when we give them their logins or when they log in for the first time, but it has typically no correlation with the actual value they're receiving. So they can get kind of confusing, but if your time to live is based on an arbitrary metric that is not correlated with retention, kill it. So we have 5 minutes left. I would love to open it up for questions. I don't know if we have mics out there, if we just want to shout any question, or go ahead.

There's a mic in the middle.

Oh, there's a mic right there.

May I have to ask you to pass it down?

Sorry, I'm getting the noise from over there too, so.

—scaling from smaller to more enterprise engagements. What are kind of best practices that you might have picked up when working at smaller orgs in terms of information sharing internally between kind of pre- and post-sale? And how do you recommend smaller organizations going through kind of that transition phase from, you know, smaller to enterprise can really work together and bridge that gap between pre- and post-sale information sharing and really having all of that full, full context.

I actually think this is a great use case of where AI works wonderfully, because face it, in a small organization, sometimes the handoff note is on like a Post-it note. It's like, hey Johnny, I just sold this deal. You know, as you grow and scale, things have to be systematized. And there actually are amazing tools out there that are listening to every call, every email, every interaction, documenting correctly. I think it's, it's architecting the system first while you're small so it's ready to scale.

But I think to answer your question, moving upstream into your customer segment, if I understand correctly, right, is servicing larger customers. Yeah. The biggest thing you need to be prepared is on the expectations larger customers have on your roadmap, actually. And how do you equip your teams to speak confidently on the decisions made in the roadmap? What happens is larger companies wanna influence your roadmap, wanna have features, they have certain expectations. That your smaller customers don't necessarily have, which slows down a lot of your processes in general. And so, and demands time of your product teams, more cross-functional representation in the way you do your account management, right? So I think that's one of the biggest differences that I've seen when you go upstream. There's many others, but I would say that is one of them. I think that's a big one to be prepared for.

The other thing I would add to that, because I agree with you, is this is— I think part of your question was also around pre- and post-sales alignment on that. Some of the rigor of discovery and selling better and deeper and understanding customer problem is where I see that break sometimes, because to your point, these are more complex use cases. These are also bigger buyer decisions. Like the bigger, the further up you go and the more enterprise you become, the more likely the thing that they're buying from you has risk to the person who's buying it from you. You can imagine going all the way to the top of this, the sort of classic software, no one gets fired for hiring IBM. You, if you're selling an SMB product that could be replaced by another one, low risk. If you fail onboarding, you churn, go somewhere else. If you are spending a huge portion of your software budget on something, saying it matters deeply. So the CSM responsibility of understanding the product deeply enough to represent and, and ensure that they're using it and know what they got, but also the sales team understanding how to more deeply sell, sell that. So if you have like any influence over whatever enablement function works or doesn't work in your organization, if you want AI to help you with that, to understand it better, the, the real problem is not understanding deeply enough what the customer outcome is and your CSM not being technical enough to accommodate it. And just put yourself in the buyer's shoes. It's a bigger decision with a bigger dollar tag on it.

The other thing you just reminded me of, which is really important, is that in larger organizations, you're always selling. And the reason being is whether you sell wall-to-wall, you're deploying through phases, or you do a land and expand, which is often the case, the either whether you organize your teams as pods, the sales motion is constant on the larger accounts, whereas in the smaller accounts you sell and deliver a lot quicker. So that's also a big difference in terms of you can hand off things in the smaller accounts where larger accounts you're kind of in a pod space. There should be.

Sorry, Ashwin, do you want to add anything to that? All good.

Thank you. Yeah, please.

So I completely agreed with your comments on NPS and, you know, was smiling there when you were saying. So could you share how you're thinking about collecting customer sentiment these days with the tools and what you think about CSAT and like customer interviews, etc.? That's a good question.

Yeah. So this is so So in founding my company, which again, we're really early stage, I truly believe data science can tell us so much of what's going on with the customer. So one of the things we do is we ingest data from every source, CRM, data warehouse, product telemetry. There are very, very specific patterns that you can get out of your product telemetry that can tell you the sentiment of your customer. You just have to have the right data model. It's actually very difficult to do. I was doing this as a consultant for a while, and I had a team of 3, like, data engineers, data scientists on my team that were creating these models. I decided to try to productize this whole thing. But if you're not sending NPS surveys, there's a few ways. One, you're getting a lot of email from your customers, you're getting other contact data points from your customers. You're probably measuring them in Gong and all your systems already, but your product telemetry data, your utilization data, can tell you a lot how they're interacting with the product. If they're logging in and giving up after an hour, after a minute, the signals are there. They just have to be like, deconstructed and correlated with something. But yeah, love to hear— So go ahead, Gaurav.

I was going to say, I find customer effort score that Danielle mentioned earlier very helpful for new product launches because people haven't figured out— the product team hasn't figured out the right end state yet. But especially in the world that we live in, there is a velocity angle that is super important for product teams to get right, which doesn't always mean that the product is the finished state. And having that customer effort score signal for how long does it take for a user to do their job on any particular part of the product is invaluable information for post-sales teams where you could put humans or AI on the problem for the time being, but it has to be solved in the product itself. And I find customer effort score very valid.

Yeah, I agree with you. CES, combining CES, CSAT, and NPS as a more holistic view. This is part of our, like at this point, weekly voice of the customer analysis is digesting all of those things. And to your point, all the data sources that are out there. It's a lot easier than it used to be, but my recommendation is don't pick one. Have more well-rounded view of all the different customer interactions.

I think in addition to those two, I would add we have with AI the ability to consume so much more information around sentiment and actionable sentiment. So one of the things that we're exploring is we record and capture every single interaction with our customers. And we, and I think we are kind of fascinated with scoring stuff. And to me, it's like, score is great, but I want to know exactly what the problem is with that customer, if it's happy or not, why they're happy, why they're not happy. So two things that we're doing is we want to bring sentiment feedback on each customer into Gainsight, which is one of the products that we're using. The other thing is we work with a vendor called Difly that does interviews at scale and one-on-one interviews on our churn accounts and we're bringing that, that information as contextual information to evaluate our churn insights, right? So it's kind of like postmortem learnings and in-flight learning. So in addition to the CISA and the CES scores is sentiment actions. And these are based on moments that matter, right? After after a QBR or after an implementation or after a moment where value was supposed to be exchanged, what the outcome perception is. But yeah, that's a good— I mean, it's easy to hate. It's like, what are you gonna do about it to replace, right?

Awesome. Oh, there's a question.

Oh, we have to— We're getting told that we're over time. Can I take your question? Can you grab us offstage? Love to answer your question, but we want to be respectful to the team here. Thank you everybody for showing up, give us your time, and appreciate it.

Thank you.

Let's grab us something.

All right.

Please welcome back John Gleason.

All right, let's, let's deposit this water and get down to business. Us here. How's everybody doing? Good. All right, we are on the home stretch now. My name is John Gleason. I'm the founder and general partner at SuccessVP. Success Venture Partners invests in applied AI, and the special thing about our firm is that we are backed by some of the best customer success leaders in the world. We're talking about companies like Anthropic, OpenAI, Harvey, Lovable, Toast, GitHub, GitLab, just to name a few. Prior to starting SuccessVP, I led customer success at a company called Motive. I led there from $1 million in ARR to over $300 million in ARR. And this combination of experience has given me this really unique vantage point point on how everything we thought to be true about customer success changes in this applied AI world. So let's get into it. Here we go. The customer success manager role grew over 700% through to Q2 2022, and then it flatlined. 4 years and counting. And I think it's left some people in customer success absolutely reeling. Meanwhile, a new role has emerged. Ford deployed engineering is up over 1,000%, a role that 5 years ago almost nobody had even heard of. And it's still climbing, peaking in Q4 of last year and climbing into this this year. And yet, when I talk about forward deployed engineering on my Substack or on LinkedIn, I get major pushback. Everybody says, yo, whoa, wait a minute, isn't this just a rebrand of things we've already seen before? Isn't this just professional services, sales engineering, and onboarding? And the answer? Maybe. But I think as customer success leaders and as a broader customer success community, we'd be naive to think that that is all that's at play here. After all, customer success was just a response to SaaS. We didn't invent the work that goes into customer success. We simply rearranged it to solve the unique unique business challenges that the SaaS business problem or business model brought up. And the truth is, even within customer success, there's never been a right way to do customer success. We all know the debates. They're endless. Should customer success own revenue, lead onboarding, focus on NPS, drive adoption? And I think All of them are right, depending on the company. Instead, what we've seen is dominant archetypes of customer success that map back to nuances within the SaaS business model. On one end of the spectrum, you see heavy enterprise deployment motions. Think Salesforce, think HubSpot. On the other end, we see product-led bottoms-up motions. Slack, Atlassian, those are great examples. And in the middle, consumption-based. Stripe and Snowflake are two canonical examples. Each one is a different orchestration of the roles, the responsibilities, and the focuses that make up a customer success organization. And this begs the question, is AI just a new archetype of something we've seen before, or is it an entirely new operating environment? Well, if you talk to CCOs, what they're dominantly going to tell you is that AI is all about productivity. But this warrants an inspection. Most of the companies where we find a CCO exist in the traditional traditional SaaS world. And while many of these companies have made great strides when it comes to AI, whether they are public or whether they are private, they've all hit serious capital-constraining head— serious capital-constraining headwinds. And so yes, to them, AI is all about productivity. But much like the shape of CSM hiring, VC funding into these traditional SaaS companies has flatlined while funding into applied AI companies has skyrocketed. And if you look at the shape of that, it maps almost identically to that of hiring Ford deployed engineers. You see, these new AI companies, these applied AI companies, they operate from an entirely different set of principles. Principles. The assumption in an applied AI company is that there are jobs to be done and that the software will actually do the work, not just the people using the software, not just making them more productive, but actually doing the work. And so with that in mind, I would argue that applied AI is more than a new archetype, which leads me to the question, to the question that I know is on everybody's mind. What happens to customer success? Well, in my opinion, there are two forces on customer success right now. One is deeply technical and one is commercial. The first is that these AI systems are largely non-deterministic. In other words, the software doesn't compute the same way every time. Time. The same prompt can give you different answers. The outputs in applied AI can be very unpredictable. The second is how customers pay. The value may not be in the software itself anymore, but actually in the work the software produces. And increasingly what we're seeing is that customers are paying directly for those outcomes. So who's ready to get into it? Here we go. All right, so we're going to go deep on these two forces, but force number one, non-deterministic systems. When we talk about non-deterministic software, what we're saying is that the same prompt with the same data can yield different answers. Deterministic software, the kind that most of us have been using for the bulk of our careers, follows instructions. Non-deterministic systems choose from a range of possibilities. That's not a bug, that's the architecture, but it changes how the software needs to be supported. All right, so we've all been here. We all know that the starting point for any great AI experience is context. Consider this, all right? First, you ask ask Claude a question, just a prompt, no context. All right, chances are you're gonna get an okay answer, right? Now you take that same prompt and you run it with access to all your files. The answer you're gonna get, we all know this, is far richer. The more organized the context and the more of it you have, the better output you can pull from a non-deterministic system. Context matters in AI, but context is only one side of the equation. The output is only usable in a business setting if it is verifiably correct. When we have both, we have the conditions for AI success— high context and verifiable correctness. They come together, and this is what forms our ability ability to create success in AI. And so let's think about this for a second. Where do we see this in the world? Where do we have both halves existing already? Well, the classic example is code. With code, we have both sides of the equation, right? Context that's centralized, standardized, and fully versioned, and correctness that's verifiable. The code either compiles or it doesn't. The test pass or it doesn't. We have guardrails like types, lint, runtime checks, all of this. The conditions in code are met by default. And that's why code was one of the first places where we saw really, really meaningful outcomes in AI. In our side of the world, the go-to-market world, tier 1 support And lead qualification are probably the two best examples. Both areas are where AI has taken serious hold because again, you can map it back to these two qualifying principles, context and verifiable correctness. But here's the rub, okay? The uncomfortable truth is that AI, while it is the biggest thing on pretty much every CEO's desk right now, only 56% of CEOs are seeing any financial gains from it. The mandate is very real, but the results in most cases are not. And it gets worse. 95% of enterprise AI pilots generate no financial return at all. I can tell you the problem is not the models. It's not the capability of the models. The, the problem is structural, right? And here's the structure that's broken: context and correctness. Code support, lead qualification— these were the easier cases. The conditions were already there from the start. Most business problems, we don't have those things baked in. We either lack context or we can't verify correctness. Somebody has to engineer both, and that job has become the job of the vendor. And you can see it, it's in plain sight. All right, when you look— and I pulled down like 100 FDE job descriptions— and when you pull them down, you either see them forming in one camp or the other. On the context side, Hebbia says hires FDEs for domain-specific solutions on top of our platform. Harvey, we heard from Tom earlier, wants them to integrate Harvey into customers' daily workflows. Rogo has FDEs deeply embedded with strategic customers. Every job description names similar work, embedding the customer, or embedding with the customer to understand their business context. On the correctness side, we see this at Glean. Glean's founding FDE owns prompt engineering, agent development, evaluation frameworks. At Cursor, they say Cursor is tracing evaluations and debugging model behavior. Hippocratic says deploy their LLM safely, effectively, and efficiently. Every job description on this side of the column alludes alludes to a similar thing, making sure that the model is verifiably correct. In every case, the FDE is fulfilling at least one of two conditions for AI success. Whether you call this person a forward-deployed engineer, a solutions engineer, an implementations lead, somebody has to do this work in an AI company. On the context side, somebody maps the data sources, captures tribal knowledge, documents undocumented workflows, and brings the stakeholders into the system. That's how the model learns the customer's business. On the correctness side, somebody builds the eval libraries, sets the success criteria, watches for drift, owns the quality bar, and this is how the model becomes trustworthy. You see, most teams don't even fully realize what they're hiring for. All that they know is that if they don't have these conditions met, the work is not getting done. I think that the title FTE is incidental, but the work is the new requirement of customer success. So let's talk about number 2, force number 2, units of work. We're moving to a world where the valuable thing isn't the software anymore. It's the outcome the software produces. And in some ways, at least to those of us who have been in customer success for a little while, this might feel like a homecoming, right? We've been to enough customer success conferences. We know the definition of Customer Success is, customer success proactively ensures customers reach their desired outcomes. But how many of us actually believe that? We've all said it, but do we actually believe that? Raise your hand if you made a slide like this. You take the seats, you multiply by usage, you layer in your hypothetical productivity gain, and you arrive at this massive number of value. And then you make your EBR and you stand in front of the customer and you say, "Oh my gosh, look how much value we've created for you and you only paid this." The problem is most of this isn't measured and it's really just storytelling. In a consumption-based business model, it's a little bit easier to tell tell the story of value. Each unit of usage in it of itself is an expression of value, or at least an expression of need. Snowflake charges per query, Stripe per transaction, Twilio per message. These companies grew fast because the gap between sale and confirmed value was very incremental, but there's still a gap. I like to think of it in this way. Every software business model can be understood through this single lens: the distance between when a customer pays, all right, when they get value, and when they pay for that value all over again. Back in the on-prem world, the gap was stretched so far, and it was largely debatable if they ever got value, that the renewal was effectively a new sale. When we came into SaaS, we compressed that in many cases to an annual contract, and that's where the vendor was finally on the hook for value. It's what created customer success in the first place. Consumption-based businesses shrunk it even further. Per unit recommitment, the value gap shrinks. And here's the thing: outcome-based pricing eliminates the gap entirely. Value and the work become the same thing. A resolved ticket isn't evidence of value, it is the value. All right, so back in Force One, here's what we said. We said when you have both conditions, high context and verifiable correction— correctness, AI can succeed. So what does this look like in a business context? Success becomes the unit of work produced, which means that both of these forces, they come together right here. High context plus verifiable correctness equals a unit of work, and that is what we are getting paid on. The recipe for AI working is the recipe for how we get paid. All right. So we've seen this chart before. In the SaaS era, it was pretty much unheard of for a company to reach $100 million in ARR in less than a year. Certainly there was outliers, but as a general rule, this was a multi-year journey. In applied AI, the numbers are steep. Cursor did it in 12, Bolt in 14, We heard from Monica this morning at Lovable. They did it in 8. Replit's AI agent ramp went from $10 million in ARR to $100 million in ARR in just 6 months. This isn't a few outliers. This is the new cohort. But here's the thing. Every one of those companies with this incredible growth is a developer tool. Code, we know, inherits the conditions, right? They have high context and verifiable correction baked right in. If any space was going to grow this fast, it was code. But most domains don't get that for free. That's the new job of customer success. We need to engineer the conditions. The conditions create the unit of work and the unit of revenue follows that. For years, Customer success has wanted to be a revenue function, and this is our path right here. And I want to pause right here before we talk about what to do, and I want to show you just the size of this prize. For every dollar a customer spends on software, 6 are spent on services to actually do the work. Sequoia, of course they did, has already named this. Services, the new software. Code is an example, and it highlights the latent demand for the work we always wanted from developers. Applied AI is not going after the software market. Applied AI is going after the work, which means customer success moves. We move from protecting an annuity, the renewal, to operating a tollbooth. SaaS sold the tool. Outcome-based software sells the work. And if the case example of these hyperscaling companies can tell us anything, it's that this— when you tap into the latent demand for work to be done, you can hit incredible growth. None of us could have ever done enough coding, right? We didn't have access to all the engineers we needed to solve our product backlog. The demand, it was already there. It was always there. The supply was the problem. Now with AI, the supply is unlocked and effectively in a lot of cases it's uncapped. And so if customer success is operating a toll booth, what does the the customer success motion actually look like? Well, the truth is not a lot of companies have truly moved to outcome-based pricing just yet, but the ones who have are doing one of two things. Either they're selling on open volume, the customer pays for outcomes they get with no ceiling, and in that world, your job is to discover new use cases, expand into adjacent workflows, and and grow the overall surface area of the work. Or they're selling a preset volume of work. The customer commits to a number upfront, and now your job is to bring the work up to that target amount. You do this by driving utilization, hitting consumption targets, and unblocking those barriers to hit your contracted volume. In both camps though, The job is no longer to tell the story of value. The job is to engineer more value into existence. Even if your company is not on outcome-based pricing, and certainly it's going to take a while for everybody to get here, the job at its core is the same. Modern customer success is to engineer the conditions for work to get done done by AI. When work gets done by AI, the revenue follows that. All right, we're on the home stretch now. So finally, what to do. Here's what I would suggest from a hiring profile, a metrics perspective, and just an overall mentality to achieve success in this applied AI customer success world. Number 1, you're going to want to start by auditing your organization. A time and motion study is the new starting point. I think it's always been the starting point for customer success, but it definitely counts now. First, you're going to want to figure out what your unit of work actually is. Understand that discrete thing, agnostic of your pricing and packaging, that a customer is paying you to do. Second, you're going to want to map the input inputs on both sides. What does context look like? What does the system need to do the work? And what does verifiable correctness actually look like? And then third, once you have those two, two components, you're going on a pressure test, all of it, right? You're going to want to see where it breaks and where the met— the model is best suited to take responsibility for, for certain components of it? And what are the unique delivery constraints that only a person, maybe an FDE, can solve for? All right, so assuming you've got that in play now, all right, you've done the audit, the next question is who actually does this work? First, you're gonna want to think of hiring in terms of both of those halves of the equation. On the context side, what What we find is that consulting backgrounds can be really good. People who can synthesize fast, ask the right questions, get to the inputs in the models that these models need to get a good outcome out of them. So consultants, they do really well here. On the correctness side, I think that's where a lot of people maybe get a little bit more confused, right? I think people get tripped up on, hey, we need to be more technical or we need somebody from tech, who can write code. And in the context of a developer tool, knowing what's right probably means being able to write code, being an engineer. But in a lot of business, verifying what's right actually probably comes from the domain itself. We heard from Tom Ronan at Harvey, and they're actually hiring former lawyers to check the work. All right, so The job becomes somebody who can verify that it's right. That's what builds trust in an applied AI company. And so finally, you've got your team in place. How do you know that this is actually working? So if I were going to measure my team and I don't have a team now, I just get to write a blog. There are probably 4 metrics that I would pay close attention to. Number 1, I would want to know what my eval pass rate is. What percentage of these AI outputs actually meet the customer's bar of quality? The famous saying is, what gets measured gets done. Well, in this case, if you can measure it, consider it done. And this is kind of our new barometer too. If this number is drifting down, in other words, eval passes are, are failing, failing, we're getting failed rate, that's probably your new signal for churn. If it's climbing, you probably have trust with your customers and you're probably gonna get a renewal. Number 2, time to first value. All right, so supposing you know what these units of work are and they're passing the eval bar. Now what you wanna understand is how long it takes you to get to that first unit of work. In the SaaS era, this was a number that we often hypothesized about, right? Time to value. Now, because the unit of value is actually so discrete, time to value actually becomes a real metric. And what you're going to want to do is you're going to track how long it takes your team to engineer that first unit of value from closed deal to outcome itself. Number 3, you're gonna wanna track velocity, all right? Time to value gives the building block, but because you're going for latent demand, what you really wanna understand is the velocity. Usage is a great starting point. It's a leading proxy. Are customers actually using the platform? But a better place to get to is actually units of work completed. Completed. Think of it, units of work completed per day on a cohort basis. These are your truer signals. And I think the last piece to this, the temptation is to measure volume. But if you're going after latent demand, what you really want to understand is what is your velocity? Are you doing more of this work done right? Velocity tells you where things are going. So that's number 3. And last, you're gonna wanna reset your NRR number. If you do all of this and you're in one of these companies, there's a world where you need to reset all of your numbers, but NRR is probably the first. You see, the ceiling in an applied AI customer's business is not the number of seats you can attach to them. It's not your entire product, product catalog that you can attach to them. You can sell to them. NRR in one of these business is structurally uncapped. The race is for latent demand, not the renewal. And it has to change our mentality around what good NRR actually looks like. All right, in closing, what I want to say is that all of this is a work in progress. We haven't reached final form yet. And it's going to take the entire customer success community to get there. But here's one thing that I am sure of. Customer success has, in fact, structurally changed. The work is different now. The metrics are different. And the hire is different. Right now, what I'm finding is I'm learning most about customer success from startups. They're the ones who are born into this business problem. And they don't have a choice but to solve it. The rest of us do have a choice. We can keep running the playbook from the last decade, or we can change our thinking and stay in front of where the work is going. If you want to do this with me, I write a great Substack. Take a picture of that. This is where I'm sharing everything that I learned from the startups I work with. Put it all out there. Write nasty comments in the bottom, pressure test my thinking. Uh, we're not gonna get there alone. All of us need to shape what this new motion looks like, and it's gonna take all of us to learn what it actually looks like. So in closing, just thank you so much. This has been an awesome saster. Really appreciate you all coming out. And we're actually gonna do Q&A. So if we have some of our other speakers come up. We can ask all of the questions from the day. So I need Tom, I need Monica. I think I see Monica. Monica, you gonna come up? I need Ryan. I think Monica, come on up. We have questions.

John, I was saying we need chairs.

Oh, we need chairs. We can stand.

We have big strong legs.

Q&A. I know I don't have heels on.

I'm sorry. These are not the Q&A heels.

Yeah. All right. So earlier today we had Ryan Seams from AssemblyAI speak. Speaking. Monica Perez from Lovable. You want to come up? Yeah. Oh, we got chairs coming too. Yeah, that was because of me. Sorry. Thank you. So have you been paying attention all day? Now is the time for all your questions. Here, Monica, grab a seat. Grab a seat. Good work.

Yeah, sure. Bring them up. Sure.

Here, guys, come on in. Come on in.

All right. I'll stand. Oh yeah, we can stand. Yeah, us tall guys will be over here.

Yeah, we'll even it out.

All right, fire away. Here we go.

Feel better? OK. Here you go.

All right. Thank you everyone for the great panel, the great speeches and presentation. So I want to ask everyone about how you do content context and correctness within the context of your organization?

Got more chairs coming.

Hmm.

Can I share like maybe a tactical example? So I was working with a company, doing some work with a company about a year ago, and we wanted to implement a an AI customer service tool. We implemented Maven, if you guys know Maven AGI. The problem is, is we had no knowledge base. We actually had no knowledge base to train the AI.

So, um, that's a good chair. Yeah, yeah, take it.

We also really did not have a team to create or curate a knowledge base right now. So what we did is I actually utilized Glean, if you know Glean. We I connected Glean to all conversational systems from our email, our Zendesk, like everything. And I actually had Glean like take every single conversation we've ever had with a customer and essentially create a knowledge base based on conversations that had really good sentiment on the responses from the customer. And we were able to use that to create this knowledge base. Now, when we did it, We went through it and we were like, I think it's probably still only 60 to 70% correct. But what would happen is that, that probably took like a 6-month job of creating a knowledge base, and we did it in about 3 or 4 days. And then we actually had this big knowledge base that we could then pass around to the CSMs, the engineers, and whatnot to validate for correctness. And we actually were able to do this whole project in a matter of a few weeks. But I think, one, when you're trying to validate correctness in the code, you have to go to the engineering team. When you're looking at validating correctness on, um, business use cases, I, I think CSM's implementation, those are probably your best point because they're the ones interacting with the use cases. But you can jumpstart that with AI, but you, you do need human validation of people that know the business and know the customer's like domain that have that domain expertise.

Yeah, maybe I would just add, I, I, I talked earlier about how we built our customer support agent, right? And we moved like the end-to-end resolution rate from 10% to 75%. So every day we look at all of the issues that the agent went through, right? We're scoring them with Claude and actually saying like, does this look right? Does this look wrong? Was it like properly solved? And especially for the ones that get escalated, right? We're looking at, okay, great. Is this like something in our docs that's broken? Is there maybe like a bug potentially in the actual code base we need to go fix? Is this actually, I don't know, some question we intentionally told the agent not to answer? For example, something we saw that was pretty crazy was it started giving legal advice to customers. It was reading the terms of service and then be like, well, you know, in this country, this is how the terms of service would apply. And we were like, yeah, no, like we don't want it to do that. And so like, if you review all of those every day and you create some sort of scoring mechanism where you're using AI to review those responses and having a human in the loop to go and make the changes that you need to make, you're just continuously updating and making that context better and also checking if they are correct and not just blindly trusting that like the agent replying and the customer being happy or not responding is like the right outcome for you.

The only other thing I'll add to that is because we're doing most of this with AI as well now, but when we rebuilt our knowledge base, we used Stoneleaf for knowledge base. We had a contest across our support team, we called it Feed the Beast. Every support interaction was an opportunity to feed feed the knowledge base when it didn't get something right. So using it based off of our current technology, when something was not deflected, came to a support rep, how they answer the question and how they had to submit sort of a process and a ticket to update the knowledge base. That was supplementing. At this point, AI is doing most of it for us already. And to your point, Bobby, it can digest all the same things now. But the human in the loop point is valid and important. Important because, to your point, there are some guardrails you don't want it to go around or over. Finding those and making sure that those are clear in context setting, only the business understands that stuff.

Another question. Thank you. Hey everyone, this is Victoria. I got to say thank you so much for being here. I definitely definitely did learn a lot. And I— this question is for Monica. It was really impressive to hear, like, you built the CS portal yourself with Loveable. So I was wondering, other than not wanting to depend on a vendor, what are other motivations behind that? My second question is, who in your CS org is responsible for the iteration? And I believe probably like every single CSM can like touch the portal like every single time they want to change something. Do you still have like operation manager to do that? Yes, great question. I think it's the golden question in AI vibe coding generally. A couple of things. Um, I very much agree that there is a spectrum of things that can be vibe coded, and there's, we all know, AI slop all the way to like full load-bearing applications. Um, and it does take work to get there, and you have to be really intentional about it. And so we could take a lot of inspiration from our product management peers where we are bringing product management principles into our teams as we build our own products and ship our own products. So we built our customer success tool. I feel like a PM because I have the requirements spec'd out. I have a list of feature requests from my CSMs, we have versioning, we have feedback cycles, and we continue to develop it and maintain it. So it does require that thinking. And then we also make it real, we make it live with data. So that does require our RevOps team to be involved, for example. So it's in heavy partnership with RevOps, our ops our data teams to make sure that it's being done safely, to make sure that it's being done accurately. But the why behind it is because we know the business and we know the customer the best. Like, we are closest to the problem, so we should be the one to build the solution. So I do think there's an element of not depending on external vendors, but I also think there's an element of we know exactly how to do it how it should work. We can dream up, like, imagine if it could do all these follow-ups for us and all of these things. Like, we can dream the wishlist that we want and move at the speed that we need to move to, like, hit all of these solutions ourselves. So I think, like, the brain behind the product really comes from the domain experts. So, like, we're building our own CS tool because we're the CS team. But you really have to put it through a product development lifecycle internally. And for that, we do that in conjunction with ops teams, even our security teams, because we have logins, we have customer information. And so, yeah, we're all mini PMs now. Thank you.

Amazing panel, guys and girls. I'm Aaron. I have a regtech company in a hyper-vertical in energy and commodity trading. I'm dealing with stakeholders that are dinosaurs. And so I am ex-consultant as well. I'm fully subscribed to the FDE strategy of changing operating model. I'm going to repeat what I think I understand as the operating model. And then my question is, what's the one or two secrets to get the dinosaurs transition to AI? So My understanding, you have an FTE, you put it in, you do workshops to get everybody understanding the operating model change and what the tech can do, you get them inspired on idea— I used to do innovation funnels at companies, so you get the innovation idea generation happening. Once you land that, you do value effort, you decide which use cases, then you deploy and you build on their behalf. I think that's that's what the playbook is. I just want to confirm that's what everyone is saying on the stage today. And then secondly is what's the secret to get these people to spend time? I mean, what stakeholders are you talking to? The CEO, the CIO? I'm doing compliance to the CCO as well. Who else? I'm just— it's a good playbook I want to take back to my branch.

Okay, so in terms— I'll answer your last question. It was a loaded question. In terms of— there are some things around changing mindsets in people, and I'll go back because I go back to my Meta. We used to sell reality, virtual reality technology, which is very out there, to pharmaceutical companies, for example. Talk about how do you bring something that's really conceptual into a use case that's supposed to be leveraged to be, to be in business. So the first thing is experiential, is get to somebody that has that mindset that you're trying to change and take them through experiencing the change. The Fox. Right, exactly. Exactly. So once you get one, it's kind of the idea of the first follower. You have to create this momentum for change, is you have to get a first follower to accompany this one person that has influence. Once you have that combination, then you can then use your business cases. But if you are trying to convince people, you have to start with one. One person and have them experience and do that. So that would be how we applied it. And then you build your models for scale, right? But that would be an example I would follow.

Maybe I'd add, so your hope is not that your FDE has to go in every account and launch 10 features. That's not a good thing. If that's where you are, that probably means that your just general product is lacking. and FDEs can help build those features and help accelerate product velocity. But the thing that you should notice, right, is if you're targeting the right customers, those asks should be similar. So it's not like if you look at 20 customers and they all have 10 requests, you have 200 new requests. You actually probably have 20, 25, right? And so you could really focus the team on, okay, great. If all 20 customers want one thing, we should probably go do that first because that's the, the highest leverage thing we can go do. To go and do. And so the FDE as a rule, it's not just like each customer's everything is custom. You're trying to use them to also build like your product, right? Your product velocity as well. And then I would just note maybe on like the other part around like, yeah, find the person who's like really excited about change. If you can also figure out, you know, at the C-suite and the leadership level who has maybe that mandate for change and who's talking about using AI and interested in changing the company and wants to have that as their initiative, That's kind of the person who's going to be your ally and your sponsor, right? And so really helping work with them to be like, hey, so I am talking to your team, but looks like only 2 people are actually using this thing. What's the disconnect between what you're saying and the team on the ground that's using it? And kind of leveraging those relationships. Yeah, exactly.

And sorry, I'm just for the love of bull. I love the idea that you customize the experience for each customer. By having a Slack channel. My people don't even use Slack.

So we're on Microsoft Teams too. On Teams too.

Okay. So, but do you find that they're actually doing that and they're using it even for the, the generation that doesn't know how to use technology, that are used to spreadsheets and notebooks, which is my customer base?

They will do it if the first, the first impression is very good. And I think you lose people on the first impression. So it is a delicate balance between, hey, let's just ship this like beta version and see who likes it and let's just test it on our customers. We should not be testing a lot of really important things on our customers. But if the first impression is so strong, at the end of the day, going back to the outcome, like they just want the right answer or they want the fastest answer, right? So just like make that really, really good. And I think you'd be surprised how much they would go back to that.

And does that sit in your team, the people that are responding, or is that another team? Is that ops?

It sits on my team, yeah.

Okay, so you have that resource.

Yeah, and this is because this is for our enterprise big B2B clients, so it's much more specialized offering, but Lovable also has a B2C side of the business, and on the consumer side, there's a lot of in-product work that's a little bit different. Blow my mind, team.

Thank you so much.

So I actually have one more quick comment that came to mind. Okay. Um, when working with these legacy folks, you— another important thing is understand what's in it for them. So I, I did some work with a company called Pearl AI. We sold computer vision AI software to dentists. Um, essentially we'd read x-rays, and yes, the AI can read x-rays better than the human. Uh, Basically, the software was bought by the PE firm that owned a conglomerate of dentists, but we had such pushback from the dentists themselves because they thought that we were challenging their, their, their education, saying we're better than you. And we had a— it would not get adopted when we were adopting through we can read x-rays better than you. But what the other outcome was, was that we now could diagnose a lot more cavities and we could increase the amount of revenue per patient. And so they cared about that. They cared that we can increase revenue per patient. So understand what's in it for them. Don't talk about what's not in it for them and drive the behaviors that are gonna be adopted.

Does that make sense? Yep. Consultative-led. Yep. Yeah. Appreciate everybody. Yep.

Thank you. I'm actually really grateful I came after this one because my audience is pharma, so also quite conservative. I wouldn't say the word dinosaur, but it is definitely more conservative and held back. A very standard old playbook of customer success, this pre-AI. Everyone talked about onboarding, and the process of onboarding self-led, and then only the face-to-face sort of personalized relationship. I noticed in all conversations today, onboarding was almost nearly not mentioned, like not, not even once, almost like as a byline text. Is that dead in the age of AI, or is that like something in FDE or customized? Is there a process to get people who are conservative through the process.

Yes, I'm very passionate about that because this is like the topic I talk about every day now to fix. When I talked about time to value, it was all about onboarding. It was all about how do you introduce this new process technology into your customer in a way that's relevant but in a way that is going to create momentum, right? And so it's launching the use case is tied to the biggest opportunity for value. And so I do think there is an opportunity to disrupt ourselves in how we approach onboarding. I think we can onboard people during trials. I think, why do we have to wait? 'til after we sell it. Like, can we accelerate, like start selling them, start the onboarding process on your trial? That would be like my first thing that I would do different.

I was gonna say something very similar. We all were. I don't, the short answer is like, is it gonna, does it go away? Probably not, but it depends. 'Cause there are places at Carta where it will definitely go away. Away, and I want it to go away. And I think we have gotten broken on that concept for a while. I might have mentioned during our panel, we had an onboarding churn problem for sure, but the problem wasn't the products, it wasn't the value of the products, it was engagement, and it was lack of understanding of customers, understanding of what they were expecting to happen from the product. So what I would tell you is like, there are places where onboarding has created their own processes that are not for the customer. They're checkpoints for yourself. If you cannot surface those to a customer, make them get value faster without them going through your prescribed motion, then kill it. Like, it shouldn't be there. However, you do need to do something with that time. And I'm sort of challenging my implementations teams to think about, being more of an implementation success function. A lot of our business that comes in every month is SMB. We can automate a lot of that now and make no friction, no my calendar's the blocker for you to get on a call with me and walk through documents. You send me, you give me access to Drive, I will go extract all of your documents, build a cap table, and send you live, but I'm not abandoning you. I'm gonna spend time with you contextually explaining what you need to know from what you bought. So I think the short answer is like onboarding as a setup guide, step by step by step, we did that for ourselves, not for our customers. Those were our own like software checkpoints. They're not value. If you can kill that and then use the time for something else, then onboarding is the most important part of the lifecycle. I think of what Bobby said earlier, this sort of like customer does first X or second Y, by whatever that is that you know means they've adopted. That implementation or onboarding should be that, not the actual procedural steps that are all backend that don't mean anything to a customer.

So basically rewriting the steps of onboarding. Yep. It's all bloggable.

Yeah. I would maybe just add, at least with the usage-based pricing, right, being kind of like the way that you're driving actual spend, at least for us, for us, right? Onboarding is like actually winning the evaluation, because in order to win the evaluation, we have to get them to implement our API, test it in production, benchmark that it works right, confirm all the configurations correct. And so previously, you'd have this whole sales process, you sign the contract, it's like, cool, let's go kick off onboarding. It's like totally reversed. You have to like do the onboarding to win the evaluation and grow the usage from there. And so it's less of this step-by-step thing and it's more like the outcome for us is how do we win the evaluation? That will include how do we actually onboard them and get them to effectively use our models.

Yeah. And there was a great differential of customer success now building the system of context and the system of correctness. Right. And so every single client will have their own history and own knowledge base. Like, I could sit there and wait for them to connect up, and that will be— sorry, I have to contest the time-to-live metric, but like, it could sit there gathering dust. Or we get them to get their ass on the line and upload those documents, test that, go through the metric of evaluation. It's like, how— where does The FDE model sounds like a solutionist that is almost doing everything, but like, yeah, I just want—

Well, I think I'm still hooked up. I'm still hooked. Oh yeah, you are. That's wonderful. In a lot of cases, like what I'm seeing is that these forward deployed engineers are actually getting pulled into the sales cycle, like what you said, Ryan. And I think onboarding like really rides rises in importance. And the reason for this is it's like you could give somebody a demo account of a SaaS piece of software and they could kind of click around and they could get the gist of it. Now to get like a real outcome out of, you know, an applied AI company, you need context or you need to know that it's actually right. So there's a lot of onboarding that goes to just making the software work. In the first place. So you can't even do a good demo, you can't even do a good trial if you don't do a certain amount of onboarding to either get that context or understand what right looks like in that business. So I don't think it goes away. I think we can automate the long tail, but I think in a lot of enterprise situations, it probably pulls more forward and rises in importance importance.

Okay, all right. So redefine the onboarding steps not to a checklist of old steps that was for ourselves, but for the first value that we can derive and to pull it forward as much as possible in the sales process. That's great. Thank you. Thank you.

Thank you for lovely panel, lot to learn. My name Bina Kimani. And looks like, as I'm hearing, there is a lot of training happening on the customer's data. And Monica talked about you don't depend on third-party vendors for creation, but do you depend on them for the data, for testing, for evaluation? And, or do you always do it with customer's data? And each customer is also unique, and how do you deal with failures and errors and, you know, thank you, dealing with all the things that happen for that particular customer.

And repeating that in like a quick, quick version because I think we got like 50% of it, so I want to make sure we got all of it. Okay, sure.

So on the technical side, I— my understanding is that you have access to customers' data and And the model is getting smarter or the algorithm is getting smarter based on customers' data. But do you also buy third-party data for evaluation, training, all that? And do you use that for hardening? How do you deal with failures in customers' environments when there are failures and how do you capture it? How do you go about it?

Yeah, I can give like the software development like vibe coding perspective. Perspective, which is, um, we— I think that model strategy is going to be a whole thing at companies, and companies are trying to figure out who do I go with for what. And so the way that we de-risk that at Lovable is that we use all the models, and for any particular task, we're using the best model for that particular task in that particular moment in time. So It's always picking the best underlying model for anything that our users are doing in the product. If one of the models goes down or like this year, I think we all agree on who the best model is, but we don't know who the best model will be in a year from now. We don't put that burden of decision on our customers. We do all of that under the hood. When they're buying Lovable, they have confidence that they don't have to figure that part out. I know I didn't directly answer answer your question on the data piece, and I think maybe it's a function of our different products, but on the model piece, we, we try to have a very diversified strategy.

I mean, we actually build models, so like, yes, of course we have to have data that we use to evaluate our models, provide benchmarks to customers. But the reality is any model company, whatever release they make, is going to tell you it's the best. That's just full stop. Like, doesn't matter which model it is. And so the reality of that means that you do need to— and this is like what our FDEs are doing during onboarding, which is like, show me your real audio files, let's run them through our API, what does the performance look like, does that match our benchmarks? Well, if it does, then like, great, everything's looking pretty good, right? If it doesn't, like, that's where the disconnect is and where our team's spending a lot of time, is like, how do we configure the setup, the API, the implementation, etc., so that it reflects those benchmarks at the end of the day because we know from those benchmarks our model is strong and good, but the implementation process could lead you to all sorts of crazy things that don't result in that same thing showing on real customer data, and so we spend a lot of time there actually. Evals, I learned that word this year. We do a lot of work.

I've used the word evals more this year than my whole life combined.

Thank you. So Monica, what is the model that we all agree upon. I'm just looking for a fight here. So let's see. Rhymes with—

no, I'm not going to say that. Hey, every model leapfrogs each other. Like, yeah, we're all in it. Don't worry. It's friendly competition. Hi, everybody. Okay. Customer health scores.

So we still have lots of SaaS companies that are like building an algorithm in Gainsight or whatever of 30% this, 40% this, et cetera, right?

And then we have a lot of vendors on the market doing that with AI.

You don't have to choose what are the criteria, the AI's gonna surface it for you, right?

But in this world, we're talking about where revenue equals value.

Do we need health scores? Do we need early indicators?

Or is it all, I mean, is it really just gonna be what percent of code was accepted or was the outcome successful? I care lots about this because I'm a digital CS person, so I've always been triggering off of health scores and like the old health score is dead. So where are we going?

I think it depends on what do you want your health score to tell you. Yeah, your health score can mean a lot. Is it an adoption health score, churn risk health score, renewal probability health score? Yeah, whatever it defines. Health scores, I think, will always be relevant, but you have to define what do you want your health score to tell you, and then are all the inputs going into it have direct correlation with whatever that output is?

Yeah, I want it to tell me likelihood to renew and expand, so like maybe that's not needed anymore. So I think leading indicators is what you need to consider, right? Leading indicators are around risk, leading indicators are around health. I think more importantly is understanding what does a successful mature, what's a sophistication path to success. That's more important than starting with a health score. Is what does success look like in your product and how do you build a path towards that? And then building the, the warmup warning systems in whichever format you want to build to help you predict likelihood to derail customers on the path to that success. But what happens is, and this is what I'm finding out operationally today, is that we don't have a clear path to a sophisticated customer in different products, but we started with the health score. Yeah. And so I think start with that model of sophistication and maturity of what the most successful customer is and then build the backwards path towards getting customers to that level faster.

I have a perspective also as a digital scale CS leader as well. On the predictability part, we've— my initial answer is don't care about health scores at all. They've never validated much of anything for me. Me. At scale, they have not. However, I have treated health scores as a forecasting methodology with our— as a joint partnership with our finance team. And so I can look at, you know, big swaths of customers, tens of thousands of customers, and leading indicators against churn and look at it as a forecasting. And then when I buy in from the finance team and the data team, and now with Claude, I can get what I want in terms of predicting large chunks of revenue moving, because that's the thing that matters to me. So I've replaced health score with predictable revenue and renewal forecasting as a proxy for the same thing. Not true for enterprise. Enterprise, we still have to measure one-to-one. The other thing I would say is, this is my argument for why, other reason why I think health scores are going away. If there's a world, and we believe this at Carda for sure, that's coming soon where a customer will use the product where they want to use the product. Yeah, the product adoption metrics are meaningless to me. The outcome is the only thing that matters. And I, we've always, maybe I don't know if this is true for you, but part of our digital journey has been product adoption is really hard as a metric when somebody logs in 2 or 3 times a year. Yeah. And that's good. That's what you want. You don't want them doing it every day because it's just not the use case for the product. I have other ways of evaluating health that are not product adoption. But if it's a world where you're giving, you know, access to a customer through cloud, through an MCP, which we are doing today, read-only for today, but we're saying all you have to do in plain language is say, run foreign NA evaluation, and it produces a live artifact that looks like CARTA. All you care about is the outcome. Was it right or not? And so the health score in that regard, if I don't know if every platform is going to go that direction, but if ours does and others do in the future, or at least they choose how they want to use the product, you at the very best have to have a mixed understanding of customer health. But generally, the thing that's going to matter in the end is, was it done accurately? Was it done fast? Did it meet your expectations? And that's the measure of health.

And we are at time. Oh, cliffhanger. We're going to hit overtime, and we can't go there. So everybody, thank you so much. Appreciate your questions. Speakers, great work. We should get a picture though.