No Lead Left Behind, Our AI Teammates in Action, and the Clay Price Slip | The Agents Ep. 1

Okay. Hey everybody. Welcome to the first episode of The Agents with Emilia LaRouche, Chief AI Officer of SaaStr, and me, Chief AI Evangelist at SaaStr. And this is a bit of an experiment that we hope work, but those of you who've been following us for the last 10 months on our agent journey have consumed a lot of our content. It's exploded. Our page views are up 5x, our Attendance to SaaStr AI Annual this year will be up 40%. Folks are very interested in our agentic journey and we're constantly asked to do consulting calls and meetings. Literally in the last week I've done it with 2 CEOs of public companies. They want to know about our agents, how they work. We are not savants. We are not better than you. It's just we're like 6 to 9 to 12 months ahead of a lot of folks. That's it. And so many folks ask me, ask us, probably 50 people a week, if we can help them think about their agents. We can't. We can't do that, but what we thought we would do is have a little fun and maybe once a week share everything that's happening with our agents on the Agents, every, all the issues and bumps we have. And if you're on this journey, whether it's just rolling out your first AI SDR, whether you want to build your own AI VP of marketing or customer success like we have, whether you want to think about adding AI for support or whatever, maybe if you listen to this every week or when you have time, we can help you avoid some issues, think through things you wouldn't otherwise think, uh, and accelerate your success in the Agentic journey. So that's the goal. If we have a tagline for this, it's, it's too long, but it's going to be accelerating your success on the Agentic journey. And as part of that, we are going to talk at least on the first one about some of our bumps, because if you hear about our bumps, I think, I think you can avoid them. And so we'll do like 6 to 10 things that we've seen this week. Some of them may be very mundane. Some of them may be topical, um, but hopefully it's helpful. Um, The first one I wanted to hit, talk about, and then get a lot of Amelia's thoughts on, and, um, which is, okay, now you can build it, but who maintains it? Now you can build it, who maintains it? So if you've been following our agent journey since this last summer on the leading vibe code platforms, and we love them, we love Replit, we love Lovable, we love Vercel v0, they're all exciting. You know, at first when I started, it was tough to get an application out the door that worked in so-called into production, and I struggled at first, even, even having co-founded a SaaS company that went from $0 to $200 million in revenue, I still struggled. As 2025 went on, the platforms got better, the models got better. It got easier and easier. And then toward the end of last year, it got great. And if you're not a developer, but you're on social media or you talk to your own teams, you'll see everything kind of exploded when Claude Sonnet and Opus 4.5 came out in December or so of last year. Productivity exploded everywhere. Every— it was just magical. You can see it because Anthropic went from $9 billion to $30 billion in revenue already this year. It exploded the quality. But even if you're not a developer, if you're using Replit, Lovable, Vercel, VCR, and others, you're benefiting from those models because they got better. And so what happened now is it's not that you can magically in one prompt build your own CRM in 5 seconds. Like that was always marketing over reality. But now you really can finish your apps. And I kind of handed the baton to Amelia at the end of last year. I built, rebuilt Saastr AI. I started our new Saastr annual website. I built our AI valuation calculator for startups that have raised $1 million. I did all that and kind of struggled and went through the bumps. Amelia took over probably in January. Amelia, what? Yeah. Yeah. And built our AI VP of marketing on her own, which is very, very good. And then our AI VP of customer success, QB, which is even better. So we're going to talk about 10K and QB like every week, all the bumps we have because they're part of our team. They check in with our Slack every day. They are part of our team. And so you'll hear us. But my, my slightly long setup is now if you're persistent, you can build a lot of B2B and AI apps in these Vibe-coded apps without an engineer. You really can. You really can get them out for real. And the meta question is who maintains them? and this is what is underdiscussed. And I want to talk about 3 things, uh, that have happened this week for us. And it's just something you're going to think about that a lot of folks, vendors, one, don't really discuss it. People don't really talk about it. Getting an app into production, building your own app, vibing your own app is this, you know, just like sales, when you close a sale, it's the start of a customer journey. Just like when you push an app out that finally works, that's finally pretty good. Every week you have to maintain it and who on your team is going to do it? Who on your team is going to do it? And I want to— and I want Amelia to talk about a few of the things we had. But I think this is the meta answer to the meme on social media of, oh, we're going to kill Salesforce because I can vibe it. Okay. Even if you could, like, which you can't, you can't do a Salesforce or a Workday today. But even if you could, who the hell is going to maintain it? You? You? But you can buy a lot of these apps for like $5 to $100 a month. You're so— Amelia and I are worth a lot more than $5 to $100 an hour, so This is, so we're gonna share our bumps maintaining it just so you can think about how you will, if you choose to build it versus buy it, how are you gonna maintain it? So first one, just to talk about, and this happens, but just today on several of our apps, the preview instance had database issues. They didn't connect, they didn't connect. It's going to get fixed. Bugs happen, issues happen. It happens with lots of vendors. But just from an end user perspective, Amelia, just, just describe in general kind of the issue and what ran through your mind.

I mean, I, you know, I literally was exasperated at the agent. I think it's tough, you know, when, when one thing goes down, your site goes down, it's frustrating. And of course, if you haven't built it yourself, you automatically go, to the person, right? Like, oh, my Salesforce isn't working. I'm going to go to Salesforce. Is my website down? If did I— if maybe you have Squarespace or WordPress, like you're going to go to those folks first when one of those things goes down. But then when you vibe something or if you have an agent that is slightly custom that you've done kind of yourself or you have your own hookups yourself, it's like, oh shoot, who do I— I was like, well, who do I go to? Like, it's like I tried to problem solve it first because I was like, well, I guess I can go to the kind of root agent and see if I can see if it's something I can just fix in the platform. And interestingly, the agent couldn't resolve it.

In this case, sometimes when you're going to have bumps, you're going to have to maintain, right? And to be clear, this all happened on the preview environment. And for folks that have built software, you know, I'm oversimplifying, but there's basically preview environments, there's staging, we'll put it aside, and development and production environments. Production, we had no issues on the production environments. But preview is where you tweak your app, you iterate your app until you push it into production. And that we weren't able to access. It was unavailable for hours. Right. And yeah, not only was it down, but if that happened at Shopify or Salesforce, right, or even Squarespace, it's on them.

Right.

The difference here is that there's millions and millions of unique apps. So whose fault is it? Whose is— who's the database administrator?

I mean, in this case, I was like, it's all of us, right? Like, I literally told you, I slapped you right away. I was like, oh, I think I want to— I think some of the agents are down. Like, is it— I was like, for At first I thought it was just me. I was like, oh, it must just be me. Maybe you can still access it. But then when we learned it was you too, it was affecting you too, it wasn't just me, then yeah, I, you know, it's this, it's this, it's the Spider-Man meme where you all point at each other and you're like, well, whose fault is it, right? Like, is it the customer's fault? Like, is it something that they do? I kind of assumed it was our fault at first. I was like, okay, well, we, we built this. I'll just assume it's our fault and I should try and fix it. Then I asked the agent what to do. It started doing some lines of code, but then I realized in that process that some of my other agents were down, right, in the same way. So I was like, okay, it's actually not just this one instance. So that was super interesting too. So then I went to our vendor that we use for this, but again, it's one of those instances where it brings up, yeah, like who maintains this? And also if you're not as online as we are, right, we're trying to keep up with our agents 24/7, like I notice this pretty much right away. I feel like other folks, if you don't have a database manager or somebody who's in charge of all the agents, you may not notice. It may lapse. Like you may not notice for a couple of days, especially if the front end is live, that some of your agents on the back end have broken. And so I think it's an important nuance now to start to think about who owns all this in perpetuity, not just again, to your point earlier, the hard part does seem like getting it, you know, fully baked and spec'd out the way we like it to do. We like to always use our agents for processes that already works like that is a lot of run-up and lead-up of work to get it already to that point. But then you have this whole other beast once you get your agents actually into production where you're like, oh shoot, what do I do now?

Yeah. So Amelia has now surpassed me in terms of her ability to vibe code apps.

Talk about that.

No, mostly she has. But in terms of this particular issue, it's interesting. I still have some domain expertise. Again, having at least co-founded an app that I still understand some issues out there better than she does. Without me, her initial conclusion of the issue was wrong. As good as she is, her initial assessment of what was wrong. Then we asked the AI agent, and it's pretty fun. When I started doing this, when I started vibe coding, you'd ask the agent, it would hallucinate the answer. It would make a, oh, the answer is, yeah, your, the answer is your, uh, your DNS. It would just make it up. Now the agents, and we'll, we'll talk about this in a different context with a little bump we had with Clay next, but the agents are pretty self-aware. They can analyze all the issues and it's caught you. They can help you debug. Like 98, 99%. So when you go into an app like Lovable or Replit or Vue Zero or whatever, and there'll be that little prompt, that little agent, not only can you come up with ideas on how to build the site and features, it can also help you debug issues, but it's not perfect and it doesn't have access to something. And so it tries, but all agents are goal-seeking. They wanna make you happy. So at the end, they blame somebody else. They'll try to fix it. They always try to fix it now. And then they're going to blame Claude or Lovable.

They definitely blamed some apps I had hooked up to it, right? It was like, oh, it must be—

it must be Qualified. Salesforce, they blame Qualified, which was not the issue. Immediately they blamed Qualified as why it couldn't come up. Right. And Amelia's like, well, we've been— we didn't change the code. We've been using Qualified for 4 months.

I was like, nothing's changed. Yeah.

Yeah.

So I was like, are you sure, agent? Because I was like, I haven't even like, you know, when I make updates to Qualified, I make them in Qualified. It doesn't really change how it integrates with this particular agent. And I knew they had just integrated with Salesforce this week. So I was like, maybe there's an outside shot they changed something on the backend. Like, that was the only thing I could think of. But then I was like, okay, let's, let's for a second try and turn Qualified off and see if that fixes it. But of course that didn't fix it. Then it started to blame other third-party tools. I was like, yeah, it's not Qualified. I think it just blamed that one because it's one of our beefier agents that this one hooks into. So it was like, let me just find the most common denominator and blame that instead of maybe looking for the actual problem.

Yeah. When we talk to folks there, I don't think you have to be technical to create your own amazing vibe-coded app today. You do not need to be technical, but you need someone pretty damn product savvy to maintain it if it's at all complicated. And we talk to a lot of folks and maybe this is the biggest meta takeaway either they don't really plan on anybody maintaining a lot of agents or vibe-coded apps, right? Or they push it to someone who's maybe very good at a line of a functional line. They're a decent marketer, they're a decent sales ops person, but they have no intuitive sense of how software is built or made. I don't think they could fix most of the issues we see. I don't think they would understand how to fix these production issues because they wouldn't be able to describe or identify them to the agent. Right.

I think for somebody like myself, right, like you've built multiple companies into like success and like for me having not had that background, right, like came up through like more of a marketing and sales go-to-market motion. For somebody who's more so in go-to-market, when this agent started failing this morning, I was like, okay, if it's not our third-party tools, which I don't think it is, that broke it, which we determined with the agent it wasn't, if it's not the agent itself, I'm like, maybe it was me. Maybe I did mess this up, right? I was like—

Or it could have been me because I did a, I did a little release on this app without telling you. I could have inadvertently— it could have been a regression, right?

Yeah, it could have been a regression.

I built a Friends of Jason M. Lemkin little page that reused some of Amelia's discounts on my own little page. Now, how could that possibly break the entire site? It shouldn't. But when you get a little deeper on how software works, you understand that regressions, when one thing can break another that don't even seem connected, right? It just is something you got to get zen about. Maybe that was the issue. Maybe I broke, maybe Amelia broke it, maybe I broke it, maybe the agent broke it, maybe the vendor broke it. And you got it. You— someone's got to own this. Okay. Two other ones on this. They sound minor. I would say for the most part related to this, generally speaking, hallucinations in our app are relatively minor compared to the old days. The old days, I would say, of summer of last year, they're pretty minor. When we started using a lot of these VibeCode and Agentic products, they would use more immature models, the 3s. In Claude and it would hallucinate a lot. It would, it would just make everything up. It was very frustrating and it made it very, it might've been one of the toughest reasons to finish anything, whether it was to get an AI SDR to work because it would hallucinate or to vibe code an app because it would hallucinate. But they're still there. And I'll tell you just, um, two stories from this week that were super frustrating. One is on our AI VP of Marketing 10-K. So to describe what it does, Amelia took 5 years of our revenue data for attendees, and sponsors for all of SaaS or annual, $100 million worth of data, every data point, and put— and gave it and created an AI VP of marketing. So it has all these hard data points. It can compare year-over-year performance, week-over-week performance across any metric that you want. It's proactive now. It tells us every day where we're behind and what we should do. It has beautiful graphs and analytics. It's great. We've demoed it and we'll, we'll keep showing more. But I built this fun fact feature where it's to kind of push us. It has a fun fact each day, pushes into Slack. And it pushed and it sends us an email of a fun fact. It keeps getting confused what year it is and what data it's comparing. So yesterday it said we were 44% ahead of plan, and then this morning it said 11%. And I asked the agent, what, what happened? It's like, oh yeah, actually I'm comparing it to the wrong year. And because I didn't have the right year, I made up the data.

Yeah, this happened before with dates, right? Like we had another agent before where it was telling folks still about London and London had already passed because it didn't know what day it was.

So like, this is making up a corner case. And then I congratulated myself on SaaStr a couple of weeks ago is I built— only Amelia and I can use it, but I built not only just 10K work, but I built a little bot, like a little Fin-like intercom type thing where we can talk. I can talk to it, I can ask it questions. And it started to hallucinate too, because it didn't have a full and updated answer to the question. So not only hallucination still an issue, I guess the point of the story on this, and I'll add one more and then we'll move on to the next topic, is this is something we have to fix every day. Our, our AI VP of Marketing is very good, but it's not as good as our VP of Customer Success that Amelia built after for a number of reasons. And it has these micro hallucinations every day. And for a while Amelia built it and I, I didn't really touch it and I think she gave up on it, whatever. Like it's still directionally correct, but I really wanted to go deep. I wanted answers to questions every single day, so I would push it And these micro-hallucinations would creep up every day. And so now I'm kind of maintaining our AI VP of Marketing for maybe 15 minutes a day, but I didn't even do it 2 weeks ago. So Amelia's got her maintenance now. I have these micro-hallucinations 15 minutes a day. It's not massive, but if no one on your team is doing it, you see drift. The agents can become— can drift more and more from reality and from your data inadvertently. You think they shouldn't, but they do. So this was my little issue where I kept comparing year-over-year data to the wrong years. And literally in the same day it said we were 44% over last year and then 11%. I'm like, this morning I wrote, which one is it? Like, it can't be both, right?

I think this is why, you know, some folks are a fan of letting— I think of letting their agents do stuff on their own. Like, okay, like I'm going to, you know, just queue up a bunch of stuff for my agent. I'm going to let it run. I actually don't like doing that. Like, It makes me nervous if I don't see exactly what the agent is doing. Again, because of the goal seeking, I never— all these things we've built, you— I've built that I'm now in, like all of our 20+ apps we have for agents, whether it's in Replit or Salesforce or Slackbot or anywhere else, I don't— or Artisan— I don't let any of them run without me looking at it for its like initial execution. Like now it obviously can send some emails and do some other stuff autonomously. Work with a lot of guardrails on its own. But when it actually comes to me deploying like new features or fixing something in the app with the agents or putting in a new sequence artisan, like, I don't let the agents do that autonomously at all. And I know a lot of people are a fan of the, um, Dispatch you can use on Claude or like OpenClaw. Like, that's why I don't use OpenClaw. This is why I don't really use Dispatch unless I can watch it on my phone closely with Claude, because I want to see exactly what it's doing. Because a few times, to your point, it has started to drift and go off the rails to start to do something I didn't want it to. And if I hadn't been watching the timeline, I, we wouldn't have seen it.

Okay. And let me just give you the third example from this week. This was a learning for me. This is a slightly unexpected one. It's model-based regressions. And what I mean is this is if you've been following us, we have a whole suite of tools to help founders with fundraising. Go to Saastr.ai, click on AIBC. We've done almost a million valuations of startups and we've done almost, we've done like 1,000 intros to VCs and we've done probably our most popular tool though is is you could upload your pitch deck and we analyze it. We tell you how good it is, what, how well you're doing compared to all your peers and competitors, and we tell you what your odds of getting funded are. Okay. It's called our, our pitch deck analyzer. We just crossed 4,000 startup pitch decks. It's very cool. Maybe it would be simpler if I built it today. I built this a while ago, but it's pretty complicated. It requires two different passes through Claude. Okay. And then after each pass, it has to do a lot of error and data correction, data analysis and extraction. It's, it's the most complicated analysis that we do. And it took me like a month of uploading deck after deck after deck and manually looking at it and manually seeing. And then I personally graded and made sure we were in alignment and they got all the numbers right. And it was pretty stable for a long time until maybe January and then again in February and then again like a week ago, I started seeing errors. And what it kept saying is it kept saying all these startups had $100,000 in revenue growing 500% again and again and again. I would get notified startups wanted to meet me. Once again, you know, whatever AI, we're doing $100,000 growing 500%. Like, this can't be right. And what I figured out is when there was an update to Claude, and it probably wasn't just like 4.7 or 4.5, it was probably a dot release or a silent release. All of a sudden hallucinations creeped in. And the models decided that when it didn't have data, it would just decide every startup was at $100,000 or $1 million in ARR growing 500%. And it drove me— and I kept fixing it and it would break again and again and again. And I, I now I have to constantly work on this again. And one, I left it alone for a while and it worked, but then it broke to Amelia's point. But also this wasn't because I added a new feature or changed the code. The code did not change. It was interestingly, probably because of subtle changes to the model introduced in this complex workflow hallucination. So again, 4,000 decks, but it went from working perfectly to maybe 5% of them producing completely anomalous results. And so who's going to manage that, right? Who's going to manage that? So the 3 examples of why set and forget does not work. And I don't mean to be too critical, but I want to set— I want to talk about something related to this. Not only do our own apps require maintenance, right, and have these micro hallucinations and drift, we had an issue with Clay and listen, we are big fans of Clay for the most part. We use it for, for what a lot of you do. We use it for enrichment and to track where our attendees and folks have gone. And we blew through all our credits and we're power users of Clay. We support it, but its agent lied to us. Yeah, its agent lied to us. So what? So there's a meta story. So what happened, Amelia?

Yeah, I'll tell you guys a story and it's all— listen, it's all resolved now because this was a couple of weeks ago. So I'll say, you know, all's well that ends well. But in the moment I was so mad, like I was mad and frustrated at a point where I literally told the Clay team, like, this is the first time I've wanted to rage quit an agent, which has never happened with all the 20-plus. I mean, I've tried more than 20. We have 20 now in deployment, right? Like, I was like, this is the first time I've legitimately wanted to rage quit because the answer that the AI agent gave me was so unacceptable, was so bad. And when I reached out to the humans at the first time, I couldn't get the help I wanted. I was like, that's a double handicap.

What happened? So walk back from it. What, what did the agent— so Clay announced cheaper, better pricing. For me, they made— they just made the pricing more complex and confusing. But they, they claimed it was cheaper, but they claimed it was cheaper for many use cases. Right. What did the agent tell you? Their own.

Yeah.

So, so in other words, their own. You went on Clay, it has basically a support agent, right?

It has an agent. Sculptor. It's to their, to their point.

Yeah.

They said this to me afterwards and made this point. It's still in beta, but I was using the sculptor because I'm used to using agents and, you know, the built-in agents and all my tools. I'll give you guys the full story. So for the last couple of weeks, we have these VIP summits we have for Saastr annual. I was using it to do lookalikes. I've talked about this before on Workshop Wednesday, I think. Clay is a really good place if you're trying to scale up your AI SDRs or anything like that, where you can make really solid lookalikes with Clay. It works really well for this, and we were seeing success with it. And so I was using it for that, doing the same thing I had always done. It was kind of late at night on a Sunday. I was building a new list of, you know, VIPs to go out to for the summit. And suddenly Clay wanted to charge me about 5x to run this list. Than it did the week previous. And so I asked—

Okay, so hold on just for a second. Just so Clay wanted to, or its agent told you you had to pay 5 times?

Its agent told me it was going to be 5 times the amount of credit.

Clay, the software application, didn't necessarily charge you 5 times as much. Its AI agent told you you had to pay 5 times as much, right?

Correct. Yes. It told me I was going to have to pay 5 times as much. And so when I pushed the Sculptor agent, Yeah, I said, how can that be? I just did this on a very similar size list, a very similar arrangement, basically almost the same parameters. And it was, let's say, something like 2,500 credits. But now it's trying to charge me $11,000, which is a huge, huge— I was like, it has to be wrong, right? It just has to be wrong because that doesn't make any sense. And so I asked a few questions and first I caught that the agent related to their price increase changed how it, how it billed, but it also defaulted to the most expensive action versus a cheaper action that would get me to the same outcome.

Let me— maybe I misunderstood what happened. Did the— I thought the agent wasn't properly trained on its— on the pricing of its own app.

Yeah, both.

Okay.

It's both. Yeah, it was both. So like, as I was talking to it, I was like, okay, one, I caught that it was trying to use this higher priced, you know, model to get to the same goal as last week. And so I said, hey, use this other action instead. It's cheaper. And so that lowered it by about half. Then it went down to like 5,000 credits to run the same thing. I was like, okay, we're, we're halfway there. And that part of the issue I thought was problematic because most people won't do that. Most people are going to look at the 11,000 that the agent tells you and say, that's too expensive. And they would give up right there.

The first issue was the agent told you to use the most expensive model, the most expensive product. So the upsell, the agent, the agent told you you had to use the most expensive version of the product that you didn't have to use.

Correct. I didn't have to use it, so I called him out on that.

The agent told you you had to, right?

He recommended. Yeah, yeah, yeah. He was like, well, if you want to achieve that outcome, this is how you should do it. Are you ready to deploy and run the Waterfall enrichment? I was like, no, not for 11,000 credits. There's, there's a better way to do this.

That was intentional. Or you think that was poorly training the agent? You think that was intentional to monetize the base better, or do you think it was bad training?

I don't know, because it honestly happened the same time they announced this price increase. So I was like, I don't know. I don't know if somebody as a human at Klay unintentionally, like that was the consequence of them putting in this release for new pricing that the agent took it and ran with it, right? Like I can totally see that as it wasn't malicious. It's just something that happened with their agent because they loaded in all the training for the new pricing. I was talking to the agent and the agent was like, well, let's use this super expensive, luxurious model versus like the normal, you know, Prius model you were using.

Yeah, because I have a— I know. And listen, we do use Klay and it's a great product, but when they announced all this new pricing, I have a rule, a classic Saster rule that I think is almost always true. When a company introduces more complicated pricing, you know, even if they say it's a better deal, it's always a hidden price increase. I've never seen it be like— there are rare— there are occasions where companies literally say, hey, listen, we've just decided to go cheaper. Um, when— but, but, and it's usually when there's something brutally competitive, but usually when pricing gets more complicated, it's a way to disguise a price increase. Even if that— sometimes not everybody pays the higher price, but by adding complexity to pricing, you can essentially make segments of your users pay more. So it feels like that's what the agent was trying to get you to do. You try to get you to pay 2 to 5 times more than you had to, at least it seemed like it, right? That was the first experience.

And it led to the se— yeah, it led to the second issue actually. So once I caught it in this, like trying to use a more expensive model, trying to burn through all my— at the time I was like, I don't know, I, you know, I have like 3,000 credits a month. You're gonna burn through.

My 5 months of credit.

Yeah, basically I was like, no freaking way. So then I got it down to about half once I caught it in the agent, but then I could really only get it down to half. And I was like, you know what, I'm so tired. Heck it. Like, I'm just going to— even though this is more expensive than it was a week ago, I'm just going to push it through because I need to get this done. I was like, I just can't keep— you know, I had already been arguing with the agent for like half an hour at like 11:00 on a Sunday. I was like, I just got to get it done. And so I asked the agent to, you know, push the enrichment live with the 2x credits because I just had to get it done. And it said, okay, you're not going to have enough credits to finish, so you're going to have to upgrade because this is— again, this is around the time they announced the price increase. So at that point I was like, you've got to be kidding me. So I just hit the upgrade button. Again, I'm tired. It shouldn't be on the customer to figure out— tired or not, it shouldn't be on me to figure out that the new plan was going to be worse for this outcome and I should have just stayed on my legacy plan at the time and just bought more credits, which in hindsight, yes, of course I would have done. But again, when the agent was suggesting to me, hey, if you just upgrade your plan, you'll have enough credits, I'm just going to click that button. I literally— that's what I did. I clicked the button from the agent, I clicked one of the new plans or whatever, and then I finished the enrichment and I went about my day and I loaded up with my AISDR. So it was a, it was a double-handed—

I get that. But just to— and then we can move on to the next topic. But I thought the second issue when you reached out to the Clay team was that, and I think this is something we all need to learn about agents, was they had actually not properly trained their beta agent on how their new pricing worked. Like they had not uploaded the documentation, the pricing. And we've seen this ourselves. If you don't keep, you know, Digital Jason, the digital version of me, it broke for 4 months. The old version, it didn't upload anything for 4 months. I didn't know it silently failed. So if you don't upload, if you don't constantly train your agents, they will give your customers wrong answers. So I think the second part was Clay changed their pricing. They didn't train their agent with the new pricing. So, so that was one of the reasons they gave you bad answers, right? Yeah.

Because when I, when I flagged this to the Clay team, I was like, hey, just so you know, like, obviously we use a lot of agents, obviously for families a lot, Varun was in London, like we're fans, but here's why I almost rage quit, right? And so I sent it to both, like all the folks I know there and was like, you know, just this double handicap of it trying to charge me more, use a more pricier model. Then you raising prices, then the agent telling me I had to upgrade to get something done, which wasn't actually even true, right? It just told me to— it just made that recommendation. It's also telling me to use credits I didn't have to use. I was like, you may want to look into this because I don't know if other people would be as patient. I don't know if they'd say, hey, this is mission critical, I got to get it done. I'm going to use the credits and pay the extra cost. Like I feel as a customer, like I stuck with it, but I think other folks would be fairly frustrated. And so it's just a reminder for the folks listening, right? Like, I don't want you guys to ever— and we're sometimes in the situation too, Jason's point, like we're sometimes on the reverse side of this where with our agents it is a lot of maintenance, it is a lot of training to keep it up to date. But because they hadn't trained their sculptor fully through every scenario, which I know is tedious, right? It's totally tedious. Like, that's why But that's the job. But that's why it was trying to get me down this upsell upgrade path when it didn't need to.

Yeah, it was another example. Then let's move on to the next topic. I was, I was using the AI agent on HubSpot's website on the homepage, right? Massive traffic, right?

Yeah.

Trying to get a pricing quote for what HubSpot would cost at our scale. I just could not get an intelligent answer.

Oh yeah, I remember this.

I don't know whether it was now. Now it might have been because they, they really want you to talk to a human. I think that was part of it. But it could have been better trained. And so all these stories are that some of the best folks, Clay here, to a smaller extent HubSpot, to others, like AI support is great, AI agents are great, but someone, you gotta read a segment of these chats every day. You like forever, you have to, you have to read these interactions with the agent because they're gonna drift. You're gonna forget to train them. They're not gonna ingest documents properly, or you're gonna think they say things that they don't, or your, your testing worked fine. And in reality it didn't. So it's just these are endless stories of agents not being as well trained as they could be that are customer-facing.

On the flip side, though, we had somebody this week reach out to our events team frustrated because they, they thought the answer they got from an AI was not good enough. And it's funny because I was like, sir, actually the answer you got was from a human being.

You flip from Human mistake or was it— well, it was from a human, right?

It, it was two. They made a mistake when they bought their ticket on Saastr and then one of our humans on our side, minor mistake, super minor mistake. We're done. Yeah. They, they, when they bought their ticket, they accidentally bought two. And so that is a human thing we have to resolve on our end. Like I don't let the agents right now, any of our agents touch our backend of ticketing and Stripe, right? Like that's something we have to do and it should be something we do. So. This was a flow where he, he said it to Amelia AI, Amelia AI escalated to a human as she should have down this path.

Yeah.

Our, one of our humans, our events team initiated a refund as they should have, the way they should have. The human on the attendee side said, hey, you refunded me, but not with the ticket processing fees, like fully. It's this weird thing in Stripe. There's like a 3% ticket processing fee.

They didn't get the 3% back.

They didn't get the 3% back because that's our, that's our standard is to usually when we do a refund. Now, this was a case where it was our fault because the website glitched and he got 2 tickets of the same. We should have just refunded that 3% in the first case. The team didn't want to bother me and be like, hey, can I have an exception for 3%? And so they just refunded him under standard. And this guy complained after the fact and said, thanks for my full refund now, but if this is how your AI support is, I'm really like worried about the content it's answering. And I was like, sir, I understand. I understand. I was like, yes, sir, I understand the frustration, but it was us. It was a person. It was multiple. It was people.

Agent. It would be interesting if Digital Amelia could have identified the issue of the Stripe fees that a human might not have thought through. The agent might have done better here, probably, right?

Yeah, and that's the irony.

The mediocre AI is pretty good.

Yes. So it is.

Okay. The next topic I want to talk about— I sometimes I'm slow, guys. I sometimes it takes me a little while to figure things out. I want to talk about— I want to share a story from this week, but A lot of folks read what we, what we write, especially on LinkedIn where people, people, you know, really like to, to be yahoos and they're like, oh, Saastr's results with its SDRs and GTM agents, those don't work at my company. Those are unique to Saastr. Saastr is such a strong brand or Saastr does events and tickets. It's not the same with me or Saastr is exaggerating or whatever. And we're just, we're just trying to help. But I see this, I see this all the time and, and What I basically tell people is like, we're just doing the same thing you should be doing and you might get better results than us. You might get worse, but try it, right? It's working for us. But, I didn't completely get why it works for us until I had a meeting this week. Even after all these months, even after AgentForce, Monaco, Artisan, Qualified, my man, I'm all— why does it work? But I was meeting this week with the CEO and CRO of a, of a great public company with thousands of sellers, thousands of sellers. And we were talking about agents and AI and You know, some folks, some folks do get a one-on-one version of this, of the show. And the first question the CEO and the CRO had is that, what do you think we should do first? And they had some Agentic products, but they said, what do you think you should do first in order? And I went through, I went through a series of what I thought they should, should do first. First, do something so that everyone can instantly get great quality questions answered on your website, going to what we've just talked about, right? Make sure they can instantly get an appointment set with sales, right? And I kept going through different things. And then I realized I went through all the things I said, you know what? You know why all of our agents work in go-to-market and sales? It's because we lead. There's no lead left behind. This is the flip side. We talk about how agents give you more coverage. We talk about how agents don't mind the B leads, but it's simpler than that. If all— if you deploy some of just some of what we do in AI agents, and then Amelia can share some details better than I can. Here's what you know. Anyone that has an issue like with that 3% Stripe fee gets to talk to an agent and then a human human. Anyone that wants to know if they can get a discount, whether they get one or not, can find out in real time. Any, any sponsor that clearly doesn't have enough budget to be at SaaStr can still find out if they can sponsor. But a human might be like, they raised $2 million at Demo Day, like, we don't want them. But the agent, the agent doesn't judge. Bootstrap, the agent doesn't judge. Any issue. And then, you know, and then for our use of AgentForce or others, you know, we were inspired by Marc Benioff to go back to are prior leads and prior ones that no one had touched. Right. And so clearly we have evidence that in many cases our agents do better than a mediocre human. I know it's a triggering way to say it, but it is true. But even if it, even if it's not, it may be simpler than that. It may be just if you touch every single lead, every prospect, every customer the way they want to be interacted with in real time, you have to do better. That's all these agents have to do to add value to the organization is that there's no lead, no prospect, no customer left behind, whether whether it's in pre-sales, sales, customer success. We can talk more about Qube, our AI, but no one gets left behind. I think it's that simple. What do you think, Amelia? I know I'm slow, but this was my aha moment. And then with this public company, we just talked about all the holes in their, in their funnel and their— and as well as they're doing, there's so many ways to not get touched.

I agree wholeheartedly because I've said now, like, there are times even with the volume we're doing between all of our agents, our agents still sit idle, right? We do a lot of outbound. And, you know, all the normal, quote unquote, normal go-to-market motions that, again, I think sometimes people look at it and say, oh, that doesn't translate from SaaS to me. I'm like, well, revenue is revenue and customer are customers. Like, that's still the same across the board. And we still have a lot of the same go-to-market motions you guys are using. They're just sometimes employed a little bit differently, or sometimes they have deadlines a little bit differently because we have events that are tied to it. But the, the behind-the-scenes motion is still very much the same. And even at our scale, I have not touched every single lead. I'll give you a good example. Just this week we have been in a production sprint because there were a lot of deadlines leading up to today and there were a few on Friday and there were a few on Monday, basically for SAS Re:ANAL that's coming up in May. And so we had these big production deadlines and so sprinting between myself, our designers, our production team, our, our customers, our sponsors to get some of their graphics in. Like we were sprinting to meet some of these deadlines that we had to meet because things take time to print and sometimes you can't pay rush fees to get these things done. But I'll give you what it led to was a major idolization across our agents because I just didn't have time for them. This is one of those extreme examples too, where like no lead left behind. Yes. But also if you have somebody who gets a little bit busy in a way a human can, that agents, you know, are working 24/7, don't you run into this issue? I've run into this issue over and over again, and that's why I'll keep talking about it, is my agents will start to sit idle. And this last week, it's the most they've been idle since I've gotten them, you know, all up into production.

Leads behind.

We're leaving some leads behind.

Even with our agents?

100%.

But our agents are ready.

They're ready. Yeah. But I, I was literally looking at it today. I was like, okay, I have a tiny bit more time today and I've got to go get my agents out of idle mode. I've got to get them back into go mode because I have leads left behind for sure. 100%.

Yeah, it's funny. It's a different topic, but maybe one that this is worth continuing in subsequent conversations. All our agents end up idle, right? It's an interesting topic, which is, you know, every human thinks they're so busy, even if they're really working 20 hours a week from Miami, but they think they're so busy, they don't have time. They don't want to follow up with the B leads or the C leads. Agents are so efficient and we have so many there. All of them are idle 90% of the time. They're sitting there waiting to do more. They're happy to do more work. They're sitting there idle. And it's a— I I haven't fully thought through where that will lead over the next 12 months. But, you know, in a sense, we have an order of magnitude more capacity than we had pre-agents and then we had with humans, right? How to fully exploit that is the best way you can exploit it now is just make sure everyone that hits your website, everyone that contacts me, everyone in your database is followed up with in real time. But it's amazing how much idle capacity we really have, right?

Yeah. And I think that's a The website one is a new workflow we added, which we could talk about more later, but we added a newer workflow where when folks hit our website, in particular for Saastr Annual, now they get like an automated follow-up, but it requires some human steps to get that data from the website to the agent. One, because some of our, again, because of how we're built, we have to have this step. You wouldn't have to have this step necessarily, but because we get so much traffic to saastra.com, we get, not to, that wasn't a braggy thing. It's just we get a lot of high volume traffic to saastra.com. We get a lot of traffic to Saastr Annual. Manual and we have to filter it out. I can't— to Jason's point, could our agents hit up every single person that hit the website if we de-anonymize them? Yes. But do I want to? No. This is the human orchestration layer of yes, no lead left behind. If I think they're a lead, right? Like there's still some qualification that needs to happen there. And I don't have an Agentic workflow yet, but it's one of those things right now where we have to do some cleanup of that data because it's just so much data. And kind of push people into different buckets. And so could we, could we have no lead left behind on the website? We could. But I think there's also an air of we have to filter out some of our stuff because not, not everybody is a qualified lead, right?

Yeah, I think it's interesting when you think about what agents can do over the next 12 months in marketing. I feel like marketing is pretty far behind sales, right? Sales is behind coding and support, but marketing, man, we— the reason we had to build our own AVP in marketing is we couldn't find anything that's any good, right? And I know folks are trying and kudos and we're We're— if you literally can do better than we're doing, email us, but it has to be better. We'll, we'll use your product if it's better than we're doing, but we don't need something that just is a geo or SEO scraper or writes content. Like, we have a lot of content, guys. Like, a lot of folks reach out to us as, hey, we can write social media content with you with our AI market. Like, we don't— we got 99 problems. That's not one of them, but it's behind. But, you know, so we get maybe half a million uniques to our SaaS or web properties. I mean, people read content on LinkedIn, they read it on Twitter, they watch YouTube, but we get about half a million uniques to our website. And those aren't leads, right? But traditionally, if we could use agents so that everyone truly has a customized experience to them, we can convert more of them to leads, right? Like we need a, we need an agentic lead magnet where everyone, just like now we have agents like Digital Amelia that you can talk to if you want on our websites, but we should go further where you don't even know you have a lead magnet because everything is like, who are you? Okay. Okay, you know, Jason would love to have an ebook on how to do our agents. Amelia's pretty advanced. She's a chief officer. Maybe what she would like is to invite to our CMO summit or our AI summit or something like— but if we had this hyper ability, not just to, to use some of the tools we use, which are really exciting, like all of our tools, you know, whether you think it's, it's, it's ideal or not, they can de-anonymize your web traffic. They can identify better than ever these agents who's coming on your properties. But that's not as actionable in many ways, right? As you might— unless you're a spam farm, right? You don't want to just collect people's random email address and send them weird LinkedIn notes. I mean, people do that stuff, but I, you know, I block them.

People do that.

But I think I am excited. But you could do something in the middle, which is it's— you don't spam them, but you provide a truly contextualized value add during their experience, or you push them at least into the very top of your funnel, these 500,000 people where they otherwise might. That would be— that once you— once that's a— maybe our current agents can't do that, but it's a meta idea of, you know, no, No champion left behind, no visitor left behind, no anything. That's the next— you got me thinking. Okay, next one seems small, but I think it's just interesting to note. How are the big guys doing? And, you know, we've been relatively close to Salesforce because we were relatively early on Asian forest on deploying it. We've done a lot of stuff together with Salesforce. It's been fun. And then I think maybe we even played a small role when Salesforce bought one of its partners, Qualified. To accelerate its agentic efforts. We thought it was a great idea because we use Qualified and we thought, you know, it has scale. So what— the deal closed. What did you see actually happen at the product level this week, Amelia?

Yeah, I thought it was interesting. The— so if you go to salesforce.com, just the homepage or any page without logging in, you'll, you'll now see a qualified agent versus it used to be a Agentforce, like support agent. Now they've swapped it over to Qualified, which I think for a lot of reasons was the right call. Right. It's— I think it's— I think it's part of why they bought it. Right. Like in many ways, some of this, some of the features and functionality that Qualified had just went a lot deeper than what Agentforce kind of natively had. Like now you could get there, but it would just take a lot of— a whole lot of like extra steps. And so again, I think that's— I think they were so smart to buy it. Because it has all this and because you had to be a Salesforce customer to be on Qualified anyway. I think it was— and a lot of the team came from Salesforce, right? But I think it's interesting because when that module was on Qualified.com, it was a real person, Blake, that they modeled, you know, Piper after, who's their— who was their mascot. And now it's like a 3D avatar version, which again, I think it's interesting. We've talked about it a little bit of like Do you have, you know, the rights to that person? If you use a person who's not the CEO or founder, there's a reason why, you know, Jason M. Lemkin and Amelia Ayers, like, we're not going anywhere. Like, we have all the rights to it. Like, we can still use our agents in our likeness. But I think it was interesting that probably for a CYA, if I had to guess, that Salesforce changed it from the actual human avatar that it was to a modeled, like, cartoon version of the same person. Probably for a lot of different reasons there. And also just for speed. I think for essence of speed, like they already had the other video model, but I think to also signify that it's now Salesforce and, you know, they have a lot more customers and a lot more security layers that they have to go to. They changed it to a cartoon version of the same person. Again, I think it was smart to get qualified on the website. The, the, the day the deal was pretty much done, they flipped the website. There's so many deep features. In Qualified that now the Salesforce team can action on.

The interesting takeaway, just we're trying, you know, Salesforce built Agentforce and they've got thousands of people working on it. We use it and it, and it, and it, and the most important thing for folks is it does work. It does work. But because it's so extensible and so broad, I mean, Salesforce has like 9 different clouds and so many uses. It is work to configure it. It is. And it is. They bought Qualified, which is just a GTM tool and classically was just to qualify. That's the name of the company, Inbound, right? So it's a very narrow set of— now does some outbound and other things. Yeah, but it's just a GTM agentic tool. So for their— especially for their CRM customers, which is still almost 20% of the revenue now, they bought a tool they can roll out to the entire customer base. Does it do as much as Agentforce could do? No, but it's pretty cool that the day the deal closed, it's in production, it's on Salesforce's website, and now there is an option to buy a Salesforce-native Agentic product that is much simpler to deploy. And if your goal is GTM, right, you can have it up and running probably in a couple of weeks rather than maybe a longer period it would take to map out and figure out a full Agentforce implementation.

Oh, for sure. Like I'll say, like for my, for myself, I had a much steeper learning curve with Agentforce than Qualified. Qualified, the UI feels more like that, right? So I think part of why they did the deal too is Okay, if you want to unlock this whole pool of people where AgentForce historically has been, you know, success is the number one use case of AgentForce, but now you want to go more go-to-market. I think that's why they both qualified and Momentum together, right? Because now you've got RevOps for salespeople and you've got this whole GTM kind of suite that is easier for folks to use, better to use out of the box. I think for GTM folks, like I talked about last week, I had to build this like Salesforce custom object for one of our agents. Pre-Agentforce, we weren't using our Salesforce that much. But also I'm not like Salesforce Trailblazer qualified. Like I was like, I'm not that good. Like I will confess to anyone listening, like I wasn't that good at Salesforce. I've gotten a lot better in the last 9 months because of how we use Agentforce. But a lot of the times I am, I like look for like I look for either loopholes or other things I can integrate into our Salesforce to make it easier because there's just so much data in there and there's so much you could do with it. That's, that's why now I use a lot of Slackbot too, because it can do some of it a little bit easier. But again, I think that's why it was smart of them to get Qualified up and running instantly. I think it's smart of them to expand and just kind of like go at a market area where it's, it's easier for folks maybe to understand, you know, Qualified from Salesforce, that's just your go-to-market agent, like where it blends with Agentforce, who cares, right? I don't care. I told the teams to just was just let people pick what they want. I was like, if you can do it in a way where people can pick either the qualified UI or the agent-first traditional UI that it had, like just let them hit a toggle and say, I'm going to use the qual— yeah, like let the customer, you know, again, go to your customer, let them pick what's easiest for them of how they want to set up the agent so that they actually get to deployed agent in production.

Yep. So, and now I want to move on to our sort of our last and final topic that we will come back to every week. But I think the, the, this, the meta thing is, I think, um, listen, there's, there's Qualified has, there's other folks like Qualified. There's a number of vendors that can do a great job with inbound and outbound, but we've used it, right? And so now there is an option that's natively in Salesforce. It's on their homepage where you can deploy this much simp— more simply than Agentforce. If you're deep on Salesforce, like, go for it. Get this thing up and, uh, like, it's, it goes back, like, whether it's the perfect vendor or not, it's native, it's built in. And, and our theme of no prospect, no customer left behind, this is one thing you can do now, get it deployed. And this is, you know, just like a lot of CMOs have patted themselves on the back the last 12 to 18 months rolling out Clay, right? And kudos to them. If you're in RevOps or SalesOps or sales and you want to build it, roll out an agentic product in Salesforce, just, just do Qualify. Like, is it still called Qualify? Did they change it to—

Qualify from Salesforce.

Qualify from Salesforce. But this is— I think there are, there are, and then maybe the competitors are better. I'm not, I'm not picking sides at the moment, but this is a quick win. This is, this is something to think about. Okay. The last thing I want to talk about is our team members. And if, and maybe we'll do this each week at the end, QB and 10K, what's up with them? As a quick reminder, QB is kind of a play on QBR. It's our AI VP of customer success that Amelia built. 10K is our VP of marketing. Her initial job was to get us to 10,000 attendees in the first $10 million of revenue this year. So we call them 10K. Now they've all joined Slack. They're up early on Slack. We now have more daily check-ins on Slack with agents than humans, which is the future here today. If you go in our Slack, like our Slack, just like our Salesforce almost died before agents, our Slack almost died before agents. Then we added Slack integration for agents. Now they're talking on Slack almost too much, but they're better check-ins than any human ever did, right? Any check-ins. Okay, so a couple things on QB and 10-K. First one, because you brought it up. You built and maybe just describe the challenges and the opportunities for our— for you added Salesforce integration to 10K for sure. Maybe, maybe to be too right.

Yeah.

And it was interesting. So we did it in Replit and Replit has a native Salesforce integration, right? But that alone didn't quite do what we needed to do. So you had to build a custom object in Salesforce which if folks have been in the Salesforce ecosystem, you'll understand what custom objects do. But this was some learning, a learning curve and work for you to figure out how actually to get the Salesforce integration to work. I want to hear about it, but to give folks context, for me, I don't log into Salesforce, but when I log into 10K, I see all of our pipeline and everything over history and time presented a way a normal human would want to see it. So Salesforce is now abstracted away from me. I see all of our Salesforce pipeline opportunities, everything done beautifully and visually, and I can drill into it right inside of our AIVP marketing, which for me is great.

Okay.

But you had to build it in there, so it was different than we'd thought, right? With the current integrations.

Yeah, it's just, listen, the learning with any of the, I think any of the VibeCoding platforms with Salesforce in particular, I get now why it happened, but it was frustrating for a few days because I just used the native because again, why wouldn't I? Like if you're VibeCoding, I'm going to do that.

One is you click on a Salesforce connector, you put in your credentials or your OAuth, then you go to, you just log in once, right? And it gets a token. Problem is the token expired every 24 hours, right?

Every 24 hours, which I didn't know. And so, and the agent didn't really know it either. It like took it a couple days of it failing.

Probably because it's not a core feature like the Clay thing. It probably wasn't working on how the integration worked.

Yeah.

Because it's a, it's a corner case. It's not a high volume integration.

I was like, okay, there's gotta be a way where the token doesn't expire every 24 hours. And it was just building a custom object. App because in a custom object app, which I asked Claude how to do, super easy. Like Claude gave me really good instructions and then I loaded up Cowork so it could watch me do it, right? I was like, just watch what I'm doing, tell me if I go wrong.

Cowork was watching you manually build the Salesforce custom object?

Yeah, it was watching the screen in the browser. Yeah, yeah, it was watching.

That's pretty cool.

Yeah.

Did it catch errors while you were doing it?

I didn't too well. It was pretty easy. So, well, he gave me really good instructions, so nothing bad happened.

So Claude gave you the instructions to build a custom app, then Claude Cowork watched you while you did it. I think again, you're ahead of me.

She was like, you're doing great.

I wouldn't have thought to have Cowork watch my work while I'm doing it, but that's pretty cool.

Yeah. You know what's funny? I haven't had Cowork watch Replit or anything. I should do that. I'll do that next time before or next episode. So yeah, I was, so I deployed this custom app object agent and Claude and Cowork were walking me through, okay, here's how you can set it so that that you don't have to— because I did get— I got stuck a couple of times. I was like, you know, Claude, what you told me to do is not exactly where you told me. Like, it's using just kind of like, you know, older UI data. So I'd be like, is this the right button? Is this the right box to check? Is this the right field I want? And it was like, yes, yes, that's, that's fine. That's the one you want. And I was like, shouldn't I do this one? It was like, yes, you're right, do that one. This is why I'm watching you. And so I did it. And it— and then I found in building the custom object a way that we could have our token refresh every year. Versus every day. And now our Salesforce integration has not broken since I did the custom object.

Cool. Well, it shows. So the good news is it worked and it's epic. It's cool, right?

It's epic. Yeah, it's fun.

But it was harder than we thought. Like, not every— there's some integrations, like my two favorites vibe coding are ElevenLabs and OpenRouter. They're so easy. Like, you can integrate these products in like 30 seconds instantly.

Yeah.

And it's a credit to both vendors, to ElevenLabs, but it's also a credit to the fact how they work. Like, it doesn't have to be as complicated as Salesforce or Clerk just by nature. But, but, but, but realize that not all of these are one-click simple, even if they look like it, right? They are complicated. And this was probably— was it a couple hours of work or how long did it take you to really get the Salesforce integration working in 10K?

Half an hour. Once we figured out the problem.

Like, I would have thrown my mic at the monitor because, because I, I am— I don't want to build custom objects. But, but I, but again, for you it's half an hour. For someone that has never really used that side of Salesforce, it could be a bigger project though, right? Yeah.

Yeah. Or you might, or you might not have access, right? Like I'm our Salesforce admin and so I have all the right protocols and permissions to do that. Now, if you're building this, you're not one of your Salesforce admins or you have certain like company restrictions for, you know, because it needed API keys and auth credentials and it needed a lot of stuff to get those custom app objects it may take you longer just because you're going to hit blockades, right? You may not have all the right permissions or somebody else may have to do it for you. And so just by nature, that may take you a bit longer. Yeah.

Okay. Last one that we had with QB and 10K, when you were vibe coding in the car and we were taking a Waymo to an event this week and you were saying how we, we have, we actually have a bunch of Chinese sponsors this year, which is great for SaaStr AI Annual. And they were complaining that Qubi, our customer, our AI customer success, which is great, but it was a little hard to interact with the app in English. So we asked Replit to localize it into Chinese. And so what, what, you'd never done this before, right? So we did it in the car. I figured this was one that could be done on the phone and it, it was partially true. What'd you learn, what'd you learn from the localization project to localize it into Chinese and Spanish? We had Replit do both of them, right?

Yeah, we had, I had 'em do both 'cause I was like, you know, I'm gonna pick all the like topple it. Obviously English is native 'cause I built it in English. And I was like, okay, I'm gonna, I'm gonna pick, 'cause this is becoming a frustration point 'cause we have last minute sponsors, we have multiples of them are, you know, native Spanish speakers. Some of them are native Chinese Mandarin, you know, speakers. And I was like, okay, I wanna make this as seamless as possible for them to get up to speed. I can't do that, but Qubii could, right? So we were vibe coding the waymo and I said to Replit, hey, can you make a toggle so that you can translate everything I've put into Qubi so far into Spanish, Chinese, and then with the toggle they can switch from English to the, you know, non-native languages that I don't know. And so it took it maybe 20 minutes cuz there's, there's a lot of content in Qubi. And so it had to, you know, translate everything. It used, I think it used OpenAI for all the translations. And then after we got outta the car, you know, I, I, I took some screenshots and checked it with Claude to make sure nothing was inappropriate or weird with its translations.

What did Claude say about the translations?

It was pretty good.

Pretty good. Yeah. So you QA'd it with Claude. You took screenshots of what Replit built with OpenAI, and then you took screenshots to QA it into Claude to see if it thought that the translated app was, uh, was, was good enough. Yeah.

Actually what I'm running now too in the background is I'm having Cowork just now run through all of it. Just spot check, make sure nothing is weird, but now Cowork is doing the full thing. So that's, that's also an interesting learning. But yeah, I think it took about 20 minutes in the car. I'm about to roll it out to those sponsors who need this, but my guess is that it will ease some friction because, you know, again, I'm using our native language. They're having to translate. They're also, you know, all of these folks are last-minute sponsors and so they're trying to get caught up really fast. And so this is something again, we wouldn't have been able to accomplish with the tool we were using previously. Like, there's no— like, we would have had to get a person to translate it manually and then program that all into our—

It's worse because you have to build a harness or a system so that each chunk of text is dynamically translated to different languages and can be updated when it's changed. It's not even— it's not because if you really understand how localization— I mean, I'm not an expert. I have done it like, you know, you do need a, you do need a framework or harness because because everything's constantly changing and being updated and it's like—

you're adding things. Yeah.

Yeah. So it's not, it's, it's not a one and done kind of, kind of deal that adds. And so, so the agent Replit, I mean, it seemed to have done it all in, in the Waymo for the most part. Now at first, at first it was, as agents are, at first it was a little lazy and only translated some of the—

it only did the menus, right?

And so you had to put, keep saying, "Mogo," like it's the school thing, keep going. It was hoping it was enough to just translate translate the core stuff and not the deeper text. And you had to push it, you had to push it actually multiple times to go deeper, right?

I did. But you know what, when I, when I think back about that now, I think it's okay because I see so many people flag the costs on Replit where like, you know, maybe if you were cost conscious, you would only want to translate the menus and something like the task list to save $20. Yeah. But still, right? So I was like, okay, to his point, maybe that's why I didn't do the 14-bit This is a complex project, right?

Relatively speaking. So to, you know, to, to if 25% of the cost gets you there, that for most people, that might be a good place to stop, right? Yeah.

Yeah. But the agent should have asked me, okay, I did these menus. Are you okay with me keeping, you know, to keep going? It would have been nice.

It's funny because, and we can wrap it, I was talking with another public company CEO this week about agents and he asked me, what do you, have you guys used any of your agents for localization? And I'm like, I get the problem, you know, I lived it in my day, but no, it hasn't really come up. And then it came up in the car. I'm like, oh, we got to do it. Like, we got it, we got it. I think this is a pretty awesome use case for the agent, but let's see if it breaks. Let's see if it can literally do something that seems simple, but it's not to fully— I mean, Shopify, I think just last year rolled out localization for a lot of its product. Okay. Shopify, I mean, it's pretty— Shopify is at what, $13 billion in revenue and they have global commerce and they just did this and we did it in 20 minutes in the car. I know in a way, Mo, on Replit, man, that's an AI story. Maybe we should end with that one. Do we get everything in a wrap?

I think it's a good one. I think my only thing that— my only other thing that happened this week with QB that I think is interesting, but I don't want to derail us.

I'll add it and then we'll wrap.

Okay. Is I— there was this interesting moment, right? So I talked a little bit earlier, we have this big production deadline not only for ourselves, but also for some of our customers who are sponsoring the SaaSRe annual. They had a print deadline this week. A few people tried to, in a very human way, trick the agent in a way where from, from the agents. I just think it's always an interesting reminder. You can't hide from the agents, right? And so a few people tried to, whether it was malicious or not, it doesn't matter, right? A few folks sent us their graphics and it was, I got a couple placeholders.

They were placeholders, right?

Placeholders. Either I got placeholders or I got renders.

Our graphic, big deadline. So they uploaded whatever the hell they had on their, on their desk.

Yeah. They were like, I could have said they pretended they met the deadline.

Right.

And they were like, okay, great. Like I sent this to you. We're all good. Right. I was like, well, actually between me and QB in a way that I couldn't previously, 'cause I didn't have enough scale or output. But now that I have the agent helping me, we did it in a way where not only did we get the, we and they got an automatic email We got an automatic email and it would send to all contacts because sometimes I would get like a random designer sending us, you know, the, the booth graphics.

Yeah.

And it would say, here's what you uploaded. And I would, between me and Claude, we would check it pretty much instantly and we would catch these things, right? We'd be like, okay, within the age of AI, you really can't hide. Like we can, we can see it. You didn't, you uploaded a placeholder or you uploaded, or a lot of times They would upload something that was incomplete, things you might have been able to hide back in the day, but that you can't hide now in the age of AI because it just checks everything. I was like, it's check between me and it, it's together. We're checking everything. Like maybe I wouldn't have got this, some of this a year ago, but we're checking everything now cuz I've got my agents running. And so I can see if your, if your graphics are incomplete or fake or real or not. And so.

It was just an interesting thing that people didn't fake the deadline, didn't fake the asset, and it can follow up in a kinder fashion than a human might be. So that because they would, they would pretend they did it right and QB would be like, would objectively say, no, I appreciate it, but that's not actually what the graphic or assets that you're supposed to. Yeah, right. In a way that a human that we do get yelled at a lot by, by folks. Right. For— I don't want to go too deep. We can wrap it up. But, but, but a lot of folks, when they miss deadlines or when they don't have the assets or things they need, they, they sort of try to blame us. And we're zen about it now. But it's human reaction, right? Oh, that's not the deadline that David told me. Well, it's in the contract. It's in the— here's the 28 emails. David said I could, I could just sketch it on my iPad and that you or David, you said this and people would just make crap up.

Yeah, but QB's calm.

Like QB doesn't get, get, get, get, get triggered, right? And QBee just says, thank you. I sent you 28 emails saying this is what we needed. It's late. Understood. Could you please do it by tomorrow?

Yeah. And that's, and that's the thing. Like in a way where I was like, I don't know if I would feel comfortable pushing back to some of our customers like they are paying us money, but in a way that's super neutral, just not caring. QBee and I would check the graphics. Yeah, better than a human. And if it was, and if there was an issue, it would just tell that person. It would just tell everybody in the org, hey, because maybe, you know, maybe the CMO didn't know that their designer just uploaded a placeholder. That happens sometimes. Like, yeah, you know, it's fine. It happens. And so in a way that it was super neutral, it would say, hey, there's an issue with your graphics. Here's the issue. I'm going to email everybody that I have from in the org as a customer, you know, in this customer org. I'm going to let them all know. And I'm going to also say in that reminder, either some of them were like already past due. So it would say you're past due, you need to upload this now to avoid a late fee or not. Or not. And for some of them, when it was approaching the deadline, it would just tell them, hey, we need your fully revised graphics. This doesn't count as a submission. Again, in a very neutral AI way. Thanks for trying. This wasn't it. This doesn't count. You might have been able to trick a human previously, but you can't hide from QBee.

Yeah, maybe we'll wrap on that. When I was talking with, with another public company CEO about AI, a great one, he said, well, your QBee, your AAV, that sounds like support on steroids. I'm like, no, no, no. Let me explain. It proactively follows up up on every possible deliverable there is and make sure it happens. And so my comment to him, and then we can wrap, I'll close and we'll wrap for this week. If you require any deployment for your product, for your company, for your application, if you have any checklist that you have to do, any training, any onboarding, do you have any onboarding for your company? If you have nothing, don't, you don't need to listen to the rest of this. If you have any, your humans are probably not giving you 100% coverage on this. They're probably being argued with. There's probably issues. It's probably somewhat your fault or someone's fault that's not getting done. I don't think there's a lot of great off-the-shelf tools for this. Build your own QB, AIVP, customer success. Amelia gave you the whole playbook. It's on saster.com. You can see our whole prompt and how we built it. Build your own one, because if you do nothing else but completely automate the onboarding, right, of your customers with no drama, no complaints, no issues like we did, your life is going to be better, right? And so that's why we'll keep talking about QB and 10K. And we'll wrap. Hopefully folks find this useful. If we find that on YouTube you all churned after 60 seconds, we may do this less often. If we find out people listen longer than average, we'll keep doing it. But we know this is the number one thing we're asked constantly, what's going on with your agents? So this is the agents. Amelia, thank you for doing this one. And we'll see you guys soon and continue the conversations.