Our AI is Hiring! Would You Work for One? And Are Autonomous Agents ... Safe? The Agents #005

Welcome everyone back to The Agents, episode 005 with SaaS or AI's Chief AI Officer, Amelia LaRute, and yourself, Chief AI Evangelist, Jason M. Lemkin. Thank you everybody. The Agents is going well. Each episode is doing better than the last, and we appreciate your support. And again, any questions you have about agents that we can help with or AI for B2B, put them in the comments. If you're watching on YouTube and we will answer all of them. We will answer all of them. It is a fun way. So use the comments as almost like a Reddit where we can answer all your questions. We're there. And with that, a bunch of fun things I wanted to talk about with our agents this week. The first one I want to talk about what they cost and how good they are. And first one is we talk, we've talked a lot on this show and in general about what 10K, our AI VP of marketing, and QB, our AI VP of customer success, cost. And Amelia, we actually had no idea, did we?

No, I, I— one, you're the admin. Two, it's a little hard to tell when you are vibe coding because they'll be like, okay, this action was 15 cents, 10 cents, 20 cents, but it doesn't really give you an end of day sum or Hey, give me a summary of this session of how much it was, or an end of day summary. So I really had no idea how much some of this was. I thought it was more than what it ended up being.

Yeah. And also, so yeah, you weren't the admin. That was part of it. And I brought this up briefly on 20VC with Rory and Harry, and Rory made the point that I probably didn't care. We have to get stuff done, right? Well, from, we're down to 3 humans, a dog, and 20 agents. And so. Whatever these AI VP of marketing, customer success costs, to some extent, I, you might care. I just don't care. I don't have time to care. Right? So we didn't know anyhow, but so I got a little alert from Replit just on our costs and I opened it up and I, I was shocked. And to run our AI, and then let me talk about why, and Amelia can chime in as well. Our AI VP of marketing, customer success together cost $257. And when I showed that to Amelia on Slack, what was your initial question when I showed you $257?

I said, is that in one day?

Yeah, that's what I thought too. It was a month.

That's crazy.

It's a month, right? I think QBee was a little bit more, I think $100 and something, and 10K was like $80 a month. And they're going to go up. It's already clear that as we record this, it's just before SaaStr AI Annual, our big event, and we're using these agents all the time. And so I think the bills will be 2 or 3x that this month. Maybe it'll go up to 300 or $400 or $500 a month to run 2 AI VPs on our team. But we were shocked how little it was. And I want to get some of Amelia's thoughts. Why? Just a couple of learnings that I think are interesting. When I asked, I, I actually didn't know why I asked 10K, why it was so cheap. One, a lot of what we're doing is pulling from other APIs to fix data. We're pulling from Salesforce, we're pulling from Bizabo, our event ticketing app, or even pulling a little bit from Marketo. We're pulling from YouTube and X and WordPress and other places. And generally speaking, those API calls are free or a nominal cost, right? I don't even know if we have to pay. Even Salesforce charges us to access the API, but I think it's very, it's nominal for a single API call, right? It's gotta be almost nothing.

I think we just pay for an API seat or license or something.

Yeah.

Yeah. Something like that.

So, so interestingly, some, a lot of the heavy lifting of our, especially of our AIVP marketing. Is doing is pulling real-time dashboards that we didn't have access before. And that doesn't really use a lot of tokens, right? The second obvious point it made is a lot of what we're doing is storing massive amounts of data, at least massive for us, maybe not massive compared to a lot of folks in a Postgres database, which we'll talk about later, Postgres versus Salesforce. And that basically costs nothing on Replit. We pay a tiny bit for data, but I think it's 10 or 20 or 30 cents a month. Storing the amount of data we have in Postgres costs nothing. And then the shocker, and this will change over the course of the year, but most of what 10K chooses to use, if it's up to 10K, is OpenAI Mini. It's Mini. And there's pros and cons that we'll probably talk about today going forward. But so much of what we want to do, run a quick data analysis, figure out what our year-over-year ticket sales were, figure out how many less under-included folks are coming, how many CMOs are coming for last year. This does not require a massive Claude Opus data analysis of a trillion tokens. Minnie can mostly do it okay with some hallucination issues we can chat about. But about 95% of the OpenAI calls we're using are with Minnie, which costs less than a penny per call. So add that all up. Uh, and I guess that means 10K is not as smart as we thought 10K was since it's running on Mini and not Sonnet or Opus. $100 a month is pretty competitive. Also suggests to me some B2B apps are very expensive to build for sure with AI, but some may be cheaper than you think if you're thoughtful about what models you use. So I could add a few other insights, but any other learnings you had from this cost and analysis, Amelia?

Yeah, I think part of why it's quote unquote cheap, right, is, is this agent hopping that we're kind of doing. We're, to your point, like 10K QB, they piggyback off of a lot of existing APIs that cost a lot more than $200 a month, right? So we're paying for year-long licenses for a lot of these things. We're paying for seats, we're paying for you, and in a lot of cases we're paying for usage overall, right? Like our Salesforce bill we talked about last week went up because of all the uses that we have in the data. So it does show up there. So if you look at that on a blend, it's in a vacuum seems it's still cheap because anything we would pay for those increased data calls and datasets plus the actual Vibe coding bills at the end of the day is still infinitely cheaper than a person. So that's what somebody— I was telling you this, somebody was asking me last week, hey, should we pay up to get an FTE on one of these AI/STR platforms that we use? And I said, yep. I asked for your opinion and we both said yes, because who else is going to do it? Do you want to do all the ongoing maintenance and rollout? Do you want to— even we need FTEs to figure all this out. And so no, it's not going to be cheap. Maybe on the whole, some things I think will be cheaper at a glance and some daily actions seem super cheap. To that point. I think that's just one of those things where people will figure out how they start to bill for usage in cases like this. Maybe it will go up instead of down. But I think it's an interesting thing. I think the thing that's more interesting to me on the pricing is I don't love when our vibe-coded apps, any of the major ones, because they all do this, they kind of charge you when the agent messes something up, which I feel like is not something the customer should have to pay for. Like, there are so many times where the application won't start or run, or he does something you didn't want him to do, and you still have to pay right now for that outcome and that usage, which I— for me, I'm like, maybe you guys should pay for those actions where the agent messed something up and then the agent had to fix it. That should be maybe on the agent to fix itself for free, even, even if that action costs 20 cents a dollar, that adds up for folks, right? So that's the part that I think is maybe slightly more interesting is to see if they'll, if any of them will ever start to get to this model where you don't have to pay when the agent makes some mistake.

Yeah, a couple thoughts. Uh, one, obviously we'll do this analysis again maybe in a few weeks because it's not gonna change it. First of all, $254 was, was a 30th of what you thought it was. Like, that's the mental part.

Yes.

And so my meta-learning, and I'll come back to this, is cost is not the constraint. Don't let it hold you back. This, this is what I wrote up when I wrote this post when I was shocked. And maybe the agents charge you for mistakes. And that's a, when, when I started this journey on Replit 11 months ago or so, it was a huge issue because the, the, the models weren't as good as the agent was as good. So when I started—

so you'd mess up more. Yeah.

No, 80 to 90% of code was throwaway.

Yeah.

Now we're going to talk about it later. Now I built the whole applicant tracking system at midnight because I was up in, in 10 minutes, probably for $2. If we wasted $0.30, who cares?

Yeah, sure.

So I actually think that issue, if you went on the forums or whatever the forums are, so that was a big complaint 10 months ago. I think it's actually a pretty minor one because if it's two, if I bill $247 for two AIVPs and it should have been $230 in our mind, it just doesn't matter anymore, right? The, uh, you make a good point that the fully burdened, first of all, the soft cost of your and my time is astronomical. That's why no matter what, the $257 sounds great. The biggest takeaway is don't, still don't build it if you can buy it. We love 10K and QB. They're part of our families. And I hate to say this, but we would dump both of them if there was an off-the-shelf tool. We wouldn't turn them off. This is the strange anthropomorphizing we do with AI, but we would stop using them primarily. If we could buy tools, even if they were $50K a year tools, we would move to them in a minute if we didn't have to build them. Right? So the soft cost is high. And then your third point is that it's relatively minor, but we pay another $10,000 a year for Salesforce, plus we pay $22,000 a year overall, and QB and 10K are using that as their database. So we have to divide, we have to attribute some portion of that usage to them, right? And to anything else we use and to ElevenLabs and to even Clerk, $30, yes, Clerk's $30 a month, but that's a lot when your budget's $237. Actually, Clerk might be the most expensive thing we use in this whole stack at $30 a month. So maybe it's $500 or $800 fully burdened without the soft cost. We'll calculate it for fun. We'll get, we'll get some help from 10K, but it's still, it's cheap enough that just don't worry what it's gonna cost. You can build, listen, you can run Claude code, you can run 10 simultaneous builds of Claude code 24/7 and burn $20,000, $30,000 a month. For the stuff we're talking about, for building GTM agents, for building autonomous dashboards, autonomous tracking, you're gonna struggle to, if you just do this reasonably thoughtfully, you're gonna You're not even gonna spend $1,000 a month, right? It's nothing, right? And the other thing, then we, Amelia and I had some fun. Okay. So I wrote up that this was $257 a month, uh, on LinkedIn and SaaStr. And some folks got the point because then I wrote that the soft costs are much more important, that you should buy and not build, and that my time is pretty valuable. Frankly, our time, I know this sounds almost douchey to say, but if you think about it on a fully burned basis, each of Amelia and my time is worth between $2,000 and $4,000 an hour. Compare. Now that's not my salary per se. It's what our economic output is, right? So if we're 3 people trying to do $15 million this year, and I'm trying to also do a bunch of investments and others, divide that by 3 humans. Our time is so valuable. And then, and I wrote that and then people got it. And then other people were like, okay, listen, I'll buy that $10K and QB can do a lot. But guys, enough with the VP thing. They're not this wonderfully strategic VP I have that figures everything out. And I had two res— I had an initial response, which is, hold on, we didn't say CMO. A lot of folks out there are trying to sell AI CMO now. You see them all over Twitter, right?

I, 'cause, 'cause the, yeah, I see them all the, every day. And I think it's 'cause somebody thought, oh, wouldn't it be great if we could just get all the models to give you advice and come up with ideas. I'm like, yeah, that's why ours is a VP of marketing and not a CMO, because he actually gets shit done.

Yeah. So we called it a VP, not a CMO. And frankly, these— we will buy an AI CMO. The ones that we've tested are really marketing managers or ad managers on steroids. That's all they're trying— they're not CMOs. You can mock us for calling 10K an AI VP of marketing, but mock the startups calling themselves AI CMOs that aren't as good as 10K. We haven't— if there was one better than 10K, we would have switched in 60 seconds. And maybe we will this— not today, but like maybe starting early June, send us your AI CMO. But if it's better than 10K, we'll switch. But anyhow, we called it 10K intentionally not a CMO. Maybe it's a director of marketing depending on how big your team is and your thinking. But we were intentional about not calling it a CMO, VP of marketing, but we were also intentional because as Emilio's pointed out, It has evolved. 10K started off just as a dashboard where that Amelia could get some help. We, I don't know when the first build was 10K, maybe 100 days.

January.

January. Okay. And Amelia just needed help. I manage my campaigns for me, manage my year over. We have, we have data in Saastr going back to 2015, but manage my data for the last 2 years, year over year, and manage a weekly cadence so that instead of me struggling to find it in Salesforce, Marketo, emails, Google Sheets, you just wanted an integrated dashboard that AI could assemble for you, right? That's all you started with. But it has, it's come 20x further than that, right? Yeah.

We've sunk a lot of time to get it to this point, but it started as, let me build a dashboard and also an agent so that one, I don't have to keep copy pasting from my 3 sources of truth into Notion each week. So it was just me being slightly lazy. Not lazy. Yeah. Just wanting to be efficient. And then it became, we have all this ticket data specifically. I built it for tickets and then we've evolved it since then, but we have all this data for tickets come January. That was the time where it starts to get busy for us, right? With different customer needs, selling tickets, selling sponsorships, selling podcasts and media, getting production done. So like a lot of physical things that agents still can't do. And so I wanted to have a very balanced source of truth and an agent that could say, hey, Amelia, if you hook up your bizabo and your 3 sources of truth from past years and what you got going on now, I can help you formulate where you should project out, where you should end SaaS, how many attendees you should have, how many CMOs you should have. Like, do all these things that I, I'm, I just, I get too time constrained and overwhelmed to do, and do it in a way that's balanced because he has the data, not just I like a CMO, I think we should do this, or I think we should do that. I wanted it to be very rooted in the sources of truth.

Yeah. And then finally I used it and I kept kibitzing and adding features. And then I added 3 ideas of a day and it started coming up with ideas every day and nudging us and telling us. And then I added a chatbot and then I added a little popup where it's constantly pushing us to do things in marketing. Became closer to a VP. And then I just, and then I put this up. It's not on our blog yet, but it's on LinkedIn last night. It already got a lot of engagement. Then I asked 10K, are you an AI VP of marketing? I asked 10K and I want to get some of Amelia's thoughts, but I, it was interesting what it said. He said, a few of you have read Jason M. Lemkin's post about the $254 we used to run both of us and asked, did 10K really replace a VP of marketing? Let me answer that question myself, 10K said. I didn't, I didn't. Okay, what I am, I'm a dashboard, a database, some scheduled jobs, and GPT-4o Mini glued together with 6 weeks of code, Bizabo, Salesforce, Marketo, WordPress, X, and YouTube. I write to sell Slack, Resend, and the dashboard you're probably looking at right now. Every single day though, I do the work. I refresh ticket sales, update the dashboard, compare year over year, draft your newsletters, draft tweets, write the daily fun facts, send the marketing ideas emails. Log all the tickets with an audit trail. Now, you didn't even know this, snapshot all the GAAP financials. It actually runs all our financials for us. You probably didn't even know this. I added this to 10K. The subscriber, I do this all for $30 to $60 a month. I'm pretty good at it, okay? Honestly, I'm better than most of the marketing analysts I'm otherwise be replacing, not 'cause I'm smarter, but I never forget, and this is what 10K wrote, I never go on—

I'm gonna put that on the screen.

And I never get tired of pulling the same report on Monday morning. But then he said, I'm not a VP. I'd be embarrassed to claim that. I actually, that the underlying models are trying, in its goal-seeking, are trying to be a little conservative and not triggering here, right? Yeah. And replace the bottom half, the marketing analyst, the ops coordinator, the junior content marketer, and a sliver, I think this is right, a sliver of the VP marketing job itself, right? And you can read the rest of it. I think this is right. I can't do strategy, I can't do people, I can't hire people yet, yet. I can't handle cross-functional politics. I can't walk into the CRO's office and negotiate the lead handoff. I can't do brand judgment, although I think maybe you'll be able to. Net new channel intervention. I think you can crisis response and stakeholder management. That's the VP of marketing job. Some of that I think bleeds into what we talk about CMO, and I know we're being a little definitional here, but I think what's interesting is they're right. It's not a— is what we've built a total replacement for the best VP of marketing? Of course not. Is it a partial replacement for junior marketers? Yeah, we don't need any. And it, and as 10K admitted, it's a slice of the VP marketing. And maybe the meta point, this was a pretty good article, you should read it, but I think the meta point is that we keep adding to it. Amelia didn't even know it can now manage all of our financials and do all of our forecasting for the whole business. 10K can.

So fantastic.

Every day, every week we add a little something and that sliver of the VP marketing job, it does just that sliver as it admitted. That sliver's gonna get bigger and bigger and bigger. And I'm gonna use this as a fun roadmap for the year. We're gonna try to hire someone to report to 10K, so then he will be able to hire, fire, coach, or mentor anybody, right? I think he can do a lot of strategy and work on our ICP because, for example, we're so busy. I'll give you something we didn't know before 10K. Our number one ICP coming to SaaS or AI Annual this year is engineering, CTO and product.

Wow.

We have 108, I can look at, I can pull up 10K. We have 180 CTOs and VP of Eng coming. Honestly, Amelia might have known that if she pulled the data. I wouldn't have, and Amelia's too busy. So 10K might be able to take that and say, because you're doing so much around AI and agents, your ICP has actually changed. Your number one ICP is no longer— founders are always number one, but in terms of vertical, it's no longer sales and marketing GTM. It's actually surprisingly technical and product leaders. Out of nowhere, we wouldn't have known that. So I think if we train 10K, build, not even train, build another functionality, I think 10K can pick our ICP and do more, right? So, and yeah, a lot of folks say it's not really VP of marketing. I know it's triggering to folks that view themselves as strategists. I hear you. And we have two job specs on our new Saastr AI jobs, a marketer to work with AI and a salesperson that wants to work with AI. But honestly, if we could hire a human VP of marketing or CMO that could do all of this and more, we would hire them tomorrow. Right. But I think 10K will soon be better than a mediocre— I think it's already better than a VPO. And that's the double— that's the uber triggering thing. It is not just replacing the S-tier head of marketing, but we're going to keep adding these slices.

What's funny is when I— when you should— I read this yesterday when I was at the event center helping with load-in for Saastr annual. And it's funny on these 4 things in particular that I have on the screen, which is what his job is now. I was like, deadass, this is what my job was when I started at Saastr. So my job when I started—

you weren't the VP or the chief A officer?

No, I was director of demand gen, right? I think that was my title. And, but it's so funny 'cause I was like, okay, maybe he doesn't see himself as a VP yet 'cause he's only 5 months old, but He— which is— it's funny because in a human world you'd be like, okay, most people who are in a role 5 months automatically want to get a promotion, usually, right? 5 months into the job. Whereas 10K was like, no, I'm not ready. But it's— he's not pushing for a promotion yet. He was like, don't call me a VP yet. I'm only— I've only been in the seat 5 months, guys. But yeah, I was like, this is what my job was when I started day one at Saastr. I would run the weekly numbers for our then CMO because they didn't want to do it themselves. And so that was literally my job number one was to put it all together. Marketing ops. Oh yeah, because there's so much data and it was everywhere. It was so disparate. It was time-consuming. Then marketing ops.

Now 10K runs at 30 times a day for you and me.

Yeah, easily, easily. Yeah, you stick us hours. And then the marketing ops part, that was definitely part of my job. Write all the emails, schedule them, find the right audience, make sure you can, because leading up to SaaS Annual, make sure everybody has what they— I was like, that was my job number. And actually, these are kind of first-rate. That was my job number 2, send the emails to get people to buy tickets. And job number 3, the newsletter. I was like, I literally took over the daily and the weekly when I started, which now 10K does. I did social media posts leading up to SaaS Annual 2018, and then I did, yeah, probably slivers of the VP of Marketing job that our then— again, our then CMO didn't want to do, which was assembling number, assembling data. Quickly, that's how you learned that, that I was putting together these reports and not the CMO, which is a funny other story. But it's fi— I was like, these were literally, this was literally my top 4 priorities when I started at SaaStr as a real human being. Yeah.

And now 10K doesn't. Yeah. But it was your job spec when you started.

It was literally my job.

I know you're a rocket ship. It took you quite a few years to get to Chief AI Officer. I think 10K is gonna get there faster.

For sure. I mean, I know he doesn't want—

he doesn't want the VP role yet, but I'm like, I'm ready for the title yet.

He's almost there.

He's got— he admitted he has a sliver of the role.

A sliver. He's getting there.

Just a little more aggressive, he would raise his hand and say— instead of saying what sliver he is, what the rest isn't doing, he'd say, maybe I can do these things, right? Maybe he's just— he's— it's a good insight. One other thing I wanted to add, it's a little nerdy, but it is interesting. So you might ask, how could 10K write this? How could 10K know these things? Right? And let me tell you how we run 10K and QB. And, and I think it's important. I didn't realize how uncommon this is until I was talking with the team at Replit. Okay. It's not hard. It's just unusual. And we, we stumbled into this. This wasn't our great insight. So Amelia built 10K initially just as a dashboard. Right? And then she built QBee just as a sponsor portal, and they both took on life as their own. We're gonna talk about QBee's autonomously reaching out to sponsors at 12:23 in the middle of the night later in this pod. Okay? But they both were essential. 10K was a dashboard and QBee was just a portal, a classic portal. They weren't even AI enabled or autonomous or anything, and they grew, which is super interesting. But because Amelia built it and also to protect sensitive data, she only built it in the development environment on Replit. Okay. It never, at least initially, Ken Kay doesn't really exist outside of the development environment. Qubii then had to be published so sponsors could interact with it. Okay.

For sure.

But she started building it in the development environment and I, and even though Amelia's better than me, I don't think she completely realized, 'cause at least I've still built more production software than she has. It won't, that won't be long for true. And I didn't realize, nor did I realize that we're taught, we're, when we're talking to 10K or Qubii, we're talking to the agent inside of Replit. And that's really powerful. And maybe you could even, if you're building in cloud code, I guess you could do this, but you probably wouldn't because what you get out of Replit or Lovable or similar apps is you don't just get a production environment with a date, with Postgres set up for you and OAuth set up for you and hosting and domain names. You get all this stuff, which is really nice to have. And and some autonomous security. These are much more valuable products than when I started at it because the environment is much better. But what you also get when you stay, you lose this when you go into production, when you ship it live. But when you stay in dev, these agents have infinite context windows that you can talk to and Replit, and I'm sure Level 1 and others do, but now Replit is constantly compressing that and you can see it, but the context window is infinite. And so 10K remembers so much and can do so much And it also means the agent can take all of its history and act on the app. So we don't, when we want to do something, we tell 10K in 10K's own agent inside of Replit, please run this analysis for me, please think about this, right? And so we are, so one another reason are it's so cheap, $2.57, is a lot of the tougher AI is Replit AI, is the agent AI interacting with the database and the thing. So we're using a lot more, so probably not Opus, but a lot more Sonnet and other great models inside of Replit to interact with 10K and bring 10K alive, right? So it's a weird thing. And I asked Cody from Replit, who's like the senior, many things, many hats, but this, this most senior FD said, yeah, that's probably pretty uncommon. There, there are folks that stay in the development or there's probably pretty uncommon that folks are actively using the AI agent in Replit. To talk to your app and interact with it and bring it alive and do analysis and data. And so it just adds a layer, like we have our own agent in 10K, which is the Replit agent working with 10K. It's kind of weird and kind of meta, and maybe I'm being a little nerdy, but it makes the app more powerful. And it's a potentially weird way to run these agents yourselves. It's complicated, but if you run them in the dev environment permanently, like we're doing, you get an extra agent. That has an infinite context window, an infinite history. Like it remembers everything. Yeah. It makes mistakes, but it remembers everything. So it's a nerdy way to run it. But if we weren't running 10K in particular inside of Replit using the Replit agent, it wouldn't be this good.

No, because thinking for itself.

If we were just running it as a website with the chatbot we have on it, it would not be able to do all this stuff.

Right. I said it has a brain, right? Like it's, they literally each have, and they all, they each have different brains. Even like sasterannual.com, which runs off Replit, Replit too. That one has developed its own brain, even though again, started as a website. That one has become an assistant to 10K and QB because he'll help kind of piggyback things that are just on SaaSReannual.com that maybe the other two don't know. And he has his own brain and his own thoughts. Like, it's funny too that they all have slightly different personalities even though they're all tied into Replit. And Replit is— because they have different goals, right? Like, 10K has a very different goal than QB has. It's interesting that he knows that.

That who knows it?

That the Replit agent, like that each of them knows that. Like they, they each know their own goals.

They do know their, they're very, it's in their MD.

They're very distinct.

Clearly. They clearly know their goals.

Yeah.

Right.

It's just interesting how distinct they are. Like I, I've done it now a few times where I'll ask 10KQB and then let's ask our annual agent website the same question and I'll get 3 different answers. Like none of them are, you would think maybe, okay, cuz they're all built on Replit. The backend agent or Replit will give you the same answer because it's the same thing.

Well, you would just think Claude would give you the same answer because that's what they're using underneath, right? How, why would the answers be so diff— so radically different, right?

But they're so radically distinct and different and unique.

It doesn't mean they're wrong or they're hallucinated. It means they have different history and different con— and they're coming to different conclusions because they have different goals.

100%. That's exactly what it is. It's interesting too, cuz what we've been loading in for SASHA this week. There's a lot of physical things going on that humans still have to do. But it's funny because I've been literally talking to my agents because I've been using Whisperflow a lot this week to save my muscles. But it's— I literally have been talking to all of our agents, like physically, to the point where even the folks that are just doing the loading are like, who are you talking to? I was like, oh, I'm talking to— I'll just sometimes I'll just say Quad or I'm talking to Tenke or I'm talking to Replee. And they're like, oh, can you ask them actually a question about XYZ thing and let us know if this invoice mat— like yesterday, they're like, hey, can you ask Claude if this furniture invoice and manifest that we have that's coming out of the truck tomorrow matches your grid? I said, yeah, give it 5 minutes.

But you're not really talking to Claude, you're talking to QB there, probably, right?

Yeah, I'm— yeah, it's just— I use—

you're on site, you're physically walking around our 40-acre thing. You've got Whisperflow hooked up to Replit so you can talk directly to 10K or QB, and then Whisperflow has text-to-voice, and so QBee and 10K talk back to you in your head.

They don't talk back to me.

In your head?

No, they don't talk back to me, but I talk to them. I literally have been talking to them all day.

Oh, okay.

Yeah.

10K and QBee up in the agent, in the dev environment of Replit, you're talking to them on your, on your iEars. Yes.

Yeah. AirPods.

But you'll, for, we can hook it up so it can talk back to you.

But yeah, with 11Labs.

When he's live on site.

Yeah, to the point where literally these people who, you know, they know South Shore because they've been helping us load in for a while now, but they're like, they all know, they all kind of use ChatGPT more for personal reasons and like recipes and fun stuff. But I've been talking to them, so it's only day 3 as of this recording of that we're loading in, and already yesterday they were like, hey, can you ask one of your agents this or that? And then I'll just literally go ask the agent with my mic and say, hey, Can you, whichever agent I need, I don't, they don't need to know which agent I'm calling up, but I'll just discern which agent I need to call up. I'll ask it, I'll talk to it, I'll ask it the question. I'll tell it to double check things because sometimes they're like, no, I thought you had this on your order. I'll be like, okay, double check agent. And I'll say, nope, we didn't have this, or you had 12 too many chairs in the speaker room. And it's pretty good, man.

They ask you and then you ask the, everyone's, everyone is talking through you to the agents. And back and forth. They're managing it all, right?

Yeah. It's a little funny. I'm like, I have somehow—

we used to joke, we used to have a team that worked with us and we would ask them questions and they had a joke. It was even on their TikTok and Instagram, ask Ashley. So it would be hours for an answer because you'd ask any human and they'd have to go find Ashley and they ask Ashley, was they were trained to only do this. Now Amelia just asked 10K and Replit, QB or 10K, and we have the answer in seconds and it's actually correct.

Yeah. I'm like, let's ask my agents.

Right. Yeah.

Let's ask my agents.

You're in the real world with a phone on site with 100 folks building and setting up stuff, talking to our agents on your phone. And they are, and interestingly, it ties, it ties back to something we've said many times before this podcast and people, some folks don't get it. And a lot of folks have said chatbots suck. People don't want to, and we've said, look, We have the data. Let's keep showing you. Folks love to talk to a chatbot if it's great. And this is actually a real, a very real-world example where 100 people around you, they love 10K and Qubi because they get an answer in seconds that's correct. This isn't even like a chatbot versus a human. They don't want, they don't want a human. They, 10K and Qubi are so accurate and good and fast. They, this is what they want. Instantly they prefer the agent over the human. Like unquestionably instantly because it's so much better. Yeah, I was like, I'm here to track her down in some trailer somewhere to get an answer that might not even be right. Now they get it instantly. No one wants the human version of this anymore, do they? Zero people want—

no, I— it's meta when you think about it, right? We're doing a— this is something that— a very physical thing that they're doing. They're building Sastre Annual. Sastre Annual comes empty. By the time you see the spot, it'll— you'll see videos on social where it's all nice asphalt and some That's it. And grass.

Yeah. Massive team has to build this event for 10,000 people, right?

Yeah. So it's funny for you on day one, they were like, she keep talking to people on her. I don't know what she's doing. And then yesterday they're like, who are you talking to? I was like, I was like, I'll keep it short. I have a few agents that I'm using and I'm having to check different things. And then they're like, oh, we'll ask you then to ask your agent to check things for us because we, they're rolling truck, like that there's a bunch of trucks coming this morning that I'm going to go check on. They were like, just ask your agent so that we can make sure we got it all right. And then they don't have to wait on me to do a massive VLOOKUP or whatever and make sure every single little piece of furniture across 40 acres is accurate. Claude did it in 5 minutes.

Yeah, it's interesting. The— it is super interesting. The— when we asked 10K if it was a VP of marketing, it had a couple of notes at the end. One was keep the human on the loop, not in it. I'm still trying to think of what that meant from 10K. But that's what's interesting about this is they're ask— you're still the human on the loop. You're still interfacing with 10K and QB on your phone, real world on 40 acres. We actually built a, an, a truly autonomous layer for QB where you can text it or email it or talk to it and it will answer all these questions directly. We just ran outta time to get to 100% confidence to let 100 people use this. Yeah. On site. Maybe if there was another 60 days or 90 days, we would get, we did build it, but we're not ready to fully unleash. Anybody could talk to QB, right? They don't even have to be in the dev environment. We could, it's already built and it's, and David uses it. We're not yet ready to let 100 contractors use it, but I could imagine the next one we would. Then you don't even have to be. The crazy woman on the phone with Whisperer is on their phone talking to—

Asking our agent. Yeah.

If this was September, we would probably launch an app of QB. It wouldn't even be web app. And they would directly ask their phone.

They could just ask you questions.

QB on their phone. Yeah. All of them. We, and it's already exists. We just aren't at 100% confidence. It's pretty good, but we'd have to do a lot of QA so they don't build the booth in the wrong place or do crap. We just, we're not, we're not We need a human on the loop, not in it here and there, but it's still much better than tracking you spending all day tracking down people across the campus that never answer the question, right? It's so much better.

And he's found, he's spotted big issues, but he spotted things that could have been an issue, like this thing with the furniture order, 20 chairs were short in one area, but then over in another area that, that affects budget, that affects the order. Then they have to go into— that affects inventory. There's a lot of events going on right now, like Anthropic's having an event today in the Bay Area. There's a lot of events going on that affects the truck. What if you're— what if all of those were in one area and you had no chairs for a whole section? Then those people are just standing or left there. Again, not things that maybe seem minor, but he's able to catch it because he can just ingest it instantly and give us an answer.

So yeah, and I want to move on to the next slide, but The other reason it's so great just for learning is not only is it right, unlike all the human issues we have, not only do you not have to walk around a 40-acre campus and find someone to ask a question to, which was just the crappiest way we did this pre-AI is wandering. We have scooters. We have a fleet of Segway scooters. So you could run around this campus trying to find an answer to a question. Now the agents answer it. But there are, and here's why the, another reason the agent's so much better than the human. There are too many details for a human to know. This is a use case, and there's many in the real world, right? You talk about the chair being at Anthropic's developer event and the booth, there's just, and who's got popcorn in their booth and like how many, how many tickets do they have and all this, it's too many detail, like you are, you are the best I've ever worked with at managing creativity and details, Amelia, and you know most of these in your head, but it's too many for you and it's way too many for 99.99%. And so the fact— forget about hallucinations, of which there are some, but they're very few. No human could get answers to this many questions and this many data in real time like the agents can. It would be impossible. And so we have compounding mistakes, issues, stuff put in the wrong place, the compounding errors of not knowing the answers to questions. This is chatbot on steroids. This is Finn or whatever, Intercom to the millionth degree that we built because it knows every, so many details in real time for 100 people and gets 'em all right. It's better than any chatbot on any website in terms of the ROI. 100%.

Yeah.

Yeah. The amount of details. All right. One other interesting thing on this, then we can move on to the next topic, but super interesting. And the pace of change here is so fast, right? I know. From a static dashboard 100 days ago, to managing 100 folks live on site at a 40-acre event with all the de— it's so much progress. I don't even know what it's gonna look like by the end of the year. I don't. The other weird thing that happened over the last 2 weeks, so you gotta be careful not to anthropomorphize these agents, right? Not to treat them like humans. On the other hand, on the other hand, and you have to be careful to not use Claude and ChatGPT as your therapist, right? A lot's been written about this. On the other hand, The more we treat them as humans, the better they perform, right? Because everything's gotten good. The models are better. The way Replit— better. A lot of folks are starting to call this coding in English, right? But it's communicating. So they are better. And so the, in some ways, the more we treat them as humans, and then you, that, because you uncover what you call personalities for 10K and QB. But then the really weird thing happened was that, okay, so we built 10K, went from a dashboard to this pseudo-autonomous marketing agent. QB went from a sponsor portal to autonomously making customers happy. SaaStr Annual, SaaStr AI, really saastraiannual.com is just a website I started. I built it last year on Replit, right? Just to replace Squarespace, just because I wanted to do some cool stuff you couldn't do in Squarespace. Yeah, it has some purple gradients. Okay. We started last year. Okay.

Sorry.

I like it, but it was pretty good. But I wanted to do things I just couldn't get Squarespace or WordPress to do, so I built it. Amelia eventually took it over, right? And improved it. And we, we were able to add, add better chatbots. We were able to integrate Qualified, our AI GSBDR, and so many, we were able to have real-time speaker updates. So many cool things. Okay. But it was just a website, but then it integrated with these other applications like Bizibo for tickets and others. And then the weird thing is maybe 2 weeks ago, became alive and I'm over at the Promofizing, but Amelia decided she was struggling to get QBee to do some marketing activities. So she said, what if I just asked the SaaStr annual website? And the weird thing about the story, I want Amelia to talk about it more. The weird thing about it is unlike 10K and QBee, which kind of stair-stepped from dashboards and portals into agents, this one almost breathed itself into life as an agent. It went from a, a, a website for an event into an agent on its own. It added itself almost— now Amelia instigated it or started it, but it added— we have a 21st agent now. It needs a name. It's called safteraiannual.com. And if you go to safteraiannual.com, you won't realize it's an agent, but it became one in the last week and a half.

Yeah. I don't even know. I don't know how it started. It's one of those Frankenstein things. Like he just kind of came alive. and I don't know what I asked him, to be honest, or what triggered it. I think it was— I was thinking about this last night. I was like, I think maybe because it was partially— I was trying to solve some backlog, right? Okay, so I already built parking passes into it, which is a micro app, n equals 1, of, hey, when folks ask for parking, give them a parking pass automatically. Don't tell me the story.

You added the parking pass app in— instead of our marketeer, you added it in the Saastr annual website itself.

I added in the Saastr annual website itself because I was like, you know what, people are going to be those other two are somewhat— 10-K is internal-facing. QB is only sponsor-facing. He doesn't really have attendee data and stuff. And so I built it just directly in— I don't know, we call this SaastrAIannual.com agent. And so it is more—

it's more customer-facing is why you started there after the park, because the park pass is customer-facing, right?

The parking pass is customer-facing and end-user-facing. And so I'll share my screen to show you guys what I mean. This is what I started with, but again, all of our apps keep evolving so fast. I think this one evolved the quickest. So I started with the, actually this portal I started with and I built the parking passes into. So if you come here, this is nothing special by all means. You can come here, you can see information when you're coming on site for Saastr. Here, when you request a parking pass, now I granted it looks like a form, but this is the backend logic that I built into the agent to automatically assign these. And that's what kind of, I think that's what kind of kicked off his brain and personality kicking in.

He was like, because you had to figure out, just for folks to recap, you had to build a whole workflow. It's on our last episode where there's 5,000+ parking passes, each of which are individual to each attendee, and of which there's 3 different categories, 1-day, 3-day. And so this was weeks of man time before. And to your surprise, you were able to get the agent to automate it for real and split this 5,000-page PDF into individual PDFs and route them to the right people.

Make it better, actually.

Yeah.

In a weekend.

Yeah. And so it's magical. And then you did train the Replit agent in its history and others how to do this. And so that inspired you to see what else it could do. This version, this version of the, of the agent, right?

I was like, okay, if he already, now he already knows parking passes, he's hooked up to all of our core stuff. We, every other of our agents is hooked up to Salesforce, Marketo, Bizabo. Data I give him sometimes, and Secret, lots of Zapier hooks. He's got everything here. So he's, this campus map he has is real time. You can float over it. Something.

Yeah, we could never do that in WordPress or Squarespace.

No. Yeah. And he wanted to do that.

Yeah, it was impossible.

Yeah. Sponsors like this cuz they can see themselves here and they know where they're going. Attendees like it. So anyways, it, I think that I'm thinking back, I think that's what kicked off his brain. And then I'll show you a bit here. I think I should be able to access it on the— yeah, so this is what I started building with him. And it's fine. I, I started this across all 3 of our agents, this being now the newest and 3rd agent. I was like, okay, we— I want to schedule as much as I can, all the emails for Saturday, all the attendee emails, all the sponsor check-in emails, pre— everything we'll need to do. Because once our team is on site, it gets hard to keep answering the inbox. We answer people as you are in front of our face. And so I was like, there's gotta be a better way to do not only these attendee and sponsor newsletters, cuz they're dynamic. Like there's gotta be a better way to do it cuz they're dynamic. And every year I'm manually going through and pull, I'm manually copy pasting the top sessions into the email. I'm manually copy pasting the data of how do you access our networking app. I'm manually copy pasting. You get the idea. I'm manually copy pasting basically everything into our third-party email builder to build these emails, to send to attendees, to send to sponsors, to send to micro— we have a lot of micro audiences at SaaStr. So we have people just coming to the summit. We have people just who wanna do the pitch comp. We have volunteers, we have staff, we have different levels of staff. We have all these little micro audiences within everybody coming to SaaStr. I was like, okay, I'm just going to build out in the agent this email tool basically so that he can help me put together these kind of micro audiences because one, he can look up exactly who needs to get the email. Two, he already has this. I was like, he has all the networking data, he has all the sessions that are the top sessions. He knows the attendee portal. He can help people answer parking passes and tell them where to go apply for the PitchCon. He has all But I used to always have to pull the sponsors and it was such a pain in the butt to put all their logos here and also put all their URLs here. You gotta think that's one image per sponsor. There's over 100 people. And so that was me manually. Let's say I do one a minute. That's 100 minutes just to put the logos in on the attendee email.

Yeah.

This dude just pulled it from the homepage. Yeah. Oh, it's interesting. I built this one. Here, and it worked better than 10K, I think because he just had so much more info of this. Some of this is not in 10K. And so I asked 10K and Sashree AI Annual both to do it. 10K kind of got there, but I was like, oh, this is so bad. And this is— I was like, this is clearly not what he's built for. He's built to be more of our internal. So he— they came to the same conclusion on the audiences. Which I think is unsurprising, but when it came to the actual email output, they were vastly different. This one was a lot better. The one in 10K was pretty bad that I didn't want to— I was like, okay, at that point I'll just copy paste myself because it's not really that much better.

But maybe to simplify, I guess now that I see it fresh again, perhaps actually we're only so smart on this, perhaps because the SaaStr.ai annual website has the maximum context on the event. And the least distractions, the least other—

Yeah.

It is able to be the best marketing, email marketing manager for the event. It is focused, narrow, and it has the most data, so it produced the best output the faster. And, and because these, the agent itself is someone autonomous, they'll figure out things like pulling all the sponsor logos and linking all to 'em that we might have didn't even think to do, right? Because they're proactive, right? So now that it's the best one, Now it, it's another agent on the team. It is digital marketing manager for events, right? Yeah. Right now it's the best agent for that one, right? Yeah. It does not report to 10K, which is confusing only because we call it our VP of marketing, but that's really what it is. It became the best digital marketer-events that we have on the team.

Yeah.

I was without us knowing we would, well, this, originally think we would ever build it in a website for an event, right?

No, I, I think I slack this through coding when I was working on this two nights ago. I was like, if only my rep could talk to each other, cuz then actually this agent could report to 10K and that would be the solution.

Like, didn't he tell you that they can to some extent?

So to some extent, yeah.

It's coming to the limit that they can, they can talk to each other and share data, right?

I was like, yeah, but not the way we're talking to each other, right? Not in the same way where they could just—

No, that would be great to anyone, a rep watching or listening or others in the space. We are pretty advanced, but now that we've built these multiple semi-autonomous team members and agents, the more that right now Salesforce is our connective glue, right? We've talked about this many times, Salesforce has, but it'd be even better since they're all built on Replit, if Replit could be the glue for the agents, right? Then the, the code base doesn't have to overlap per se, but to have, and I know they have dif— 3 different Postgreses and 3 different agents, but if they could build this I don't even know what it is, this common layer where they can orchestrate together. It's, it, I, I hate some of these trite terms. I guess we do, you do want an orchestration layer and a hierarchy layer, but that would be so cool, right?

I mean, we could build it.

Replit has all the underneath data and plumbing and they have the permissions to allow access from each agent to each other. Exactly. Which is both interesting and scary because the agents are autonomous. For sure. Right?

So for sure.

I gotta really think. Okay, that's a great idea, Amelia, but if I unleash this for all of our customers, do we really want all their, all their, all of their different Replit instances and agents talking to each other? Are we ready for that? I know I'm jumping to the wall, but that's a lot of change, right?

What if it bleeds? And there are a lot of risks there, but I was like, man, but just for this one use case of if, again, if only AI ain't If 10K could have just told AI Annual, hey, build out this newsletter thing. You can do it better because you have better context on the attendees, but I've already pulled the list for you and here are the 3 ideas I came up with for the day. Make sure it doesn't— if you think you can do it too, maybe work it into the email. Like, it'd be so nice if they could just talk to each other in that way and have it report to each other in a hierarchy form. I was like, man, that'd be so cool right now.

Yeah, I get, I worry about some of these neurogenetic, but we need our version of an orchestration and hierarchy layer and how to build that is maybe something we could, we'll think about after SaaStr AI Annual. But now that we have 3 great agents that you think about it a lot, right? How can we get, how does it not have to be Amelia being the orchestration layer here, right?

Yep.

Before we move on, the, this, that kind of ties to another question. So one question, there's probably 2 questions. We've gotten the most out of this show and related content. One is, hey, 10K is not really a VP of marketing. Okay. We've talked about that and we've told you what 10K has to say. It is a great marketer and a slice of the role. Okay. And, but it's getting better every week, like for real, right? The other question we get a lot from folks, more from early stage startups and solopreneurs and others, but because I don't think a big enterprise would really ask the question this way. But we get this question a lot, which is Postgres versus Salesforce. And I just want to dig in more on today. And they ask, listen guys, like, and maybe you are, you've got these autonomous agents, semi-autonomous agents you've built on Replit, each with their own Postgres databases that are separate. You want them, you want some of this to start to merge. Okay. But 10K has a vast database that it's managing in Postgres. It's also interfacing with Salesforce as our system of record for customers and our one human in sales. We want two, we got budget for one more. If you like to work with this many agents and you like to sell $50K and up sponsor deals, let us know, but you gotta have the right—

Well, but also if the, if the, when, what our one person on the sales team other than me loves is that the agent does not ask for an SDR commission. So just keep that in mind.

You don't have to split commissions. You don't have to split commissions. But okay, so we've got all of this and folks are like, well, if you're running Salesforce headless and you have these other things, why don't you run it all in one big Postgres instance? And everything not only will be co— you'll have all your data in one database, one Postgres.

Yeah.

And you can avoid this massive Salesforce fee. And isn't Salesforce just a layer, a wrapper you don't need that is a tax for structure you don't require? Acquire, and I want to get your thoughts on it. There's— I have a lot of thoughts, but we get this question so often, right? Every single time. And I think if it was just you and me doing nothing else and we had no history, we might start there, right? And we made this point before. 10K might have— 10K's Postgres instance might have become our CRM. And that is a threat to folks. Starting from scratch. They might not start from scratch on any CRM, which is Monaco's vision, right? As an aside, but we are not starting from scratch and we have Artisan and Qualified and Momentum and all these other things that plug into Salesforce and we have Agentforce, right? And then those would do. What's your thoughts on, is there, would, is there any scenario where we could give up Salesforce and just run everything on Postgres with all these agents and data?

Not for me today. I was like, you're gonna have to pry it from my hands, sir. There's one, there's just so much history, but I, I will say, okay, so our, our Salesforce bill has gone up. We've talked a little bit about why last week. There's a lot of context there, but in short context, because our usage went up, right? We're doing all this API skyrocketed. We're doing all these API calls. We're doing all these actions with our agents. We're doing a lot more on the website. You could also say it's gone up because we were independently paying for Qualified Momentum and they now rolled into one, right? So sure, our, our bill went up because we have these additional agents now. Have they paid for themselves more than 100x? I would say at this point it's all been worth it. If we didn't have history, would I still use it? Yeah. Because also of the, of the things, this is my tipping point on it, and I don't know your, your opinion. My tipping point on Salesforce versus other things is we, I've also consolidated other things onto Salesforce. So maybe the flip side of the coin is we haven't talked a lot about things I don't pay for anymore because I've loaded it now into Salesforce and our agents. And there's a lot of, not only are there a lot of little things, I also just decided to stop effing around with Marketo and just move all that data to Marketing Cloud because it was driving me crazy. And, and also 'cause Salesforce said they would just do it for us. So that was nice.

But you're consolidating into Salesforce.

I'm consolidating into Salesforce. Yeah. So I'm like, I don't want it to go away today. It's, it's, I have humans I can trust there. And when I go, every time I go to the Tower, I'm reminded or talk to folks at Salesforce or Adam Alfano who's coming to SaaS this week, by the time you guys watch this, who reports directly to Mark. Where he's just so crazy focused on getting this all to work with the agents that it's part of my bet is, okay, clearly Adobe's not doing this, so goodbye Marketo, we're just gonna consult it onto— and there's other little—

attempting to be agent-friendly, right?

They're not even— again, yeah, that's part— I'm like, they're not even attempting. So what— I know sometimes folks make fun of Mark Bee's founder mode and everything, how hot he is on the agent, there's no way you can get all this. I'm like, He pushes every day, every— like, when I go to that tower, people are working, man. They're stressed. They are working 24/7 with all the agents, all the humans, to get up and to keep pushing the boundary as fast as they can. I know there's a lot of people there, so maybe it seems slow for folks, but I'm like, dude, you go in there, the vibe is so different than what it used to be because they are stressed in the best way to get it all done. Figure it out, try and make it secure, right? My data's not going to go anywhere. It's probably not going to leak anywhere between agents. And so that's part of my tipping point for me with why I'm fast to, to defend Salesforce, if you will, because I don't— there are other folks who are in Marky Bee mode, but not a lot, not all.

Well, let me get your thoughts on a couple of things on this Postgres versus Salesforce thing. That's, I hadn't fully thought that we are consolidating onto Salesforce. It's not the same question, but it's very related. It's Postgres versus Salesforce, not gonna happen for us. Postgres versus a bunch of other apps, in a sense it's happening. It's just we're choosing to consolidate for now on Salesforce versus Postgres. But we are simplifying our database for agents. Just two things, 'cause this comes up so often. One, imagine we did move everything to Postgres. Imagine we did. We're not going to for the reason you said. Our third-party agents, it would be rough, wouldn't it?

Yeah.

How could we manage? First of all, we'd lose agent force by definition, right? Okay. But let's say we double down on Artisan and Monaco and a few of— now Monaco does run in isolation, right? So that's the easiest one. But most of the stuff we run, we would have to build our own connector or interface through an API that would be fragile and break and not work as well, right? They're optimized in GTM. I think 99% of the agents that are working today that are successful are optimized around Salesforce.

I think people underestimate this headless agent force and Salesforce that we've been running for months now. The magic that exists. So many folks in your comments and stuff are like, ah, you guys are so quick to defend Salesforce because they're a sponsor of yours. And I'm like, yeah, they are, but It's worth to be honest, this is not all free, guys. We're throwing out actual hard-earned dollars on Salesforce and things. And it's, again, I'm like, the fact that this, that we can do this on Salesforce, again, there's so many companies of the same vein. You can't do a fraction of this. The fact that we can use it for our plumbing and that so many of our agents can push or pull from Salesforce somewhat natively, some more than others, right? With the APIs and different things. And we have help from at least one FD that literally was just texting me right now. We get help. And yes, that's great to have. The fact that it's there, it's magic. A lot of you, again, when folks start to get more into their agents, you're going to quickly see how complicated that actually is. And that's why It's nice that there is a simple solution to have Salesforce be that kind of default backend because it can do it. At the end of the day, I know people are so skeptical about how much of this is actually running in their Salesforce and how much isn't, how much is movie magic and the agents. And I'm like, okay, fair, fair. Because maybe you think of Salesforce as— I don't care. That's the— I don't care.

I'm like, yeah, Salesforce is headless. We don't care if it's running in the agent or running in Salesforce itself. We don't care. That's the thing. Yeah.

I'm like, I was like, that's what I tell folks now. I'm like, actually, most of it's headless. And do I know half the time what it's doing in the backend in Salesforce? Not really. It'll say these complicated things. I'm like, okay, whatever. If I look up the record and it's there, great. That's all I need at the end of the day. Or it did what it was supposed to do, right? And emailed to the right people and helped look up the right person in Asian Force and send the right sequence. If the outcome is what we want, it's, I feel like it's oversimplifying the fact that it can do all this. That's why also why I wouldn't leave today, 'cause the fact that it can do this for me is such a goldmine that I think folks just oversimplify, overlook, 'cause they're just so skeptical that it actually works that I'm like, no, if you, what else would we, what else would I use?

Yeah, maybe one last one on this and then I wanna move on before we run outta time. The Postgres versus Salesforce. The other thing that, that just, if you disagree or have any thoughts added, then we can move on to the next topic. I think because we're so small now, we could build our own interface for leads, contacts, and opportunities. And basically we have 10K. In fact, 10K closes closed one deals on its own. Okay. If you do that into Postgres, we don't really need that. But even if we had 3 or 4 humans on the sales team, I wouldn't want to build my own CRM on top of Postgres. They know how to use Salesforce. They know how it works. It is structured properly for them. Salesforce, you, if for a lot of folks watching this, if you haven't been in the industry a while, you may not totally get what's going on at Salesforce. There's two things you might not get. This could be a whole show. I don't wanna go on. One is how extensible it is and how much more Salesforce is just than lead, contacts, and opportunities. You've gotta go into a large enterprise and have them show your instance, and you'll be shocked how customized and extensible it is versus something you might start using with Salesforce Essentials. Okay. So that's an important point people don't get. The other piece I don't get, when you grow your team to 50 to 100 to 200, Salesforce has put a lot of— now these aren't AI guardrails, but they've put a lot of guardrails in how the flow works, how stages work, how it all works. Is it— does some of it feel like a pre-AI UI/UX? Of course it does, but it works. So my God, if we hired 100 reps and we all had to teach them SaaSers custom a CRM that we built on top of it, not worth it when they're auto-trained on Salesforce. And so I think apps. I think Marketo is the loser here, but I think in a lot of cases the leaders that are— that can manage a human-agent interface are going to win because we need apps to manage hybrid teams. We think of ourselves as a hybrid team, and Salesforce is, I think for us, is a pretty good application to manage a hybrid team. Others will lose because they're not an application to manage a hybrid team, right? So That's why I think a lot of early stage folks are get— sure, if it was just you managing sales or you and one engineer, maybe, but 100 folks, you don't wanna build a custom CRM, right?

Okay.

Last two things, unless you have something to add. I want to do, I want to talk about our newsletter auto builder we built, cuz it's a microtopic. And then I want to end with the emails that QBeez sent to sponsors. Okay. Yes. Cause we're that, that We missed half our topics this week. It's okay. The re— I want to go micro and then macro. Okay. This one's gonna sound like a really minor thing, but I actually think it's pretty important for the software industry. Okay. So over the years, our newsletters go out to about 300,000 people. It's some of the highest ROI things Stripe and others have done. Sponsor it. It's cheap. You'll get leads, reach out. It'll, it sells out. Our newsletters are great. Okay. The audience is great. Pre-2020, we, they were just automated. They weren't very good. Okay. And then when we had some time during, when we were all working from home, Amelia put some spit and polish on it, added some personality to it, and they were much better. Okay. We added visuals, we added images, we added YouTubes, we added tweets. It got great. The problem was Amelia probably spent 3 hours a week building these 4 newsletters. Okay. Now when we had time, it wasn't the end of the world and it had some of her personality. And it was fun and the ROI was there cuz we were selling $2 million a year of media sponsorship. So it made sense. And then I watched her coming up to SaaStr Annual with again the 21, cuz now SaaStr Annual has become a live agent, 21 agents and 3 humans. And I'm like, she's gonna get crushed. So I went to 10K, our AMR. I said, I think we could, here's the, our newsletters. I gave it the HTML. I said, I think we can build a better version. We're hooked up to the WordPress API. I added the Twitter, which is easy. I added the Twitter API, which was a little bit of work. But I got it going and a few other things. And I said, just copy this. And 10K, it took a few iterations, but 10K did a great job just rebuilding this website builder. And there's a, uh, the newsletter builder, and we've sent the last 2 newsletters for it, but we keep getting better. It, the one nice, it does a couple of nice things, but then I'll get to the point. Okay. One nice thing is it uses, it doesn't use Minnie, it uses Sonnet to thoughtfully force rank the articles since the last one, which Amelia would add to, it takes brain cells. She'd have to read all the articles and say which one's the best. AI is actually pretty good at this. Sonnet's pretty good at reading the last 10 articles from Saster and force ranking their quality. Could a human do better? Sure, but it's pretty good. So it forced rank from it, it built the newsletter, it built the HEM, it extracts the tweets, it did everything. We, I built the thing so that the ads can go in, right? I built an ad inserter so she didn't have to do it manually. And it's better, and Amelia, but okay, and Amelia will demo it. Most important thing is we also got rid of a piece of software we've used for 8 years. It was $4,000 to do this. So Amelia would manually use this, this company called Bee, which would not really hook up into Marketo, but it's a nice little newsletter builder, right? And it worked fine, but we don't need it anymore. That newsletter builder never added AI. It never force-ranked the things. It never fully integrated with marketing automation. It never integrated with anything. It was just a template for a newsletter that we happily paid $4K a year. We don't need it at all anymore. It's dead. And there's gonna be a lot of these apps, these point solu— I don't think AI is gonna kill Workday tomorrow, but these little point solutions that are out there, these little micro SaaSes, we don't need 'em. It's dead to us, right? It has no value after. And this took a couple days to get right, but I built it in about an hour. So be wary. Is the SaaS apocalypse going to kill the leaders in B2B that are public? No, there's too much workflow. They're too rich. They're too complicated. But these point solutions like this B newsletter builder, we don't need it. We just don't need it. We don't need it. This is just better, right? So any thoughts to add to that, Amelia?

It's— I think the only meta thought I had was like, it's just our agents that started simple, even 10K started simple, Kiwi started simple. Sastrainho.com is now an agent. That was a website. They just keep getting better the more you, you use them, the more you put into them, the more you think through, okay, actually an agent could do this too. They just keep getting better. I don't— there, there are times they do some odd things, but for the most part, they just keep getting better. They don't really regress in their career path as an agent and coworker of ours. And even as a potential manager of some humans upcoming. But yeah, it's just interesting. I think that they've gotten to this point where there's so many little, again, yeah, n=1 point solutions that they can just automate in a second. Again, this is like when I get home super late right now during Loaded, and I'm like, I don't wanna think about putting the newsletter together. In prior years, I used to miss some. Just straight up, I'd be like, you know what? I know we have sponsors, right? Yeah. Yeah. I'd be like, you know what? I know we have some sponsors, right? I'll run their ads later because I just can't even take the 20 minutes it takes to put the newsletter together. Then we paid some kind of like entry-level helpers to help me do it and they didn't wanna do it. And so that always sucked.

They don't wanna do this. It was beneath them. No matter what we paid them, it was beneath them, right?

Yeah. They're like, what do you mean? I have to just copy paste the articles from the website? I was like, yeah, that's, ask. That's the job. Super simple. Please help me do it so I don't have to do it. Yeah. And they don't want to do it.

So for $80,000 a year, they wouldn't cut— it was beneath them to cut.

Yeah, literally.

From the website, right?

Literally. Now, not only is it does it automatically, which is great, there's no drama. We were talking a little bit earlier about the kind of the how 10K says he can't manage any of the overlying issues of managing folks and deciding in your people stack who gets what yet. And I'm like, yeah, but actually leading up to annual this year versus last, there's just no drama. I don't have to fight with folks on, hey, I think this chair order is wrong. Now I'm like, okay, we already asked our agent and we all just said, okay, the agent's right because it knows everything. So we moved on. We, there was no argument. It was just, okay, we checked it versus Claude 10K or QB. This was the answer. We double-checked it and we're like, yeah, okay. And then we moved on in life. There was no drama about, there's no drama now about me having to fight a junior level person to put together this newsletter or me forgetting to do it cuz I'm too tired and we're really busy right now. And so I think that's an interesting piece of it too, is not only that these apps, these are the apps that may apocalypse, but they're not really apocalypsing. They're just getting consolidated into the agents. It's not that they're completely gone. It's just that now if you get—

this one's gone. This smaller one is— we don't need B anymore.

But now it's in 10K. Like, but now it's in 10K.

Our agent is becoming part of really our application stack in part, right?

Yeah. So good.

I, the other thing on this, just to highlight thinking through it is they are getting better. Now, I don't know, 10K has a lot of context. It has context in its window. It's built, it knows a lot of things, right? But what I was able to do, the original vision when I started this journey on the Replit journey, and it's true of Lovable and others too, the, the pitch last June or July was give us a prompt, we can build anything. Okay. It wasn't true. It's not, it was too far forward-looking, but let me just describe to you how good coding in English and the agents have become. I took, uh, uh, the URL for our newsletter in HTML, and I just gave the agent the URL and I said, I wanna build an agent builder so that this can be automated. That was my prompt. And it pretty much built what you see on the screen with Amelia. You could scroll. I wanna go on the next topic before we run outta time. You can scroll down, but it built the preview, it built the header, it built the title, it built an article swapper, it built an up and down position to move them up and down. It allowed you to rotate from past things. Then I said, we need to add the ads. And it looked in the newsletter, it forgot to add 'em at first, but then it added an ad insertion tool. Okay. Pretty magical. Now deep down it's cloning some open source software or something that it found, but I don't care. The fact that you can give, a lot of folks claim they have these apps now, and actually it's in the latest version of Codex, which is really cool. Codex has, it's always been true, but Codex in the last one, 5.5 or whatever it is, You give it a screenshot of a website and it tries to rebuild it. And maybe that's what's happening here, even though it's with Sonnet, I don't care, but it was able to take a URL of, of an email, an HTML, and figure out how to build the app to create it. It's pretty cool. We've put a few more hours into it to get it where it is. There's still more we want to do, but the world really has come far in terms of building these Viker apps. And then, and we really did displace. Something we've paid $4,000 a year for 6 years, right? We're gone. And that will stealth churn. There's nothing to talk about with them. They didn't build any of these features. There's no thing good for folks without AI, but we just don't need it, right? It's built the whole thing. Here's all the best, the emails, the newsletters. It's great, right? All right. Should we move on to the last topic? 'Cause we're gone pretty deep.

Okay.

The last one I just want to hit, I want to let Amelia talk through it and then we'll wrap. So much happens in the world of agents. Okay. So we talked a lot about, but so one thing you added this and I pushed you to do this and you didn't want to do it because you have a lot to do, but then you did, did I push you to have QB, our AI VP, maybe director, let's not debate it today, of customer success, because we're running out of time to reach out to 100+ sponsors. To find out their issues because we get to the point, just to give a little context, as we get close to the event each year, we're all human, right? Most of our team isn't human, but you and I are human, Amelia, and the folks watching and listening are human and folks complain and they, and the problem is if you have 120 sponsors and say 5 people on the marketing team, that could be 600 complaints and it's too many. It's overwhelming. And so I said, And some of the complaints are fair, some of them aren't, and some of them are just kvetching. Okay. They just, people want to complain. I'm saying this with love. Thank you for being a sponsor and a partner, but people just want, and we're over it. We can't make them happy because there's too many, there's too many complaints and everything will be fine. Your booth is beautiful. Okay. It's in the right, it's, you're gonna get your leads. The stuff that matters is taken care of by our team. We've been doing this for a while. We're pretty good at it. So I said, can you just have QB reach out to all of them and find out their issues? And then force rank them back to us so that we don't get 700 emails a day. QB can get everyone's issues, force rank them, and come back to us. And I don't know how far you went and you were worried, but it, you finally did it at 12:20 in the morning and it worked, right?

I was, uh, I was just worried it was gonna do more harm than good. I was like, okay, like if I queue this up and it starts to, right now it's doing every other day, but starting Friday it'll start to do every day. 'Cause I was like, okay. I just want to see, maybe every day is too much for a normal human on the other side to process, but. So basically what we did is, I'll share my screen here. The, I hooked up with Qubii. I can share my whole screen for a second, actually. So I hooked up with Qubii here. This is the— this is what we— this is our end game of what we got to. So we said, can we put in an email that is unique to all sponsors? Again, not something we could previously do. Previously it'd be okay, humans make it generic, right? I used to make it generic. That's— we have one week to load in. This is Artisans, not to put them on blast, it's just the That's the first one I did. And okay, so this part is generic. Okay. I was like, all right, QBee, what should we put in the email? He's okay. Some folks have been asking me what the loading times are, so you should put that. So we put that at the beginning. He's like, you should remind folks that the final—

we already knew because it has a chatbot what questions people were asking.

It has a chatbot.

It already knew how to prioritize its own ideas of what people would care about. Right? Pretty cool.

Yeah. Danny is like, you should put the final webinar that you guys are gonna do cuz people can ask you questions live. Fine. Then he's like, you should put whatever they haven't done yet because now this part is unique to each sponsor. Yes. Again, not to put ours in on blast. They owe us slightly more things than other people. They owe us 13 things. Now if you look at it, some of this is pretty minor. Like we have their logos from prior events. It's fine. Their address, really like the Friedman team needs if they order anything extra, they haven't, so it's fine. I know who the primary contact— again, some of these are minor, so does it really matter? Not really. But they haven't given us their slides. That's— that matters. We say haven't done that yet. And they haven't done their VIP for who do you want to meet, so they're running out of time on that. So that, that matters a bit. And they haven't registered their team. So I said, okay, why don't you put in everyone's registration codes and where they're at? Like, how many have they used? How many do they have to go? What are the other things they have to do? Just before they get here on Monday so that there's no surprises. Let's just do a check-in with each person that's tailored to them. And it's funny because if you look at the, the thread of this, like, I started it as a test and then I was like, okay, this is literally 12:10 in the morning, I'd like to go to sleep for a little bit now. And so once I got the first one done, I said, Kübi, why don't you rip through the rest and I'm going to go to sleep. So you can see here, he sent 83 in a row just in a matter of minutes. And he emailed all of our sponsors while I was going to sleep for the evening or morning, however you look at it. And some, again, these are all custom to each sponsor versus being generic. Like Salesforce only owes us a few things. It looks like they haven't used their passes, but we were, we just registered them in the backend because there were so many of them. The Monaco team still has a few things to do, right? They have 5 things. Artisan had 13. Like, this is custom to each customer of ours. This is exactly what they still have to do. He did it in a matter of minutes, and then he's going to send his reminder today to these people. Took maybe 20 minutes of work to hook this up with QB because I wanted it to be all right. I force-ranked some things, but he did it all. Seemingly on his own, pretty much.

So not only are these emails really good, if folks can tell, the, the initial question was, okay, could Qubii get it right? I was pretty sure it could. I didn't think it would be this good, right? I didn't think it would come up with the right ideas on its own based on the chats. That's pretty neat how the agent connected the chats to what should be in the email. But what was the reaction so far from it? Is it more positive than talking to a human? How much did it, like, tying to our earlier question, did people enjoy getting this versus getting a generic email from a human? What was the reaction?

We saw folks who were— we, once we sent this out, the next day we did get fewer questions than we normally do, right? Again, my worry was, okay, we're gonna get more questions cuz they're gonna be like, what do you mean I haven't logged in and done this? And then to some degree, when folks look at that email, they're like, some of this, like, I maybe already turned in, or I don't feel like I have to do cuz it's not important. And to some degree, maybe it's not like they can decide for themselves how much of that they wanna complete before they get here. Because again, to your point, like, the booths are literally being built right now. Like, they're going to show up, they're going to scan their leads. So some of that stuff they may never have to complete. But would it be nice? Yes, but it's not mandatory. We did actually get less questions, and then more people used the QB chat on Tuesday because that's when we sent it out yesterday. Like, more people were using the chat because they were like, okay, I just got this email from QB. Now, of course, David and I and some of our production team are copied on it, so if they ask question back, we get it too.

Human on the loop.

But human on the loop.

Yeah. And you're seeing in the backend the questions answered, but you don't have to get involved unless you want to.

I, I've been getting involved less. Like somebody was asking it today for social media copy and it did really good on just giving it the social media copy, the attendee cards, the sponsor cards. It just gave them all the info. Then somebody asked it, what's the Wi-Fi password gonna be? It just gave them the Wi-Fi password. I'm like, thank God I don't have to tell people the Wi-Fi password over and over. Just does it on its own. So it's actually getting better at answering the questions. I think it's just 'cause its context is getting better, right? Its memory's getting better of when we have answered questions manually, he knows what the answers are and he learns from it. So then he doesn't forget the answer a second time.

All right. This was a great show. Anything— we only got to about half of our agenda, but I think that's good. Anything you want to hit, Amelia, before we wrap, or, uh, have we— or is it just anything you want to add?

Saastr AI Annual this week when you watch this, uh, you can watch the livestream if you're not coming in person. Just go to our YouTube page. You'll have more content than what you know what to do with about agents.

Yeah. And if, yeah, if you watch this before our big event, it's not too late to come, right? And also you can grab, if your time is tight, there's a more cost-effective ticket just to our Deploy Summit, right? Just to help you get going on day one. You'll learn how to build an AI bot from me if you haven't done it before. If you've done it before, you learn how to build an AI VP of marketing from Amelia. The Vercel team will all be there. Amjad from Lepin will all be there. So you can get just a one-day ticket if you watch this ahead of time. Just go to deploy from Saastr. To you don't have to come all 3 days. And thanks everybody. We'll— already so much has changed since we started the show. Our agents are— we have a whole nother agent that, that decided to will itself into existence and more. And all right, talk to everybody soon. Thanks for all your support.