Stop Letting Your Best Model Do The Work

Cursor rebuilt the exact same database 6 different ways, and the costs ranged from $1,339 to over $10,000 on an identical task. And here's the part nobody understands. The win wasn't just optimizing cheaper models. It's really the meta around plan with a frontier intelligence model and then execute with lower range, lower compute, lower cost models. And what Cursor proved is you can do that and get ahead. Every enterprise tech executive and staff engineer is looking at their token budgets and they're trying to forecast the next year and not become the next Uber where you run outta your token budgets in a quarter. So today we're going to examine enterprise AI DLC transformations from the original software development lifecycle or SDLC. We're also going to go into demos of the new Groq bot, that came out. And we're going to go into as well, how do you calculate the financial ROI if you're at an enterprise technology company to present a case for why you should transform into an AI DLC and align everybody on success metrics down to the bottom line? As always, this is the Self-Aligned Podcast with your weekly AI clarity. My name is Robert Ta, CEO and co-founder of Clarity by Epistemic.me, joined by my CTO and co-founder Jonathan McCoy. Today's episode is really concentrated on this concept of AI-native transformation at companies, and people are calling this the AI DLC. Every single pain point that I've heard from prospects and customers, we'll talk about today, of transformations not working well, people not being aligned, experts not being able to scale across the business. And as usual, we go into AI news, some memes, and then what's the meta. So we got a lot of stuff today, and let's go ahead and start with this, which is AI will get cheaper and enterprise AI bills Probably won't. This is a commentary from Brian Murphy, CEO of Smartling, and he's leading an AI translation platform. And so in this, they kind of mentioned Pylon's annual Anthropic bill was on track to rise from approximately $400,000 to $1.4 million when the company crossed 150 seats. The projected increase was not the result of a sudden surge in usage; it reflected a change in pricing. So beyond the 150 seats, Pylon would move from a plan that included usage to an enterprise plan under which tokens were billed separately at standard. API rates. His conclusion was blunt. After years of encouraging employees to use more AI, Pylon had begun introducing spending limits and requiring approval for additional consumption. Pylon's experience reflects one part of a broader shift. Many companies began adopting AI under unusually favorable conditions: bundled usage, introductory pricing, generous enterprise discounts, and relatively limited deployment. As those arrangements expire and experimentation gives way to production, the full cost of enterprise AI is becoming more visible. At the same time, the Underlying technology is becoming cheaper. The cost of inference has fallen substantially. Competition amongst model providers remains intense, and companies have more opportunities to route work to smaller models. And this totally aligns with what we've seen a few weeks ago when Cursor released their agent swarms and a new model economics, where they're actually showing different metas, for example, using Opus 4.8 plus Composer 2.5, where they have sort of a frontier intelligence. planning as the brain, and then lower intelligence models are not as expensive models. They use them as the hands. So in this case, you could see if it's pure Fable 5 to do the SQLite rewrite, it's $20,000. If it's Opus 4.8, Composer 2.5, it's $1,339. Anybody who is a tech executive or engineer who's responsible for token budgets and optimizing this across, you know, Dozens, hundreds, thousands of AI builders or engineers at your companies, you're going to be thinking about this more. Because that one story from this CEO of, hey, we went from $400,000 to $1.4 million in capital expenditures just for these licenses. It's like, oh, holy shit, can't really control that. I go to the CFO, they're gonna rail me, right? Like, that's what's happening. Jonathan, I know we've talked about on the podcast a lot of this Hey, there's all the geopolitics. Hey, there's all this stuff going on around the world. There's the fight for compute and electricity. Elon's making this new gigantic, like, I think he's trying to pump out like 6 gigawatts next year or something like that in Texas. So there's all these things going on and at its core, our customers and prospects, they're all wondering, how do I get everybody to be able to be productive with AI? Because now the costs are running away from me and I can't really prove the productivity. Jonathan, I'm curious from your CTO perspective, what you think about token and cost optimization and the value part of the equation.

I mean, I guess it's all emergent, right? Like things are changing pretty quickly and it's hard to tell exactly where things will be 6 months from now, much less like a couple of years from now. I think on one hand, the productivity equation, I think, is something where we're seeing task landscapes reshape, roles are reshaping actively. And I think we're actually in a period of more R&D investment. We're still kind of like, you know, on the dip in the J-curve. Most organizations are still trying to figure out like, okay, if we use Frontier intelligence, where can we get value? How does it reshape tasks? How can we get more work done? And then what can we delegate to like non-frontier intelligence is becoming more and more of an issue because there's like these models that are not that expensive that you can sort of trust now. So I think there's a couple of things that are just like dynamic over time that we're trying to get a feeling for.

Yeah, totally. The way that I kind of see it is every board, at least in America, I'm guessing around the world too, is tech companies are like, use AI. Right. And especially at tech companies, it's use AI to be able to deliver higher feature velocity and then capture incrementally more market share and also drive down costs to serve. And so there's this analogy that I have, which is if you're a tech executive and you have hundreds, thousands of employees, engineers, builders, just like this CEO, they're all using AI and you have the board being like, everybody use AI, go use it. TokenMax, right? That was like a few months ago. What it is, is it's just a casino. You're telling everybody to go play, go throw in tokens at the casino, and hopefully you get something good out of it. And there's this feeling at the micro scale, which is when you're using it, you do feel productive because you get a dopamine hit every time the Claude code or whatever comes back. But there's a question there, which is, was the thing you made any good that you spent a bunch of tokens? Is this a useful PR? Does this actually land in production? And, you know, leaders like Stripe and Replit, clearly they've kind of figured out, okay, if we have domain-specific harnesses and set up our teams well, get them to trust the AI tooling, then it's kind of less of a casino because we've created the avows, the private avows, as Satya from Microsoft says, is really the moat, your private avows in your own domain. If you create those well and you get the harnesses around the problems that you are trying to solve for your customers and your own workflows to solve those problems, AKA ship more software to solve customer problems as a technology org, then it becomes less of a casino, right? Like, you can kind of think of it like, if I'm putting in tokens and the house wins 51% of the time, and I lose 49% of the time, it's like, well, how do we win like 90% of the time? That's sort of the question that I think every tech executive wants is, please, anytime you use Claude code, it comes up with something valuable that increases enterprise productivity or value. So I kind of like that analogy. I think that's gone over well with the customers and prospects I've talked to. Okay. Question here is there's this tension between now, you know, everybody was like, go adopt AI, go use AI. And now it's like, wait a second, my bill more than tripled. Like this guy's bill more than tripled. And so we're seeing like sort of this pullback. And I'm kind of curious if you've heard of any stories where this has been done well. And the reason I ask this is because me and you have talked about like how Marty Cagan has said, like most companies he's talked to hasn't really figured this out. So could you speak more to that, how this is an evolving landscape and a lot of people just haven't figured this out, this problem of token optimization and winning the casino every time?

Yeah, I mean, I think maybe some non-obvious aspects to like how the token economics play out, right? Like I think maybe the thought model initially was like, oh, automation will save cost and that's going to trickle through to your ROI right away. I think the thinking I'm hearing more of is that's not actually where any enterprises are finding Or many enterprises are finding efficiency. It's in discovering new things that they couldn't do before, where there's actually like additional upside, new task creation, new ways to build products. That's the place where, like, you know, you're seeing more potential for ROI. And that's where I think that when you talk about the casino and like the random reward, it's like, you're not really trying to find like an automation that's going to save the company money. You're trying to say like, what can we automate so that we free up new bandwidth so that we can test new hypotheses? And what of those new hypotheses are actually like showing, you know, gains? And I think there's also like distance from the customer problem here, like Meta Google have reported increases in ROI based on use of AI and ad technology, right? The ROI equation is very specific. There's a customer, there's an ad you put in front of them. Can we do this more efficiently? That's the type of thing where you're close to the customer. Sales, you're close to the customer. Customer service, you're close to the customer. When you start to product management and software development, you're a couple of layers removed, right? And there's always been this equation of like, how does engineering output translate into customer value? That is an unsolved problem. And that's what Marty Cagan talks about is like the measurable outcomes that we're gaining. He says on one side you have prototyping tools like Lovable and Versa. D0 and this kind of thing. On the other side, you have Claude Code and other tools that you're using to actually do real engineering and deliver products. And those are 2 islands of sort of innovation that don't connect well with one another. There's still no bridge that the whole software development process leads to outcome-driven results that AI is clearly feeding into. And I'll go back to what I said the first time around. I think we're a couple layers removed of task decomposition and role redefinition. You're just going to have to push through R&D for maybe a couple more years before these human roles get redefined in such a way that It's obvious like where ROI is. That feels like the moment that we're at right now.

Yeah. Let's talk about ROI. That's something that I think is very elusive to get down. And obviously every CFO wants to be like, what is the ROI of this AI? Anybody that I talk to, they want to figure that out. And actually we've done some work recently for a customer of ours around this. So if you are a tech executive that's trying to build a business case and really understand what's the ROI for moving from a software development lifecycle to an AI DLC or AI development lifecycle. Take a look at this. We built this little calculator of what's the cost of this, what the returns are, and I'm gonna explain that real quick. The real things that really compound are we unlock, like Jonathan said, the ability for people to do more things after automating and augmenting certain workflows. And that increases your best decision maker's intellectual capacity to go drive more decisions towards Whatever, might be product management to discover more opportunity, might be engineering to be able to increase feature velocity towards the right thing, right? And so really here, the thesis in terms of ROI is more people can become builders and contribute at a tech company. Like think about just PRs as a proxy, right? If you get more PRs, and that is some proxy for feature velocity, not perfect. And you say, hey, there's a percentage of that, that incrementally moves the needle on incremental market share capture or revenue. This is sort of how I modeled it is think about how big your team is. So this calculator, we've kind of had it at like 20 or below. So let's say we have an organization of like 500. And the thinking here is you could put in a fully loaded FTE cost of one of your builders. And right now, like in Silicon Valley, I typically see $250K. Haters don't tell me like it's more or less or whatever. This is just what I've heard from enterprise tech companies. And that's what we use as a basis. And we say there's a certain amount of agent runs per day. Let's call it 20. And that gives an inference cost. And then we say, Hey, there's a certain amount of spend that we have to put on the CapEx side for total AI tech stack, right? So coding assistance, platform licenses, seats, API spend, et cetera. And then there's how many people are actually dedicated full-time to running this AI DLC transformation thing. That's something that I've seen at companies is unless it's like really CEO-driven, then this program team is just part-time people. And so you really want full-time people, and that's what this model has. So let's say you have like 10 program team members that are central driving sort of a federated adoption approach. And then there's this realization rate of how much more time are we freed up by automation and augmentation? And how much revenue do we start with? And how much more revenue can we capture through that velocity that I mentioned? So here we just have like a 5% revenue capture conversion metric. So you could kind of play around with this. We'll put this on our website if you're interested, and calculate this business case for your own. There's another part of this equation. So there's like ROI, right? Investment is like the CapEx, the OpEx, the people, the tools, the technology, and then there's money with that. There's a revenue generation aspect of it. And then there's, can we drive down cost? Get more cost efficiencies. And one of the things I've heard from customers and prospects, especially with like hundreds or thousands of people in the technology or R&D org, is duplicated effort. How many times do you sit in a meeting and then you're like, holy shit, I should have seen what this team has done months ago, because we're doing something similar and the Venn diagram is too close. And so we duplicated effort. That's a huge thing, big technology teams. And so you could kind of like forecast, okay, if I put in This reduces some amount of duplicative effort, let's say 10%. What does that look like and what's the break-even point for this investment? And then headcount savings. So this is the OpEx part, right? Like, and this is a sensitive topic, but for everybody out there, you know what's coming and we're seeing this in the news constantly, like layoffs and stuff like that. And I expect more in Q4 coming into Q1 next year. That's usually when these things happen after the teams have gotten some time to figure out who's really AI-pilled and who's on the bus and who's off the bus. Those are the conversations I'm hearing at the leadership level. So you could put this in there. too, for your CFO. They definitely want that number. Another use case that I've seen in particular for the enterprise story is, oh my gosh, Lovable.ai or Replit or whatever, they make cool prototypes, but holy shit, what if we like took one of those or Claude or any of these models and made like a little vibe coding tool for ourselves with our own design system? Can we arm our sales teams to be able to go to a prospect, they need more additive features, and then instead of going to a designer on the R&D team and 2 weeks later, then you get a prototype that looks in your design with your brand guidelines, and then you close the deal, you could just arm that sales team to just go close on their own. And then the prototype they show to the customer could be in the design system that you have grounded in what the brand guidelines actually are. So then if the sales team closes more, which we've seen, I just saw a story about this from one of our customers where they took a $6 million lost opportunity with a prospect. And then within 24 hours, the CEO was able to, with their own tool, with their own harness, prototype something in their taste and design system and brand guidelines. And they actually won the deal and then extended the ACV. So they won a $12 million deal. So it was a lost deal. They came back within 24 hours because of the speed of using these AI tools specific to their organization, specific to their own product and how it looks. And so this prototype ended up looking like exactly how it would look in the product, like not like a Lovable or a Replit or whatever with like different tokens and typography. You know, because getting that stuff is kind of hard, but they made their own investment. And so it's like, holy crap, if you're able to invest, you know, $500,000 into a tool to prototype for your sales team to close deals, and then you could turn around a loss, $6 million to a won $12 million, insane value. It's a very easy thing for B2B sales-led organizations to look at as a tip of the spear for your own ROI case. I would recommend looking at it this way and definitely sales-led orgs. If you're a product manager or an engineer at sales-led orgs, And you're trying to ground your ROI use case, look at enabling more sales. The CEO will love you, the CFO will love you, and the ROI case is going to be very simple. They're going to be like, oh wait, we invest more people or tooling technology costs into enabling sales to sell better and close more deals. That's like very easy to see top line, right? So we put in some numbers there assuming that how many more deals can you win per year and what's your average deal size and what does that return What's your win rate lift based on that story, like ACV lift as well. So turning around like something that was a loss into like a higher ACV win. That's what this model shows. And then down here, we kind of have like some baseline. And this is, there's a lot of assumptions here. Down here is all of our assumptions and all of that research. You could take a look at that. But there's some workflows in the traditional SDLC to look at and then prioritize. So if you're a product manager trying to build a use case, or if you're a program manager trying to drive transformation at your enterprise tech company and you're responsible for transforming the SDLC to an AIDLC, go look at the workflows that your designers, engineers, and product managers do. Here are some, and go put an economic number, like a financial number on it, and then stack rank it. And this should be, hey, can we drive down cycle time? And then can we do that and replicate that success across the organization, dozens of builders, hundreds, thousands, and then get to an ROI that your CFO could really understand based on driving down cycle time. Anyway, just wanted to show this. And so based on this 500 engineering, you know, team and product size, here's like the bear, neutral, bull case. So bear case, just with the numbers I went over, is net benefit in terms of productivity, incremental revenue capture, cost efficiencies, deduplicating effort is about $5 million from a steady state point of view. I think this is really important because when I talk to prospects and customers, when you go inside the organization, there's lots of people trying to figure out AIDLC. And usually what happens is What I'm seeing is there are pockets of AI-pilled people in marketing and sales and product and design and engineering, and those AI-pilled people really get it. But then the rest of the org maybe doesn't, or they've tried these tools and then they have so many turns with the tools and it doesn't generate productivity, so they don't trust it. If you are struggling with driving change and championing change and getting people aligned, really get down to what is the success metric in a way that your CFO can understand. And when you do this and you go through this exercise, I find that it becomes way more clear and cuts through the noise of any qualitative, like, hey, I feel like we should do this. Hey, I feel like if we adopt more here, we'll get more productivity. It's like, what does that even mean? And if you need resources, you're going to have to justify the ask to the CEO, the CFO, or the powers that be at your organization. And here's a little tool that you could use to do that. So drop a comment, YouTube or whatever you're seeing this on, if you're seeing a clip about this. And let me know if you want a link to the calculator to get a business case for yourself. So just drop a comment for calculator and I'll get it to you. Okay. I know I just like spoke a lot, but I'm really stoked about this ROI model. And I do think it is something that, I mean, as CEO of our own company, you know, you always push me to be way more like ROI driven. And, you know, I'm growing there. I'm spreading my wings a little bit. What do you think of this lens? Hey, we are trying to make sure AI DLC, AI adoption works. Like you were saying in the Google case, like ads are proven. There's certain cases that are proven. For a tech team, getting the alignment down to the bottom line, I think is really important. And so I'm really curious, this is the first time I've shown you this ROI model. I'm really curious, like what you think about this from a CTO perspective. And I think that might help other CTOs as well with similar challenges.

Yeah, it's definitely a cool simulation. And I think this is the kind of like, you think about creating a digital twin of your company itself, and you're looking at like, you know, how does the inputs match the outputs? Have we talked about the book The Goal? By Eliyahu Goldratt. This is one of the first books I read on business and it's about called The Theory of Constraints. So, you know, he basically is saying, you know, your job as a business is to like do the things that are going to make your company money. And as an executive in your company, what you can do is identify what are the constraints at any given time or the bottlenecks clearly. And I think that would be the next kind of like iteration or evolution of this simulation would be is if can we actually identify where the next constraint is and then can we see if this AI simulation will help us break through that or like kind of overcome it? I think that's the farther way you get from the customer, the more constraint modeling becomes like the thing that you need to be, to have a predictive digital simulation or digital twin of your organization. And that's what I think is so hard about like predicting ROI is that it requires a model of your organization where you understand the cascading set of constraints. And really like all you can do is take that R&D money and put it to the next constraint that you're trying to overcome. That's just like practically where a simulation meets the reality of like, do more sales, do we make more money, do we get higher ROI?

Yeah, totally. And actually towards that customer world model company world model. Let me actually just share kind of how— I think I made a little image of this. So for folks that didn't hear, Jack Dorsey came out with this months ago from Block, which is, hey, we are moving from hierarchy to organizations of intelligence where you can query, learning, and simulate your company world model and your customer world model at scale. So when Jonathan talks about digital twins, I kind of see any ROI modeling, like financial modeling, it's all modeling. It's all modeling all the way down. down, kind of like turtles all the way down, right? Like, when you're doing these scenarios, you're trying to figure out, this is the business, this is what we know about the landscape, about our, you know, how we operate. And then there's the customer world model, how does that meet us? And this graphic just really shows we're going from Roman army, where super nested hierarchy, spans of control, that concept came from there. And we're now moving to this like world model, customer model-centric era where anybody could contribute to these models and then query against them. So really thinking to me is like the before world is you have fragmented learning. You know, if you have 1,000 people going off and doing product management, design, sales, marketing, they're all learning and experimenting about how do we evolve the business to incrementally capture more revenue at a lower cost. And it's a very simple way that I think about this that like has resonated is the traditional business or the traditional enterprise is human capital shits out real capital. That's, that's, that's business, right? Nowadays, it's human capital plus token capital equals More capital, how you optimize your human and token capital allocation. And the way that me and Jonathan think about it is if you model your stakeholders, your company, and your customers really well, and then make that modeling queryable, contributable, curatable at the enterprise level, then it kind of goes from this where the left side is like we fragmented our learning, right? And on the right side, we have a shared model of how we view the world. And so we can compound things. In this graphic, it calls— we're saying let's compound product. It's very product-centric, but like you can compound marketing too. Like at Clarity, we have a content market fit engine, and I'm trying to get every agent run that informs our podcast scripts or our content clips with social media analytics to be able to inform private evals to then compound our— what we learn from marketing hypotheses. So really good points, Jonathan. Really good points. And those were some of the things I just wanted to add on. The theory of constraints is a really good— just showing this graphic. The theory of constraints really applied to like, what do we know and what don't we know? And what are the constraints? And how do we uncover that? And that's really where you could see higher leverage for the business. So going back to the case here with ROI calculator, you know, in enterprise B2B enterprise software, in particular, the sales cycles are really long. They're like 6 to 18 months, potentially more if you're going for government or higher education, like public institutions. So being able to arm your salespeople so that they're as good as your designers to close deals. And if you shorten those sales cycles and you bring the revenue forward, the business case is much easier. So I kind of recommend all enterprise software companies that are B2B. If you are a product manager, engineer, tech executive, designer, whatever, and you're struggling with like, how do I get AI adoption? Really look at the sales prototyping case. Like think about building a harness, a minimal harness that just ties your design system and the tokens and brand guidelines there with like a lovable type experience to generate like mocks of the product. And you will certainly win more deals faster. And once you win one faster, just like the one I spoke about earlier from one of our customers, they turned a $6 million loss to a $12 million win with such a tool. This kind of like justifies the ROI case for you. And then you could buy time to figure out how do we get cost efficiencies across these like workflows for the SDLC to transform to an AIDLC. So anyway, that's like tactically what I would recommend people trying to drive towards ROI and like champion more adoption of AI. So hit us up if you want to get this ROI model, we'll put it up on our Let's take a look at the Cursor agent swarms. So this next topic, Cursor rebuilt SQLite 6 ways and the costs range from $1,339 to over $10,000 on the exact same task. And here's what it means for engineering leaders staring at their runaway AI bills. So in this segment, we're just going to talk through how Cursor rebuilt SQLite. And for a lot of people out there, we've talked about this before and people show up in the comments and they're like, well, it was trained on SQLite. It knows about SQLite. It's like, yeah, that's not really true. the point of these benchmarks. Like, a benchmark is a benchmark. It's not perfect. I'm just going to say that. But what these benchmarks do, and I'm glad Cursor did this research, is because we're builders trying different metas, they actually experimented and showed the cost efficiencies or lack thereof of this particular meta. Let's just let a frontier intelligence run away and rebuild SQLite. You see Fable 5 at $20,000 in cost, GPT-5 at about $10,000, Opus 4.8 at $5,000. So the question becomes, Can we get the same result using a different meta, which is the frontier intelligence is the brains and the lower cost models are the hands? What Cursor indeed found is you could do this, like Opus 4.8 and Composer 2.5 together as planner and worker here. You can get it down to, looks like, 1/18th to 1/20th of the cost. And this is one use case understood, but this is how it kind of looks. And I thought this was really interesting is the agent swarm concept. One thing I'll read down here that I really like about this agent swarm concept is they talk about specs as prompts. So let me just read this out. Each jump in AI capability— this is from Cursor— has raised the level of abstraction at which an engineer can work. Autocomplete let engineers work one line of code at a time. Early models raised that to a block of code, and agents raised it to a file or a feature. With swarms, the unit of work becomes the spec. For that to work, the swarm has to actually follow the spec, which is what much of this post is about. Taking a step back, right? In all of traditional software development, it always comes down to how good are the requirements? How well understood is the specification? And if you have hundreds of engineers working on it, are they actually following the spec? And so you can kind of akin this to, well, if we had a bunch of engineers, we've got a cat herd towards a specific spec. This is just cat herding your agents with swarms, right? So Cursor says, we gave the swarm 835 pages of prose and it came back with a database. What was scarce in this experiment and what we expect to be scarce in software engineering going forward is the right description of intent. Seen this way, the swarm starts to resemble a compiler. A compiler translates source code down to machine code through a series of intermediate steps. The swarm does something similar with intent. Planners parse a goal into task trees, then lower it step by step into executable work. The difference Is that a compiler preserves meaning at every step while the swarm is probabilistic at every one. Everything described in this post by Cursor exists to close that gap. They actually have the swarm output here, which is really cool. You can go take a look at their SQLite implementation from this 830+ pages of prose. I will say I have never written an 835-page spec, but now it's making me think, holy shit, maybe I should be. Jonathan, thoughts on agent swarming and thoughts on the unit of work becoming the spec? Here with what Cursor proved.

Yeah, I guess one thing, you know, we just talked about Jack Dorsey's kind of like evolution from the Roman army and the hierarchy of organizations. They point to Coase's work on the nature of the firm as a guide to like, how do these agent swarms like actually form hierarchy of the trees and how do they communicate? Let's see, where is that in the article? Coase. The economist Ronald Coase, asking why firms should exist at all, argued that coordination costs grows faster than the work itself. So organizations settle into tiers of bounded units Rather than letting everyone talk to everyone. So this whole thesis is about like how the hierarchical communication system forms in the swarms and how they can like reduce the overhead. I'm not sure I caught that the first time I read this article. That was like how they evolved their model right here.

The diagram I'm showing, just the hierarchy and the information routing.

Right.

So there's the planners here and the swarm splits the context across the tree and they have to be bounded by a certain constraint, which is the context limit. that every agent has, right? So I think that's really their thesis is, okay, if that's the constraint and if we kind of model this with like, how would humans execute on this? And humans have a very limited context window too. And it's also variable across humans, right? So anyway, just wanted to say, I'm not through with the diagram on there if you want to continue, Jonathan, with, with what you were saying, drawing the parallels.

Yeah. So this is all about like complex behavior at scale, how these agents communicate with one another and how to most efficiently break down that spec. into what each individual agent needs to know so that it's doing the right thing for the collective versus some arbitrary goal or some locally defined goal. It's a really interesting way to break down the problem and saying that they're moving through that constraint of the overhead of communication to get this complex spec built across a set of coordinated agents. So yeah. And so Jack Dorsey is looking at the same thing from his organization standpoint. How do I define each node as a human, right? And redefine the firm. And they even mentioned models of termites and ants in this where the feedback From the nodes actually helps to shape the tree, right? So it's not like a hierarchical, like top-down system that becomes the most effective over time. And this is actually a dynamic that's been seen in combined arms warfare, where like command and control tends to like yield, you know, getting information from agentic actors on the front line who then bring back information. You especially see that in like drone warfare today on the Ukrainian and Russian front. This is the way that organizations sort of have to evolve, you know, is that the shape of the tree needs to like push a little bit more autonomy towards the nodes, but it's actually the tree shaping itself where like these The last thing I'll say is these type of swarms are like probably several orders of magnitude less complex than an organism like a human being that has to coordinate cells. And there's probably lessons to be learned about how like cells differentiate, like, you know, incrementally through the embryonic process and feedback for information. And I imagine as these swarms get more capable, we'll be borrowing not just from economics like Coase, but also from like embryonics, like a Michael Levin style. Say like, how does gigantic swarm solve these problems, break down the spec? And nature had to solve this problem of like how individual cells act towards the collective. And it solved it 3 different ways. And those are what we know as plants, animals, and fungi. It's actually, those are 3 different strategies by which this tree sort of like unfolds in order to like exploit resources in the environment and act in the organism's best interest. Anyway, super interesting, super interesting how they break it down.

Let's go into that a little bit because actually for the pod, everybody that's listening to this pod knows, hopefully, and if you're listening to this the first time, me and Jonathan, we built a company creating digital twins of everything. So company world models, customer world models, like the marketing positioning is we build company brains so that every agent run makes the next agent run better. And you could optimize your human and token capital towards more enterprise value and revenue capture and all the things that matter for your business outcomes and eventually get a closed loop around more and more of your workflows. So closed loop around sales prototyping, the first agent run around a sales prototype, make the next one better, a closed loop around your product management Function, right? To pass off specs to engineering or requirements or PRDs to engineering, and then further close loop around how do we take the prototype to the PRD, to the spec, to the PR, and then quality regressions. And then further, do people adopt the thing like that we ship to prod, like we merged, right? And use it. And does that lead to deepening average contract value in the enterprise space or more stickiness, retention, engagement, or you know like referral loyalty? So anyway, backing up a little bit, we draw our philosophy. a lot from Michael Levin and Karl Frischmann's work in computational biology and neuroscience. So when Jonathan speaks about, there's this model of agent swarms, and it sort of mirrors what we see around hierarchies of intelligence and how does information get routed from the front lines of combined arms warfare. And we actually see the same thing when we look at cells. And this is a paper that me and Jonathan talked about on a pod a while ago. We will throw this in the show notes. Aging as a loss of goal-directedness. All of this is about goals. Warfare, there's a goal right here. There's a goal. Business, there's a goal. Software development lifecycle, AI, DLC, there's a goal, right? And so that's really the interesting part about this Cursor article to me is they talk about goals and intent. And we go back to this, this graph here or this diagram here. They're really showing that, hey, if we have one agent walking the entire tree, there's like a context limit. And so therefore there's sort of a fidelity loss on the execution towards a goal if it's such a giant goal. But what if we have this giant goal and then we optimize the planning just like in military or just like organizations of human or just like cells that make up the structure of our bodies and such? Then we could look at thinner slices and ideally the information is maximally routed so that the thinner slices could actually work towards the goal. And And that's like a company, that's like a team, right? It's the same thing. And so they talk about intent a lot and how the fidelity of the intent, the fidelity of the goal being communicated and cascaded through the hierarchy is the most important thing that they found. Like they even say here, we suspect the ability to scale the agent swarm comes from this context efficiency more than the parallelism itself. That's what I sort of read as that intent or goal capture. Anyway, we could totally side quest and nerd out a little bit about the Michael Levin work. I'll just really quickly say here and then tee up Jonathan for some of his thoughts is what we found from this paper, Michael Levin, this was last year or early January. What they found is when cells don't have a goal, they start dying. That's what happens. That's kind of the TL;DR. If there's no goal-directedness, the cell starts aging, which is so interesting. I believe it's been a while since I read this, but they also found that when the goal becomes reintroduced and the cell is directed towards a goal, The aging stops. So that's such an interesting thing, right? When you draw the parallel there with your best people at an organization, if you are a technology leader or executive, your most AI-pilled people, you're the people that are high agency. If they don't have a goal, they're gonna do whatever. And how does that go into enterprise value, right? So if people aren't aligned on the OKR, they're not marching to the same beat of the drum, right? You're not concentrating on the goal. And sort of the metaphor is your company just might be aging. While all these other companies are trying to win and direct all their resources, human and token capital towards a particular goal or set of goals. Jonathan, do you wanna speak a little bit more to Michael Levin's work here and drawing the parallels between, you know, cellular structures to military combined warfare and then, you know, Harkins of intelligence and humans and now with agent swarms?

Yeah, it's a really cool analogy. I didn't see that coming in actually, is that the spec we were talking about in the tree of the cursor agent swarm is the goal, right? In the same way that like when an embryo in Michael Levin's world, like each cell differs. differentiation is working towards a goal. And that goal in the biological sense is communicated through like epigenetics and bioelectric fields. And so that when the cell differentiates, it has no subjective experience and decides, how am I going to help out to the goal the most? It has this information, it has like a blueprint of what it's supposed to do to be able to impact the goal. If you isolate any of these agents in the swarm, like you look at their local context, you look at like, what do they know about the system? How do they think about impacting the goal? And then you have this kind of like these network effects that come at scale where actually like, you know, the cumulative set of agents acting is actually informing the next agent to act towards the goal. So I think that's a really cool metaphor and, you know, something that's like helpful with the shape. Now, in the aging example, like Michael Levin and David Sinclair have both used this kind of theory in an information theoretic way, actually, of how a goal is defined. Sinclair does it at the epigenetic and genetic level of each individual cell, and Levin does it at the organism level, or communicating goals at high level of complexity. And one thing that Sinclair pointed out, if you look at Claude Shannon's observer loop, Which is like, how do I take the communication of a network and then observe like whether or not the input and output of a node is matching the goal specification that we have? There's actually a theory, an information theory that says like the amount of communication that you have to have available to measure the goal. This is the diagram. So like that observer there, you think about, okay, I specified this 835-page specification. It's a goal. I need to monitor the agent swarms. There's a requisite amount of information that needs to be like observed in the system to know and to calibrate the system towards a goal at any point in time. And when they talked about Coase's Law of the Firm and the communication overhead of coordination, something like this has to happen. In Shannon's work, he had measured it specifically, you know, so that channel bandwidth is like specific amount of channel bandwidth available in order for the observer to be able to manage the input and the output of the goal specification. Anyway, these are some of the things like I've been trying to pull together, but, you know, it's kind of cool to talk about in this context where like agentic swarms now are taking on this same sort of like goal specification modality. How do we like measure and manage the channel bandwidth? within the system and minimize it. Those seem to be vectors by which agent swarms and then models as a whole become more effective at solving goal definition and propagation to subagents.

Yeah, absolutely. So fidelity of the goal, making it to all nodes of some orchestrated execution, whether it be on the cellular level, whether it be on the people, human level, or now we're seeing an agent swarm level and probably also extending into the human plus agent swarm level. Coordination layer. So yeah, I think that makes a lot of sense. And it's interesting to draw the parallels between just like nodes and information routing and goals. That's kind of what this comes down to. And how do we optimally do that to some sort of like benefit, aka the goal? So I love what Cursor did here. They say this too. There are echoes of this structure elsewhere. The economist Ronald Coase, asking why firms exist at all, argued that coordination costs grow faster than the work itself. So organizations settle into tiers of bounded units rather than letting everyone talk to everyone. And this kind of, they wrap this up into their thesis at the end, which is that with agent swarms, the unit of work becomes the spec. So for this particular case, right, they spent a lot of time on this 835 pages of prose as a specification to run this swarm against. And you could sort of, what's cool to think about is, If you think about humans working towards a goal, hey, I'm a startup, I'm trying to write a pitch deck and then I get money and then we execute on it. That's like you have a goal, you have a plan, you're trying to bring together a team, you get people aligned on it and you try to go make money or make a product, get to product-market fit. Or on cells, it's like, hey, can we grow and could we have more longevity and stuff like that? Or in military, can we win a war? So it's really fascinating to just see the parallels between nodes, information routing, And goals across different domains. One last thing I'll end with on reviewing this article is Cursor really they say if the planner work split is this dominant, does every serious agent system become a two-tier swarm where it's the meta is one expensive brain frontier intelligence and then many cheap hands doing more of the execution and potentially rote work? And it's going to look like this decomposing work so that every agent's context is small. But there's a good planner and orchestrator with the frontier intelligence. And that way you can get to the goal at a lower cost. And your CFO or CEO is gonna love you for this. Definitely take a look at this. And I'd also say take a look at our AI DLC, like calculator that we had put together. And you could probably start getting to what's the financial goal. Hey, this is a good meta. Let's start experimenting across workflows we have or like features we gotta build. And then actually benchmark it at your own organization with your own harnesses that are domain specific. And I'm sure with this meta, like me and Jonathan are finding a lot of success with the plan with Fable 5, execute with like GPT-5.5, 5.6 sort of thing. And that seems to be working. And Cursor is proving that, hey, when we benchmark it for real too, what you're feeling as a developer, it actually works. So thank you Cursor for helping us with the meta of expensive brain frontier intelligence, cheap hands. That's how I sort of summarize this. Okay, let's move to AI memes. Let's see what we got on deck today from the community in terms of memes. This was really funny. Claude Opus 5 over-engineers simple tasks. That's our first AI meme of today. A viral r/cloudeaithread calls Opus 5 a paranoid over-engineering mess. Here is what developers are complaining about and the workaround patterns that actually work. They mentioned this: a fix my sitemap request turned into a full site rebuild. New color palette, new copy, replaced images, a broken animation. The only backup of the original files deleted in the process. That is the incident circulating from a widely upvoted Claude AI thread posted on August 6th. So that was about a week ago, titled My Opus 5 Experience in a Nutshell. It's not an isolated complaint. It is the clearest public crystallization yet of a pattern developers have been trading in smaller threads since Opus 5 launched on July 24th. Here's the thread itself. Add this to the show notes. My Opus 5 experience in a nutshell. So excited. It's better than Fable. Fuck yeah. It's doing too much. Oh no. Oh wait, what is it? What the fuck is it doing? This graph, when I saw this, I was like, this actually does explain my whole experience with Opus 5 so far. I can't trust this guy. I can't trust this guy to do anything. At first I was so stoked. I was like, wait a second, It's better than Fable. It's cheaper than Fable. It's the same benchmarks. Hell yeah. And then it was like, wait a second, it keeps asserting more things it doesn't know. It lies to me. And I'm like, damn, this is a bad employee. If this was an employee, I would fire them right away. And this WTF thing is so true. Curious if anybody is watching the pod or listening to the pod or seeing this in a clip, is this your experience with Opus 5 so far? If you're an AI builder using Opus 5, what have your experiences been? And does this graph really translate? It definitely does for me and Jonathan. So I'm very curious to hear from the community. Jonathan, thoughts on this AI meme? Does this mirror your experience?

Yeah, it's pretty perfect. I'm curious, did you implement those system prompts on that Opus 5 fix article that you sent me? I did. I have a story about that.

Yeah, let me pull that up because me and Jonathan have been, obviously we're builders and we're always trying to build and get on the latest meta and then share what we learn and get signal from the noise for community. So do, do, do, let me pull up the fixes that we tried to make and me and Jonathan can both relay our experiences with that. Okay. So this is from Powell, the Product Compass. It's a good newsletter. I would recommend people go check it out if you're an AIPM or aspiring AIPM. He writes a lot about AIPM. So Claude Opus 5, the best Opus yet once you heal it. So remember we were just talking about how this is the experience with Opus 5 is like, oh my gosh, hell yeah. And then what the fuck is it doing? So I got this in my inbox and I sent it to Jonathan. And Powell also goes through the same experience. At first I love the Wiz 5 and then I used it for a few days. The more I worked with it, the less I liked it. And here's, here's a little X thread. I'm going to cancel Claude. It's just so bad. I can't believe it. It's just lazy. The most recent example, I have Claude check my inbox for important emails, summarize them, work with them, and send out replies if necessary. I caught Claude again, simply not Wow. And many other complaints, like there are lots of other complaints here. And so Powell is like, hey, you can heal this. And he never switched away from Opus 5. Every time a failure repeated, he wrote a rule that stopped it, then kept working. After 2 weeks, his CloudMD, 8 blocks of the CloudMD survived. So throw this in the show notes, but we could first talk about how's your experience going with this, Jonathan? Did you try this out? Yeah, I did.

Yeah, I did. So at the bottom, he summarizes all the intent engineering into like one copy and paste prompt. And I put that in my user level Claude MD. So not the repo level, but the user level. And I think it's, yeah, it's 10. So that was the actual thing you copy and paste, the whole file. Yeah. And it was pretty funny. Like I actually used an Opus 5 session to like update the system prompt. And then it said, okay, but I'm going to, it asked me to do something. And I said, well, given the new rules we just talked about, What do you think about that? Do you think you can figure it out? And it said, oh yes, I can. And it turned out to be of the form of like, I can make a cheap reversible return, cheap reversible change and see if it works and it won't cost anything. And then I can solve the problem. I was like, yeah, we'll do that. And it did it. Kind of like this trained haplessness that it has. All of a sudden it's like, wait, it had a new tool to like rethink how to do it. So I've seen that in a few of my Opus sessions now where it's just like these little nudges on the intent engineering. And by the way, this guy just did a great job of like working with the model. Decomposing its intent into a prompt that is actually like visibly improves the model. I think that's like a really strong skill that this guy demonstrated, the intent engineering. But, uh, yeah, anyway, the fix, the fix works for the most part. Opus is mostly usable. Opus 5 is mostly usable now with this fix. And yeah, it's still not my favorite model, but it's like practically works, you know.

Dude, it took me a little bit too. I had the same experience where I read this and I took this prompt and threw it in my Claude code session and had Modify the MD, and I have been finding a better developer experience using it where the before was literally like this, where I'm like, I ran a goal loop with 4.8 and it did better than you at 5. Like, what are you doing? With the goal loops I was running before on Opus 4.8, it could go all night and it won't stop. And it's pretty high agency. What I found with Opus 5 is it'll stop every 5 minutes and be like, look, we just can't do this. And I'm like, wait, but we can, we actually planned This, we already researched the state of the codebase. We can absolutely do this. Also, you shouldn't be going back to me with this stuff. It was kind of like intern-level engineer going to me to be like, hey, is this the right plan? Are you sure? And then it's also lying to me. And it's like, dude, that's like the worst teammate, coworker, person to manage ever. It like went from great staff-level engineer I could trust with like a giant autonomous goal loop, your high agency, to like, this thing is just so hapless, as you said, super low agency and kind of annoying to work So if you're an AI builder and you're struggling with Opus 5, here's the fix is this prompt. Just put this into your user Claude.md harness. Throw this into a Claude code session, modify your Claude.md user-level harness. Don't, don't go and change your team-level harness yet. Test this out. Me and Jonathan have found great results from a developer experience standpoint where Claude Opus 5 will just become way more high agency. I think the meta is still though going to be, you know, plan with, you know, a frontier intelligence level thing like Fable 5 and then execute with something else. And this, this makes Opus 5 a bit better of an executor, I'd say, and a better planner. But I, I'd still go with the Fable 5 plus Opus 5. Anyway, we'll throw this into show notes here, a link here and the prompt. So give a holler and drop prompt in the comments if you want me to DM you.

Yeah.

Very cool. Okay, so that was an educational AI meme where we could laugh a little bit at Opus 5 being a little hapless and then also give people some meta on how to improve it. Because we've been through the weeds with it trying to ship production code, like we're shipping production code every single day for customers. And so we're really on it with our developer experience. We really feel it when the model is not good. So this has helped us. Hopefully it helps you too. Sweet. Let's move into demos. So Elon Musk and Grok releases multiplayer AI. Elon Musk and Grok release GrokBot multiplayer AI. This is when agents join your team. Take a look at this. For 2 years, every AI product gives you your own assistant. And this week, Elon Musk's released your team of always-on agents with GrokBot. AI teammates you can give real work to bots. Can sign into your tools, use them just like you do, and come back with finished work. And here's what this means for any AI builder that wants the AI faster and better than the competition. You can kind of see here what this sort of looks like, and we're about to demo this for you because Jonathan was able to get in on this early beta. Some of the value props before we go into the demo, you could message your bots like teammates So you can give tasks to bots like you would a teammate on desktop or iOS. They take projects from start to end, keep context on how you work, and get smarter over time and come back when your approval is needed. You could work with many bots at once. You could sort of see here GrokBot works where you work. So you log into GrokBot once, it uses your apps and websites just like you would. So there's like a computer use. So you could sign into Zendesk, GrokBot could actually work the support queue. For you, you can ask a bot to follow along as you complete a workflow once. It would then save a routine and runs it on its own next time. That's something I'm seeing come out of Codex, and now Grok is pulling that feature as well. Because I think that's like a really easy way to get a more functional user to start automating their workflows. The bots get smarter over time, so they compound their context, and you connect the bots in the same thread, and they could pass work between themselves. The cool part about this when I was reading this is. The bots can use a single computer together versus if you're a human and you need another human to work on the same computer, like that's literally never gonna work. So here's a bunch of bots using the same computer, which is super cool. And their sort of marketing messaging I loved as well. SpaceX AI team runs on GrokBot. I like this. Yeah, this is really cool cuz it's just showing people walking around if they're on their phone and you can get GrokBot right now on macOS as a dedicated app as well as on the iOS For mobile. Cool. So Jonathan, you've been playing around with GrokBot. Let's take a look at GrokBot live as a demo.

I downloaded GrokBot. They actually have a 1-week free trial, which was cool, at least because I had the Cursor like pro plan and I was able to try this out. Created a few different agents over here. The one I'll start with was scanning X for me. And I said, you know, like, hey, can you scan my X content of interesting content? Look for some categories that match my X bookmarks. So I'm like asking it to like look at my bookmarks and predict like what are gonna be new bookmarks, you know, coming from my X feed. And what you have over here on the right is really interesting. This is its own computer. So it has its own like virtual environment here with the Chrome instance. And that's gonna be the difference between OpenCLAW and GroqBot is that GroqBot runs in the cloud. It has its own virtual machine that it works with to do a lot of this stuff. It can also access your machine for computer use, but it's, it's kind of limited. And I'll tell you the things I found about that. But overall, this was a pretty cool case. First of all, GroqBot has a lot of plugins, you know, like the familiar things you're used to seeing like Gmail and Granola, Google Calendar. And the ones it doesn't have, like it didn't have Auto Otter in here. So I asked it to connect to Otter and it handled it seamlessly. And I was like really impressed with how well GrokBot just like lets you connect all your stuff into it. That seems to work pretty well. On the downside, like 1Password, it can't use your 1Password credentials because it actually can't access that in the local system. It's trying to like go through its own virtual machine and access your 1Password and it can't do that. So it actually has a plugin called 1Password that GrokBot can't use. And so those are kind of like little quirks that you see when you're using this new product. Yeah, this did a really great job. It, when it can't use the xAPI, it just opens up its own browser window here. I authenticate into my account, and then it was able to like look at my bookmark folders from there. And then it created this, it actually gives you like screenshots of like what it's doing while it's searching. Kind of cool.

It screenshots the computer to keep you in the loop about like, hey, this is how I'm doing it. So then you could steer, right? How did you find that experience? Because that is, that is really different because this is doing it by default. I usually have to get, whenever I'm doing it sort of any testing, I have Claude or Codex screenshot and put that into a folder for me to review. But this is cool that as it's going, it screenshots for you. How have you found that experience with GrokBot to help with this agentic steering?

Yeah, I mean, it really helps with the coherence of like, what are you doing? GrokBot, it's a little bit like, it doesn't give you the same provenance. It doesn't tell you, think aloud and tell you what it's doing for the most part, but it does give you these visual indicators of like, here's what I'm looking at right now. And you'll see that in Codex too. It has like a picture in a picture. But I found that these snapshots of the picture in the picture, like, you know, is actually like, okay, cool. This is like, we have, you know, we're, I mean, I can understand what you're doing here. Another note about GrokBot is it doesn't tell you what model it's using. It's just like, It's probably the new Groq models, Groq 4.6 or something like that, but it doesn't actually tell you and you can't configure it. So that's an interesting aspect of it.

That is a really interesting design choice. You have to look at that as a very intentional design choice, right? Because every other codec and et cetera, they very clearly show you which model, so you have a bit more agency. I'm wondering if that's just a beta thing because this is early beta and early access. So I wonder if that's just going to come with time. But that's interesting. Interesting as a design choice. Who do you think this is built for, Jonathan Grobbat?

Yeah, I think it's everybody who is like kind of interested in OpenCLAW, but maybe didn't have the hardware or the cycles to invest in it. This makes it really easy to just get started building agents. And I think that's gonna like, whether that means that you can run this in an enterprise environment or that means you can run this independently, I think that's gonna be more accessible to a broader set of people. So a really progressive, interesting design.

Wait, so this use case here, we're seeing that it pulled out Other things you might be interested in. Scanning your timeline and recent X articles for new bookmark candidates in those buckets. I won't resuggest ones you've already saved. Did you find this useful, the output?

Yeah, so this was a great output. So it characterized my bookmarks. So these are like some of my bookmarks: philosophy and perception, evolution science, and so on. It searched through the X feed and it like extracted these links and said like, hey, these are good candidates for you to bookmark. This was like genuinely useful. Probably 80% of these articles were things that I was interested in. And, you know, then I thought like, well, okay, how far can I take this? I said, this is Can you present it as HTML document with preview images instead of the links that you just gave? And that's where it had sort of a shortcoming. It did create an HTML document here. This is a picture that it took, but I have to go into the virtual machine and I have to click on this. And then by that point, like not nearly as usable as my own machine. Had more trouble like creating an HTML file locally that I could browse in my own browser. Those are some of the kind of trade-offs that I was seeing where it's like I was really impressed and then take it to the next level. I can't quite do exactly what I want yet. It's interesting paradigm.

Quick shout out there to Hamel Hussein. I see he came back in your feed. there? Hamel data agent benchmark. Wait, if you go back to the previous screen, I saw it. I actually, funny enough, I was just reading this. Because if you think about it, really right context, right time is about how do we get the right data from the right data source and serve it into the context window? And I thought this was a great use case for us to look at because we're essentially doing that. So anyway, yeah, in particular for that one, I think me and you should read that and then maybe talk to Hamel about it. But yeah, they introduced the whole data agent Uh, benchmark, which is cool. Tell me more about on the left here. It looks like these are all different Grok bots, like daily standup, or are these threads like on the left?

These are threads, but they're also agents. And so I think you can reuse this agent to have multiple threads, but I actually have like a unique agent per thread. I think you can have up to 20 or 30 of them. This, uh, standup is a good example of one that has a routine. And so this one run this at like we every weekday at 10:45 AM. And I woke up, you know, and I came here and it'd already run it and like it did a pretty good job of running the standup. up, which is substantial because like, if you look at, I had to connect Otter, I had to connect Linear, I had to connect my Google Calendar, Slack. So it was able to like just pick up on everything Claude Code was already doing. Yeah, look at all this stuff it does. And, and then produce an output report. Where it failed is I actually have like another, you know, tool that I use, a Node.js script that like creates the walk that we talked about for me to annotate the standup. And it couldn't run Node. So that was like one of the shortcomings is it can run Node on its own machine, but it can't It couldn't actually access the files on my computer, run the Node script. And that was something that OpenCLAW can do that GroqBot couldn't.

Definitely trade-offs in the design here where they're trying to have a sandboxed environment. That's what I see around this is like super IT cybersecurity-friendly way to let your workers use AI and you won't get as many headaches about data breaches or risks. That leads to limitations, which I'm curious about how that evolves Over time with these limitations where you'd like to be able to compile like an HTML on your local, for example, based on some of the work the agent did in its little computer sandbox. So I'm sure they'll figure that out. This is a meta that I've seen with Databricks as well, is going with, hey, we have a thesis that agents probably need some sort of VM or sandbox, and enterprise companies want that so that their IT teams and cybersecurity teams could sleep well at night with that. Thousands of people using these agents connected to the fucking internet. So, oh, you were just looking at plugins there?

Yeah. There's a GroqBot for iOS. This is great. It's just an app on your iOS and it works a lot like the Codex app does where you can view your remote sessions. So everything I view here, I can get like complete fidelity on my iPhone and I can control it through my iPhone too. So that's like the big win. And of course that was like the original OG OpenClaw, you know, cases. I could just use Telegram. The settings are, are very simple. You know, there's not like a whole lot I had to mess with here to get started. I think that's by design, you know, you can just get started. And I can take you through one more use case that I stress tested last night. It's kind of interesting. I had like 5 or 6 PRs open and I was like, can you tell me which Codex or Chat or Claude code session these originated from and just direct me there? You know, you probably do this too, where you just have like all these PRs open and like you have to hunt down the session that you were working on, right? And so I said like, can you just route me there? And so I kind of created this system where I took my open PRs.

Just to tee it up for the audience, so if you're an AI builder and you have like, you know, 4 to like 20 agent Cloud Code sessions, Codex sessions going on at once. It's like, oh, PR here, PR here in this repo, PR, multiple PRs in this repo. Where the fuck? Like, and so you're saying you use Groq bot to help make sense of this and point you down to hunt down the right session and PRs. Is that what you're saying?

Yeah. Straddle across Cloud Code and Codex and my PRs and then like direct my attention to the one that I should work on next.

Oh, that's interesting. So you are orchestrating and then you're like, hey, little chief of Staff Orchestrator, where— help me orchestrate better, right?

That's the use case? Yeah. Yeah. That was my use case. Yeah.

And how'd it do?

What I wanted was something I could, like, would've pulled the PR and I could click on this and then go to the Codex app in the session that I was using. And that didn't work. But what it could do is like, it could route me and it could say like, look, this PR 425 here is in this Codex session. And it could open up the session to the right session on Codex. And I'm like right there. And so I ended up working with it in a way that I was like, okay, I'm fixing some bugs on this one. What's the next session? It would like redirect me to either the Cloud Code Terminal Or the Codex session accurately. And that was impressive. So it wasn't quite the user experience that I initially set out to do, but it actually ended up being a useful tool and a way for me to like just kind of chief of staff, you know, all my AI agents and like navigate to the right session that was controlling a PR, which was pretty cool. It was like pretty successful in that case.

That's really awesome. That's a use case that I wouldn't have thought of, but I definitely feel that pain point. And any other AI builder that has the pain point of like, oh my gosh, I have this cluster of PRs that are ongoing. Going and I'm steering all these agents. And for me, I have like, ready next time, but I have like 4 32-inch quad, like quad monitor setup. So this chief of staff meta of use Groq bot to pass me what sessions and then open the actual session on Codex, Claude Code. I think that's really great and speaks to a pain point that I've been feeling, but I had no real solution for other than just get it together and try to remember. My, my own human context window is limited. You know, so it's cool to have this like chief of staff Grokbot help me find the right session, right PR to keep steering Meta. That's awesome. So anybody else try it out? Let us know what you think about Grokbot, and thank you, Jonathan, for the demo. So there was this talk by Jeff Dean, and just tying into AI news a little bit, Jeff Dean just left Google. He was employee number 30, and there was a lot of news around this, which is: Is Google AI dying? And a lot of famous names that have left Google DeepMind and their AI initiatives recently, Geoff Dean being one of them. He was the chief scientist for Google Research and Google DeepMind. This is his talk on important trends in AI and how did we get here. And I'm just going to summarize real quick the talk and we'll give a link in the show notes for this. But I thought this was really interesting and I'll just scroll around the talk right now. But he goes through distributed training on many computers, specialized TPU supercomputers and neural network training. Transformer model architecture, which was the original thing that kicked off all this stuff and this generative AI wave. So just real quick, because I thought this was such a good talk, and AI progress has been driven. Okay, so backing up a bit. This is Jeff Dean. If you're an AI builder and you want to know what is the history of AI, what are the most important trends, how do you get up to speed, how do we get here, how what can we do now, where are we headed? This is from Jeff Dean. So a few points. Number one, AI progress has been driven by scale and algorithms, not just bigger models. And he argues that the last 15 years of AI came from 2 multiplicative forces, dramatically more compute and data, and dramatically better architectures. So that leads to not just incremental additive gains, but also compounded orders of magnitude step function improvements. Google's AI infrastructure evolved alongside the model. So Dean kind of walks through Google's progression. From distributed network training with their Disbelief tool, unsupervised learning on millions of YouTube frames, Word2Vec as a tool, sequence-to-sequence models, and eventually the TPU program that he talks about here. And then he goes on to explain transformers and sparse models and pathways and how that fundamentally paved the road for larger-scale AI and changes there in industry. So he explains why transformers replaced LSTMs by eliminating sequential bottlenecks through attention mechanisms. And everybody, I think, has seen like the whole transformer paper now. If you're in the AI game, like, you know where this came from. He also discusses sparse MOE or mixture of experts models that activate only relevant portions of a network, reducing compute while increasing capacities. And that Google's Pathway systems abstracts tens of thousands of TPUs into one virtual computer, and that automatically handles hardware failures during massive distributed training. So really cool behind-the-scenes peek at like what does massive distributed training look like. He then talks about modern LLM capabilities coming from many complementary innovations, not like a single breakthrough, but things like— he kind of talks about chain-of-thought reasoning, which improved difficult problem solving by allocating more inference compute, knowledge distillation, which compresses powerful models into cheaper ones, aka Gemini Pro model distilling into Flash, and then DeepSeek and GLM and all of those that we've seen in the US versus China AI war. Reinforcement learning, how that shapes model behavior, speculative decoding, which dramatically speeds up inference by using smaller draft models that predict multiple tokens ahead before verification of a larger model. So a lot of technical stuff. And a few more notes here. He mentions that scaling AI requires organizational innovation as as technical innovation. And I thought that was one of the coolest parts of the talk. What he learned about Gemini is that in the execution of Gemini for Google, Jeff says, you know, scaling AI requires more organizational innovation because Gemini involved over 1,000 researchers distributed across the globe. Anybody who's just even trying to get family members together across the globe or even together for dinner, you know, it's a cat-herding exercise. So thousands of researchers trying to innovate at the front lines. Oh my gosh. So Google coordinates development through thousands of internal RFCs, requests for comments, shared benchmarks, continuous experimentation across multiple model sizes, and async collaboration. And he emphasizes that the most important ideas really just start at like small pilot experimental scales. So any AI builders out there thinking about big bangs and stuff, always ground in really good engineering research. Start small, right? Small hypotheses. And Dean also believes that the next frontier of AI agent collaboration is trillion-token knowledge access, and future systems will likely involve humans coordinating many specialized AI agents rather than interacting with a single chatbot. Jeff Dean then goes on to say he predicts advances in retrieval systems that effectively extend context from today's million-token windows— this is at the start of the year, so he was saying million-token windows— towards trillion-token accessible knowledge bases with efficient inference, specialized hardware, and AI-assisted chip design. One last thing he said is misinformation. He sees misinformation, not artificial general intelligence or AGI, as the most immediate AI risk, which is really interesting coming from, from Dean. During the Q&A of this, he argues that engineering and guardrails can mitigate many long-term AI safety concerns, while realistic synthetic media poses a more immediate challenge. He talks about reducing hallucinations, Google increasingly relying on multiple reasoning rollouts, self-verification, additional inference time computation. So his advice to people, researchers out there, builders out there without access to massive compute, is to pursue fundamentally different ideas at a small scale rather than chasing incremental benchmark improvements. Really great talk. Would highly recommend anybody who's an AI builder that wants to know the, how did we even get here as an industry? What is possible today and where's the industry headed? This is about 6 months old, but I find Well, a lot of what he says still sticks and the principles and thought leadership, definitely I see that evolving in the landscape and I'm really excited to see what he does next here. Okay. It's been kind of a long pod. Me and Jonathan gotta go build the company. So wanted to do more today, but we gotta go do shit and we got customer delivery to do. Really great news. Quick update. Clarity Harness is gonna drop soon. Stay tuned. And we're about to start a little marketing campaign for that. So you're gonna see that across the board. across all the socials. If you have not already subscribed to the podcast, please give us a follow. Follow my founder brand on all social media. That's where you could hear more of clarity. And if you want signal from the noise of the front lines of AI, definitely tune in every week with this podcast. This is your weekly AI clarity. Thank you, Jonathan, my co-founder and co-host who's so wonderful and always has great demos and thought leadership to share. And thank you to the audience. Thanks for supporting the podcast. Thanks for having me. Thanks for supporting the show. Keep tuning in. We want to build a great community of AI builders. We're about to release our open source Clarity Harness. I'm really stoked about it. So it's getting good feedback from customers right now and from people that just see it. It like really clicks. Like how do you get one agent run annotated by an expert and make the next agent run by any expert in your organization better? That's what we're really trying to solve. So if you're interested in CompanyBrain, if you're interested in reusable, shareable harnesses across your team and the enterprise, hit us up. You know where to find me. All right, everybody, take care and tune in next time for your weekly AI Clarity. Have a good day. Bye-bye. Bye-bye.