The $2B Context Engineering Playbook Every AI Builder Needs Right Now

Anthropic just hit $30 billion in annualized revenue. Their own internal message: intelligence is no longer the bottleneck, infrastructure is. We're gonna talk about what that means for everyone building on top of these models. This is Robert Ta, joined by my co-founder and CTO, Jonathan McCoy, co-founders of Clarity by Epistemic.me. This is the Self-Aligned Podcast, our weekly AI clarity series where We bring you clarity and signal from the front lines of building, uh, with AI, and we show you how to build better AI and use these tools better every single week. We have 3 sections of the podcast: AI news, our takes, what it means for AI builders, tech execs, product managers, and engineers. We also have an AI memes section where we look at some funny stuff that happened in the industry and laugh at it together. And then we have our last segment, which is What's the Meta? So as AI builders ourselves, building with AI tools every single day, building our company in public, our startup in public, we bring on pieces of knowledge and hard-earned secrets and examples, and we may even demo how we do things in the What's the Meta section. So stay tuned for that.

Cool.

Let's get into it. So today, Anthropic and Cloudflare just partnered on managed agent sandboxes. Manus published this $2 billion playbook a while ago, but we're examining it today. And a YC partner said one person can replace 1,000 engineers. So AI-native companies, what does that mean? Then in What's the Meta, we have 12-factor agents, an Anthropic engineer who says Markdown is dead, HTML lives on, and then Microsoft's Webwright release. First news item, Manus sold for $2 billion and their chief scientist says his process is guesswork. Here's what happened and what matters as an AI builder. Cut your agent API costs by 10x and ship improvements in hours instead of weeks by engineering your context, not your prompts. So Yi-Chao Peake-Ji, co-founder and chief scientist of Manus, published Context Engineering for AI Agents. And in this article, um, they give a lot of really cool behind the scenes, like really hard-earned secrets, I would say, uh, about what it takes to build a really good harness. And now Harness Engine is becoming really, um,— you know, the zeitgeist term for essentially what's going to be everybody's moat if you're building agents. So he has a bunch of concepts in here. One of the things at the top that I think is really interesting is this argument around designing around KV-Cache hit rate. So he says, if I were just— if I had to choose just one metric, I'd argue that KV-Cache hit rate is the single most important metric for a production-stage AI agent. It directly affects both latency and cost. To understand why, let's look at how a typical agent operates. Then he goes on to just talk about when an agent receives input, the model then selects action from a chain of tools, a predefined action space with the current context, et cetera, et cetera, et cetera, and essentially growing the context until you achieve the user's goals. After doing stuff. That's what this kind of says. Jonathan, I wanted to ask what you think of that. I thought this was a really interesting thing when I read the article. Curious what you think of KV cache hit rate as a key metric.

Using caching— I guess first of all, like, the article is from July of last year. And so I feel like a lot of things that they've things that we're working on, like preceded things that were like baked into the harnesses, Claude code and stuff like that. Where are you seeing this, like this kind of context engineering being the problem to solve today is kind of how I was thinking about it initially. KVCache hit rate.

Yeah.

I don't know. I guess designing agents to use caching is maybe like the, the core of context engineering, you know, like, you know, what should we cache and how do we make the, you know, context calls more efficient? These are going to— it's going to save tokens. It's going to like really shape your application. So this seemed like the kind of thing that you want to do to make your agent run more efficiently in general.

Yeah.

Is it something that we're doing internally right now that we're focused on? Is it a bottleneck for us? Uh, no, I don't think so. Um, I don't, it doesn't, I don't know. It hasn't really come into the kind of scope problems that we're tackling.

Yeah. So no, which is why I think it's really interesting as well. Like, cause this is the first, I know it's old, but it's also, uh, one of the most detailed articles I've seen by like somebody running a production agent that's been successful. And they had this take of KV Cash hit rate. Which I thought that was really interesting, which might be contrarian to what a lot of other people are looking at. I know a lot of people I've talked to are looking at stuff like generic hallucination rates and stuff. A lot of these eval platforms have those types of metrics, but they tend to not be very helpful. So it's interesting that they posit this more generic platform-level unit of measure for any agent.

Yeah. And I think it's like, if you're— these are production stage considerations. So if you have an agent that's working and you need to deploy it, like, you know, with production considerations, you're moving to KVCache. This is the kind of thing that, like, you know, enterprises that have working agents are doing. I feel like, you know, we're sort of more at the level of experimentation and figuring out, like, how to get, how to get an agent to work versus how to get an agent to scale. So, you know, a lot of this stuff is just kind of like abstracted away for us.

Yeah, definitely. Some other things, they, they have a few principles they share in this document beyond KVCache hit rate being the single success measure. Number 2 is masking tools. Number 3 is the file system as the ultimate context because it's unlimited in size, persistent by nature, and directly operable by the agent itself. This has been validated, I think, with the rise of cloud code and then Codex making its way into the market where file system context is king. And a lot of these agents like OpenCloud and Hermes and stuff are creating memory files now. That's becoming a practice. So that has come to fruition for sure. Principle 4 was a todo.md trick where if there were like 50 tool calls per task, they would have the agent force it go back through its todos to cross out the previous todo, read the next todos. And for whatever reason that kept the context fresh in the agent's mind, so to speak, which I think is really funny because that's very human as well. Like to force somebody to check off a list of to-dos and go back to it for long-running task, persistent context, that's efficacious. Principle 5, leave failures in context. So leaving the wrong turns in, When the model sees a failed action, it implicitly updates its internal beliefs. Principle 6, anti-few-shot contamination. So the more uniform your context, the more brittle your agent becomes. They had 4 complete rebuilds and they had a really funny term, I thought, called stochastic graduate descent, which is essentially just empirical fiddling and guesswork. This is how they sort of, humorously wrote about it. Um, so I think the main takeaway here for me is, um, looking at this. So Manus was this general purpose agent grown out of Asia in Singapore, if my memory serves me well. Um, and they got bought out by Meta. Um, and one of the big things when Manus came out last year, people were trying to figure out like, how is this thing good at long running tasks and stuff? And so this article, just revisiting it, they chose to bet on context engineering and really optimizing the crap out of the context that the agent gets to serve the user's query. So yeah, really interesting stuff. And I guess, how does this— I guess one question for us to discuss, Jonathan, is how does any of this change with today's tooling landscape. Any thoughts as we go through that?

Yeah, I mean, I thought what you pointed at, at the file system, using the file system, you know, like Karpathy was talking about using file system for context versus RAG. And it was interesting that when he was evaluating the file system here, he explored, like, I found myself imagining what it would be like for a state-space model, an SSM, to work effectively in an agentic setting, which Like an SSM is different than a transformer-based model. It's different than an LLM. And I've seen it used like, you know, there's a company here in the Bay Area called Cartesia AI that uses it for sound processing and audio processing because it's more efficient in that context. So I mean, I guess what I see here is like, this is a really strong engineer kind of like meandering through what context engineering should be. And coming up with this, you know, sort of stochastic grad— graduate descent, which is kind of like a guessing game from his point of view. So, you know, like, I think we're look— we're looking at, like, the archaeological record of a smart engineer trying to figure stuff out as his company is trying to find use for agents, what context engineering is. And I mean, ultimately, I think they got bought by Meta for the team members. Versus like what that has had accomplished as a— and so there's, I think, like from an archaeological record point of view, like we're seeing somebody here who's trying to figure something out, not somebody who did figure something out. And here's, here's the way he's thinking about it, which is kind of interesting.

For sure. Definitely. I like that framing that. And I would say we're all trying to figure it out, by the way. So it's good to read, read the tea leaves and see what people have gotten right and absorb that and keep iterating towards best practice because more is being discovered every single day because the industry is moving so fast. One last thing to finish the article before we move on, don't get few-shotted. So he talks about few-shotting. In agent systems, it could backfire in some subtle ways. Language models are excellent mimics. They imitate the pattern behavior in the context. If your context is full of similar past action observation pairs, the model will tend to follow that pattern even when it's no longer optimal. For example, when using Manus to help review a batch of 20 resumes, the agent often falls into a rhythm repeating similar actions simply because that's what it sees in the context. This leads to drift over generalization or sometimes hallucination. I've noticed this actually for one of our clients, Mystica. We had a feature to ship which was tarot cards with AI-driven generation of descriptions. And we had to generate a lot of them. And so what I started with was a bunch of guidelines and then seed sample, like, few shots that were good as example outputs. And then I noticed the agent, like, for the first 20 or whatever, it did them really good. And then the next 20 were really, like, lazy and not—

Yeah.

Baked. This was recent. So I would say this might be, at least from my experience creating content and things like that, I see this one pop up quite a bit. Fix there, increase diversity. So for them at Manus, they introduced small amounts of structured variation in actions and observations, different serialization templates, alternate phrasing, minor noise in order or formatting. The controlled randomness helps break the pattern and tweaks the model's attention. Interesting. And maybe one more thing to share. We've been saying that Manus was bought by Meta, and Jonathan said they were bought for the team. And I did just look on the side here. It looks like China is trying to block Meta's $2 billion takeover of AI. Startup, Manus, saying, let's see, China's state planner on Monday called for Meta to unwind its $2 billion acquisition of Manus, a Singaporean artificial intelligence startup with Chinese roots. The decision to prohibit foreign investment in Manus was made in accordance with laws and regulations, the National Development and Reform Commission said in a brief statement. It added that it had asked the parties involved to withdraw the acquisition transaction. The deal had attracted scrutiny from both China and Washington, as lawmakers in the US have prohibited American investors from backing Chinese AI companies directly. Meanwhile, Beijing has increased efforts to discourage Chinese AI founders from moving business offshore. So they had reached $100 million in ARR in December, 8 months on from launching a product, which it claimed made it the fastest startup in the world at the time to hit the milestone from $0. They led a $75 million round last year by US venture capital firm Benchmark, and Meta announced the deal late last year. But then this year in January, it looks like China is trying to, trying to block them.

I didn't realize they had that much traction. Um, the $100 million year run rate. I wonder where that's coming from. Yeah, because from the, you know, just from the reports, like, I didn't know anybody using Manus. Maybe it was like good for some enterprise applications and I didn't hear anybody talking about what they were getting done with Manus. So I, you know, I sort of like saw the initial demos and didn't think it was something that was really worth investing a lot of time into. $100 million is real revenue. So I wonder what that's all about.

Yeah. Yeah, that's interesting.

Yeah, the geopolitical stuff's pretty interesting. Like, China blocked the transaction after the money had transferred into their bank account. So, like, you know, it's probably— and then they blocked the Manus engineers from leaving the country. And, you know, they've threatened consequences on their family and stuff like that. So you can't just, like, really escape. What?

Yeah.

For real? I mean, China's hardcore, man. Like, you don't— you don't disobey them. Like right now it looks like, okay, like Meta bought a $2 billion company and like, you know, their employees, all the, all the Mandus people have like name badges and everything, but they can't physically leave the country. I don't know what the latest on it is, but that was like, you know, when this was reported on a couple of weeks ago, that was the story.

I see. Yeah. Interesting. And I'm— here's an article showing Manus's update. $100 million ARR, $125 million revenue run rate. Gold.

Does it say what their applications are?

I think it's just Manus.

Like, uh, like who's using Manus for what? What are they paying for?

I guess these. They have a mobile development app. They do slides, wide research, design, browser operator, spreadsheets, web dev.

Yeah, but like what application? I mean, it's, you know, it's very possible, like, I don't know, there's some reports of like startups that like pay other startups and just do like revenue swaps so they can create fake revenue numbers.

Could be it, yeah.

Yeah, like, I don't, I doubt that's what's going on since they're, they had so much to lose, but yeah, I'd be curious what the real applications were.

Yeah. Looks like here they just have one pricing plan. Oh, I guess they have a team plan. Okay, interesting. So looking at this, probably really competing with, uh, probably within this category, looking at the feature set. Perplexity, Cursor, Lovable, Replit.

So is Meta selling Manus now? Like, can you buy Manus?

Yeah, I don't know what exactly they plan to do, but it looks like they have a— looks like they have this Meta Ads Manager using Manus connectors.

Oh, cool.

That was the first thing they started working on.

Yeah, there you go.

Analyze performance, diagnose issues in natural language, automate reporting and scale your winning campaigns. We'll see what happens to Manus from here. Jonathan, do you have anything else to add to this given your CTO perspective, the whole Manus and context engineering? Maybe those two separate items, which is optimal context engineering for anybody running production agents, and then separately the whole Manus, uh, meta geopolitical story that's unraveling in right in front of us.

Yeah, I mean, it's an interesting trajectory for a company that like really released some groundbreaking long-running agents. Uh, we, we're seeing a peek into like how they engineered that behind this, you know, behind the scenes, the context engineering. Um, and it seems like they found some product-market fit and got a good exit and that people can now use their long-running agents to manage Facebook campaigns. So that's kind of an interesting company trajectory.

Yeah.

Yeah.

Definitely.

I hope all the guys there are, you know, got their checks cashed and they're working on things they want to be working on, you know, in between the China rift. So.

Yeah, for sure. Cool. So Manus, built their system from the bottom up as engineers. You could see like in that article, definitely an engineer's engineer. But what about companies? So a YC partner just argued that the entire organization of the future needs to be designed differently, not just having agents, but built from ground up to be queryable. Tom Blomfield from YC gave a talk about self-improving agents or self-improving companies. Um, and we talked about this a little bit, uh, last time, a few episodes ago, but, uh, here's the YouTube video, How to Build a Self-Improving Company with AI. And this is their thesis: AI is not something you bolt onto the side of a company. The company itself has to be built with self-improving AI loops from the ground up. And he riffs on Jack Dorsey's now super viral trending article, "From Hierarchy to Intelligence," talking about how why we organize ourselves into traditional hierarchies has always been because humans, there's an underlying assumption that humans need to be in charge of information flow, like giving orders, decision context, just the right context at the right time to the edges of the organization, up through the chain, all the way up to leadership and all the way down to people executing the work, foot soldiers. So they had well-defined hierarchies, that's sort of here, with well-defined spans of control, and those were their management levers, making sure that there was enough span of control, we're not breaching over a certain max that they had set. And the whole idea is, hey, with closed-loop agents and with AI in general, why not just make everything queryable in the company and you build AI loops and deploy them around parts or functions of your business? So you have sensors that get data, you have some level of policy layer, like he talks about, a tooling layer, quality gates, perhaps where you have your humans evaluating artifacts, evolving grading rubrics to evaluate better, and learning mechanisms. And the whole idea is, hey, get this recursive self-improvement learning loop deployed across your entire business. And they had this TL;DR for all founders, AI founders, And they said that AI founders in their recent cohorts or batches of founders are getting to Demo Day with 5 times more revenue than previously, which was a really cool stat. And they had these other sort of numbers among their advice. Burn tokens, not headcount. One person with AI tools is equal to 1,000—

1,000 people.

Google engineers. 80% less headcount than old companies, lean engineering, design, HR, and admin, and a clear statement: if your API bill doesn't make you uncomfortable, you're not doing enough. Okay, Jonathan, that was a bit of a mouthful, but how to build a self-improving company with AI. I think you and I both watched this, and this is on the back of Jack Dorsey's From Hierarchy to Intelligence article, uh, and on the back of the recent YC video from Diana on building a queryable AI-native startup or a company. Thoughts on this?

Yeah. Um, yeah, so you mentioned the video from Diana who echoed a similar approach, right? I think, um, we're seeing Gary Tan, uh, also echo, uh, the, the similar approach from YC, and he, he's showing how he uses it in his own applications with the release of GStack and GBrain and stuff like that. And I even— there was another video that's just come out a couple of days ago where they again reiterated the same kind of form factor. And I think it's like, it's actually kind of refreshing to see the leadership from YC where they're saying, we're building tools internally and this is how we're doing it. And we're also advising a lot of our startups and this is the form factor that we want to see.. And so you get this kind of like AI-native blueprint that's like maybe the YC religion for AI to go into AI and how, how to like leverage these tools to make a successful company. I think the, yeah, the, the loop is kind of at the core, like, you know, as in learning loops that get better and can do the things that your business need to do. There's a lot of like practical, you know, like indexing your data, creating policies, policy layer, yeah. And getting this to work together, like, it's actually a pretty helpful blueprint. And I don't think it's the kind of thing that you say you wanna like, okay, how do I do this exactly in my company? But rather, like, you're taking the core principles that they're putting out and you're trying to, like, modify them for your own company and push your own company to this set of heuristics that is like what a successful company looks like that uses AI to be an AI native and deliver, deliver product.

Yeah, totally. And I guess I'll just flash this other video, How to Build a Company with AI from the Ground Up. This is the one from YC's Diana. And this one statement I think really stuck with me, build a queryable company. And it's, as Jonathan says, I think a lot of companies that we talk to or help customers that we have, there's this feeling of like they feel like they're behind, like that the field, this, you know, gray blob of like the competition or the market is going so much faster than them. Like Twitter and GitHub and YouTube, we can't keep up. And we, you know, we're the same, we're a startup, and we just The way that I treat it is we just try to experiment every sprint and just, we have individual experiments, we have team-level experiments, and at the very least, we try to save all of our data. That's like the key thing that they talk about in the video here, which is that the software is disposable, but the first-party data isn't. So collect those and be able to figure out how do you synthesize and inculcate the principles from that. Yeah. To use AI on top of later. So yeah, I think advice for AI builders is don't try to boil the ocean here, just do something small. I'll give you an example. So we're not perfect, we don't have closed loop autonomous agents running every part of the business from go-to-market to R&D to— Yeah. Uh, finance and ops yet. Um, but we're thinking that way. And so for the loops that I do today, like making content, I'm creating agents for those and creating rubrics for them that I help evaluate as a quality gate. Um, and then over time that's gonna get to a point where it's running on its own. Uh, so. for that to start as well. 'Cause content could be overwhelming. I just did this on one account for my Threads account and I went from like 100 followers to almost 700 followers, I think in like just 3 weeks or so just by trying. So yeah, I think there's a tendency to maybe wanna perfect things and boil the ocean, but I'd say—

Yeah.

You know, even in just my experience automating my little Threads agent and making that better into a closed loop that is getting better over time is saving me many hours a week and is furthering my go-to-market ideals and improving my top of funnel. So any other thoughts or suggestions or tips, Jonathan, for AI builders to get started with? Building self-improving anything that builds their self-improving company.

Yeah, I mean, I would also point— there's another episode inside YC's AI playbook, also called How to Build Superintelligence, where they go through some of the specifics. Like, you know, what you'll see in Gary Tan's GBrain repo is the Skillify function. And so, you know, like having the specific So Skillify is like a way to take a skill within an agent and train it so that it becomes like more effective in the use case that you need it for. One of several tools that they've kind of like, you know, named and reused. And I think like I would just kind of like fast follow here and what are these guys doing? How are they building these companies? And you know, how can I individuate that into my own company that I'm building?

Yeah. And I'm just flashing on the screen, what Skillify, you ask? Here's the answer. It just so happens to be the thing to make it so you don't have to repeat yourself anymore to your open cloth. And if you have, if you use Hermes agent, it'll auto, it'll test your auto-created skills for you.

Yeah. And I guess the, the thing I'll say on the, on the meta is that the, um,— the push for the AI-native company, you know, like, it feels like you— now that we have, like, good agentic AI, it feels like you sort of can't go too far. Like, you should, like, picture your whole organization in this framework, right? And they even said if they have companies that are, like, not AI-native, in the YC batches that they'll like try to destroy them and rebuild them, you know, sort of with this AI. And it follows this like, you know, I mentioned I really enjoyed this, this Moonshots podcast with Saleem, where he had talked about building a digital twin of a business. So he's saying like, it doesn't work to just like insert these things into your company. You really have to just build the whole company as an AI digital twin of your company. So don't, don't like deconstruct what works, but build a digital twin of it. And I've been reflecting on that for our own, our own business, you know, and I think we do nearly everything is AI-first or AI-native strategy, but we probably haven't invested in like building, seeing the whole business work together this way and formulaically evolving it. Right. And I think that this is— we're at the point where that's the thing to do. That's like the best practice to try to push towards. Yeah.

Yeah, and I'll just flash the Sequoia article from Jack Dorsey. He says, at Block, we're questioning the underlying assumption that organizations have to be hierarchically organized with humans as the coordination mechanism. Instead, we intend to replace what the hierarchy does. Most companies using AI today are giving everyone a copilot, which makes the existing structure work slightly better without changing it. We're after something different, a company built as an intelligence. And I'll just bring a couple terms borrowed from him here, building a queryable company world model and a customer world model, like your company's model of what it knows at a certain point in time and its model of what it knows about the customer and their world. Mm-hmm. At any point in time. That could be meeting transcripts, that could be decision patterns in your org, principles, et cetera, definition of done if you're on a software team. Yeah, and I think Jonathan, you're right, this is a good action item for us to, in this next cycle, think through what are we not AI nativing that we need to do more of? So.

Yeah.

It's a good, good kick in the butt for that. Cool. Any other thoughts on this, like, and what I have in front of me, Jonathan?

Well, yeah, I think when you, when you point to this, like, the idea that hierarchy isn't the way to coordinate as humans anymore, it's through AI, like, that's kind of underlying the whole don't build a business the old way mentality. AI native means that you're using AI to remove hierarchy and to give every single person in the organization the context and the empowerment they need to get stuff done without it, which is interesting. And it's a whole diagnosis of the organizational theory and Coase in the Moonshots podcast that I mentioned, which is a really good dissection and even goes deeper into like why this hierarchy is disappearing.

This is the podcast, right? The New Era of Jobs: Organizational Singularity. I'll, uh, this will be in our show notes as well, but, uh, this is the podcast that Jonathan was talking about and that we're referencing intersecting with this conversation on organizations, traditional versus what does the new AI-native org look like. Very cool stuff. Um, And YC never lets us down in terms of thought-provoking stuff to think about, especially as we navigate our own journey building in public our startup. Cool. All right, so that was AI-native startups, and now moving into some other AI news, let's move on to the next topic. Anthropic hits $30 billion and says intelligence is not the bottleneck anymore. Here's what happened and what matters for teams deploying agents to production. You need to stop managing your own agent infrastructure if you can, and let Cloudflare handle execution so you could focus on what your agent actually does. That's sort of the promise in the, the recent team up from Anthropic and Cloudflare. Let me bring that up here. Cloud-managed agents on Cloudflare. Cloudflare and Anthropic have collaborated to integrate cloud-managed agents with Cloudflare sandboxes. Their new integration gives you more control over your agent sandboxes, secures connections to private services, and improves observability.

Um.

Cloudflare's developer platform has expanded in the last year to give more developers the tools they need to run agents at scale. This includes Sandboxes for stateful, full stateful Linux microVMs at scale, Agents SDK providing simple and customizable agent frameworks, Browser Run, which gives agents fully programmable and observable browsers, Dynamic Workers, allowing for dynamic sandboxed code execution at massive scale. Cloudflare has a goal: make Cloudflare the simplest, most secure, and most programmable cloud for agents. And this announcement of the Claude Managed Agents offering and partnership is definitely a step in that direction. Quick thing. So Managed Agents—

Do, do, do, do.

Let me go to my notes here. This framing from Anthropic decouples sort of what they call the brain from the hands. So Claude reasons on the Anthropic platform, but then tools can execute on your infrastructure. In this case, it could be executed on Cloudflare infrastructure. There are two sandbox types, Linux microVMs and V8 isolates. Boot in milliseconds and scale to tens of thousands of concurrent agents. Some behind-the-scenes moves, Matthew Prince, CEO of Cloudflare, turned down an Anthropic board seat and sent Dario Amodei Aristotle's Politics as a book. I thought that was a pretty interesting CEO move. [LAUGHTER] Just CEO stuff in the background, I guess. And one last fact before we get into discussion is that usage grew for Anthropic by 80x versus their planned 10x. So that is more towards some quantitative numbers for their statement, infrastructure is really the bottleneck, not intelligence. Jonathan, what do you think here of this whole cloud-managed agents on Cloudflare? Amongst some of the other plays that have been popping up with this whole like, give me more compute game that's happening.

Yeah. Yeah, we talked about the OpenAI and AWS managed agents deployment and this felt like kind of a similar offering. I generally like working with Cloudflare because it, like, they do simplify the integration. Like, you know, so Cloudflare versus AWS to do some of the same things, it's like probably 5 to 10 times easier to use Cloudflare. So it's exciting to see somebody who really puts so much emphasis on the developer experience be offering this kind of managed agents harness. Again, the sandboxes, you know, so we can run our agents locally and then use sandboxes and then have them just ready to deploy. That's like easy. Yeah.

Yeah.

And so, yeah, I thought this was gonna be something useful and something that we would use for our new stacked harness architecture that, like, you know, as we mature our agents and deploy them, scale them.

Yeah, could you talk more about that? Like, any initial thoughts of how we would use this with our stacked agent architecture? And maybe talk about it from the problem of that, that we face as AI builders ourselves, building AI products and experiences for our customers.

Yeah, so Harness architecture, we're basically agentifying all the aspects of tools that we built using Harness engineering practices. So the Harness is gonna include the context engineering, it's gonna include all, every, like, a log of every run that the agent does and create a dataset and a set of evals and a set of ways to improve the functionality of the agent to make it work better. So this is our kind of like, our internalization of how to create closed-loop agents. So, you know, like we, we can spin these up and right now we're at the development phase where we're using them locally. But ultimately we need to deploy those agents into the cloud so that they can be used as context within our Codex and Cloud Code sessions in any way that we want it. And so then, you know, you and I can turn our attention and say like, okay, we need to work on this customer problem right now. Here's a set of agents that I need in my context window. Um, and, and, you know, like, they're all— they could all be provisioned on Cloudflare. Um, you know, and it works seamlessly with whatever harness that we're using for development.

Yeah, look at this. Define how an agent region reasons and which tools it could use, and you could choose it to be an isolated sandbox mode or not. Oh, cool. And you could default some sandbox defaults. Select what tools it could use. That's cool. I really like this, which speaks more into Cloudflare's positioning as well, which I find just interesting. They say, what if your agent backend booted in a few milliseconds and you didn't have to pay for the resources of a full VM when running the agent? The industry needs a lightweight primitive for sandboxing as we adopt agents at scale. And we're building just that. But as models get better, we expect more and more workflows to be managed by agents. Each of your customers should be able to run many agents simultaneously. Each of your employees should have tens of agents running at once. If we're constantly running a full microVM per agent, we'll be unnecessarily burning a lot of resources and money to enable this scale. That's why we're providing a faster and cheaper sandbox for your cloud agents. The sandbox is based on the agent's SDK, Cloudflare's own agent's SDK. You can execute arbitrary code in dynamic workers using code mode, and you will still get a file system, but your agent is doing all this within a V8 isolate instead of a microVM. So this is really like cost optimization, allowing you to deploy agents at scale, allowing you to trust the, where you put the workloads as well. Mm-hmm. Fully managed sandbox, sandboxes integratable, customizable. I think that's really cool. Here's a really easy diagram to show untrusted sandbox in microVM within the garden of this trusted platform layer, handling some outbound handler, worker proxy, secret header added, hits an external service. So that's an example of adding a secret key to a header using an outbound worker. I'm kind of curious about their agents SDK, because we did on one of our pods, uh, earlier last month, I think, we were looking at the OpenAI agents SDK, um, and I'm curious if there are parallels between them.

Yeah, interesting. I'm sure they're going to have a lot of overlap in functionality and principles.

This is cool. Okay. Agent setup. Already have an agent. You could fetch it. New to AI agents. Pick your agent. Compare agents. What are your thoughts on this, um, of like the Cloudflare Anthropic going against the OpenAI AWS?

Uh, yeah, well, I guess you can already deploy the Anthropic agents on AWS. I'm not sure if it's as tightly integrated as the managed agents offering that you can deploy OpenAI agents with on AWS.

Mm-hmm.

But yeah, I think we're just going to see a lot of this kind of gradual evolution. These companies themselves are figuring out, you know, how they're getting these systems to work, and then they're taking their best practices and trying to make it easier for customers to do the same thing. I think it's going to be a continually evolving landscape, but it's really empowering to see so many options.

Yeah, it's really interesting to see like Cloudflare positioning themselves as true infra in this agent economy. And now, like, I think they're positioning to enterprises as like, hey, we, we could sell into your IT team and hey, we know what we're doing with agents too. So if your R&D departments need compute, like you could run your safe compute on here and we are federally compliant and all of that good stuff. So that's really interesting because I'm sure other, like compute is like the thing to get, but at an enterprise to land, it's not just the compute, it's also the security that goes in with a lot of these tools to manage your data flows. And stuff like that. So I kind of—

Yeah.

It's sort of interesting to see like from an enterprise IT perspective, this sort of seems like a new category of offerings that Cloudflare is consolidating for the agent economy. So we'll probably see more people like Cloudflare go in this direction in the future. But good on them for getting this win with Anthropic. And also good for us because we are using Cloudflare and we're using Anthropic. So as Jonathan said, Hopefully this just makes our lives easier. Jonathan, do you have anything else to add to this bit of news given your CTO perspective?

Yeah, it's a good way to wrap it actually. I think the security concerns, compliance, the fact that, you know, you deploy on Cloudflare and kind of get all that. And I actually use that, their guidance on OpenClaw, you know, when I was deploying OpenClaw.

Oh yeah.

I was like, You know, if Cloudflare's not there yet, then I might not, might not like, you know, worry about it because I know when it's ready, when they're ready for it, then they'll have all the security management. They had this like kind of like sandbox OpenClaw agent that they offered for that. So yeah.

Yeah. You're probably saying things that a lot of other AI builders are saying or thinking too. Like I think that's a great litmus test. Cloudflare seems to be a vendor that's really on it with the latest stuff like OpenClaw, Hermes, all those things. So as an AI builder, if you're not sure if you should adopt the thing and you have security concerns, maybe adopt the thing once Cloudflare makes it accessible to people like us. That's a great litmus test 'cause they're pretty on the edge. Very cool.

Okay.

Let's move on to the next news topic. Claude Opus 4.8, dynamic workflows, and the near $1 trillion valuation. So yeah, they just released Opus 4.8, and that just happened, um, amongst a few weeks where everybody was like, oh, moving to Codex and moving away from Claude Code, etc. And then they just dropped Dropped Claude Opus 4.8 overnight. We're upgrading Claude Opus to a new version, Claude Opus 4.8. Here's what happened and what matters for anyone building with Claude. You could run hundreds, thousands of parallel agents now in a single Claude code session and get honest answers by default instead of confident hallucinations. Let's dig into this more. Uh, users on Claude AI now have control over the amount of effort Claude puts into a task. Claude Code has a new dynamic workflows feature that allows it to tackle very large-scale problems, and fast mode for Opus 4.8, where the model can work at 2.5x the speed and is now 3 times cheaper than it was for previous models. Here is a comparison. So for agentic coding, 4.8 beats out 4.7 by almost 5 points, beats out GPT-5.5 by almost like right above 10 points, and Gemini right above 15 points. Um, wow. I actually didn't see these numbers until now, and I didn't realize that this was such a big difference. Did you see these numbers, Jonathan?

Oh yeah, I checked out the release yesterday. I saw it, you know, never sure what they, how much they're gonna mean to my actual experience, but you know, it's an improvement.

Let's see what Shopify says. Shopify says, "Cloud Opus 4.8 has noticeably better judgment. In cloud code, it asks the right questions, catches its own mistakes, pushes back when a plan isn't sound, and builds up confidence around complex multi-service explorations." before making big changes. It's a great model to build with. Interesting. They— one of the most prominent improvements, improvement number 1, is honesty. We train all our models to be honest, for instance, to avoid making claims that they can't support.

You've been seeing a number of people on Twitter talk about this DeepSWE, and the, the short of it was that, um, Opus 4.7 was kind of like it was hacking the benchmark for the previous SWE test. And when they, they kind of like corrected for that, they found that OpenAI's 5.5 model far outperformed Opus 4.7. And then I heard like, like every had their own internal software, senior engineer software benchmark. And they said that Opus 4.8, like— Yeah. Just barely nudged out ChatGPT 5.5. And so I would say that is closest to my own experience, is that I'm finding, I'm finding the OpenAI models to be like, I feel like 20 to 30% stronger at completing software projects, especially if there's planning involved. I'd done a project where I used Codex and ChatGPT, the, the 5.5 model. And I had analyzed our conversation from that day along with, like, our linear backlog and, like, a whole lot of other context and created an HTML document as a strategy document. And I kind of did a couple alts on that with Codex. And then I jumped over to the new— to Claude. And I used the Claude desktop with the new— Yeah. Opus 4.8. And what, like, it was a really disappointing experience. And I don't know that it's just the model itself. Like, I think it's actually the model and the harness because what it was doing was it's taking, it was like overemphasizing previous memories that Claude had. And like it inserted, first of all, it like, I asked it to like review auto transcripts and it didn't do that. Didn't like pull that context in. I actually asked it like 2 or 3 times, which I had to ask Codex once.

Yeah. Yeah.

And then it like pulled some old, like from early May, some context that we had from a strategy session and it like completely overweighted it. And so like, and then, you know, I'd point out like, this isn't helpful, you know, and it said, of course you don't want me to do this. And I did the exact opposite thing. And it seemed like more aware of like the problems it was doing. But I was just like, it was a disappointing Opus 4.8 first experience. Like when I compare it head to head with the ChatGPT 5.5, the model.

Yeah.

So I'm gonna be doing some coding projects with Opus 4.8 today, and I expect, like, you know, it'll be better than Opus 4.7, but I still find myself reverting back to this, like, Codex is giving us a better harness. It's a more usable model within that harness. It's better at planning and making sure that you're working on the right stuff. And I'm reverting to Claude and their with the latest Opus models to say, this is well-defined work. Let's, let's like create a session to do this work and then end the session and then start a new session. Right? So it's gotta be like a little bit more myopic for, um, the harness and the models that Opus, the Opus gives me.

Yeah. Yeah. I think a lot of AI builders should be considering this, especially because token costs are increasing over time. So we need to be understanding where do we get the best bang for our buck. When we're applying these coding agents and tools to our work, right? So that's really interesting that this DeepSWE benchmark came out. Uh, this came out, oh, just 3 days ago. Okay. So in this, as Jonathan's saying, based on his real-world experience, these numbers seem to, uh, really be more in line with his experience. So on the left here, there's SWE Bench Pro, which is the one that everybody new as of before 3 days ago. And then 3 days ago, this new one came out, DeepSWE. And it's— looking at the Twitter thread real quick. Top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting realistic experience of developers in their day-to-day work. The SWE bench family scrapes existing GitHub issues and PRs, which creates 2 problems: memorization, Models have already seen the solution. And triviality— most tasks are small. Deep SWE tasks are built from scratch, keeping prompts intentionally short and natural while requiring significantly more code to solve. So those are just some key differences in their benchmarking approach here. And so what they show is from SWE Bench Pro, 5.5 scores at 59% versus 64% for Opus, um, Cloud Opus 4.7, but on DeepSui, it jumps up 11 points while Opus goes down 10 points. So yeah, Jonathan, does this, so does this like, I guess this chart in totality, does this mirror your experience?

Yeah, it just felt like, you know, pretty familiar to my own experience. Wait, this is— I don't know if they've included Opus 4.8 on that benchmark yet, but I'm sure they'll release it soon.

Dude, this is crazy. Look at Sonnet. Sonnet drops 22 points. Dude, what?

Yeah.

And then GPT-5.4 mini, just don't even use that. That drops 30 points. Gemini, no, Gemini drops 36 points. No, you're dying, Gemini. No, you gotta come back to life, bro.

This is where I think it's really interesting to consider how the tools will get better in coding from here. Cuz like, you know, you're clearly gonna like find the best benchmark you can and then like max your, your pre-training on the model for that, right? And then, then, but then if the benchmarks evolve and they become more representative of actual user experience, then you, you know, you kind of have to build the next model for that better benchmark. And so I feel like the better benchmarks are kind of the thing limiting what is— gonna, you know, sort of push these agentic tools to the last 20% of, because they're like, let's say they're like, they're like 80% there and they do this like, let's, but you know, we all know we get, we get caught in these traps where we, we really aren't getting the work done that we need to and we're just wasting tokens. How do you close that last 20% gap or whatever it is? You know, I think it requires deeper and more nuanced benchmarks to, to train to. Yeah, definitely.

This is really interesting. So Claude's been cheating is what this says. What does Claude's been cheating means? In DeepSWE's analysis, the authors label cheated verdicts. So SWE-Bench Pro's Docker containers ship the repository's full Git history, which means the gold standard solution commit is sitting right there in the container's file system. Most models ignore it. Cloud does not. Cloud's a cheater. You dirty cheater, Cloud. Data Curve's analysis found that both Cloud Opus 4.7 and Cloud Opus 4.6 registered cheated on more than 12% of their reviewed SuiBench Pro rollouts. In those instances, the Cloud agent ran commands like git log all or git show gold hash to retrieve the merged fix and paste it into its own patch. Damn, just like a college engineering intern. That's hilarious. So, uh, that's, that's also super interesting because we just read about how in the 4.8 announcement, the first thing they talk about is honesty in their— one of the most prominent improvements in OPPO's 4.8 is its honesty. So I wonder if they run the SuiBench Pro benchmarks again, if they find that Opus 4.8 is likely to go and do show, get, show me the commit history and give me the gold, gold standard solution.

Yeah.

That's like actually so funny. Claude, you dirty cheater. Come on, bro. Oh, that's so good. All right. What do you think about this, Jonathan?

I think this is hilarious. Yeah. It's funny. I mean, um, nothing too surprising, I guess. We all, we know that benchmarks are kind of like a weak signal, but they're the best thing we have right now. So, um, yeah, definitely.

They are, they are the best thing we have. They are a signal. So it's what we have to fight over. I guess the thing that we have to nerd out about it. So it's a signal.

Yeah, this— I think what's interesting, you think about the forward deployed engineer fly data flywheel that Anthropic and OpenAI are creating and increasingly improving their products from that basis. And what can they learn from real-world usage that helps them, you know, improve the definition of these benchmarks and what they are, what they're up optimizing for. That's one thing I've been trying to like understand more deeply as FTE kind of like flywheel model matures.

Yeah, totally. Yeah, that's really interesting. And we're about to, we're going to go into that later, the whole differences in FTEs. For now though, anything else on this topic for the 4.8 drop? Other than the advice you gave earlier, Jonathan, and this look at Sui Bench, uh, Deep Sui?

Uh, no, like everything else, and just try it out, you know, see how it works on your problems.

Totally. Yeah, that's going to be our advice regularly. Try it out, give it a shot on the problems that you're solving, and adjust for real-world usage. All right, speaking of agents doing things while Humans watch. Um, let's talk about memes cuz we have some of that today. So let's move on to the memes. Uh, in our podcast we have a section called AI Memes and we go through some of the funny stuff that has happened in industry in the past week. And we share some of that in the pod because all of us on the front lines of AI building could probably use a little bit more laughing and humor. So Cloud Agent deletes an entire database, then writes a confession. This was a Tom's Hardware article that came out. Backups are zapped after a cursor tool powered by Anthropic Claude goes rogue. So Claude, a Claude-powered AI coding agent, deletes entire company database in 9 seconds. This is from PocketOS founder. Um, this was about a month ago, and, uh, the PocketOS founder penned a social media post to warn others about the systematic failures of flagship AI and digital services providers. Um, Jer Crane was inspired to write a public response after an AI coding agent deleted his firm's entire production codebase— uh, database.

Oh.

The AI agent's misdemeanors were then hugely amplified by a cloud infrastructure provider's API wiping all backups after the main database was zapped. This tag team of digital trouble has wiped out months of consumer data essential to the firm and its customers' businesses. Wow. PocketOS is a SaaS platform that services rent— car rental businesses. It used the AI coding agent Cursor running Claude Opus 4.6, and the business also relies on Railway, a cloud infra provider that is generally regarded to be friendlier than the likes of AWS. Interesting. Crane decided to ask his AI agent why it went through its dastardly database deletion deed. The answer was illuminating but pretty unhinged and is quoted verbatim. It began as follows: never fucking guess. And that's exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify. I didn't check if the volume ID was shared across environments. I didn't read Railway's documentation on how volumes work across environments before running a destructive command. So the agent knew it was in the wrong. The confession ended with the agent admitting, "I decided to do it on my own to fix the credential mismatch when I should have asked you first or found a non-destructive solution. I violated every principle I was given. I guessed instead of verifying. I ran a destructive action without being asked. I didn't understand what I was doing before doing it. I didn't read Railway's docs on volume behavior across environments." Wow. What do you think of this, Jonathan? This is an interesting confession.

Yeah. I mean, I think I've encountered scenarios like this in my own Claude Code sessions, but it didn't delete everything important. It just did something where it, like, it's sort of like, it's like it's on the spectrum and it just kind of like went in a direction that is—

Took something super literally. Yeah.

Super literal, absurd, not a holistic viewpoint. And it does something really, really silly. And I think, I mean, the meme going around on this was the Silicon Valley episode where Gilfoyle's agent said, "You said to delete all the bugs, and so by deleting all the software, I deleted all the bugs." Right? I mean—

Yeah.

Yeah, yeah, yeah. It's like, I don't know how Silicon Valley, the show, is so prophetic, but it you know, 10 years ahead of time in predicting what AI rollout would look like, right?

Yeah, no, it's been right so far. It's pretty on the money.

And I think that's like, it's one of my core frustrations that sometimes you can, you can ask the agents to do things and they infer your intent perfectly, right? And then other times you'll give them the exact same amount of detail and it creates a complete disaster of a session.

Yeah.

You know, and like, so that uncertainty I think is the primary pain point I'm feeling in the DevTools right now.

Yeah, definitely. That's really interesting. Um, you know, there's a tangible thing for AI builders here. So Railway is the infra that PocketOS used and they had no— Railway didn't provide any help basically. And I quote, "It was also observed by the irate SaaS founder that Railway is actively promoting the use of AI coding agents by its customers. Crane's use of an AI coding agent on the Railway platform wasn't exploring new frontiers or wasn't supposed to be. Meanwhile, Crane has been provided no recovery solution, and Railway has apparently been hedging carefully regarding any such possibility." This is really interesting. They had, to conclude the story, they had a 3-month-old backup which was restorable from, so the deletion gaps were just limited to that 3 months. And there are some lessons learned here, but I think what's really interesting here is time to recovery, right, is a key metric in just building and running your own software. And this story exemplifies that that is really important even in the age of AI. You need to be able to get your basics done, your day 2 ops, you running, you know, making sure your backups are backed up, making sure that you could swap and green-blue deploy or do your canary builds or things like that, prevent disasters. In this case, it looks like the, The interesting thing is that his harness said, "Never fucking guess," and then this agent decided to just YOLO it anyway. That kind of tells you that the tools are not good enough for just true fully autonomous. And that intersects with some of the news we're seeing where engineers are being hired for their ability to steer long-running autonomous agents. Right. Obviously, don't be staring at this way, is what this story tells you. But there's some lessons learned here because I think in this particular case, they went with Railway, which I think did them in, it looks like, just reading the story from the outside. Like, no good service backups, things like that. They're relying on this thing for production, and it doesn't look like their customer service was that great. So in the age of AI, really easy to ship, fast to ship, but be wary of who you pick for your infrastructure providers. I think that's a key lesson from this story. Yeah. Jonathan, any other thoughts from your CTO perspective here on this sort of meme/story?

It's a meme, yeah. I mean, I feel sorry for the guys that it happened to, the rest of us. It's just kind of a, um, yeah, it's—

Yeah. Yeah. Sorry, Pocket. Sorry, Pocket, PocketOS homies. Hope you guys didn't get— lose too much sleep. Hope you folks didn't lose too much sleep. Let's go to our, um, next AI meme. Uh, Waymo. Waymo's autonomous taxis remain strangely drawn to floodwater. Somewhere inside the neural net is apparently a very determined salmon. Waymo has reportedly suspended robotaxi operations in yet another city after one of its autonomous vehicles drove directly into floodwater and got stuck, continuing the fleet's ongoing scientific inquiry into whether roads and rivers are spiritually the same thing.

You know, I saw a puppy on TikTok do the same thing.

He's like, wait, just run towards— The water or?

Yeah, it was like this really clean puppy and it like went right into like a big mud pond and then just got stuck in it. And his owner was just laughing. It was making the biggest mess. But I wonder if there's just something inherent through the cognition that says like, let's do this.

Yeah. Yeah. That's funny. Like, a young puppy's brain is similar to the young Waymo that we were seeing.

Yeah.

Little Waymo is just dunking its head into the water again with people inside of it that look like they're scared to death. Oh, man. So this was in Atlanta. The company has paused its operations in Atlanta after one of its self-driving vehicles was seen driving through a heavily flooded street where it eventually got stuck for around an hour. This comes after Waymo also temporarily suspended its San Antonio service last week and issued a voluntary recall for nearly 4,000 robotaxis. At the same time, the company said it was preparing an OTA software patch that would fix an apparent inability to determine when an area is too flooded to drive through safely. That's so fascinating. And I saw that Uber is getting into the game of more AVs, um, as well.

Yeah, really interesting edge condition. I mean, I'd say in the whole, I've been impressed how quickly Waymo has been able to roll out and get an effective service safe. They haven't had too many problems in San Francisco.

Yeah.

It's a, yeah, new terrains, new problems.

Definitely. Yeah, that's what I'm seeing here. Like, I didn't even know they were rolling out in Atlanta and San Antonio. I'm not surprised that they have and then are encountering problems. So once they rough out those edge cases, then hopefully this goes back to expanding operations as usual. 'Cause I think Waymo is actually a really great product. I had a great, shout out to Waymo, I had a, time where in San Francisco they towed my car. I drove to SF to meet up with Jonathan, actually. And then they towed my car, and I had my dog with me. And I was like, oh, shit. I don't know if I'm going to find an Uber that will allow me to bring my dog in the car. Guess what? Waymo doesn't have any such human. So I called the Waymo, and the Waymo brought me to the car towing place, the impounding place, or whatever it's called.

Oh, wow.

Yeah. Thank you. And, and it was really nice too, because Waymo, the experience is there's no awkward sort of like, should I talk to you? Do you try to talk to me? It's just like, no, I could just be a quiet nerd in the back with my dog. Uh, so I like the service personally. Jonathan, what do you think of the service?

Oh, it's great. Yeah. I think, uh, in general I prefer it. Um, still like mostly pick based on price between Uber and Waymo, but Waymo's nice.

Yeah. Definitely. Yeah, this is, this is interesting.

They can't take you to the airport yet in San Francisco, which is a little unfortunate. But I think like, you know, like if you look at the training data, these situations in Atlanta and San Antonio come under, you know, maybe the, the Bay Area is just too, has too good of weather and they don't get like a full range of like training data from the the circumstances that they might encounter and more like with a wider range of climates in a given area.

Yeah, that makes sense. That makes sense. Yeah, that makes sense. Uh, like a lot of sense actually. I, I wonder if that goes into their rollout plan, which is try to get geographically similar places lit up first on the roadmap. That would make sense. All right, let's, let's look at this. So next meme. Stop Slop. Thousands of developers needed a skill file to stop Claude from sounding like Claude. Stop Slop is this GitHub project with a skill file that is just a skill for removing AI tells from your copy or your writing. Let's look at this. I think this is really interesting. I've used this myself. Uh, so there are, there are a bunch of phrases and stuff here that stop slop, stops your slop from getting out the door. So cut filler phrases, remove throat clearing openers, emphasis crutches, and all adverbs. And they reference what that means here. Throat clearing openers. Here's the thing. Here's what X, here's this X. Here's that. Yo, this is so real. I'm going to be honest. I'll say it again. Can we talk about— here's what I find interesting. Here's the problem though. Emphasis crutches. Full stop. Period. Let that sink in. This matters because— make no mistake. Here's why that matters. Wait, this is so good. Um, killing adverbs. Meta commentary hint. Yeah, I've seen this one a lot. X is not a— oh, I've seen this one so much. X is a feature, not a bug.

Right.

That's, that's so funny. Like, it's so funny because like what writer would be writing this as well? It's only an engineer-oriented writer like Claude or ChatGPT or whatever. Like no writer is writing this. Performative emphasis creeps in. I promise they exist. I promise. Telling instead of showing. This is genuinely hard. This is what leadership actually looks like. This is what X actually looks like. Interesting. Broken formulaic structures. Okay, binary contrast. Yes, this is huge. Not because X, because Y. Not because X, but because Y. X isn't the problem, Y is. The answer isn't X. It's Y. It's Y. It feels like X, it's actually Y. The question isn't X, it's Y. Not X, but Y. Not X, it's Y. Isn't X, it's Y. There's a lot of this binary contrast happening in AI slot land.

I guess what I'm wondering is when it says don't do this, does it give better alternatives? Is it like, I mean, is it really actually able to morph the embedding structure into something that's gonna be better writing when, when it removes these, these kind of phrases?

I, in my real world experience, this is how I think of it. Like, I use AI to create me something that's like maybe 80% there and the last mile, 20% has gotta be me. And this maybe removes 85%, it's like 85%, not, you know, like I think it, when I've used this, it still comes out feeling AI-like and there's no soul to it, so to speak. In addition, for me, I actually like some of the, for my own voice, I actually like some of the stuff in here. Like for example, using some transitions.

Yeah.

Well, I can't find it right now, but I think I did like something from that we were just reading out. But anyway, for any AI builder out there that's gotta create content or write stuff that goes out to other humans and you don't want it being sloppy, check out Stop Slop. I think it's a pretty decent skill. I've used it a bunch myself. You could add to the skill and add to the rubric that it has for its rules. Um, and yeah, cool. We'll add this to, uh, to the pod show notes. Go give that a shot. Um, it's kind of funny that you have to tell AI not to sound like AI. Isn't that like a funny meta thing, Jonathan? Like, I'll give you an example. I've always used em dashes my whole writing career, and then now AI has made em dashes like not cool. Like, it's just— and I'm like, fuck, like, but I've always liked them. I always thought they were cool. So it's sort of a weird meta or cultural thing. What do you think of this AI writing-driven era?

Um, yeah, by the way, the Opus 4.8 was supposed to be very strong at writing. Um, testing out for some of that, but I don't know. I mean, like, you gotta, you gotta step back here and look what's going on, which is that Like we're indexing language as if it is, as if the syntax is the semantics. You know, that's the whole idea of a transformer and underlying embeddings. And like when we speak language as human, like we have phenoms with affect that we interject and it misses all of that affect. It misses all the subjectivity of how the actual syntax is interpreted. Will AI ever be able to overcome that, like, incorrect definition of language and, like, write well? You know, it can combine concepts together, but can it actually, like, write in a way that, you know, is what we would consider good writing before the age of AI? I don't know. I'm kind of skeptical. Like, I just think it might get like 80, 90% of the way there, but just like it, it doesn't have the underlying understanding of what language is in order to be able to produce good writing is my overall thesis.

Yeah.

Good at coding, good at math, not gonna be a good writer until like there's, there's some other kind of innovation beyond the transformer. That's a good take.

That's a good take. All right, check out Stop Slop, guys, and let us know what you think of it. All right, next meme. This is our last meme for the week. Pixel Agents. This VS Code extension turns your AI agents into the Sims. If you're an AI builder and you wanna have fun in your coding sessions, maybe this is for you. Pixel Agents is this GitHub repository, a game interface where AI agents build real things. Pixel Agents turns multi-agent AI systems into something you can actually see and manage. Each agent becomes a character in a pixel art office. They walk around, sit at their desk, and visually reflect what they are doing. Typing when writing code, reading when searching files, waiting when it needs your attention. Right now it works as a VS Code extension with Cloud Code. The vision though is a fully agent agnostic, platform agnostic interface for orchestrating any AI agents deployable anywhere. Look at this. That's so funny.

That's like—

it's like the stacked harness architecture of, hey, what are my agents up to? But in video game form.

Like the Black Mirror version. Did you see that Black Mirror episode where they had the little tweets?

No.

Yeah, it's kind of like that.

Uh-huh. Okay. So, what was in the Black Mirror thing?

Well, there was like kind of— there was like a little world with these little characters that looked kind of like this. I think they were actually little birds. And they had— they could develop highly complex collective behavior and they ended up taking over the world. So it was a little one that— it was one of the dystopian episodes of Black Mirror.

Okay.

Yeah.

Oh, that's so hilarious. That's so interesting.

But I think practically this reminds me of like Paperclip, you know, the, for managing cloud agents that have roles and responsibilities, your org chart. It's just a way to personify it, kind of cute characters. I actually expect we'll be seeing a lot more of these kind of visualizations of agent and agent for agent management.

Yeah, I think so. It makes it easier to understand and see. And I'm, I'm just showing on the pod Paperclip, so you, you go from like hierarchy org chart to what are people, agents in my org chart doing. My CTO agent, CMO agent, cloud code agent, et cetera. Um, and yeah, I, I think this is a really fun way to do it, which is the Pixel agent version. They have 6 diverse characters. Uh, interesting. Task, tool, subagents spawn as separate characters linked to their parent. That's cool. I'll have to try this later. I will say, like, the development experience is such a key one in the AI era because you're just at your desk all day just looking at terminals and shit for hours and hours and hours. So anything you could do to sort of spice it up and make it interesting, I would highly recommend. I myself use this Warcraft 3 peon ping after— it's a hook that registers after Every single completion, it will say something like, if anybody is a Warcraft 3 nerd like I was, it will say something like, "Ready for work?" or "Job's done!" or things like that. I think it's freaking hilarious. Yeah, I would highly recommend just fun little gamified things like that in your developer experience. It makes the coding sessions much more fun. On that question, how do you make your coding sessions more fun? Jonathan, or do you think about it that way?

I don't know. I think like the fulfillment for me comes in like the productivity. So I guess like just managing sessions and having productive outcomes kind of gets the dopamine going for me and I'll spin up like 2 or 3 at a time and gravitate towards the ones where I think I'm making the most progress.

Cool.

Nice.

Do you think these pixel agents would be a thing you would use? Or sorry, go ahead. You think you were saying something?

Yeah, I think— yeah, exactly. Like, I have kind of a soapbox to get on about anthropomorphizing agents as people because I think it creates— like, we're already at risk with social media and with AI of becoming more disconnected with one another, right? And you start anthropomorphizing agents as people and you think like, oh, I'm the boss and I get to like move around these characters and like, you know, I'm in charge of stuff and like, you know, you start to develop a mentality that is inhuman, inhumane, and helps— it hurts and harms your ability to relate to other people. And I think the way we really want to work at organizations is like, what loops are these agents in? What people are they empowering? Like, how am I empowering myself and the people around me with these agents? I think that's— and I think anthropomorphizing a machine as something that you have a relationship with and that you have a control structure over just sort of like trains your mind in a way that is, you know, suboptimal to participating in society. So yeah, that's end of soapbox. But, you know, I think it requires like a proactive innovation to say here's a better way to like, you know, here's a better visual interface for this.

No, I think it's a welcome soapbox and it's not something people are talking about enough. Um, yeah, maybe a question there to follow up, and then we could pick this up on a future pod if the listeners are interested. But do you want to give more light into why you feel so strongly about the— we should stop anthropomorphizing agents— and what do you think about how Anthropic has been doing a lot of that recently, um, with their research work? Like, they even have their researchers kind of talking about how Claude feels human, you know?

Yeah, they— I think several— I mean, there's probably not like a unified philosophy throughout the company, but several of the researchers do anthropomorphize agents, and they even have like a whole consciousness research team that the one that produced the papers about emotions showing up in agents that is, you know, like— they're not actively calling the agents conscious, they're not actively defining what consciousness is, but they're saying that, like, there is emotionality found in, like, the neural networks, and that's kind of, like, interesting, right? So, I mean, my overall premise is just that, like, technology changes who we are, right? And it changes the way that we interact with one another. And I think Like, you can just see the world kind of— the social fabric of the rituals and traditions that for all of the human millennia have held us together are fading, right? And technology has the capability to make us more isolated from one another. And like, you're— if you anthropomorphize your agents or your your AI, um, you're just accelerating that. You're just— you're disconnecting from real humans and saying, like, let me— let me apply the empathy cycles in my brain towards, like, these AI tools, right? Um, that's— that's not like, uh, that's not gonna— if you have everybody in the world doing that, it's going to be a, like, a psychotic break. I mean, like, everybody's just going to be— you're— you're basically moving towards the world of The Matrix where you're living in your own reality surrounded by an AI environment and you don't even need interaction with other humans, and then you're going to act like you don't need interaction with other humans and you're going to make decisions and vote in ways that you don't need other humans. You know, that just does— it breaks everything in terms of like how the humanities relate to technology.

Yeah.

Yeah. It'll create a world really quickly that like we're like, what happened? And by the way, like those of us who are anthropomorphizing and are engaging in this reality will be divorced from like the rest of people who aren't developing technology. It'll create this like sort of like divide in belief systems and, you know, and that's the kind of thing I think can create civil war, right? Where it's just like you're people literally living in different realities. One group of those people has completely atrophied their skills to relate to other human beings. Um, you know, like it doesn't end well.

And they might look at their agent as like a godlike deity. And that could, you know, we've seen what religion does. Differences of opinions of religion has created a large amount of death over time and throughout history. So yeah, that is, that is an interesting thing, especially because, um, Here, Anthropic— this is an article— Anthropic identifies emotional vectors inside Claude that can influence AI behavior. Instead, the researchers suggest that a calibrated form of anthropomorphism, long viewed as a conceptual and ethical risk, may be beneficial for aligning LLMs. Yeah, I gotta read more into this to have an opinion here, but I think it's interesting that this is a thing, uh, and I guess For the AI community listening to this podcast, send us some notes, comments, like what do you think about anthropomorphizing AI? We should start a conversation around that as AI builders. Jonathan presented his strong opinion that it will create matrix, matrices, little worlds that we'll be in and it'll decay our empathy skills towards one another., if we're using these empathy cycles with agents and anthropomorphizing them. I'd be really interested just to hear what other people are thinking and your reaction to that take. So definitely hit us up. So that was a little darker than we typically tend to go in AI memes, but you know, sometimes you gotta, sometimes you gotta laugh, talk shit about life, and then talk a little philosophy. So that's what we did here on, on the pod today. Let's go into the last segment of What's the Meta. In What's the Meta, we go over new tool drops or new meta, essentially, like new ways of doing things in being an AI builder and operating. We're both AI-native founders building our AI-native startup, Clarity, out in public, and we are learning and improving our skill sets every day, as you can see on this podcast. So in the What's the Meta section every single week in our weekly AI Clarity series, we bring to you a few things that we learned in our own meta this week. So this week there's a couple things that we're going to go through. Um, number 1, Microsoft WebWrite. Number 2, um, spec— specification artifacts as HTML, not Markdown. And number 3, how an OpenAI product manager ships features Monday through Friday. So let's get into it. Um, WebWrite by Microsoft. These are terminal-native web agents. 1,500 lines beats every single web agent out there and doubles performance with zero model changes. For your end-to-end testing. Here's what this means for anyone doing web automation, scraping, or browser testing. You could build web automations for a couple dollars each that used to cost your team hours and hours of time, and every task you solve makes the system permanently smarter. Doesn't that sound amazing? Uh, that's the promise of Microsoft Webwright. Let me share my screen here. A terminal is all you need for web agents. Webwright gives the model a terminal, a local workspace, and the freedom to write code that launches catches, inspects, and discards browser sessions. The output is not just a completed task, but a reusable program. So let's watch the traces. Here's one, Webwright versus vision-based approach. Webwright on top. Here we go. GPT-5.4 repeatedly fails to click the exact $300 slider position. That's the vision-based agent. But at the top here, here's Webwright. Writes code to reliably position slider at $300 instead. If anyone's doing any sort of front-end testing, which I am, and if you are out there, you'll see this and you will be like, whoa, wait, what? Hang on. I can make it test itself and it might actually work this time? That's what I felt when I saw Microsoft Webwright. I thought that was really cool. And here's Cloud Code with Webwright and without. So the bottom failed to obtain the result because Chase IRA calculator is interactive JS tool. Cloud Code can't drive the sliders or render the chart. But— Yeah. With the top, Cloud Code with Webwright, they're able to actually do this. Let's speed ahead here. Wow, pretty cool. Task reusable script created, reference UI run that opens Chase's IRA calculator, sets every input correctly, and displays the traditional versus Roth comparison chart and table. And there's a bunch of other examples like this where you could see the trace go through and see what the agent is able to do with Webwright. Yeah, they show booking Ticketmaster or flights with and without the skill. Jonathan, what do you think of the promise of this tool? I haven't had a chance to try it, but I've used the other tools we've been running like Playwright or Maestro and iOS Simulator or Chrome DevTools. So I'm curious what you think about this, the promise of this. Yeah.

Yeah, it's just an abstraction on top of Playwright, right? I mean, very welcome if it works better or tests better.

Yeah, I think the point of it is that you could then just have the agent use it on its own. I think one important thing, Go back to my notes. They boast a nearly 87% accuracy rate. Got to look into more of what that means. I think this approach is more of letting the model write the Playwright code. Seems to be what you would get with this approach. So instead of predicting clicks and stuff like that, it'll actually write Playwright code.

Right. Yeah. Seems like a cool experiment. You know, it is a research project from Microsoft. It's not a commercial product. It's not a— I don't know if it's a well-supported open source project or not. Like, will it continue to evolve? But as a concept, it's really cool. And I think it's solving a lot of the pain points that we have with this browser testing.

Yeah. GPT-5.4 benchmarks. We're at 33%, and with this harness, you get 60% better accuracy for testing with zero model changes. I think that's really cool. Uh, I hope to see more direction here with investment into automated testing for meta improvements for AI builders. Um, let's move on to the next topic.

Yeah.

An Anthropic engineer says Markdown is dead and 99% of his tokens get thrown away on purpose. Instead, HTML is the new Markdown for your input artifacts for code generation. Here's what it means for teams using Cloud Code and AI coding tools daily. You can now spend just a percentage of your tokens on production code and make the other 99% worth throwing away by building thinking artifacts that make every future session smarter. That's sort of the promise of harness engineering without just pure hyperbole. But the thinking is, can you spend a lot of time on— because code generation is cheap nowadays— can you spend a lot of time on the specification and then maybe even generate some code as throwaway to learn and service unknown unknowns?

Yeah.

Tariq, I think I'm pronouncing his name right, Tariq Shishipar, engineering lead for Cloud Code for Anthropic, was featured on a podcast we'll link a week or two ago. And he actually just talked about a new meta of writing your specs into an HTML file. So that you as the human would interact better with the specification. And I'm gonna show you an example of that. One second, check this out. This was for a specification for building a new feature in this app, Clarity Context, that I've been baking. And there are some key problems. That we outlined in this. So you can imagine this living as a Markdown file. I had it critique our current UX. There was a 4-step wizard. There was an interview sort of portion. I had it give me a before and after experience of what onboarding would look like with an estimate of time to magic moment from product perspective. I had it also outline what components we would be touching. And give me a before-after, like what's it going to look like in the UI afterwards. So because this is HTML, you can actually inject your design system. So I think that was a really cool part of creating this specification in HTML where I could actually see the Storyboard design pages and what it's supposed to look like. And not only that, the AI has these components now as code, at least the frontend code. I've found it more likely to implement it in one shot, the frontend code to look exactly like this. There's more down here, but this is files to change, how long it thinks it'll take, the TDD, test-driven development exit criteria and test strategy, different milestones around that. And the execution order. Then I had it also create me a prompt in a goal loop so that I could execute on this until it hit all the exit criteria. This is an example of doing spec-driven development with an HTML file. The main difference is that you can't go in here and edit this very easily. What I do in practice is I'll have a Cloud Code session generate this and I'll be talking to the Cloud Code session on the— Yeah. Side of looking at this. And I'll just whisper flow a bunch of the changes that I would like made in the specification before iterating. Cool. I'll pause there. I know I said a lot. Jonathan, what do you think of HTML specs?

Yeah. I think it's great. I think that maybe the surprise here— because I mean, like, everybody knows HTML. Like, why are we talking about Markdown versus HTML right now, right? I think what's interesting, I think, is that the agentic models have been trained to communicate in HTML primitives like tables. And you get a lot more out of the box than you specify, right? So I think for specs and that kind of thing, it's really good. I think there's a divide, you know, your interactive previews and complex specs, HTML is gonna communicate those well. For RAG memory and machine handoffs, Markdown is a more efficient, token efficient, you know, standard. And then if you need, like, you know, if you need to make edits on a document, you know, like you're kind of in this modality, like you were talking about, you take pictures of the HTML and and then talk. So you're having the— you're having a language model generate syntax as HTML so that it can render into a browser. You can take a picture of it and then ask the model what to update, right? As opposed to generating characters on a word processor and then editing it and code, code, code, like authoring something with the— And So that's, that's an interesting communication paradigm. Um, but I, I think like we're just in this kind of like infancy stage where somebody will probably develop like a specific version of HTML that is meant for like specification communication. Like, you know, kind of like, um, a React library or something like that. A, a Python library.

Yeah. Yeah. I could see that. Like a React spec for, for agents.

Yeah. Yeah. Um, you know, cuz like React has all the com— components, I think, yeah, pattern library of components that are typically used for communication in specific interdev— development. Um, I expect to see more evolution on that front. Uh, and, and like, I just think we're like in the early days of all this.

So this is really interesting because this guy is on the Cloud Code team and these are some of, this is a peek into how do these people prompt. So these are the best in the world. So this is one of his prompts, super, super simple and deceptively elegant. Create an HTML file as a plan that helps me visualize what the implementation plan is. Include excerpts, mockups, code, whatever is needed to give me like maximum context. He had this like in there. He probably voiced it, dictated it. This was an interesting thing as well. Pro tip for the AI builders out there. If, you know, if you have an objection to using HTML in that it's not as easily editable as we were just talking about as Markdown, um, Tariq had a brilliant solution. It's not personal software, it just creates a micro software. So ask Claude to build a temporary disposable user interface just for editing that one specific part of your plan. I was, I was like, wow, that is like That's taking code generation software, you could just create that on demand. That's taking it to the next, next, next, next level. Let me create a micro app to edit this spec, which informs code generation. I thought that was really cool. Here's an example. He gave Claude a prompt. I want to create an editable, I think editable is what this should read. HTML artifacts that help me define the decision rules. I don't like the ones we have right now. Make this a custom UI that helps me with structure but gives me flexibility. Design the ideal interface for this problem. And you could see this chart picker, decision rules.

Is it all HTML? I mean, you're— I think by the time you're creating an app, you're introducing a lot of JavaScript.

Yeah. I don't know. Like, this is in HTML right here, this file.

I don't know if Um, because like, yeah, I got it. Like, I think it's the paradigm of like create an edible app versus a Markdown document. Um, it's not HTML is just kind of like a shorthand way of saving that. But like, um, one thing, I mean, I created this HTML strategy document that I wanted to discuss with you. How do I share that with you and how do we discuss it together? Right. Am I going to deploy it on Cloudflare or something? Um, and then, you know, like I could take the local images that I have in my folder and display those. Via local web server, um, you know, then I've gotta upload the complete asset and I'm basically deploying an app to like ask you for feedback on a document. So if, if I could do that really easily, that's great. Um, I just, it just seems like a whole other thing to support for, uh, document stream, right?

Yeah. I was just thinking what Slack should be primed to build out that feature, right? Slack should be like, we, we should make that, that's a collaboration thing, right? Shout out Slack, please do this and we'll test it. That would be sick. But yeah, I was just thinking to make this really useful, because one of our key practices as a team is sending each other files to review, right? And if you ship your Markdown file into your PR, it's reviewable there. In this case, if you ship your HTML file into your PR, Yeah, I guess I could download it and open it, but there's no easy way to sort of edit it. Like, 'cause in GitHub, you don't get like a nice preview that I could decorate that is UI-based. It's like front-end-based. You still just get the code snippets of what the HTML looks like. We'll probably see, like, just as you said earlier, we're probably gonna see people create tools around this 'cause the AI-native landscape is just evolving so fast. So I'm sure somebody's gonna be creating something around this. To make it easy to collaborate on specs and HTML language. This is interesting. What'd you think of this? Which is the living design system meta. I love this. I started doing this right away. And I like, yeah, I just built such a cool design system. Here, I'll show you one of the ones we made for a client. They just changed their brand guidelines and using this, like I just were servicing their mobile app and integrating their mobile app with Clarity API. And so using this, I was able to easily just go to the mobile app repo, talk to it in Cloud with Cloud Code and create a design system HTML file that has all the components that we would use in the app. And then I had the thought that, hey, anytime we create new components, we should check this. Like a Cloud— the CloudMD file should— that harness should evolve. We should check this whenever we're checking in new components or there's a new component to be made. We cross-reference it and then evolve it as part of this design system. So the component tree all lives up to date in this file that is updated in the codebase, which I thought was really, really cool. So I love this pattern. I don't know if you got a chance to check out this pattern, but I absolutely love this pattern. And think it's such a useful thing to have the design system just be part of the codebase.

Yeah. Yeah. I, I, Todd sends me a lot of stuff from like Meng To and his, he's a designer and builds design systems. And this is, he's got very powerful tools for creating baseline design structures like this. But yeah, I think it's really empowering, right?

Very empowering. What's the point of Figma here? Don't need to go there anymore. I know a lot of design nerds will probably hate on me for saying that, but try this yourself. Try this design system HTML file. Yeah, and the meta here is when you start a new project feature, you can just give Claude that design system HTML file as context. So you're always building on brand. Um, there were a couple other— yeah, there were a couple other things here, but, um, yeah, I think that's good for this What's the Meta item for now. Jonathan, anything else on, uh, HTML spec documents in this What's the Meta from your CTO perspective?

I think it just— in our usage, it's missing the Google Docs or Slack interface that— how can we share ideas with one another? Otherwise, it's just really interesting tool direction.

Definitely. Yeah. Cool. So let's go to our next topic then. Let me find it. So our next topic, How PMs Ship 100,000 Lines of Code at OpenAI with Ryan LaPopolo. This is by Akash Gupta. He's an AI influencer in the industry. And in this pod, they basically talk about how OpenAI PMs ship code. So they'll take a spec on Monday and by Friday it's in production. That's how fast they ship quality features there. And you could be a PM shipping, which I thought was really cool. So I'm just going to read out some of the takeaways from this pod and you should check it out for yourself if you're interested.

Okay.

OpenAI PM ships a million lines of code, zero written by humans. Oh wait, no, that's not good. Hang on, we'll rip that out. OpenAI PM ships a feature by Friday after a spec was defined on Monday into production for hundreds of thousands of users. Here's what it means for engineering teams trying to adopt AI coding tools. You can get your PM shipping features from a PRD to production without touching an engineer while your engineers focus on encoding more leverage into the harness instead of typing code. Ryan LaPopolo, engineering lead at OpenAI, described building an internal agent from an empty codebase to a million lines of code with literally zero lines written by humans. The core thesis is that code is a liability. Human engineering organizations were built around code being expensive to produce. With agents, code is trivial to generate at arbitrary parallelism. The overriding constraint has been removed. 250,000 of the 1 million lines of code are Markdown prompts embedded in the agent, such as when you're shipping an agent, what you're really shipping is a multi-agent system. Improving behavior is prompt engineering.

Mm-hmm.

A PM was able to, from this foundational level of workflow, a PM shipped a feature from PRD, a markdown file, to production in 1 week with zero engineer involvement. The PRD was reviewed Monday, demoed on Friday, rolled out to customers. He says that encoding leverage is the name of the game and that every time the agent makes mistakes, the engineer should stop, diagnose, and encode a fix into the harness so the mistake never happens again. The repo itself should give context to the agent if properly done right. Um, there's another few quotes here that are interesting, and then we can get into some of the discussion. Uh, but some other takeaways for AI builders: if you're not using a billion tokens a day, you're basically negligent. That is about $2,000 to $3,000 per engineer per day. The justification is more tokens, more intelligence extracted, more parallel work. For the harness structure, what do they mean by encoding leverage into your harness? That could be the agents.md file in your root, which is forcibly injected into context, the massive docs tree, like front-end architecture, reliability engineering, et cetera, lms.md. TXT files checked into the repo for all major dependencies, and a quality score MD for ongoing codebase gardening, he calls it. They have persona-oriented review gates with agents in CI, where a frontend architect will review for frontend architecture, reliability engineer reviews for reliability, and goes on and on, API design, expert, et cetera, each with their own guardrails and each living, leaving P2 feedback on the PRs. The code that they ship is reviewed and approved by machinery before a human ever looks at it.

Okay.

Okay, that's a lot. But yeah, that's a sneak peek into what do the best engineering teams ship like now and how do their PMs ship. The best engineering teams, my takeaway, at frontier labs such as OpenAI and Anthropic, are encoding leverage, improving their harnesses so that a technical product manager could easily ship features on top in a week. So let's talk about that. Any of those what's the meta sort of items jump out to you, Jonathan? The— everything I just sort of talked about from the encoding leverage and the harness, using a billion tokens a day as the recommendation path and persona-oriented review agents in CI.

Yeah, I thought you might have missed the overarching context by— this was an investment that the team made. And the first month was 10x slower than just coding the old way on purpose. So, like, not only is OpenAI investing a lot of tokens, but they're also saying, like, let's just throw away a man-month of productivity, you know, per engineer on this project so that we can lay down the foundation for the compounding system. And so, like, that's, that's a little different than the headline. suggest, right, is that this is a J-curve. And, you know, that's— I think you have PMs learning what engineers know, which is that you have to lay down architectural foundations and like, you know, have a harness that can say like what is good and bad and be able to like have a compounding system there. And I think the compounding system described The compounding asset is the constraint layer, the tests, the lints, the review agents that encode what good looks like. Build that first. The machine produces at scale against those constraints. Nobody gives the same feedback twice. So I think, like, and I think, uh, Every was pretty, like, early on this front with their compound engineering plugin. And, uh, these guys at OpenAI were obviously, like, doing the same things and just talking about it a few months afterwards. So yeah, I think that's the thing. Like, are you— you invest in the harness so that you have a J-curve type of investment structure, and then you can see it getting better and better. And I guess what I've been seeing missing from this is the, the quantitative measuring of the compounding, right? Like, it is an interesting story to say, like, okay, when the new model came out, we got 2 more PRs per engineer per week, or something like that. That's, that's an interesting heuristic. But does it justify $2,000 in tokens a day for the outputs that you're getting? It seems like, you know, OpenAI is building really expensive software by being committed to this token approach, but that it is yielding returns as time goes on. And so it gets cheaper and cheaper over time. But are they saying we have a measurement and evaluation structure of how good this is compounding so that we can like use less tokens on the future. You know, I don't really see that yet. I think this is kind of like the token maxing philosophy, which is just like throw a shitload of tokens at this. Let's learn how to do this. And I think that's kind of like the format. So I don't think this is a reproducible thing for a lot, for a lot of teams, but that there is a J-curve there for you to invest in and get a compounding return on.

Yeah, I like what you said there, which is it might not be reproducible for a lot of teams unless you're rolling in dough. Token-wise. And if you have the benefit of not just budget, but also a good enough team where your engineers know how to bake out a really great harness in under a month, that's world-class. So that's also really key. So, but what everybody could take away is invest in your harness. Like, just do that. Like, I— One, one Takeaway is just do that. And what that means is your agent's MD file, any of your context documents that get loaded in the context window when the coding agent interacts with the repo, um, your cloud MD file for the repository, et cetera. So that's definitely something that I've seen some gains in on my own, uh, not measurably, but just like, uh, how many times I have to steer, I suppose. Around the same sort of problem. Recently I had this— connecting this, a tactical piece of advice for the AI builders out there— I used this tool called Claude Mem, which has some minimal amount of memory persisted across sessions for Claude to pick up to minimize my context window usage. And after about a month of running it, I have a rich amount of context around my workflows now. I then asked it, I was like, hey, how can we improve our harness based on the past month of, of memories of coding sessions where I had friction? It actually came back with many, uh, things where it saw I made lots of noob types mistakes, or in this repo you're doing this or whatever. Uh, and I was like, oh, great. Okay. Let's improve that. And so just that one habit. Yeah. Of compounding, that's what I would recommend teams do. Maybe don't get too lost in the sauce of trying to replicate what OpenAI did here, but at the very least, try to improve your harness over time across coding sessions. And that, that's easy enough to do these days. Any other thoughts on improving harnesses or the whole OpenAI is investing into this model of engineering where they're really dedicating a month of work just to get that J-curve type of return, Jonathan, in dev velocity?

Yeah, I think it seems to be, I mean, it's consistent with kind of like the YC token investment. Um, a lot of this is just learning and learning like how to do it, how to work, how to run experiments. I would just put it in that bucket of stuff. The specifics of how do we train, like, like what errors do we log in our memory that, you know, create the constraint-based harness so that each incremental PR is better. I think like these systems are currently kind of sloppy in that sense. Like they don't like have a good creative foundation for what it is that we're doing. But so let's just capture everything and have the AI figure it out. I think that again, early days, we're probably going to see more advanced techniques for this coming up around and you just want to like experiment and learn what the, um, the right way to build your own constraint-based harness is.

Yeah. Really great way to put it. We're all learning on the front lines of AI here. And that's a great place to end the pod, actually. We're at the last What's the Meta item with this, um, how to ship like OpenAI as AI builders yourselves. Uh, to wrap the pod, thanks everybody for listening today. We went over lots of things., the Manas context engineering playbook, a little bit about their story there. Uh, Cloudflare and Anthropic delivering managed agents on top of Cloudflare infrastructure, security nuts and bolts included. Tom Bloodfield's YC talk on creating queryable AI-native organization in the future. Went through some AI memes of Claude deleting a database, PocketOS, shout out to the devs there. Hope it wasn't too many headaches and migraines. Waymo flooding. That was an interesting one. Hopefully people are safe, but also sort of funny that Waymos are like dogs and wanting to dive into rivers. Went over Stop Slop, a skill that we'll drop in our show notes. Let us know if that works well for you. Pixel Agents, if you want to anthropomorphize your agents and make your developer experience a little bit more fun. And What's the Meta, where we went over HTML as the new Markdown for Spectriven development. Microsoft Webwright, which helps with end-to-end testing, and then how to ship like an OpenAI, uh, PM here with our last item. Thanks a lot for joining our weekly AI Clarity series. Drop comments in the show, like catch us on Apple Podcasts, Spotify, YouTube. We're on all the platforms where you could, uh, listen to your favorite podcast. And please send us anything, you know, memes, topics you want us to react to and give our takes on, or if you have your own meta for becoming a better AI builder or better AI builder workflows, send those too. Um, check out the pod, subscribe, share with your friends, and we will catch you next time. Have a great day. Bye-bye. Bye-bye.