How We Shipped a Production App Using an AI Swarm

You have 13 days. Anthropic released its most powerful model ever free on paid plans until June 23rd, days after publicly warning that AI is getting too dangerous. I spent the first 4 hours since release running it on a real production cutover. This week, Jonathan, my CTO and co-founder, on the Self-Aligned Podcast had a customer launch for a big mobile app update for a customer, and we'll share some of our, tools and tactics and how we did this cutover. And in today's podcast, this is your weekly AI clarity. We have 3 segments: AI news, AI memes, because we gotta laugh being in industry, and then what's the meta. So the meta as AI builders for AI builders. So tools, techniques, tips. And today we have a bunch of demos that we're going to show about how we ran this, production release and some very tactical things such as specification as an HTML file or, uh, proposals in HTML files, um, how to consider your spec and then even like doing App Store optimization with Codex. OpenAI just turned Codex into an OS for office workers with Sites and other updates and plugins in this latest release. 5 million weekly users are ready in less than 10 days, and here's what happened and what matters for AI builders. Engineers and engineering leaders whose non-technical colleagues are about to get a huge upgrade in their ability to create prototypes, sites with agents. So here's a tweet, uh, OpenAI: Building apps has never been easier. So with Sites, Codex can turn your work, ideas, and plans into an interactive website or app your team can explore, use, and share with URL. So basically trying to go eat Lovable and Base44 and all the vibe coding, uh, application layer companies lunch money. Pretty cool. Let's watch the video. Super cool. So rolling out to business and enterprise plans before expanding more broadly? Official release Codex for every role, tool, and workflow. New role-specific plugins, sites, and annotations help teams do more with Codex. So lots released. Jonathan, first takes on sites that we saw the marketing video for here?

Yeah, I thought it was an exciting feature. I haven't used it yet. And I'd say it reminded me a bit of Anthropic's Artifacts when you use Claude. But like, it feels a little bit more robust. At the same time. One thing, you know, we talked about this product from Shopify called Qwik, which was an internal tool for sharing prototypes. And I think when you can get away from, you know, like we've been doing like a lot of HTML generation and, you know, then you have to like share the file and like use an internal server to like view the file. And so not having to use local server just by itself is a nice feature. Um, so I can see that being useful. And, um, I was curious if when you see this, do you think like, is Base44 obsolete? Um, you know, are these, are these like vibe coding tools for building sites like, uh, still going to be useful or, um, you just do all that within Codex now?

Yeah. Let me answer that from the broader perspective of the new AI development lifecycle. Instead of software development lifecycle, we're seeing AI builders move towards AI development lifecycle or AI DLC, which we've had some engagements on for, uh, different clients. So how do you build faster? So taking a step back, why do we even build software? Software is an asset we create for the company to increase enterprise value and capture market share and reliably make money, cash flow, right? Like, that's the whole point. So why do we ship a PR? Ideally, it's to increase our capability set, or protect the asset we have to build a moat, to either capture more market share or protect the market share that we have. So why does this matter? I think why Sites matters is in the bigger play of I am trying to be an AI-native company where my new lifecycle is I learn as fast as I execute to capture market share, which I think is the real goal of any AI-native company. Sites just brings the prototyping aspect, all that context. You, you show, if you think about it, right, a designer could now go to Codex and they make a site with it and they could share the link to a user and there's some analytics built in. Now you could do a jobs to be done or Teresa Torres style opportunity solution tree style, you know, sort of interview continues discovery habits sort of interview, and then pointedly point to that prototype and get people to walk through it. You could do the mom test, right? Like, and all the analytics are there, that context is right there when you go to engineering, right? And within Codex, you can just do all these things. And their latest model is so good at looking at a design, and then taking that and shipping working code. I think their play is to kill Base44 and Lovable, because those have remained as really quick and easy applications. But all the stories say once these scale to any level of production, they're going to have to like rebuild the whole thing, right? So it's been a faster way to learn from market what to build and even build an initial application you might monetize with these vibe coding tools. Base44, I would say I even prefer over Lovable, just like quick pro tip for the AI builders. The reason why is because they integrate database, uh, API, all the contracts, like, uh, that stuff, all that stuff is out of the box versus Lovable. You got to integrate with like Supabase and stuff. So inherently you have a dependency chain. Base44, you don't. Codex, I think, is taking the Base44 path, which is an opinionated take that this is the way to like, you know, go from the initial discovery or like create some prototypes or sites or whatever to get an idea and then figure out what to productionize with Codex all the way to production level engineering standards and quality. I think they also tangentially like at the intersection of that for more functional colleagues, this is like an opening for more functional colleagues to come in and feel like a magic moment right away, similar to like a Lovable or a Base44. More, right? And once they have that magic moment, like what we're seeing in industry is product managers who are more functional from their careers come in, they have a Claude Code magic moment, they realize they could start shipping as well, which is what every tech company and executive wants. You want your people to like start getting AI-pilled. I think this is like an on-ramp to, uh, being AI-pilled. And then also going back to my original point and thesis, um, I think they're trying to make it to where everybody is using Codex, for the AI DLC or AI native, uh, development lifecycle, moving from SDLC, uh, to learn as fast as you ship to capture market share and protect that market share. Um, those are some of my thoughts though. So yes, uh, in short, killing Base44, killing Lovable— I see that happening with this move. I think those teams are going to have to pivot. That's my take. There's no way they have as much data, um, as, as these, uh, frontier labs on these problems. That's my take. What do you think, Jonathan?

Yeah, it's, it's curious. I, I think I tend to agree with you on the big picture, and, but I also wonder, is, uh, like OpenAI and, and Codex going to continue to support this, uh, specific of functionality and keep it good? And like, when we come back a year from now, will this be something that was built once and shipped and they moved the engineers to another project, or will this be something that like really, um, you know, takes on all the, all the incremental feature requests that users will need to make it something powerful and part of their daily workflows? It's not clear, but Codex is releasing a bunch of features into the product and they're all worth playing with now. So it's, uh, it's good to see.

Yeah, I agree. Um, as a full-on AI-pilled AI nerd, like, it's just cool to see like the rapid change coming out. It's a little overwhelming and makes me anxious sometimes, but that's always on the other side of excitement. So, uh, yeah, it's interesting. Like, I keep going back and forth. It's like, oh, Fable 5 came out. Oh, let's, let's try it out with this, you know, uh, contract and like the work to be done ahead of us. And then now it's like, oh, Codex made sites. Okay. Maybe we should, our next prototype should be on sites. And then we, we get a grind rep to see how good are these tools when they meet reality versus the press release.

Right.

And that's really what all the AI builders want to know is like, how good are these tools versus the press release? And that's exactly the scoop we give you on the pod. So, uh, keep listening to the pod and real quick, if you're listening and you've listened this far, please hit us, hit subscribe. It is like the number one thing you could do for us to keep this going, to give you more signal and clarity from the news. From the AI news and noise every single week. So that's super cool about the Codex new functionality insights. What do you, you know, like, I'm curious what functions do you think changed the most with the new Codex features, Jonathan? Like go-to-market teams, marketing, sales, like product development, like design, like what functions do you think change the most with this?

I think the general push is just to be a better coding product, building product than Cloud Code. So I think there's all the aspects of that, that like, you know, OpenAI probably uses internally for their product building with product managers, with engineers, with designers, that we'll just see building reinvented incrementally, you know, through the Codex releases.

Just going over the press release, there's role-specific plugins that they have released out into the wild. We'll add these links to our show notes. Role-specific plugins make Codex easier to customize for a team's day-to-day work. These template packages— these templates package domain-specific skills, connector bindings, and a starter asset, uh, so teams can adapt Codex for roles like sales, data analytics, product design, and financial markets. Very interesting. They were built with OpenAI subject matter experts around workflows that are already helping teams move faster internally and with alpha partners. Over the coming weeks, we'll continue expanding this collection with more roles, workflows, and examples. Oh cool, so this is like they're starting to roll out harnesses around different workflows essentially for these domains. That's pretty cool, makes a lot of sense. They want— like I was saying, I think that this move with Sites and some of these other make Codex easier to use tools is like a mirror to Cloud Code with Cowork. And some of that coming together to dominate enterprise. Like, they're trying to win in enterprise, same as Anthropic. That aligns with their moves for the consulting services ventures that they both have poured billions of dollars into. It's cool to see a data analytics plugin, a creative production plugin, a sales plugin, product design plugin. This one stood out: public equity investing plugin. What the fuck? Like, what?

Oh, it's for fund managers. Yeah. It's for investors.

Oh, okay. Yeah. I guess I was thinking about that wrong, but I think that would make sense as a competitive intelligence to then power on the product side. Like, okay. Like the market info, but I guess I was reading this thinking that somebody might use this to create financial plans for a consumer. That's what I immediately read it. And I was like, that's so weird. Like that's out of, that's a little bit. Little bit out of left field, but I think they're trying to make sense. Yeah.

You see Hebbia at the end. They specifically make tools for like the finance sector.

Oh, cool. Okay. The finance sector is huge. So getting more investment banking plugin helps bankers turn research and diligence into client-ready materials. Oh, cool. You get to see a bunch of use cases here. Okay. Let's go to that. Oh, they don't, they're not showing the banking one. Uh, of course they're not showing that. I want to see that one. Oh wait, no, here. Public equity investing. Let's look at this. What you would work on: do a post-earnings deep dive on Kava using its latest report quarter. Cover growth drivers, new unit expansion, et cetera. And it looks like Codex is then going to work with this plugin or harness around the financial sector. And then what do you get? Oh, an artifact-like view of a really good deep dive. Wow, that's so cool. They're using this, um, HTML paradigm that we're actually going to show in the What's the Meta. They're using this for conveying information in other industries as well, including financial markets. So that's so cool that they baked this in as a native meta. Like, like, they, they see this as a way to share information and synthesize research even outside of tech workflows. I really love that. That's so cool. Like, just as a nerd, like, me and you have been sharing HTML docs with each other for various reasons, like strategy, business, and specifically like engineering product. But this is cool that they're doing it here specifically and they actually have it in the press release, like a visualization of a financial diligence report. What do you think of that? Like that move to bring in this, um, these plugins into Codex and then bring functional users in and then have them communicate through HTML.

Yeah, I have I have mixed feelings. I think it's a cool demo. Um, and you know, Anthropic's doing the same thing, like Claude Code for business, for small business, you know, and they have these like packages. Um, is like, it kind of reminds me like when you go into Google Sheets or Excel and there's like, there's like a template that you're supposed to, that you can use. And I find that like, it's never quite right for what I'm trying to do. And I have a feeling that these plugins are kind of going to be like that because once users who are in sales or are in private equity investing, they have their own data sources that may not be included here and they're going to have to, it's, it's so idiosyncratic and the model is so good at like helping you build it specific to yourself that I think like probably few, few people would like use these plugins, but they would say it like, oh, that's an interesting template. Let me like use that as inspiration. And I can tell Codex based on this template, can you build something personalized for me? Right. And I think that would be like the, the way more people would actually use it in practice.

You know, with Codex and Cloud Code dominating the enterprise, like, interface game, like, that is going to be the entry point for a knowledge worker doing lots of different knowledge work across lots of different systems. We feel that ourselves. We're using Cloud Code or Codex. Um, and if there isn't an integration with the system, we probably are not going to be integrating with that system. Like, right? Like, it's now become the standard that if you don't have an API or an MCP where I could easily talk to you from my IDE, Cloud code in the terminal or Ghostie or whatever you run, or Codex, um, I'm not going to be using your software. Like, that's now the bar. Uh, it's very quickly becoming the bar, I should say, and it's definitely the bar for, for us as an AI-native, uh, team. Um, what you, what you say makes me think of a question, which is, are harnesses at the application level just going to be a commodity? And then the next thing will just be, how do we get the user to build their own user harness as possible, as best as possible for any given agent they're interacting with. Because we could see these plugins as just out-of-the-box harnesses, right? And you're basically saying like, yeah, but it doesn't quite get what I want. And that's the user harness level. So there's a question there of with these interfaces, Cloud Code and Codex, uh, for the enterprise, um, as on-ramps for knowledge worker to do lots of different, uh, breadth and depth of knowledge work cross-functionally. Do you see that as the next problem to solve is how can we get these users their own individual n=1 hyper-personalized harness as fast as possible in the usage of our products.

I have this debate with a friend of mine, whether or not Anthropic and OpenAI would just be able to build all this domain-specific applications for like all people, and they're just going to be good enough. And like, as in, is there actually room for entrepreneurship when it comes to like creating domain-specific evals and creating special specialized harnesses for people? And I think that OpenAI actually doesn't have the end-user workflow data to create the specificities and the, the personalization that's needed for people to, to, to like say, you know, here you go, like here's the whole harness that you need for your job. And, and you're like, you know, we fully understand all the workflow you need to build. Like they don't get that data in their FDE flywheel, you know, uh, their forward deployed engineer flywheel. Like they, they just, uh, don't get to that level of nuance, that level of specificity. And, uh, they probably won't, I think. The labs, like the frontier models do want that data. The enterprises for the most part probably don't want to give them that data. And I think in the remainder of that, that like what they don't get is just like, it's that, is that everybody needs a personalized experience when you actually get to like, what's the work I need to get done today? And how does this specifically work for me and my data? What are my workflows? You know, like I think it would, there will be a separation between what entrepreneurs can deliver in value and like what, where you can deliver value on the edge and what, OpenAI could deliver through one of these plugin harnesses.

Yeah, and on the topic of harnesses, this is really what we mean by harnesses. It could sound really opaque and like, what the fuck is a harness? The harness is the new mode. Oh, or harness as a service. But just real quick clarification for the audience, this is simply it. If you're using a coding agent like Cloud Code or Codex, as we're describing, um, an agent is simply equal to the model plus a harness. So a model down here in the gray area, an LLM of choice, maybe one of the frontier models, maybe an open-source Chinese one. And then the harness on top, um, you know, the curated evals and datasets that actually power the different user queries, workflows, toolchain calls, all these things, that's this purple area. And then Jonathan and I are thinking like this purple area is really where, um, you know, you see Codex and Cloud Code, they're competing on all these benchmarks. Like Sui Bench, Terminal Bench. Uh, the new one is Deep Sui, um, for longer running, uh, autonomous loops. And then we have, uh, this concept of a user harness. So the user steering your own personal Claude MD file, like your global settings. Uh, and we're seeing that the Frontier Labs, like in the latest Codex release, they shipped a bunch of plugins, a bunch of skills as harnesses for different use cases. And those use cases are things like data analytics, or sales plugins, or how to take a prototype and get it to engineering, like all those different plugins. And we saw some financial ones. The real, I think, competition when we zoom out on the timeline in years' time, I think will be really at this edge. How do we get the user a harness? And then ideally, and this is where we come in, how do we make that harness portable? Across different agentic surfaces. I think that's going to be a really cool thing we see. So the take is purple, you know, harnesses, that, that's where the competition is now, that, that's going to be commoditized eventually. We'll get to some level of compute cost quality that feels good for particular workflows. And then above that, how do you use it for your own, you know, the rubber meets the road, this agent meets reality. I think the user harness actually is a, is a huge thing. We're going to go into that today on what's the meta with some of the deep dives into our own native workflows. But just a thought on harnesses to round that out for the audience there. And Jonathan, yeah, Jonathan, really good perspective that, you know, that middle part there is definitely the competition now, but we see that longer game, take the time slider to infinity. Like, I think it's going to be that hyper-personalization layer at the top there, the user harness. Across domains?

I think, uh, just the way things will evolve, I think like there'll be sort of infinite like unlock of, of, you know, new subjective experiences, like value proposition workflows that you're using in your job. Like, um, I don't think it's going to be this thing where software eats the world and it's gone. Software will like make things incrementally easier and then you say like, oh, I can do new things. And like this outer ring of the harness becomes something like, I wasn't doing this last week, now I'm doing this new thing. And like, like able to make more money or more productive, or, um, I think that's more— it's just more and more opportunity that the sort of more capable these models get. That's the attitude that I take about it.

Yeah, same. And I think that's the opposite of a doomer take, so to speak, right? They're, they're, you know, we're essentially seeing like, at least our takes, um, for this team, our team, is that we're going into more abundance. We definitely have to solve big problems in humanity— climate change, you know, the token consumption, perhaps like the arbitrage of knowledge, which creeps up on every technology revolution, which then leads to wealth gap distribution. So definitely acknowledging there are still lots of societal problems to solve. While acknowledging that, I am a glass half full person, or at least we have a glass kind of person. And so I see more abundance coming from us solving really hard problems, we're starting to see more and more people are able to create software than ever, which used to be an inaccessible field. And we're seeing lots of stories of people making millions of dollars, uh, in applications, vibecoding them, and they never had coding experience. So like, that trend to me is like a very positive thing, uh, and, and I think that goes in line with exactly what you're saying, that, um, people that then use these agents or harnesses then develop their own harnesses and their own workflows and start refining those like we do for the pod or like content creation or like any, any other thing. We use it to like close our books actually financially. So, so it's really cool the opportunity that these harnesses and agents allow us to dip our hands into problems we never even would think. Like I never thought in my lifetime before, like some of these agents and the new tools have come out that I would be like closing books with Claude Code, you know, like that's, that's so cool. So just taking a moment to geek out about that. Um, pretty, pretty grateful to be alive at this point. Cool. Uh, just to move along the pod, we are moving into our AI memes section. So today we have a bunch of funny stuff and we're gonna sort of rapid-fire through them. In the AI meme section, we just bring up funny things that the community has like sent us on comments or whatever, um, and funny things that we've seen in the news in the past, uh, week or so, and then we give them a highlight very quickly on the pod, just to keep you in tune with what's clarity and signal from noise, with a humorous bent. All right, to start the AI meme section, bye-bye Gemini CLI, Google's gone and swapped you for a closed source AI. So Jonathan and I talk about this a lot, but it's, Jonathan's given this perspective that it's really hard to trust Google products because you never know when it's just going to get the plug taken out where you might start building up a habit of using a Google tool. You know, there's been various in the past, uh, and this is the latest with Gemini CLI, where they actually had accepted 6,000+ merged community PRs and cited that labor as proof of success. And then they're announcing June 18th they're going to shut it down and have a closed-source replacement called Antigravity CLI. So they're basically saying, thanks for all the PRs, guys. We're going to close-source this and make money off of it. Appreciate y'all contributing. Oh man, there's a Hacker News on this. How does anyone internally at Google justify these decisions? Even if there are competing implementations, in terms of brand recognition, I feel like Gemini is more closely associated with Google than Antigravity. Why pick the more obscure option? Perhaps they felt the sentiment on Gemini CLI was beyond repair, but surely there must be some voice on the inside saying developers will never adopt our products if we keep killing them. Jonathan, I think this would resonate a lot with you. What do you have to say about this?

Yeah, I completely agree. And I think, I bet there's people within Google too are just like, I don't want to do this. And it's just like this big machine that, you know, generates AdWords and makes lots of money and then throws everything else to this arbitrary sort of like, you know, capitalistic machine. And I don't think anybody likes it, but it's the world we live in.

So yeah, there are so many comments about this. I would love to sign up for anti-gravity CLI, but when I click on get plan, it says this account isn't eligible for for Google Anti-Gravity benefits included with Google AI plans. Google Anti-Gravity. Yeah. They're not available in some countries or for people under a certain age. Learn more about Google One feature eligibility with a button that says explore more Google AI plans that when I click on it, it takes me to my Drive. Like, it's like this dude, this dude is just trying to buy it. Like, it's like, can I give you my money, Google? And Google's like, no, we'll redirect you everywhere into our ecosystem and not take your money. I wish I had that problem where we make such fuck you money that we're like, ah, we could just mess up a product launch really bad and like kill apps and waste tons of money building assets that— no, no, we get users and then we just throw them all away, but we're still printing money. Like, what a place to be, Google. Uh, hands up for them. All right. Here's the last AI meme. Next meme. Today we are saying goodbye to Windsurf. This is Jeff Wang, and we are transforming it to Devin Desktop. So Windsurf dead, but Devin Desktop alive. He goes on to say, Windsurf has been an absolutely amazing experience for me and the team, though it has been rocky at times. We have seen every phase of AI coding and we want to keep embracing where things are going. That means we need to once again reorient ourselves towards a more focused goal and remove the Windsurf branding. Believe it or not, the Windsor brand has been around less than a year and a half, and before that, the previous name Codium was only around a similar timeframe as well. I've actually had to change my email every year all the way to the eventual acquisition to Cognition. In AI, most products only have a year lifespan before you need to drastically change it to the next. Devin now encompasses all our form factors, whether it's a cloud agent, the agent command center with IDE, CLI, review, or our other products. This way we can really focus our efforts around one name. We're doubling down on our neutrality and making Devin desktop compatible with other agents via ACP. We may be the only Switzerland of AI left, and we embrace this role. As for me, I'll be transitioning from CEO of Windsurf to Cognition's President of New Enterprise, helping open new regions and verticals, accelerating velocity, and filling in gaps as usual. The story of Windsurf doesn't end here and continues on as part of Devin's journey. So that's interesting. Users opened their IDE and overnight it had renamed itself to Devin. So that means somebody woke up in enterprise IT land and they had WinSurf on the approved list, and the binary now phones home as Devon, and the endpoint management blocked it at 9 AM starting with this rebrand. Uh, sorry for all the enterprise IT people that had to deal with that rollout.

Um, so, well, I think the Devon agent is probably more popular than WinSurf most recently anyway.

Yes, so I agree. Yeah, I agree with that.

Yeah, yeah, that's the one I haven't tried out, Devin. Um, I've heard good things about it. It's kind of expensive. Um, but, uh, yeah, I haven't— like, Devin and Blitzy are both platforms I hear about for coding that I haven't, um, had a chance to invest in a lot yet.

Yeah, I haven't tried Devin, but I heard from one of the founding engineers at Gamma who I'm friends with that Devon is her go-to for everything. And I'm like, whoa, okay, I gotta check this out. And I showed her my Claude code setup and she was kind of interested in checking that out. Um, and I, yeah, it's really cool to see like the meta being driven, um, around these agents and then continuously seeing more and more features pump out. It's definitely an exciting time to be alive. This was a little bit of a funny, uh, item for our AI memes segment. Mainly from the perspective of, you know, Windsurf just becoming Devin overnight. Like that, I just didn't really see that coming. So that's cool. Let's, since we're running out of time, let's move into What's the Meta. For those new to the show, in this segment of What's the Meta, we will go through things like new tools, new tactics, new techniques as AI builders getting into the weeds, really rolling up your sleeves and the nitty-gritty details of getting production systems live. And getting them to compound and make them better over time. In today's segment, we're going to go over a few things. We're going to go over our recent Fable 5 touchpoints and some of the things that we've done with them for our production cutover. And then we're going to show spec HTML as a methodology for spec-driven development, and then proposals as HTML for the forward-deployed engineers out there. I've had some success with this, and I'll share with you what I've been doing work around that topic space with. And then we'll go into harnesses. So we spoke about harnesses a little bit, but I actually want to double down on this from previous podcasts and like really dive into the anatomy of a harness. And we'll focus on steering coding agents. And that segment is going to be really specific to AI builders and operators and engineers and product managers, either at startups or big enterprises who are steering coding agents to do your knowledge work. Lastly, we'll dive into the weeds of the 25 Patterns of Agentic Engineering by Greg Ciccarelli from the Spec Story team. We went over patterns 1 through 9 last time. We're going to go over patterns 10 through 25 this time. So let's get into it. Um, spec-driven development, what is it? Spec-driven development is spending a ton of time, 80% of your brainpower, uh, you know, with your cross-functional colleagues, with customers, to inform a highly considered input artifact to give to a coding to give to a coding agent for code generation. Um, spec-driven development has been a meta for a while now, and recently an Anthropic engineer went on a podcast, uh, on Claire Vaux's podcast, and demonstrated how they use HTML files to be in the loop as a human to really easily visualize specs and get aligned very quickly with what's going to change. So I took this meta, and for our recent product launch for a customer of ours, I went ahead and created a specification in HTML. You can see that here. Um, what we normally do on the team for our workflows is we start with a ticket in Linear, uh, and every single PR follows a particular lifecycle. We have a feature branch cut off of staging for this particular client. Um, and then we'll go and start a PR that is a Papertrail PR and we'll add commits to it. Uh, so we, we do that so that we're always inspecting the input artifacts and for code generation. So in this HTML spec file, it's really cool. So what I'm demonstrating here is as a builder, um, you might be reading a lot of markdown and that might be getting tiring. So I find it really nice to create an HTML spec file. That's the new meta. You can even have it create these nice architecture diagrams because you have the power of HTML now and some level of front end, uh, to create very readable and highly understandable, um, specs. So it's even showing me the backend data model, which referred to a different ticket. So I could then take this and I go to Linear if I want to get the context of that ticket. Look at that, uh, it showed a bunch of options for me to pick, um, from the IDE, and I had chosen option C, and it inculcated option C for the user journey. Um, it showed me what's existing. If I go through this, it kind of shows the implementation plan the exact code changes that are going to happen so I could scan this as a staff engineer or architect and very quickly understand things. It also showed me the lifecycle of changes. For this, we really had to have a highly considered understanding across systems of state of a particular order to fulfill. And so this visualization was really nice. It helped me understand like, okay, what's going to get triggered by what, et cetera. What should the user actually see on the screen here and what UI component are they seeing. That way, me as a human, I can go in, I could be like, oh wait, this is wrong, this is right, or whatever. My meta is usually like I'm talking to Claude Code and I'll say, hey, this part of the spec, this part of the spec is wrong. I even had it create, so this is a really long file, I'm just going to scroll through it real quick, but I even had it create different drafts of the UI. These were examples of how users would reach this new screen and what it would look like. What's really nice about this is because in this HTML, it would be highly relevant for the next agent handoff to be like, okay, that is the front-end specification. It's got to look like that. Obviously, that would be part of the exit criteria or test strategy. You can go verify that with iOS Simulator MCP, for example, for end-to-end testing. Okay, I'll pause there. I just shared a bunch on spec-driven development with an HTML spin to visualize your spec. I've been finding it very, very useful as an AI builder for both PM work, engineering work, to better get into the weeds of the nitty-gritty on what is the exit criteria and what's the problem we're solving. What are we actually doing here for this pull request? Jonathan, have you tried this? Spec-driven HTML meta, and what do you think of it? Curious as to your thoughts from a CTO perspective.

Yeah, I thought this was a really great example of something where it's pretty strong. I mean, there's a lot of moving pieces in the app that you're building. I've used it for backend architecture too. And for me, it's a good abstraction layer to talk with the session, with the coding agent. Like, I mean, a lot of times you can get, like, say in our linear backlog, you know, there might be a project and it's got like a lot of interrelated tickets. There's a lot of, uh, a lot of pieces going on that I don't want to pay a lot of cognition to. And there's other things that are like really important. And I think this kind of abstraction is a way of talking about like the output artifact as a system. And you can, you can, you know, you can look at this, you can copy and paste the image back and say like, I don't understand this part or this part's wrong. Or, you know, this doesn't, you forgot to include this source document when you considered this part and you can shape this, um, you know, into an artifact that you understand and the agent understands. And then, you know, just, you're pretty safe to just let it build at that point. And just make sure it does what the spec says, right? So it's really powerful abstraction language, in my opinion.

Yeah, it is very powerful. And, you know, as AI builders, we're running as 4 deployed engineers, we have 3 customers now, and all businesses, we're all B2B and separate codebases, each of them. And so switching into context, and then leveraging the optimal context with a coding agent for all of our different streams of work. Jonathan said it best, right? There's a cognitive tax load. Uh, we as AI builders could do so much more than ever. Um, and that kind of increases the breadth and depth of your scope inherently. I've talked to, uh, customers where their management team now expects more throughput, right? And I think every engineer, and maybe even at the very AI-pilled companies, maybe even the PMs and designers are feeling this more, which is a desire from management to see a higher quality and a higher volume of throughput of knowledge work artifacts than eventually like code generation in compounding on your software asset, your product. So for me, diving in and out of different code bases and different releases and stuff like that, and being able to operate this way as an AI-native builder with specs that are very visual using HTML as language, I'm finding it very useful. So I'm going to continue this, and I highly recommend other builders do too. Stepping into the next meta item, I'll just go ahead and share. For the same client, I was running into this challenge where, oh shit, I gotta generate so much front-end code. And okay, I don't have a designer on my team, and we gotta run pretty fast because the expectations of the client are really fast. So I have to play the role of like product manager, designer, engineer, right? I'm an AI builder, so I'm just doing whatever. There's a question there, which is how can we compound the front end beyond just the test coverage, unit tests, integration tests, et cetera? But from a design and taste perspective, how do we compound the front end? And that kind of brings me into design system as an HTML, extending on the idea of specs as an HTML and visualization language layer between the human and the agent. I now have been getting into the practice of every project I, I work on, I create a design system.html for that particular codebase and application. And with that, what's really nice is I just, for the harness interacting with that codebase, like the agents.md file, their Claude.md file for that codebase, I always have some lines in there to anytime there's new frontend code, check with the design system HTML and re-leverage everything there. Or if we need to create new front-end components to satisfy, um, the acceptance criteria of the ticket or the project at hand, then propose changes to the design system HTML. So that way this evolves over time. And you can see this is also a big file, but it has all the different CTAs, primary, secondary, dark CTA, uh, ghost text, cards, and containers. And so that way as well, it's easier for the agent to always verify Did I make the component right? Right? Because you have this source of truth for all the different components. And I'm finding that I don't need to go to Figma. I could just— this is good enough, honestly. And then if I go and I need to go make something bespoke, I'll go to Magic Patterns or Claude Design or a dedicated tool. I haven't been using Figma these days, and I'll go and generate using this as context, any sort of new page or UI or UX components towards the jobs to be done identified with the new feature set. And then I go and extend this base design system HTML file for the rest of the codebase so that as developers in the codebase can compound this work. So this is just an example of the updates to the design system for this new launch. We had all this new branding, we had new, do, do, do, option cards, progress dots, messages for notification or inbox feature. And all this is now fully under one source of truth in the codebase for anybody to use. Jonathan, thoughts on design system as an HTML file in the codebase that compounds and evolves with every PR. I've loved this meta and it's really changed one of all my front-end development workflows. So I'm really curious, like, what you think from your CTO perspective of this meta. Yeah.

I mean, to give you an idea. I remember pre-AI, I have a project where we used this. So we used a design template system called Polaris from Shopify and it was very helpful in providing the design system components, but also versioning issues and inconsistencies. We always ran into problems with it. And now you can just build your own design system. And I think for one, that's a really powerful thing in itself. You don't have to use these older, more rigid ways of using a design system. You can just have it on the fly. You can version it. So you're on version 4 of this particular design system. That's awesome. And then combined with the way that we're using HTML as a communication tool, you can even use it for non-app things like just communication documents and this kind style template, I, I think it's like, so, so basically the, the coding agents solved design systems and here's what it looks like. And, and I think it's a beautiful thing, you know?

Oh, it definitely is a beautiful thing for sure. Um, yeah, the days of going back and forth in Figma are over and Figma dev seats. It's like, what the fuck is the point of a Figma dev seat anymore? Remember how they actually I, I actually don't know if they still charge developer accounts, but they used to have this when I was using Figma before the birth of all these coding tools, uh, and AI native ways to bring design into coding or closer to coding. They would sell developer accounts so that you could go from design quickly to code and do that handoff cleaner. And they would actually price the seats higher for developer accounts. So now it's just like, okay, just like have the coding agent making an HTML and it just blows away all of that. It's like, why do you need any of this? Right. So it's really interesting to see, like, I would bet that many companies have design and research departments or product departments where they're really heavy on Figma use. And then maybe wanting to protect that capability and like not want to move to an agent or something like, like if you have so much invested in a design system, like I've seen this. Um, then people get really protective of that instead of like just embracing what new tools could do where it's like, hey, we don't have to do that entire process anymore. We just go straight to, uh, codebase, like design system HTML. Um, really interesting, uh, from this, what's the meta is an intersecting topic of like, is Figma gonna die? Like, what are they gonna do? I actually haven't kept up with that, but, uh, let's go look at there. Market cap.

I think their stock price is doing pretty good, uh, to be honest. And, um, yeah, I don't think it's a tool that's going to die quickly.

Uh, year to date down 50%, so—

oh, oh yeah, so it sort of made a comeback and then it took another dive down.

5 years down 85%. I don't know, dude, I think that sounds pretty terrible.

They released a new plugin today. Where you can use a Chrome extension and then like copy a website into Figma. And the tweet I saw it on said, yeah, so all original design is officially dead. Yeah, I think if you're a designer, I think this is still like the Figma is still like a necessary tool. Like let's say you take that design, the design pattern that you just created, the component set, and you need, you want to do something like that was really nuance, like move this 5 pixels, right?

Like that's—

yeah, yeah, move it a couple pixels, like put some particular like, uh, visualization, you know, visual design, uh, onto it, tweak the, the fonts in ways that are specialized. Like you want to—

you could—

you take it into Figma, you do that, you bring it back over to your AI system, you know, as a way to communicate with it. So I think we're still going to need tools like that. Yeah, but it's probably not as dominant position that Figma might have been in, um, you know, otherwise.

Yeah. I, I think what's going to actually happen from this is it's going to be really easy to see, maybe like products that are AI generated in terms of design versus products that are like, I have handpicked every pixel. You know how in today's world you go to Berkeley, for example, and you go to a farmer's market and there's like pottery for a cup for $60. It's because it's handcrafted. It's like, you know, a human made it. There's a social sort of like human connection element where you want to support this artist and this artisan at their craft. I think we're going to kind of sort of see something like that, right? Where you could also go to IKEA and get, you know, a mug that is way cheaper and probably like works about the same as the $60 cup for like $5. But this is not human. It's like soulless. It's like, you know. It's IKEA, um, but they're different markets, right? So I'm, I'm kind of curious to see how the design game, as we invite more people and the baseline of AI-generated design goes up, I think just because of the way LLMs work, like, we're going to be able to see those look similar, like they're using ShadCN or some of these base libraries, um, that the models seem to really like. And then we're going to start to see— this is my prediction— differentiation again in design. Like, it's going to be like a roller coaster, like up where all expectations are higher because baseline is higher, but then because expectations are higher, then expectations for new entrants are even higher. And then maybe it's like, it's whoever is pixel perfect at their craft. And so I, I kind of, I kind of just had that thought I wanted to share, um, analogous to the Berkeley Farmer Markets that I like go to, where these like super expensive handcrafted goods, but it's more for the artisan sort of bent. I see the same thing happening with software very soon.

It's kind of a question of like, how does the bitter lesson get applied across various domains of expertise? And, you know, like, I think when we're talking about design systems and you have like a hierarchy of assets that are meant to like, you know, from fonts to like, you know, web components to pages and, you know, when you have human expertise that's meant to define all the layers of those things, and then you have a tool like an AI coding tool that can just kind of regenerate them all from first principles, then the bitter lesson is fully applied. And the expertise of how to scaffold the design system layer by layer is all of a sudden just an artisan adventure. And maybe you're going to be used as training data, but generative patterns is what everybody's going to use in their day-to-day work to build stuff at some point or another. I think that's like a nice way to think about like what's the tipping point to the bitter lesson is like, is there a generative pattern here that, you know, rescaffolds the expertise in a way that is equal to or better than? And this is a pretty good example of like where it's a bitter lesson is gradually kind of creeping in here.

Yeah. No, I think so too. Yeah. And we see it with Figma's stock price, actually. Definitely, I think the better lesson is creeping up into—

I mean, reverse NVIDIA. Sure, you're, um, like, yeah, you lost a lot of money.

Yeah, if you were at NVIDIA 5 years ago, you have a 1,000% return. If you were in NVIDIA a year ago, you have a 2x return. So maybe the real lesson is being in NVIDIA. This is not, this is not a stock recommendation. Everybody make your own choices.

Yeah.

Okay. Last, last HTML thing that I'll go over and then we'll move into more on harnesses in our What's the Meta section. As for deployed engineers, and if you're early, early stage startup, like we are, and we're AI builders and founders. I am our salesperson. I am the one salesperson on the team. I am powered by Claude Code and my OpenClaw instance running on my Mac Mini. Um, and so some of the most annoying work that I have to do is I get on a call with a prospect. That's not the annoying part, but after the call is all the collateral. Like I gotta make a deck, I gotta make a proposal, I gotta make an SOW. I gotta like, you know, do all these things, right? Send the follow-up email., maybe research a code base if they gave me access to it under an NDA, and then I could assess a scope, um, to then give a price. So, uh, in B2B founder-led sales for early-stage startups, if you're a founder and you don't have AEs, like account executives or business development teams, um, I have found this to be really useful, which is go from like, say, a meeting transcription tool like Granola all the way to an SOW, within, you know, 15 minutes after the, uh, actual meeting. So let me show you how we did that. Okay, so here's like a granola transcript, uh, from a recent prospect and, uh, customer we closed. Um, and we'll actually show you, uh, from this granola, what we'll do is like go to Claude Code and then have Claude Code— what I'll typically do is I'll do something like this. Uh, let me go to, okay, so I have a meeting skill where I'll just take this, pop in, copy paste the transcript and just yeet this with Claude Code into my Obsidian. What I typically do as well is at the same time as doing that, what I'll do is process transcript. This Cloud Code session is pointed at Clarity API, which is our product for digital twins. And so in this, I'm having it create digital twins. In this, I'm having it just process the meeting notes into my personal knowledge management system, Obsidian. And yes, I could do it with both. I haven't done that yet, but I should actually consolidate the skill. What I end up doing after I do this—

Do you have the MCP for the transcript source? You copy and paste it, but—

Oh yeah, I copy and paste it. I just wanted to show for the audience, very easy to do is just copy and paste it. And then After that, just to show you on the founder-led sales side, what we can do once we structure the context, that's the idea, so structure the context, then from the same Cloud Code session with that structured context, and in particular, once I get the digital twins of the prospects or customers in there, I can interrogate that and create downstream artifacts such as an SOW, or I'll show you, this is a recent Gamma deck that we created from Claude Code via their API. And this basically got the whole SOW down in— this is a different one, but in like, like it was 95%. I took like 5 minutes to adjust things. So if you get your settings right on the Claude Code, uh, user harness, uh, and skills, you can also go from prospect call to, oh, I got to generate a bunch of assets and downstream artifacts for review to try to close a sale. Within 15 to 20 minutes of the call. That's, that's really what I'm trying to show here. So let me go back to terminal here. Okay, so in this, what it's doing is it's actually going to my people notes and it says, hey, we have a Mustafa one, we have Jonathan McCoy. Like, that's kind of funny. Systematic Optimizer is what it has you number one on Brian Johnson's app leaderboard. We kind of talked about that in the meeting. So it captured that context. And then here is the, uh, digital twin, uh, creation. So, uh, it's pulling from that same transcript, talking about struggling moments from a Jobs to Be Done lens, the jobs and forces, the push and pull from a Jobs to Be Done lens, um, their switch energy. So how highly likely do they seem to want to close right away? Like, uh, where are they in the decision process? And a bunch of other contexts. So, uh, this is just massage structure context into some format to where, uh, we can then create these downstream artifacts. My point here being, if you find yourself to be in a position where you're having to be the only salesperson on the team in an early-stage startup like I am, uh, then the problem becomes I want to close as fast as possible and spend as little time on high-quality artifacts, which I need to create anyway to try to close. So I want to reduce my admin time on these artifacts that are very necessary for the sales cycle. So I just want to show that real quick. This is what we do. And if you want to see more of this kind of thing, I know most of the times we, we spend time on like technical topics, but just wanted to show, uh, more on the functional side. So drop a comment if this was helpful to you for founder-led sales. Uh, more on like how do we update the CRM and stuff like that on another pod. Uh, that's, that's a whole other part of it, but just want to show these steps. All right. Uh, let's go into second to last meta item. Uh, next topic, anatomy of a harness, AI DLC, and steering coding agents. So what is harness and what are coding agents? Agents equal model plus harness. What the fuck does that mean? Harness engineering. What does that mean? Harness as a service. What does that mean? I'm going to pull up a Martin Fowler article and share my screen here so that we can all be on the same page. About this. And then Jonathan and I are just going to share some knowledge around some questions about harnesses. So, so I really like this image here of, um, what is a harness. And also, actually, I really love, uh, the 'What is an agent harness?' by Arise. Um, let me pull that up. What is an agent? Agent is really just an LLM in a while loop at a really high level.. And so when we talk about an agent, uh, an agent's harness, um, in conjunction with a model, this is what we can sort of see is user types a message and you have this inner loop, a tool loop, machine-paced. Step 1, call LLM with some level of assembled context. That's an optimization problem in itself. Uh, the LLM is like, should I call a tool? Yes, I should call it. No, I shouldn't. Break. Text-only response. If it needs to call a tool, such as call this MCP server or get some data, or call this MCP server, write some data, you know, grep, blob, glob, bash commands, call a subagent, that kind of thing. Keep going until you're done, right? And that, that's really this loop, is keep going until you fulfilled the user's query from this input. So how well you fulfill the user's query is really that harness, um, and that's this part right here. When we talk about a coding agent like Cloud Code, uh, or Codex, you can think of it as the model plus a harness. So, uh, in Cloud Code, it would be the harness built by builders of that coding agent— the system prompt, the code search tools, orchestration, any of the markdown file context that comes with it, uh, in the assembly. And then the user harness— this is another term to define— user harness is the feed-forward or feedback controls per Martin Fowler and the language that, uh, they— his, his team brings to the table. That's how are you steering the thing, and then what inputs are you giving it? What's your prompt? And then what data are you giving it? What logs are you feeding it? That kind of thing. And we see that really clearly with, uh, this visual here. So you as the human might be steering, and you might drive principles, rules, reference documents, how-tos, and that's feed forward for the coding agent. Feedback is after point of generation. How do we like make that user harness, um, part of that original image better with static analysis or review agents like an adversarial agent, uh, review or logs or the browser, uh, with Chrome DevTools and stuff like that. And that way that you give that feedback. So then this purple part gets better and you're steering in this part and you get better at that too. So I'll pause there. Um, I think there's some, some real meat to the bones with the topic of harness engineering, and I just covered like agent equals harness plus model. Harness is where we see competition, the, you know, Claude Code versus Codex, all the, you know, different, um, applications there. You could also think about it in functional domains, like I just showed you a bunch of the founder-led sales items. I built skills and commands for that, that you could also think of that as a harness. And then there's the user harness, like the, the human behind controlling and steering this thing. Jonathan, this is a good time to invite your opinion from a CTO perspective. What are the things that you think AI builders need to be thinking about right now, um, in this moment when it comes to harness engineering?

Yeah, um, and so we, we did an episode on harness engineering, uh, maybe a few weeks ago, and Even since then, I've seen like a just constant rollout of papers of people saying like the anatomy of a harness, how to, how to do harness engineering. I think it's an active topic that people are trying to figure out, A, what it means, and then like what it means to them. So, um, the, uh, I think the basis of the harness is nothing too complicated. It's just like, how do I get the right context in place, and how do I get the, you know, all my upstream files in my company or for my workflows to be accessible by my coding agent while I'm working with it. And then, yeah, basically those are indexed for queries and those are indexed for the things that I want. I think that's like the first layer of what a good harness is. Where we've invested more time in what I think is ultimately, like, if you ask where is this all going and like what is a harness going to be a year from now, I think we're moving towards being able to provision closed-loop agents. These are agents that have a harness that can collect data about every run that they have and whether or not the behavior of it is, is like achieving the— is verified as being like, you know, the achieved result of what's going to happen. And you're seeing this in like harnesses that are longer running, that are autonomous looping patterns, like you're slash goal capabilities and agents and this kind of thing. And where we've been, you know, thinking about is like, how can we build harnesses that will gradually evolve towards being closed-loop systems? And I don't know if that's the right decision for you and your company today, but that is, that's like, you know, how we're architecting and engineering our systems is to be able to like evolve towards closed-loop systems. And what that means is that we need to be able to create a log of every run that the agent has. We need to be able to evaluate whether it was good or bad and have evaluations in place. And we want to work towards training an LLM judge so that we have a log with enough evaluations that we can actually determine whether or not the agent can run and be judged by an LLM judge that can then say whether or not it is indeed doing a good job. And I thought, like, the example that you just showed about processing transcripts and creating statements of work, Robert, that's a good example. Something like, how do we verify that the results were, A, good in specificity, and B, how do we, every time that we run the agent, how do we know that's creating, like, a better SOW or a better Gamma presentation? And that's ultimately, I think, where the harness is like your tool, if you configure it correctly, it should just like, you know, continue to sort of evolve with the use case and the data you feed in and get better and better.

Yeah. And for AI builders in particular, there's this concept of like minimum viable evals or minimum viable harness that you can think about. And from first principles, the way I think about it is time and money around an economically valuable workflow, like that, those are the vectors, right? So when we talked about founder-led sales. If I could spend 2 hours less every single time I have to generate a proposal in terms of an SOW, in terms, in a Google Doc to be signed, in terms of a slide deck, that's a visual form of the same document for a group of stakeholders and different types of stakeholders that are part of the buying decision-making process. And then perhaps a prototype to go along with that proposal. What Jonathan is saying is, what's the harness for each of those commands? Like, what is the quality bar for that? And if you're thinking about minimum viable evals or minimum viable harness for each of those workflows that are economically valuable, assign a time or materially, like, cost, like dollars, to that workflow and how much, you know, you get back in terms of time or money to create the thing and have the agent create the thing for you. And for me, it became really obvious that if I just spent a little bit of time making better harnesses and a curated dataset, around the evals of what is a good SOW, and that takes some taste, that takes some labeling, open coding, annotation from my end as the human to make the machine better and make the harness better, then it meets my goal faster, right? Which is really decrease time spent in sales cycle while maintaining high quality asset delivery to still maintain the same close rate. So AI builders out there, really think about your minimum viable evals and minimum viable harness. And those things are expensive to do. It's expensive to spend your cognitive load and human expertise to label data. So really just do it for your areas where you're doing the workflow a lot. There's a material amount of time and money to that workflow for you, your team, and your business. That's how I think about minimum viable evals or harness and when you should be thinking about that as an AI builder or an operator. Towards your goals and closed-loop autonomous agents, I had to bring up the Claude Code docs around /goal. You had brought up that you tried /goal when it first came out and it kind of sucked. So have you tried it since, uh, Jonathan, speaking of closed-loop autonomous agents, have you been trying, um, I haven't tried it with Claude.

I have tried the Codex slash goal. Oh, um, yeah.

Tell me about it.

Yeah.

Like a closed-loop Claude, uh, uh, Codex command. Hold up right here. Wait, goals in Codex. Yes. Persistent. So goals in Codex, just to tee up Jonathan, goals in Codex are persistent objectives in Codex that keep a thread working toward a defined outcome across turns. A goal gives Codex a completion condition. What should be true, how success should be checked, and what constraints must stay intact. Jonathan, from your CTO perspective, you were mentioning you've been using goals in Codex, or you, you had tried it. What are your thoughts?

I just tested it out. I still don't consider it like a valuable tool that I use regularly in my workflow. Um, it's more like, you know, okay, I'll, I'll pick up a ticket from Linear, and if it has really clear acceptance criteria that I can feed into the goal, I can say like, you know, here, allocate 20 or 30 turns towards like trying to accomplish this whole thing by yourself and see how far it can get. And I'm really not expecting it's the best way to get my work done. I'm just kind of seeing like what's the model capable of at this moment in time. And I think that the, um, the, the idea here is good. Uh, the— I posted an article on our team learning section about like, uh, It's called What the Fuck Is a Loop? Peter Steinberger versus Boris Tourney. And one of the key ideas there is like, you know, these looping agents are cool, but the underlying primitive that needs to get better is the skill. Like for each loop, how does the skill, you know, sort of get good enough to achieve the thing that it's supposed to do during the loop? So you have this Yeah, I think the skill insight is maybe near the end of the article. Yeah, it's not the loop, it's the skills.

Yeah, I see. The asset is the skill. It calls Steinberger's other recurring point pairs with the loop's one and is the more durable half. If you do something more than once, turn it into an automated skill, and if you do something hard, turn into a skill afterwards so next time it's free. I like that. That makes a lot of sense.

Yeah, yeah. And we've seen a lot of tools to improve skills like, uh, GBrain's, uh, Skillifier, uh, function. We've seen auto research applied towards skill improvement. And so I think if you're gonna run /goal, you know, you sort of want to have some like well-tuned skills that are already proven that can do that. You're gonna wanna have output criteria that's highly measurable by the LLM that it can know whether or not it's getting closer to it. And not every problem is a good fit for that, you know, but the, you know, the range of like what output criteria can a coding agent consider is getting like broader and broader with each release. I think about this a lot, actually. And, you know, like in Elon's algorithm, they talk about that every requirement for a system. So if you think about like specifying the requirement that is the endpoint for a slash goal, in Elon's companies, he said that every requirement has to have a human's name by it, right? Like if they're being really serious about engineering, really saying that this requirement matters, then who does it matter to? And to what, you know, what does it mean that it matters to them? Like, what do they want to see? Right. And so I think when we think about like how to specify a goal, I think getting to that level of pedanticness about like, this is, this is the self-model of the user that we need to see that needs to see this result in the requirement specification sort of there. But also like the reality of goals is that we successfully approximate the definition of them as, as we define them. And so having an adaptive goal. Having it is like, you know, where I think these things will get useful. The looping pattern, the autonomous looping pattern with an adaptive goal that has a requirement set that has a subjective, like, you know, like judge of like what is achieved or not achieved. That's when it's going to get super useful. And that's when, yeah, so I'm thinking about ways of how to improve the autonomous looping pattern so that it can sort of like fill in some of those gaps. Right now. But in the meantime, I just kind of play with it as a toy.

Yeah, no, I think /goal is, um, the goal— closed-loop autonomous agents— like the /goal functionality coming out of Codex and Cloud Code is really just a sneak peek of what the future is looking like. And it's tea leaves that the, uh, Frontier Labs are showing us that they believe that that's the future as well. Uh, intersecting that with how, you know, the Cloud Code team, they talk about how 80% of features internally are like driven by AI coding agents. Like, they really believe that, and they're really dogfooding that ethos, uh, that closed-loop agents around goals, uh, no matter how subjective, is, is really the endgame. And towards that, Elon Musk's algorithm here— you spoke about the 5 steps of the algorithm. For the audience, these, these are the 5 steps. You question every requirement. You make requirements less dumb, so always assign a specific person's name, not some you know, foggy department like legal or safety to each requirement. Delete any part or process. So this is vigorously stripping away unnecessary steps and always questioning whether the step should exist. Always question the assumptions that led to the step because it might not need to exist anymore in today's timeframe versus, you know, yesterday, 10 years ago, 20 years ago kind of thing. Step 3, simplify and optimize. Do this after deleting unneeded parts. The most common mistake of smart engineers is optimizing a process or process— optimizing a part or process that shouldn't exist in the first place. Very true. Over-engineering is a thing. Uh, step 4: accelerate cycle time. So speed up the remaining steps that are left. However, never speed up something until you have determined it is necessary. And necessary is a really key word there, um, because a lot of his ethos is like simplify, simplify, simplify, simplify, delete, delete, delete, right? So really understand like what is necessary and what is not in your cycle or your process or your company's algorithm. Uh, automate, and that's the last step, step 5. That's the final step. Musk has personally made the mistake of automating bad or unnecessary processes first and warns against applying technology to broken workflows. So it's, it's cool to think about Elon Musk's algorithm for building companies, these 5 steps, um, as principled ways of decreasing the cycle time to production of the service or good um, you know, to go compete in market as a business. And it's cool to think about that in terms of setting an agent towards some sort of like closed loop, right? Because business, you ideally have a closed loop— go make money, business— like, and this is Elon's sort of principles around that, uh, to build a durable business that makes money. Um, and it's kind of cool to talk about that at the parallel of these agents. So I wonder if, uh, any of these, uh, logical principles are making their way into, um, the underlying, uh, goal loops at all. So yeah, that's an interesting thought.

I think that's our job with the harness. I'm glad you brought this up because I think that, um, when it comes to like practical use of harness and, and, you know, you can look at the systems that Elon's used to build his business, and why would it be any different? The harness is just a tool to capture those learnings and to like, you know, graduate processes from being something that happens manually and then happens with reduced cost and then happens over and over again repeatedly, and then can be automated, and those are— that's what you're trying to use your agents for. Premature optimization is going to fail every time. And, you know, it's just like, uh, you shouldn't expect these things to be magic. You should expect that a harness is a way to, like, you know, gradually automate a process, just like you would in any other aspect of engineering, um, in a business.

Yeah, agreed, agreed. And I'll throw in a quick story there. When I first was, um, building out the harness for my OpenClaw, I was thinking, okay, so we should automate client onboarding. And I spent like 5 hours trying to automate what if we got a prospect, closes a sale, and then I want them to automate, I want my Open Cloud to interface with them to automate onboarding for like a cool AI-native services kind of experience. Like it onboards you to Slack, it has an input form for all the creds we need to like manage your codebase and your deployment and your infra and stuff like that on the dev side. And then when it came down to it, I realized I just wasted 5 or 6 hours because all the time, like, humans want to be onboarded by other humans. So, like, especially for a business contract. So it's like, well, what the fuck was the point of this? So, so that would have just added a step, which I now see is like, nobody wants to be onboarded by an agent when you've signed a contract with tens or hundreds of thousands of dollars. They want to be onboarded by a fucking human, right? So that was like, ah, stupid Robert, why'd you even overengineer this? This is not even a problem to solve. So one must control themselves when it comes to the dopamine releases that is the addiction of cloud coding too far. That's the lesson learned there. Okay, anything else on harnesses around agents and harnesses around company building, uh, since we're on the topic of like company building at the intersection of like this agentic harness engineering topic?

Yeah, I get— I mean, um, the Actually, did you show the statement of work that you drafted off of our like multi-phase architecture plan, the 5-phase architecture plan?

Oh no, I didn't. Yeah, so yeah, that's for this AI DLC proposal where we were breaking up, hey, this team is at a certain maturity level and we'll give you a model, crawl, walk, run, sprint, fly. And if you want to get to digital twin-driven AI-native product management, which is what this proposal was for, we were kind of saying, hey, like, to start with crawling, you probably want just people using portable, like, context, like sharing portable context skills, like that kind of thing. And then you probably want those things to get better. So making sure you have accurate twins or harnesses or agents. And then, hey, like, you get to running and then you actually are using these agents and they're better., from an efficacy standpoint and effectiveness standpoint than your humans without the agents reliably across more complex workflows, uh, such as, you know, creating PRDs and things like that. And then sprinting is, hey, you have a world-class evals sort of pipeline, uh, and you're annotating, open coding, um, around all the different agents that service those complicated workflows that save you money and give you back time. And then flying is you institutionalize knowledge that is queryable to any part of the business and contributable and curatable by certain key stakeholders of that institutionalized context. Anyway, yeah, so, and then we like sort of cataloged like what are the agents to build in the different phases. And this was like a proposal that made it really easy to sort of understand for for the person. So yeah, what did, uh, you, you were asking about this, um, the 5 stages, so I wanted to bring it up, but I also wanted to share with the audience like what are they looking at. So anyway, off to you, Jonathan. What about this?

Yeah, I think, um, you know, there's a foundation document to this where really thinking through, um, you know, what, how does our positioning align with our architecture and align for the outcomes that we want to achieve?. And so it created a, you know, a framework where we can say like, you know, here's, let's look forward and say these are the, this is the ROI driven outcomes that we want to achieve as a business with our investment in agents. Breaks it down to exactly which agents that we need to build in order to achieve that, that outcome. Exactly which harnesses need to accom— accompany those agents. And then it gives us like kind of a blueprint that we can spin out a state, you know, a proposal for a customer that says like, you know, if you want to realize ROI on your AI investment, here's a blueprint for that. And so, you know, I think it ultimately comes down to like achieving the business outcome, right? And this is our latest attempt to sort of like get on the same page and create an architecture, a positioning, and, you know, some case study outcomes that we want to create.

Yeah, totally. Thanks for reminding me about the central artifact that drove that other artifact I showed, which is a real proposal. So this is the central artifact. Quick meta thing for builders, especially on small founding teams to get people aligned, right? Like, especially if you're on the path of like trying to find PMF and getting your first customers and stuff like that. Your founders and founding team, they're all going make sense of the world, right? They're all going to learn, they all make contact with reality. And we're finding that creating not just specs to get aligned with the coding agent, but specs as HTML documents or like business strategy documents as HTML documents, really align the team. So Jonathan was just speaking about this, and I'm just scrolling through it, uh, but this is sort of the maturity model that we give. If you are a tech executive or you're at a company where you need your team to be more AI native or increase their AI fluency, or you want them all AI-pilled and you want some behavior change, you want some ROI out of these licenses you bought and you're in the middle of your enterprise deployment, I'm seeing lots of companies fail and fall on their faces around the adoption story for AI at enterprise, because they don't really have a sense of like, what does good look like? What does mature look like? There are agents, there are harnesses, like, what the fuck do I do? And so we created this internally to align on, which is, what is the value to an organization like that? If we believe in a world where companies will be ran with humans and agents, closed-loop autonomous agents, that are long-running against complex goals like business goals, revenue goals, then we need some way to coherently as humans understand like a maturity ladder to growing towards that AI-native competency. So we thought of this idea of every company should probably like start with you have some portable context, portable twins is how we think about it. So you might like have a portable twin of a persona that you share from institutionalized learning about a payroll specialist or a recruiter or an AI builder or an AI engineer, whoever your persona is. And then you get to, okay, well, how do we make that more accurate so that this agent could query different artifacts that we create as part of our knowledge work, for example? And then it becomes, okay, that thing gets better. And then it becomes, okay, it doesn't just evaluate artifacts, it makes, maybe it creates a bunch of artifacts. Maybe evaluates a bunch of artifacts in the path of creating something like code or design or prototype or servicing like a customer, right? And then the later kind of stages, alignment scores. So making sure that as we improve the agent, we can simulate and incubate like before we promote improvements that, hey, this thing does work and it's aligned to a stakeholder's subjective goal. And there's a lot more to that. We can go that into another pod.— that's a whole pot in itself. Like, how do we think about subjective goals and the ontology of humans and their subjective experience and sensemaking? But that's the core of what we think to get to AI personalization and building user-level harnesses. And the last one is, can we make a digital twin of your whole business? Not just the users, not just the people operating the business, the humans, and all the different stakeholders and vendor relationships, but like the whole business itself with all these levels of harnesses., at the individual level and then across the whole company. So we created this to get aligned on that, and we really believe and are in the middle of execution on this for clients, and we believe that this is a good way to increase your AI-native, uh, competency at an organization. Okay, I know I just babbled a lot, but Jonathan, any other thoughts on AI-native maturity, uh, that we're really showing here?

Um, yeah, I think you covered a lot there. I I think in terms of like starting the conversation, talking about what is a Harness and then saying like, well, this is how we conceptualized achieving ROI-based outcomes for our company and our clients using Harness engineering. This is the hypothesis. So surely meant to evolve, but yeah, I think these 5 steps really took like the tangible problems we were seeing about building agents and getting them delivered into a 5-step workflow that I feel is pretty, really powerful and, you know, like gives us a footprint to kind of keep iterating, testing, and evolving, uh, these problems.

So catch us in a future pod and we'll see if this changes. We'll be very upfront about it because we're building in public. So that's some of the benefit if you're an AI builder or operator or founder, uh, listen to this pod. We're very transparent about our work as much as we can be, and we keep, you know, client, uh, privileged information confidential, like the clients we serve and customers we serve. But like our approach and abstract methodology, we'll always share so that you also can get clarity from the noise of AI news and tactics. So you always have the right meta to be building the right way. Cool. So we are running so over on this pod. And I think we'll have to save the last patterns of agentic engineering for next time. Again, fuck. Pod audience, I know you have been waiting for that. And you probably were waiting through the end of this podcast for that. But I'm looking at the time, we're about 2 hours in, we can't be making this pod 3 or 4 hours, yo. So we're next time, We are going to be focused on that. And, uh, yeah, we will close out the top 5, 25, uh, patterns of agentic engineering on the next Self-Aligned pod. Um, for this, uh, maybe the last question to close this out before I round out the pod: do you think harness engineer is a job title in 2 years, Jonathan? Maybe 1 year, maybe 6 months, but do you think it will become its own role?

Um, I'm gonna guess No, it's just like part of this full stack that we all need to integrate, you know, in building the tools. So more of a skill set than a role.

What about Harness Monkey? Like we've talked about Harness Monkeys. Yeah, I think like we got like that's for AI native services agencies that want to go in there and just monkey around. You get the plumbing in, you build a harness, you glue things together and get the team going. Do you think that's going to be a thing?

Yeah, I think like it's— there's like that's like a version of a forward deployed engineer, the Harness Monkey. Monkey. Yeah, probably so. I guess that is a way people would hire, like, you know, set our harnesses up so they're all ready for our engineers to do their higher-value work kind of thing. I could see that.

Yeah, we'll go in, we'll do the open coding, we'll talk to domain experts, we'll glue systems together, normalize the data, do the data science work, munge through it, and then your engineers can compound from there. I think that's a specific kind of forward-deployed engineer, which on this pod anyway, we're going to try to make cool as the harness monkey. Concept. I almost threw up like a landing page of Harness Monkeys. In fact, right now, I'm just gonna, to end the pod, I'll create an image of a Harness Monkey. Let's check this out. So I said, make a Harness Monkey company logo styled for AI builders. I don't, we'll see if it even combines the concept of like harness, like we thinking of it like harness engineering, or if it's actually just a monkey that's in some medieval harness meant for a harness or something like that.

Like, the monkey is too biological. Potentially dangerous.

You couldn't.

No, we'd be—

we're creating bioweapons here by asking about Harness Monkey company logo, clearly. So we got to be flagged down for a huge amount of risk. Harness Monkey AI infrastructure built to ship. Wait, this is actually— this is our new business, guys. Oh, this is sick.

It looks like it gets the job done. This looks like a really serious like Harness Monkey that we're looking at right here.

He's smiling too. He's like optimistic. Usually like when there's unknown unknowns, people are not optimistic, but I could tell this monkey is ready to code. Get in there, get your plumbing done. You know, he's like, he's smiling. Like I bet behind these command line interface goggles, I bet he also has bright eyes. Like he's really excited. That's what I sense with this logo.

Yeah. Made of the finest HTML colors, you know, cause of the For sure. Brackets on his goggles.

Yeah, for sure. Cool. Well, that has been harness engineering, uh, and probably more than you would want to know, but we covered, you know, what is an agent. An agent is model plus harness. What is a harness? There's, um, you know, the ones shipped by the Frontier Labs in the form of Claude Code or Codex or Cursor, uh, as a company. And there are several other harnesses out there. You could think about them as just the skills and commands that you might use, those portable ones. And in particular, we covered like the personalized harness is really the ultimate goal in your domain as an expert practitioner, whether you be a builder or an operator, you're really looking for your niche domain evaluation or grading rubric. And you can Google some of those things like Eval's grading rubric and failure taxonomies and open coding, axial coding, that kind of thing, to make sure that you have good data to kind of like feed into these tools as you do things like founder-led sales or SOW generation or any of these other artifacts you got to create in your work that take a level of subjective qualitative assessment. Hope you got a lot out of that. And if you have more questions on harness engineering, if you need a Harness Monkey, hit us up, hit us up in the comments and drop comments, questions, we'll take them. If you got memes you want us to go over, more meta you want us to go over, problems you go through as an AI builder, we are here for you every week. This is your weekly AI clarity, giving you signal from the front lines of AI building. Have a great day.

Bye-bye.