The $5.5B Proof That APIs Are Dead for Enterprise

Anthropic and OpenAI just poured $5.5 billion into enterprise AI services in one week. That's two ecosystems forming, and you're gonna have to pick a side. We'll break down what that means, plus the visual feedback loop that's killing traditional QA and design, and why "code is free as puppies" is the most important sentence in AI engineering right now. Welcome to the Self-Aligned Podcast, where we share stories of people building the future from the front lines. This is our weekly AI Clarity series. I'm Robert Ta, CEO and co-founder of Epistemic Me, here with my co-founder and CTO, 12-time founder Jonathan McCoy. We have 3 segments in this series: AI news and our takes, AI memes and what's funny in industry right now, because we all need a laugh, and what's the meta from an AI builder perspective. So let's just hop into it. The very first thing. So Anthropic and OpenAI are both launching joint ventures for enterprise AI services. Here's what happened, what they got right, and what nobody's talking about. So Anthropic's joint venture is valued at about $1.5 billion, and the breakdown in capital is Anthropic, Blackstone, Hellman Friedman each contributing about $300 million, Goldman Sachs contributing about $150 million, with additional backing from General Atlantic, Sequoia Capital, and more. OpenAI's venture, the development company, is raising $4 billion from 19 investors against a $10 billion valuation, including TPG, Brookfield, and Bain Capital. There's zero investor overlap between the two ventures, indicating two distinct financial ecosystems forming around services. And both labs are adopting the Palantir model for, for deployed engineers, embedding very skilled engineers who understand product and engineering directly inside client organizations to accelerate enterprise adoption. From Anthropic's own CFO, Krishna Rao, exact quote, "Enterprise demand for Claude is significantly outpacing any single delivery model. Our partnerships with the world's leading systems integrators are central to how Claude reaches large enterprises." And Fortune reports the consulting industry disruption thesis: for every dollar companies are spending on software, they spend 6 on services. That's the market both labs are targeting strategically. So Jonathan, as a 12-time founder and our CTO, what's, what are some of your thoughts on this play from both OpenAI and Anthropic?

Yeah, it's an interesting story, right? Um, the— I think Anthropic and OpenAI both had announced various ways of, you know, having forward-deployed engineers in these larger enterprises. And this is kind of like maybe the longer tail of, uh, the, the, you know, the companies that they couldn't address through the first round of strategies. And I think OpenAI's first strategy was partnering with big consulting firms like McKinsey and PwC and, and those— the likes of those. And, um, so this is kind of like the longer tail effort of that. And a lot of interesting things. I mean, one is how does the value of AI get realized in enterprise? The need for a forward-deployed engineer shows that some of the frictions of, say, applying a model in the actual context of work and getting return out of it. And what Anthropic's been doing with these forward-deployed engineers is going in and understanding the domain, creating software services that are specific to like how the forward-deployed engineer is configuring the model to be used and trained. And I think that sort of like last mile of delivery is opportunity, you know, to like take these powerful models and apply them to like specific business problem with ROI. So it's interesting, this is the form factor that it's, it's coming in. I think from one standpoint you could say, well, they have to do this because it's not a proven thing yet, and this is actually like a business risk to actually get that last mile of value and get the applications working in enterprise. And we see a lot of stories about enterprise dysfunction and the lack of ROI on that front line. And I think the other thing for us, our company, is what role do we play in this world if we're trying to work with enterprise customers and, you know, like, do we, do we need to like engage with these partnerships? Do we need to, you know, work with these companies or is there going to be room for smaller players to, you know, engage in, you know, on top of OpenAI and Anthropic and deliver for deployed engineering services?

Yeah, that makes a lot of sense. I like what you said about the last mile, which is they're, they're racing for enterprise adoption as a story. Circling around their IPOs impending, both Anthropic and OpenAI. And they both, both raised a shitload of money to go win an enterprise, right? Um, and it also intersects with OpenAI's move of dropping Sora, uh, and the consumer angle there to just focus their resources to capture enterprise market. All the, all the strategic moves that we could see from the outside from these labs show that they see the money is in capturing value in enterprise for knowledge work. It makes a lot of sense. They're building intelligence layers. They want those things last mile sort of installed in enterprises so that they could accelerate the value these companies are getting in automating a lot of knowledge work. And I think, I think there's a bigger thing there. The bigger story to me, having worked in enterprise software for a large majority of my career at Workday and then as chief product architect at Dayforce, is that they're really admitting in these moves, OpenAI and Anthropic, that APIs are not enough for enterprise. And we've talked about this before, which is in enterprise software, it comes down to a few things. Um, there are four problems to solve in getting new enterprise software up and running. It's gonna be integrations to existing systems that your company's dependent on. It's gonna be reporting to get the right data to the right decision makers so that they have data-driven decisions. There's going to be data conversions from old system to new system, insert whatever systems you want, X and Y there. And the last one is change management. And I actually think that's where the forward deployed engineer role does best is it's a champion that goes in on the inside, builds trust quickly. You kind of have this sort of person that could do product and do engineering and help with change management as a trusted champion embedded. And then they could bring those learnings from the field directly from the enterprise companies they're trying to install capabilities in back to Anthropic, back to OpenAI. So the learning loop should be quite high in terms of learning velocity. Well, what do you think about that? The whole, like, I don't know, my read on it is APIs are not enough. You kind of mentioned the last mile thing. What does that tell you about kind of state of enterprise AI adoption? Any takes there, Jonathan?

Yeah, I mean, I think the reports that are coming out, they're kind of— it's a jagged surface area in terms of like who is getting return from AI, what does it look like? Like we know that developers are, you know, like getting a lot of value individually, whether or not that's trickling up to like a top-line return on investment number for the enterprise. Is kind of a mixed bag. And I think there's two kinds of applications. Like, there's people, there's enterprises that are looking for AI to, like, make things more productive. And there's looking for, then there's enterprises looking for AI to, like, let them do new things in new ways that are unlocked by AI. And I think the proof point's more in the latter, that there are firms finding ROI. And, but again, it's quite scattered and the playbook isn't, like, kind of, uh, solid yet. So we're kind of looking at a situation where there's massive investment in data centers, and now like there's kind of like massive investment all the way down the supply chain to deliver these tokens to people that are using them on the front line, right? Um, and all along that, there's still, it's still kind of like an unproven ROI on token, uh, mathematics that is universal. It's quite like, you know, um, it's quite jagged across where that it's supplied. And so it's a bet. I mean, it's a bet on the future and it follows— it fast follows the largest infrastructure investment in human history, which is these data centers. And so this kind of follow-on investment just makes sense. It's like kind of pot odds for the bets that, you know, that enterprise is making as a whole around AI.

Yeah, that makes sense. And for AI builders like us, clearly there's opportunity there. And we mentioned this on the last pod, which is OpenAI's partnership with AWS Bedrock and trying to build up services ecosystem there. Your question earlier, which was, is there opportunity for smaller players? I'll, I'll cite something. I was talking to Hamel Hussein, who's now become a good friend, fellow founder entrepreneur, um, who moved from corporate to do his own thing. And he's the most famous guy in the evals, AI evals game. Uh, I was texting him about this actually. Um, and you know what's funny is he was saying that, yeah, Accenture, PwC, these big players, uh, in consulting, he basically gets the deals where they go mess it up because they don't know what they're doing, and then it falls— then he gets a fast follow, like they get a call and it's like, hey, these Accenture, PwC guys don't know what they're doing. Hamel, like, you seem to know what you're doing, kind of get some help. And then he's like, here's my retainer. And so for the people out there thinking that PwC, Accenture, et cetera, know their shit and could install AI with speed in an enterprise, don't make that assumption. Also, this is coming from, uh, I, I worked at Workday. I started my career at Workday as an integrations consulting, uh, consultant. And I remember working with some Accenture folks and PwC folks, um, side by side on projects. And I basically had to go clean up their mess. And 10 years later, uh, I'll say it's the same thing because what they're, what they're trying to do actually is just get some college new grads to get staffed on projects that are managed by partners where the partner gets a giant cut to sell it. And then they staff it minimally to get the highest margins they can on the service engagement. What that means is they're not really incented on your outcome. They're incented to get you on recurring service contract retainer to keep them there. Uh, and I, I'm very excited to see that disrupted where smaller players like us on our services side and then other AI builders out there, you can go make this happen. You can go and eat some of their lunch money because from what I'm hearing from industry insiders, uh, a lot of these big contracts by big players, they fall apart because when the rubber meets the road, they don't understand how to stand up an LLM product with a world-class evals pipeline. That's what I'm hearing anyway. So, um, Jonathan, any thoughts on that before we move to the next topic, which is really around the whole SpaceX, uh, Anthropic Claude code usage limits, um, going up and data centers, which you just kind of alluded to?

Yeah, it's interesting what you say about your experience. I just say I think there's a lot of, um, opportunity on that for deployed engineer front. Like there's a shortage of engineers and the shortage of like case studies that have achieved ROI outcomes. And I think that provides an opportunity landscape for entrepreneurs like you and I.

Absolutely. So got to go get that money. Yeah. Sweet. Okay. Next thing, let's, let's talk about this. So Anthropic raises Claude code usage limits, credits new deal with SpaceX. Here's what happened and here's what nobody is talking about. So, uh, we'll talk about this in a second, but, um, Elon Musk is really excited to partner with Anthropic via SpaceX, uh, to get more compute essentially. So Uh, at its Code with Claude developer conference on Wednesday this week, Anthropic announced a deal with SpaceX to utilize the entire compute capacity of SpaceX's data center in Memphis, Tennessee. And on stage at the conference, CEO Dario said the deal was intended to increase usage limits for Anthropic's Pro and Max plan subscribers. That's us. And for other AI builders out there, you must be happy too when you saw the announcement that we will get more usage limits. And I am really tired of, hey, ran out of tokens, gotta wait till the next cycle. I know everybody else is tired of that too. I think it's become a meme looking at Twitter. And so in this, you could see there's a table that kind of outlines the changes for Opus from tier 1 to tier 4 usage limits as of May 6th, a couple days ago. So maximum input tokens per minute at tier 1 went from 30,000 to 500,000., and then maximum output tokens per minute, 8,000 to 80,000. And then on tier 4, it's 2 million to 10 million and 400,000 to 800,000 respectively. Anthropic claims that the deal gives the company access to more than 300 megawatts of new compute capacity. SpaceX focused its announcement on the capability of the Colossus One supercomputer that's at the center of the deal. Quote, Colossus One features over 220,000 NVIDIA GPUs, including dense deployments of H100, H200, and the next-gen GB200 accelerators. Anthropic also expressed interest in working with SpaceX to build up multiple gigawatts of orbital compute capacity. I think that's like the kind of underlying story there, which is, oh crap, like we're gonna start getting compute, uh, from orbit, which is just like, from a nerd perspective, that's just super sick. So like, we're just living in the future and this kind of stuff is becoming normal to talk about. Cool. So there's more here to talk about, but that's kind of the headline. And Jonathan, from your CTO perspective, kind of strategically and tactically, like thoughts on this move? Obviously we're happy about it from a builder perspective. Usage limits go up. I'm gonna be cloud coding a lot more.

Yeah. You know, as a builder, you immediately get this kind of like, oh, that's great. Like I've run into that problem and maybe I won't have that problem anymore. And like has affected my usage where it's one of those things nudging me closer to code using Codex more frequently. Um, on the other hand, like, you look at the background of what's going on here and you wonder, uh, okay, you know, like, Musk is basically giving SpaceX's leftovers to Anthropic in this moment, you know. And how long is that deal going to last? He says, you know, we're using Colossus-1, so you can use our leftover, you can use Colossus-2, we have some extra capacity. Um, and I think there's a broader thing going on here where there's— there are constraints to how these AI services delivered. There's there's energy and there's compute, which includes the hardware. And we're going to see shortages of both of these ongoing over the, over the years. And the people that sort of hold the most GPUs, that hold the most energy to deliver them, uh, are being kind of sort of power brokers. And I think that's where Elon has found himself, or, or explicitly positioned himself. Um, so, uh, that, that's really interesting. And I think there's, um, people that are saying, you know, like that OpenAI and Elon Musk are playing the long game of looking at like, how do we secure power? How do we secure semiconductor supply chain? How do we get the data center capacity? Uh, and they're very well positioned, whereas Anthropic took a slower approach. And so, um, they're more reliant on people that have those resources in order to like continue to, to provision at scale. And we're just seeing one of the first deals. I think this is being an ongoing thing and probably a source of a lot of geo geopolitics kind of drama as time goes on.

Say more on the geopolitics drama for those that are not geopolitics nerds. I think you have a really great view on that. And we talk about this and you share your thoughts about this. Jonathan, I feel like Jonathan knows everything that's going on in the world, like a lot of the times, at least when it relates to existential risk of humanity. So tell us, tell the audience more about like why this power broker game is really just one domino, the first to fall, in your prediction of, hey, there's gonna be some geopolitical implications from this, where from, from my view, it's simply put, we cannot get enough compute to meet demand, and, uh, these frontier labs are going to look for more compute. So talk more about that.

Yeah, uh, well, I guess you can look at like where the, where the potential shortages, you know, in, in power and, and, and micro, semiconductors and even raw materials that go into building semiconductors. And one interesting bifurcation is the, you know, the energy strategy of the United States versus China, for instance. China's producing more energy than us. Um, and so they have like more overall electricity capacity, and it's going to take time for the US grid to evolve and, and like respond to the demand of the energy that these data centers have.. And it's kind of like building a building in San Francisco. You know, it's very slow. There's a lot of permitting. There's a lot of people who have to say like, this is okay. It's a long process. We can't do this overnight. And it's one of the reasons that Musk in particular is talking about space data centers, because then you have an access to solar power that not sort of, sort of like where the regulation isn't there yet. And you can build potentially massive data centers that are just floating around in space., and it's kind of like interesting in theory, but that's also a, a really risky sort of project vector that could take years to, to unfold. So what you get is like, there's a, there's an energy constraint. We can only unblock it slowly. Um, then, then you move to the, uh, semiconductor constraint and GPUs. And like, you know, everybody knows that, that, that NVIDIA GPUs are very difficult to get right now. Um, and capacity is sort of like booked up over the coming years. One of the constraints is manufacturing and TSMC in Taiwan is the singular choke point to all semiconductors being built. And not only is it— is that like kind of a monopolistic type of situation, but it's also under threat by geopolitics from China being interested in taking over Taiwan. So there's like significant threat and there's a slow process there of unbundling what's going on in TSMC and bringing it into the United States, into Arizona fabs. And there's also processes like like Elon Musk's TerraFab, you know, which would plan to like build at scale. These things will all take time. So I think it's like there's a known set of constraints. All of them are slow to unbundle. And that doesn't even get into like the raw material constraints that could, could hit us because of global supply chains being disrupted and us just not having enough like, you know, cadmium, for instance, to like, you know, support the semiconductor production process. So this is going to be like a supply chain whack-a-mole. And, um, you know, I think we're going to see this and feel this as consumers, uh, as time goes on.

You heard it here, folks. You heard it here first, folks. Supply chain whack-a-mole is impending, and this is probably the first of many dominoes to fall. And I'm predicting there will be many more partnerships around this. This was an interesting thing, and thanks, Jonathan, for commenting on the geopolitics of the situation, especially with the supply constraints in terms of raw materials and then the semiconductor supply chain with TSMC being the main bottleneck and then the, you know, whole tension with China and Taiwan and the US, etc. And with the US kind of moving more towards manufacturing in-house, so to speak, definitely seeing those trends. This was interesting. So in this article, he tweeted on X here, Elon Musk: Anthropic hates Western civilization. This was a couple of months ago. This just kind of shows you how fast people could change their minds, slash how, uh, people are looking towards alliances in this game, right? Um, so Anthropic hates Western civilization. That's, uh, what— that's, that's what—

you got a community, did it?

Yeah, that's funny. Oh gosh, this version of Claude's Constitution of 2023 is still accessible on Anthropic's website. That's funny. That— so So clearly he doesn't hate them anymore. And this is really interesting. The latest tweet is that— this is from Elon 2 days ago, right on the heels of the announcement. By way of background for those who care, I spent a lot of time last week with senior members of the Anthropic team to understand what they do to ensure Claude is good for humanity and was impressed. Everyone I met was highly competent and cared a great deal about doing the right thing. No one set off my evil detector. So long as they engage in critical self-examination, Claude will probably be good. Probably is doing a lot of work there, but probably, probably be good. It's good to me right now building. But this is literally 2 and a half months later from the original tweet of they're evil. Any thoughts on that? This, I mean, this goes into the topic of how Elon really hates OpenAI, right? And that trial is going on right now. Um, go ahead, Jonathan.

It's a— the tweet is straight up kingmaking, right? Like, yeah, it's just, uh, very kind of feudalistic. And, um, and I mean, what's the eval function on Elon's, like, evil detector, right? Has he actually curated the evals and tested them against datasets? And like, you know, That's the eval function that's basically decide like, you know, who we get AI from right now. It's quite arbitrary and it's a little scary.

Yo, that's hilarious to think about. I'll talk to Hamel about this. How do you eval around evil? Yeah. This goes into probably philosophy meets technology. We have a lot of thinking about the axiology of I should say, the axiomatic problems around morals and ethics. And before we move off this topic of, uh, Claude Code raising usage limits and crediting the New Deal with SpaceX and the first domino to fall here around harnessing as much compute as possible and the power broker game, anything else before we move into the next topic?

Uh, I'll get here. Yeah, cool.

Sweet. So next thing we should probably, because we're on the heels of this, we should probably just talk about the Elon Musk, uh, Sam Altman, like, OpenAI trial. You've been following that, Jonathan? You even told me, I didn't know this, I've been so heads down building. I live in Oakland, for the audience, uh, Jonathan lives in SF. He was like, dude, you can go the Alameda Courthouse and just check it out. And I was like, whoa, it's that close to home? You know, because you see these headlines and stuff like that. I was like, I literally could I could drive 10 minutes and go there and see this for myself. But yeah, let's talk about that. So OpenAI versus Elon Musk. Here's the latest of what happened and here's what nobody's talking about and our takes. So there have been a series of texts that have come out from Sam Altman and Mira Murati, I think that's how you pronounce it, who ended up leaving. And this was, honestly, this was, I don't know, I read these texts and I'm like, this is how these people text? Like, I just, I was just like, what? Have you seen this? By the way, I saw this.

I haven't seen this.

No, I was— Oh, this is the most hilarious. This should be a meme. So this was during that weekend where Sam got ousted and then basically was like, where am I being brought back? What the fuck's going on? Remember that? Uh, and so they began texting at 5:43 PM Pacific time on November 19th. Sam Altman, can you indicate directionally good or bad? Satya and others are anxious. No, I actually inserted R there. He just didn't even have the right grammar. Satya and others anxious. Mira said Directionally very bad. Sam says, okay. Altman offers help to make it better. He says, what do you want to make it better? I'm still willing to just walk away if that helps. If they are ramped up for crazy lawsuits against me, then I'm not sure what. Can you please tell them I just want to resolve this however and would like to join? Mira says, they're convinced about their decision. Sam says, for me to be fired or some new thing? Mira, yes, for you to be gone. And then Sam says, can you wrap up soon? Lots of pressure from Microsoft for an update. Mira says, Sam, this is very bad. Sam says, can I come in? You can't make this shit up. I'm like, are these real texts? These are real texts from the people deciding the future. Isn't this wild? Like, it's like bad grammar and all. And I'm like, damn, my parents text like this. This is actually comedy. Sam then says, then can I come in and talk about a path forward with them? Mira says, they're saying no and they need more time. Uh, Sam says, more time for what? Mira says, they've walked me through all the reasons and the issues with you and why you can't be CEO. Sam then says, can you ask why they've been saying all weekend they wanted me back? Mira says, they want new CEO in place. Like, I just— look at this. This is the next text. Oh my God, this had me rolling when I, when I read this. Sam says, Still don't want me? No way. What? Mira then says, new guy is rando Twitch guy. Speaking of, uh, Emmett, Emmett Shearer. Mira then says, they don't want you. Sam says, Emmett? Mira says, yeah. Rando Twitch guy. Like, this is, this is how they're talking about it. Yeah. Oh shit. Mira then says they don't care if everyone quits. Sam then says, is that what they want? Is the IP going to Anthropic? That's what team thinks. No time for that's what the team thinks. It's got to be that's what team thinks. Mira then says, just not your hand on AGI. Sam says, I cannot come back, exclamation point. That's a big exclamation point. Okay, I'll pause there. What do you think of all this so far? This is— I thought this was one of the funniest things I've seen in recent memory.

Yeah, it's kind of wild, just like the, the board drama, um, and the situation that the company put itself in. And remember, like, who is on the other side of the board? I remember Ilya Seshgover was one of the like key holdouts, uh, you know, uh, maybe Greg Brockman was involved. Um, it— yeah, it was like kind of crazy to see the way the company evolved. Uh, I, you know, I have some like overarching thoughts on the whole, like, the whole basis of this, but like, go for it.

Yeah, thoughts. What are the overarching thoughts on the situation?

Yeah, I think, you know, if you go back to when, um, like Elon and Sam founded this OpenAI and they raised a whole bunch of money for, to like have OpenAI, the, I guess, I think there is a certain amount of naivety that you didn't need a for-profit model in order to bring this, this technologies to the public. Right. And, um, and I think like if you extend the thinking and like how well do the people that are building these models understand how the models will impact the rest of the world, like they sort of don't have a thought model for it. You know, like they're like, this is going to be really powerful. AGI is going to be this thing that changes everything. But like beyond that, they don't really have like an economic perspective, a humanitarian perspective, even an MBA business perspective, like, you know, to like, you know, be savvy about how to introduce this technology to the world. You know, they're just guys figuring out just like everybody else. And I think they got themselves in this hole just like that. I think nobody would make that mistake today and say, like, let's start a nonprofit for like world-changing technology. They would say we have to be a for-profit B Corp, for instance, if they, you know, even want to have a corporate mission about it. Um, and so it's, it's just, it's just interesting to see how such smart people back themselves into a corner and get into these, these like things that seem like sort of obvious in retrospect that they were going to cause friction.

Yeah, hindsight's always 20/20. The most hilarious thing to me is that these texts reveal to me anyone could go do anything. Like, it just, it just shows you people at the top are just figuring it out, just like Jonathan said. And, uh, they don't even grammar check themselves in texts. Um, yeah, just reading this next part. Sam says, did Satya get on a call? Yes, with him. Sam then says, wait, I have an interesting idea. This part was fucking hilarious. Mira then says, still with Satya, go ahead. Sam then says, what if Microsoft acquires OpenAI? Would that provide the governance the board wants? Is Satya making progress at all? Mira then says Satya is being diplomatic. That's wild that this is just a text thread to be like, yo, like maybe Microsoft acquires us and this solves this little problem we got ourselves into. And I think there's even, yeah, you're just YOLO, YOLO thought. That's, that's what you see here, folks, is these are how big decisions get made. By people baking the bread or making the sausage. One other thing I think is an undercurrent here is Mira was trying to take his job. That was another thing, part of this. That's what this sort of revealed. Because she was CEO, I think, for like a day, if I remember that right. Let me look that up just so I could fact-check myself.

Mira CEO for— I don't know if she was trying to take it as much as she was most likely candidate for the position given the situation.

Yeah, but looking at this, like, if she was trying to balance this and trying to talk to Sam about this in this way where she's trying to say, I don't know, whatever, like, I think— I don't know, it seemed really clear to me on the outside, you know, asterisk on that, I'm on the outside, that it seemed like a land grab by her. Makes a lot of sense. Um, okay, I thought about your proposal, I'm on board. Sam says, KK, I think it's the best shot to get company back and keep people. And if that doesn't work, we have a new thing in the compute and the IP. But I feel that has a good chance of winning. I think you all need to get a petition of everyone saying they will quit and join. Um, yeah, that's interesting. I, I guess backing up a little bit, let's, let's go to the overall trial. So this is a Yahoo article, everything you need to know from the first 2 weeks of the Musk versus OpenAI trial. Musk's lawsuit against one of the world's most valuable private companies has provided a number of details about the inner workings of OpenAI and the relationships between Musk, his fellow co-founders, CEO Sam Altman, President Greg Brockman, former Chief Technology Officer Mira Murati, and former board member and mother of 4 of Musk's children, Shivon Zilis. Musk contends that Altman and Brockman duped him into donating millions of dollars to get the company off the ground with the understanding that would— it would remain a nonprofit, only to flip it to a for-profit later. OpenAI, however, says Musk is angry that the company rejected his offer to merge it with Tesla and name him CEO of the new entity. Musk is seeking damages and a reversal of OpenAI's transition to a for-profit. So earlier this week, OpenAI filed court documents claiming that Musk tried to gauge Brockman's interest in a settlement before the start of the trial. When Brockman suggested both sides drop their claims against each other, Musk allegedly told Brockman he would turn him and Altman into the most hated men in America. The case will decide the future of OpenAI and whether it will continue to operate as a for-profit entity or revert to a nonprofit structure. That's really interesting. Um, what do you think is going to happen here? I think the world is open to hot takes on this one. I don't really see them going back to a nonprofit structure. Like, I just don't think that's going to happen. What do you think?

Yeah, I mean, I think the, the question for that lies in like case law and like, you know, the general case of, uh, a nonprofit converting to a for-profit and one of the boundaries around that. And I think it'll be some specificities there that might be interesting case law for years to come. Um, and I think, uh, like sort of, you know, if you look on Polymarket and who's, who's favored to win, I think, you know, like Musk is the underdog, maybe is like 30% chance. I don't know what the details say today, but, you know, I think— but he kind of wins even if he just makes OpenAI look bad, if he gets their recruiting funnel kind of like, you know, puzzled about the underpinnings of the company and the way that these executives have behaved. And there may be key AI researchers who OpenAI wants to recruit or wants to retain that are affected by this and affected by the stigma that it creates. And I think that's the game that Musk can play., so, you know, like, I think that text, that, that text to, to Brockman was pretty revealing in the sense that, like, I'll make you guys look bad. Um, that's really the game, you know, he's playing at the end of the day. Mm-hmm.

Yeah. That's interesting.

Mm-hmm.

Yeah. Just reading more. Brockman also disclosed that he holds a roughly $30 billion stake in the AI company. As well as holdings in other Altman-backed companies. This is getting weirder. Zilis revealed that she became pregnant via in vitro fertilization in 2021, but didn't tell him that Musk was the sperm donor. Brockman said he found out later when the media reported on the matter. Dude, you can't make this shit up. Like, this is wild. Like, what is this? Let's just throw that shit in there too. Yeah, the case. The case of the century. To sort of determine a big power grab or struggle that will determine a lot of things for humanity. And let's just inject this in there too. Why not? Yeah.

I wonder, you know, what Zillis's eval function is for sperm, right? Like, you know, I wonder what their selection criteria were. Like, you know, of all the sperm she met, this is the sperm that she wanted.

Like, um, yeah. Yeah, probably having to do with wealth and power somehow. You got to break down the grading rubric a certain way, but—

but that's not epigenetically expressed.

No, I don't think so. It's a very subjective thing. That's got to be a human-in-the-loop sort of evaluation. Oh man. Okay, so that's, that's the latest on the, uh, Musk versus OpenAI trial. We'll continue to cover this in our weekly pod. Um, moving on, there is another thing that we saw this, this past couple weeks. We just were made aware of it. Um, it's about simulations, which we're really interested in because we build digital twins of people and quantify their subjective experience. For AI product teams trying to build agents, closed-loop agents in particular, with hyper-personalized experiences to end users. So that's what we do with Clarity, our API, to inject the right context at the right time. And simulations is definitely a case that we have come up with in our engagements with customers and prospects. And underlying it, you know, there's always this desire to capture market signal as a business, whether you're small, whether you're big, and then convert that market signal and learning, uh, into decision-making for creating an asset or a service to capture, um, that demand and capitalize on it. So simulations is obviously like a very interesting angle. Can you simulate the world and people in the world and what they might do? And with AI, that kind of, uh, at that intersection, it's a sort of interesting story there. So CVS Health and Simile I think that's what they're called. Uh, CVS Health and Simile, simulations for faster, safer decisions. Here's what happened. Here's what they got right and what nobody is talking about. So there's a case study here with medication adherence, um, research compressed from weeks into hours, and they found trust was a higher barrier to entry than cost. Uh, similarly, AI raised about $100 million Series A. To build generative agent simulations trained on, uh, a few million responses. So in this article, this is from Simuli themselves, uh, Simuli simulation is evolving from faster surveys and concept tests to a full-stack capability modeling individuals, then journeys over time, then interactions, and ultimately entire markets. The frontier is no longer predicting what a customer might say at a moment in time, but understanding people, how people update beliefs, influence one another, and respond to incentives inside design systems. This is obviously really interesting to Jonathan and I from our own philosophy of subjective experience and being, which grounds on Karl Friston's and Michael Levin's work around predictive processing and the free energy principle. So TL;DR, what is the core primitive that drives agentic decision-making? Your beliefs and how you evidence your beliefs. That's, that's our belief. And here they actually have a real-world use case where CVS Health has used generative agent simulations to help guide decision-making with the support of Simile, the first company to build and bring this technology to market. Built on about 3 million consented responses from more than 400,000 participants across 200+ behavioral scenarios. These agents are modeled on data from real people, including interview responses, past choices, and so they act as safe, accurate stand-ins for the individuals they represent. For the past year, CVS Health partnered with the Simile team to build a significant simulation capability that allowed them to get a high-fidelity lens on customers and competitors, and that gave 4 key tangible advantages for them in business. Number 1, Faster validation of known insights. Number 2, sharper NPS and experienced drivers. Number 3, adherence and behavior change under real constraints. And number 4, differentiation and competitive positioning. Jonathan, from your perspective, having worked on user modeling for the longest time and also before it was cool, what do you think of this simulation use case meeting the enterprise?

Yeah, it caught my attention because of their approach of simulating humans in their environments. And I think there's a lot of simulation startups right now that are raising a lot of money. And you look at the way that they are modeling humans, and it's a very sort of shallow model. This comes from demographic information, or like, you know, it might model be predictive about the population of decision makers, but it's not— you can't You can't like zoom into somebody, put yourself in somebody's shoes and understand how they're going to make a decision in a certain situation. And this is the first approach that I've seen that the company's raised significant capital. They seem to have a strong team backing them and they have, you know, this sort of like human-oriented way of like, you know, understanding how people make decisions. And I think what that gets them is that they have like a more representative vector to be predictive and say, where are simulations going to be predictive and what, in what situations? And I think that they're going to have a lot broader set of use cases that it works in than some of their competitors that I'm seeing.

Yeah, and I love that this is successful where they are driving business outcomes, not just dinking around creating simulations for no reason. I think that's the coolest part about this, which is CVS Health saw value. They partnered for a year, and now they've collapsed the throughput of quality decision-making by thinking, hey, what's the— what might consumers react to in our ecosystem? Okay, well, we have all this data on previous consumer behavior and decision-making. Can we make sense of that and simulate potential scenarios and how that impacts the bottom line, right? That's like a— that's a great use case. And it makes a callback to me about how NASA used digital twins where rocket making is expensive. You don't really want to mess that up. So you simulate a digital twin of the rocket and the different parts. This is something made in the mid-1900s, right, when we were trying to compete to go to space during the Cold War. And they pioneered digital twins and simulations to make better decisions on the engineering front. This is just a different application with a different technology layer to me around digital twins of people across a business, an ecosystem of consumers, to allow the business to decide what direction should we go if we have so many different ways to go? What are the levers to pull? And can we aggregate some model of our domain-specific consumer behaviors to then inform that decision-making so that our bottom line is more profitable? Ultimately, that's what you want as a business is maximize throughput of capturing market signal clarified through to the business lens, right? And then you could activate your teams around it. So I love this because it also intersects with use cases we're seeing around employee experience, right? Like, I'm seeing quite literally in some of the work we're doing with one of our customers that HR leaders, they want to make great decisions about their 1,000, 10,000, 20-plus thousand, 30-plus thousand human workforces. And even that's going to change with now we're moving towards humans plus agents as workforces providing enterprise productivity. So the landscape of simulations and what-ifs based on the what-is, like, I think that's such an underdeveloped area. And this very real business case study by a big giant, CVS Health, partnering with Simile, proves that there is a market. For simulations, um, around quantifying, uh, the users and their individual experiences and leading to behavioral data. Um, yeah, anything else to follow up on that, Jonathan, around this, this case?

Yeah, I mean, it's an interesting juxtaposition, us talking about the— some of the arbitrary decisions that we just saw in this, like, huge game of like OpenAI versus Musk. And, and now I think, um, you know, if we can— if we count on simulations to like back our decisions, um, I think that's, that's the overarching sort of, sort of thing. And then, uh, how can we make these simulations representative of the humans that we're working with? That lets us like understand people better. It lets us make better decisions as a whole. And I think that's going to be— it's going to be sort of like, um, a decision maker's best friend to have these kind of simulations and, and trust that they can understand that the, uh, the predictions they're making are representative of the people that they're talking about. In this case, the CVS customers that are, you know, working in the prescription pipeline.

There's an interesting question there, which is, if simulated agents can replace focus groups, what happens to the $80 billion market research industry? And I saw some other cases where people were able to reproduce— I saw this other startup, I think you showed it to me, We'll throw, try to throw it in the pod notes where they replicated one of the big consulting firms' market research studies over a 6 to 8 month horizon, if I recall correctly, overnight. And they produced even better results than the original market research. This is a really lucrative industry. Like if you ever worked in enterprise software and you went to go Google some shit about like a market, then you're going to see reports worth thousands of dollars and people pay that business pay for that because it's. Signal from market, um, of the territory and how the territory is changing, to capture a snapshot of a map of like what the territory is, to bring that back to the people on the signal to ultimately impact decision-making towards the bottom line. I think that $80 billion market research industry was just proven to be disruptible with this case study and with the other ones that we're seeing, uh, for, for the simulation use case. It, it makes sense. It makes a lot of sense. Why not run at-scale simulations that are domain-specific, tied to a particular problem for your own business and ecosystem you would care about, versus paying for some more generic market research conducted by people out of your company? Right, like, why not just bring that capability in-house with tools like this? Um, makes sense. Yeah, really interesting stuff.

Yeah, and one thing I'll say is that because a lot of the examples you give, um, you know, you're kind of looking at a group of people in the abstract and you're creating stereotypes or like stick figure sort of representations of who people are. And I think What the, what Simile, what the, the language that they use about their simulations appealed to me is that you can put yourself as, as a human in their environment and like, what is, what is this customer feeling when they're like given the adherence protocols for a prescription, for instance? And I think that the ability to like, um, put ourselves in the shoes of one another and to like take not an abstract representation of a human, but a phenomenological and, uh, a felt relationship with the customer is where I hope that AI brings us to. And because there's a lot of risk about just continuing to look at populations of people in the abstract as these stereotypes. And I think what it has the power to do is bring us closer to one another and closer to the felt experience of being a human, being a consumer, making decision in a certain situation.

Yeah, yeah, definitely. And I think that's for the AI doomers out there. What we're really saying is I think, I think there's a doomer case, which is, oh, we're gonna be more disconnected than ever. Everybody's just doom scrolling addicted to their phones. AI's gonna make it even worse. Okay. In terms of our AI meme section, we're just gonna look at memes together. And in the past, uh, week or so, we'll just bring up memes and we'll talk about it. One of the interesting things that came about from ChatGPT recently is, uh, what I'll call evaling against goblins. So it was revealed that there was a lot of, uh, I guess goblin references, uh, underlying the GPT model. And so there's this whole phenomena around, uh, goblins. This is— so hang on, let me look up— this is an art piece um, from the ChatGPT handle. Let me show you. ChatGPT has Goblin Mania in the US. Uh, this was a thing. Hang on. Okay, ChatGPT has a Goblin problem. An expert says it's bigger than an AI quirk. So here's what happened, and here's, here's what nobody's talking about in terms of, uh, this AI meme. Starting sometime in November, people who use ChatGPT began noticing some peculiar behavior. The AI chatbot would not shut up about goblins. So OpenAI, the company behind the chatbot, began looking into it. The mythical creatures, along with gremlins and others of that ilk, started showing up in metaphors that ChatGPT would use in its responses and even in images it would generate from non-goblin-related prompts. Over the next few months, the goblins spread. The problem became so pervasive that OpenAI acknowledged it in a recent public post, and an investigation revealed the use of goblin had jumped by 175% after the launch of ChatGPT 5.1. At the time, the company found that the goblins did not look especially alarming, but as newer versions of ChatGPT rolled out, they noticed an even bigger uptick. For most people, ChatGPT's abundance of goblins is a benign AI quirk, and they've taken measures since then. Anyway, backing up a little bit, what do you think of this whole goblin phenomenon, Jonathan?

Oh yeah, it's not just goblins, it's creatures of all kinds. Yeah, yeah, mythical creatures. Yeah, if you read the OpenAI, uh, kind of history behind it. I think it came from a personality for OpenAI, for ChatGPT, they created that was like more of a nerdy personality and was trying to communicate sort of like sci-fi or fantasy type of terms. And then that personality kind of leaked into their main, you know, corpus. Yeah. So that's, it's really interesting that such a thing would happen. And I thought, It's like, like, I've— somebody pulled up the prompt where they have the, um, like, you know, the system prompt that they have that guards against goblins and creatures of all kinds. It's, it's like hilarious if you just read it out loud.

OpenAI Codex system prompt includes explicit directive to never talk about goblins. This is hilarious. The system prompt for OpenAI's Codex CLI contains a perplexing and repeated warning for the most recent GPT model to never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query. This explicit operational warning was made public last week as part of the latest open source code for Codex CLI that OpenAI posted on GitHub. The prohibition is repeated twice in a 3,500+ word set of base instructions for the recently released GPT-5.5. Alongside more anodyne reminders not to use emojis or em dashes unless explicitly stated. So in addition to no em dashes and to never use destructive commands like git reset --hard or git checkout unless the users ask for it, we're just talking about no goblins, gremlins, raccoons, trolls, and ogres.

What's up with the raccoons and pigeons? Yeah, they're a little bit different than the other category, right?

Yeah, no, like, how did these get tied together? So you were saying like it was some nerd personality that leaked itself into, uh, the model itself from training. Uh, I think that's sort of hilarious, but look at this. This is, this is it. This is the prompt. You can go check it out yourself. We'll throw, uh, we'll throw this into the show notes. That's so funny. Yeah.

And did you see this? Apparently there's a Chinese equivalent that's happening too. Oh, really?

Yeah. Chinese equivalent that's happening as well.

In China, it will catch you steadily. Catch you.

Wired magazine was reporting on that yesterday. Oh, yeah. In China, it will catch you steadily. That's so interesting. I mean, it kind of talks about just like the levels of nuance there are to building quality products and these models. Um, it's just sort of funny that this has happened in this way where it's a goblin thing and then in China it's, we'll catch you steadily. We'll see what's next. I saw that, um, in the same article they talk about how XAI's Grok frequently brings up white genocide in South Africa. Oh, it's also interesting that Elon Musk is South African. Like, that's— I don't know if there are actual correlations there, but yeah, public statements more heavily in their training data. It could— XAI's Grok suddenly can't stop bringing up white genocide in South Africa. The topic has long been a hobby horse of ex-owner Elon Musk. So this was, this was last year, um, and you could see this is— hey Grok, is this accurate towards baseball? And then Grok comes in: the claim of white genocide in South Africa is highly debated. Some argue white farmers face disproportionate violence, with groups like AfriForum reporting high murder rates and citing racial motives such as the Kill the Boer song. However, South African courts— That's interesting. For AI builders out there that are looking to test your AI products and your harnesses, I guess this is maybe something else to look out for. I wonder if anybody is experiencing such weird behaviors from their AI products built on these foundation models. Yeah, I guess I would have, from a meme perspective, in my 10+ years in tech, I would have never thought we'd be building regressions against goblins. I'll say that. Yeah, that's a great way to say it. Yeah, that's wild.

Yeah, like I'd heard a perspective yesterday from a writer, um, that, uh, said that ChatGPT keeps on getting better and worse since GPT-3. And he doesn't feel like it's actually evolving and to, to get better at writing. Um, and this is the kind of thing that, you know, this is the kind of regression that you see and you're like, well, how, how close are we? How much— what is actually getting better without like, uh, you know, uh, an objective evaluation function like you have in math and coding? Um, you know, is it even getting better or is it just like we're throwing more tokens at the same problem and it sort of— you sort of get like a distribution of of random results.

Yeah, totally. Goblins, folks. Protect against goblins. Hide your kids. Hide your wife. Um, all right, next meme. This is also from OpenAI. There's a viral prompt: redraw the attached image in the most clumsy, scribbly, and utterly pathetic way possible. Use a white background and make it look like it was drawn in an old computer painting program with a mouse. It should be vaguely similar, but also not really. Kind of matching, but also off in a confusing, awkward way with that low-quality pixel-by-pixel feel that really emphasizes how ridiculously bad it is. Actually, you know what? Whatever. Just draw it however you want. That is actually kind of hilarious that this is part of the prompt. Actually, you know what? Whatever. Just draw it however you want. I'm going to go ahead and just toss my PFP into ChatGPT, and we'll just do this live. Like, cool. I'm gonna get a picture of me and my dog Kenji, and then we'll, uh, here's— okay, here's the picture that I'm going to use. This— I just took this selfie yesterday of me and Kenji being ridiculous. Let's see how this goes into ChatGPT land. All right. Let's check it out. I haven't done this yet, so we'll see what this does. While this generates, let's look at ChatGPT's own, their own Instagram basically. Like they did this for ChatGPT, then look at their Instagram account. This is actually hilarious. I think this is great. It just, it's just ridiculous that this is like meme velocity has reached an all-time high where something appears and then anybody could try it with AI now. And then people will try it and then post their own version of this. I think that's actually from a Meta perspective, this is one of the best growth marketing hacks I could see where the product itself creates these sort of meme moments. That people want to participate in and create community around. So I'd love to see more of this kind of stuff from Claude, but maybe, maybe they're too intellectual for that. I don't know.

Yeah, well, it's, uh, if you remember Studio, uh, the Ghibli, uh, images, you know, was the thing when image generation first came out. And that, I think, pushed OpenAI direction. They said like, oh, consumers, and let's build Sora. And then, you know, we're actually in a mode today where Like, no, it's like, never mind about consumers. Uh, let's just focus on coding agents. Um, and so yeah, it's kind of an interesting footnote in, in the development of all this stuff.

AI meme history, y'all. Y'all heard it here first. All right, check it out. This is, uh, this is the original. Me and Kenji being weirdos. He was yawning and I just took this selfie of me pretending to yawn. And this is, I don't know, it's really good. Yeah, that's kind of funny. So that, that, I love the Kenji one. You know what's funny is my aunt always gets on me for how big my nostrils are, and then boom, like massive in this MS Paint, uh, drawing of me. Gigantic nostrils. Like, oh, it even got my, I wear some like a bracelet bead thing. It got that. I'd also noticed that I have some peach fuzz facial hair, kind of drew that in. It's a bit more— honestly, it's giving me more credit than I got here. Uh, look at Kenji, he's actually like, this is so cute. Actually, the Kenji— Kenji looks great.

Like, this is hilarious.

His eyes are very vivid. Yeah, very vivid. It's like very cartoony. I wonder, you know, just in terms of prompt engineering, I wonder if I did the same thing but didn't put in the actually, you know what, just draw whatever you want. Here, let me try that again.

I mean, chat, it was about what I was expecting.

Yeah, but like, why did they need the draw whatever you want thing? Like, why is that in the prompt that went viral?

So I'm removing that sentence. For upvotes on Reddit and stuff. I mean, it's just— Oh, okay.

Like, yeah. Okay. Maybe that's what it is. So let me just try again without that last sentence. So that last sentence being, um, actually, you know what, whatever, just draw it however you want. That's an interesting goal to give the, uh, AI. So I'm curious, like, if this actually turns out different without that.

In the embedding space, I think you can count on that being ignored. Like it's already got strong guidance, and so you could probably, you know, write something superfluous at the end, um, you know, and it gets translated into like a lower weight set of embeddings.

Well, I wonder if I tell it to draw us as goblins, I wonder what it would do.

I think you'd get a good goblin.

Yeah. Oh, hey, look, interesting. So quick side by side here where actually Oh, interesting. It didn't get the bead bracelet well. It also only gave me a chin hair, not slight goatee. Kenji looks worse here. Yeah, he looks like, I don't know, not as cartoony. I think maybe that last sentence does more work than we give it credit for. Now draw us as goblins. Let's see how this goes, given what we just learned about the anti-goblining. Let's see what a canine goblin looks like.

Yeah, in Microsoft Paint, of course.

Of course, of course. That is actually— it's, it's the new Mona Lisa-esque style that's, uh, in vogue right now. Part of the zeitgeist. Making the first draft. I do like— while we're waiting for this, I do like the new The new image generator, we're going to get into this in a sec. I like this AI is thinking thing. Before when I was using image generator before they came out with this thinking animation, I don't know, it just always felt like, I don't know, when are you doing something? I'll just say for AI builders, people want to know if the AI is doing something for them. Design touches like this, where you show not just, you know, there's provenance. That's a different use case where it's, you know, you're trying to figure out how is this thing thinking, but also just an in-progress thing. The in-progress animations, I just, as in terms of design nerdery, I like how that's evolving. They chose to go with dot dot dot, like moving around pixels. That's pretty good. That's not bad.

Yeah. There's your Christmas card right there.

Yeah, I'm downloading this guy. Sweet. Yo, that's pretty good. I'm gonna throw this onto Twitter. That's so good. Um, it messed up Kenji's ears, but you know, it's, uh, it's MS Paint level-esque. Yeah, it's a little too much goblin on his, uh, left nostril, but super goblin. Also, my nostrils don't look as absurdly big in this one. Actually, here there's Goblinified. Yeah, they've been Goblinified. Hang on, let me see. Do we have— is this it? Is this the Goblin? No, this is Clarity stuff. Meme ideas for AI builders. Is this it? No. Wait, hold on. Where did that go? Tech Kremlin. No, I lost the other thread. It's okay though. That's it for, uh, AI memes this week. Please try out— we'll throw the MS Draw paint prompt into, um, the show notes. Please try it out. And if you have memes that you want us to share and go over and laugh about collectively as a community, throw them onto the comments. Uh, you could email us at podcast@epistemicme.ai. I'll read every single one and I'll take your memes and we'll highlight it here on the show. We'll highlight the best ones. Feel free to send that over and I would love to see your Goblin MS Paint pictures of yourselves. Cool. Moving away from memes, let's move into our last segment, which is What's the Meta? In this segment of the show, we'll go over as native, AI-native builders ourselves building AI products on the front lines with AI, Uh, we really seek to help people build better AI products and use AI better. Um, we have a couple things to go over today, which are Codex Desktop, a visual feedback use case to close loop on QA and design with their new image gen capabilities. And then we also have a feature around the new Claude Code capabilities that have come out this week. So we'll go over managed agents, Claude code now being able to run programmatically via AI, those managed agents, multi-agent sessions, self-improving agents with defined rubrics and outcomes, and then agents dreaming. So here's what it means if you're building. Let's start with Codex Desktop visual feedback. I'll just go to that thread. That we have looked at, and I'll share my screen on that. This is the design loop that makes codex a magician at UX and UI, the full guide by Meta Alchemist on X or Twitter. And here's what it means for AI builders. So a really good image to ground on is this codex design loop. So a good codex design happens through an iterative loop, not a single prompt. And the whole idea is you could iterate, validate, and ship. Uh, you number one prompt, define the goal, audience constraints. Codex builds the thing, runs the thing, you get a preview locally, it creates a screenshot. This is a key thing. Uh, Codex reviews the visuals for hierarchy, spacing, text fit, contrast, states, consistency, et cetera. It will click and test it like a real user, revise, um, and then compare. And then choose the best option or iteration, and then go back and forth until it reaches, uh, the stated outcome or a goal. Um, this is really cool because here's a really quick example, and even this image itself was generated from Codex. You could see just like, you know, a year ago we couldn't get accurate text on something as complicated as this, or even very, very simple images. But now you can see the text is, it works, it just works. And one of the metas that we've been going over as a team is just creating images of complicated process flows and sharing them with each other. So Jonathan recently did that with our stacked agent harness architecture that we're building for other AI builders to be able to easily incubate agents and see with quality gates when it's ready for primetime to help you with our harness engineering. So in this article, it goes over ImageGen versus Vision. It goes over the loop around how this is probably going to change things from a design perspective. Like, where's Figma in any of this? You don't even need it here. And for the designers that are pixel-perfect, perfectionists, which I know a lot are, please don't take that as blasphemy. It's more like, I think we can get really good one-shots with an agent running in a big autonomous loop if you set a good goal. Um, there's a really cool prompt here around using Codex Desktop as a visual design loop to get that closed loop around a target UX/UI outcome. Here's an example of a game UI loop. Yeah, this reminds me of detail generates, right? Like, Jonathan, do you play— what video games do you play growing up, or were some of your favorites?

Oh yeah, I spent a lot of time on StarCraft 2, um, and, uh, Mario Kart. Oh, fuck yeah, 64 days. Yeah, yeah.

Wait, StarCraft 2, what, uh, what race did you play?

Uh, StarCraft 2. Um, I, I think I was the, uh, who was the Zealots, right?

Oh, Protoss. You're Protoss. Protoss. Yeah, yeah, yeah. There's Protoss, Zerg, and, uh, Terran, right? Yeah, good memory.

It's been like 15 years since I played that game, dude.

Yeah, dude, I was the biggest StarCraft 2 nerd. I don't know if I told this about you, uh, told you this about myself, but In StarCraft II, I was like semi-competitive in college because I had way more free time, and I basically skipped all my classes and just played that game. And, you know, and then like tinkered with hacking my Android phone on the side. Like, that's all I did. So, uh, and then I was hustling, you know, I had a private tutoring business to get me through college. But like, I spent too much time in StarCraft II. It was embarrassing how many hours. I probably too much, like thousands. Um, and it got to the point where This sucked. I never got paid for it, but I did enter some paid tournaments, and I reached, uh, for this one NVIDIA ran, I think there were 20,000 applicants. I reached the round of 32, I believe, or maybe 64, and it was right before the round where you get— got paid a couple thousand dollars. I like, I remember this vividly how, uh, I fucking lost and made so many bad moves that I just wouldn't normally make. And my key mistake was my friend who I was also a StarCraft 2 nerd with, he was like, hey, can I come over and watch you play in this tournament? And before that, I was just like, nobody's watching me, I'm just playing, I'm cruising along, crushing noobs in the tournament. And then he came over and watched, and I was just like, uh, pure programming pressure, fucking like, why am I making the Colossus right now? Like, shit, they're going— like, you know, it was just stupid mistakes. Like, and I remember this, this Zerg I was playing was so bad but I ended up being worse because I was just a scrub that folded under, I don't know, my friend watching me. It just sucked, dude. Like, my APM dropped like by half. Like, I was— it was just— it was terrible. I remember it very vividly. Um, yeah, I even talked to my friend after and I was like, bro, I don't think you should have come over. And he was like, dude, you sucked. Yeah. Anyway, so that's, uh, when I decided I'm not going to be a pro gamer.

Um, yeah, wow. No, totally. It's like the, the training harness, right? Like you trained in this one environment and then you put yourself in a different one for the competition.

Yeah, exactly. And, um, I wasn't good enough. The— my— I just wasn't good enough. Oh man. But anyway, uh, the point is game UIs are hard to create details around. I also had a, gaming company. My first tech startup was in gaming. And so this kind of stuff used to take forever and was very specialized. But look what Codex could do now, straight from Codex. You don't have to leave Codex to go to Figma, wherever the hell else, which is very time-consuming. And then also the generation of these types of assets, like these assets, you would have had to normally pay graphic designers a good amount of money, uh, for these 2D assets. Nowadays, it's just the cost of tokens. OpenAI introduces AI-generated pets for its Codex app. Here's what happened, and here's what it means for AI builders. So vibe coding just got a whole lot more adorable. I think there's a meta right now where, I don't know about you, but I played with Tamagotchis growing up, and I like, like Pokémon and shit like that. And so definitely, as I'm spending all my time addicted to the terminal, it would be cool to raise a pet at the same time as a side quest, right? And I think Codex and Cloud Code are in particular— I think Codex is pulling a play from Cloud Code where they introduced their little Tamagotchi pet thing. And Codex just basically did this same thing. Look, hatch pet, make me a cute goblin pet. And they're just leaning into the whole goblin story. Users can type /pet into the Codex app to summon or dismiss the companion. There are 8 built-in pets to choose from. You can also generate your own with the help of AI with the hatch command, like a cute goblin companion. Early adopters have already uploaded a bunch of options, and there are some versions of Microsoft Clippy. What? Wait, for real? Is this for real? Here's the Codex Pets store. Let me look up Clippy. This is becoming a thing. Wow. Oh my gosh. Ooh, he's pixelated. Clippy. Could be coming to life in Codex. Like, what? Very cool. Okay. What are, what are people most, what are the most liked ones on this? Dario. Oh my God. No way. Oh my God. A tiny frustrated Codex pet inspired by Dario, CEO of Anthropic with dark curly dark hair, black glasses, furrowed brow, and skeptical grimace.

Oh dude, have you seen those little robot dogs that they put, uh, Mark Zuckerberg, Elon Musk faces on? No, I haven't. That's funny. I think this is going to be a thing.

Robot dogs, Elon Musk face.

It's like the obvious application of the, you know, the next AI tool, next robot.

Here we go.

Oh, dogs hate those things.

Oh my God, dude. Oh, and it's meeting a Waymo. Oh dude. Oh, that's— oh, that's freaky and hilarious all at the same time. This is what humanity has come to. Wow.

I like the Zuck robot because it's like, it's pretty much just like Zuck.

It is like Zuck. Oh my gosh. They had a lost dog sign. Oh my. Yo, you know, I can't help but think about Camus' absurdism philosophy here, right? Which is, we see shit like this, money is going to shit like this, and then people are still hungry in the world. Right? Like, first principles, that shouldn't be a thing. We'll hopefully solve that someday. Yeah. But, um, like a lot of money. Anyway. Yeah, exactly. No, yeah, a lot of money. This is wild. Um, but hey, it's supply and demand and that's capitalism. So, okay, let's get back to the pod notes. Uh, next thing, Cloud Code changelog. So, new Cloud Code features. In Claude managed agents, they've added multi-agent orchestration and outcomes loop for rubric-driven self-improvement and dreaming for self-learning, as well as webhooks. Here's what happened, and here's what it means for AI builders. So just reading this real quick, the Claude Devs Twitter handle— I would suggest people follow this— multi-agent orchestration enables task delegation to specialized subagents. Agents can self-improve using a rubric that describes task completion. A dedicated greater sub-agent then uses the rubric to evaluate any work done, returns feedback for the next iteration. And then agents can post updates to webhooks configured in your Claude workspace, and that removes the need to keep an SSE stream open or poll for completion. I've been waiting for that one, actually. That's really cool. Agents write to memory in a session, and dreaming is a background process that reflects over many sessions to curate memories. It can edit them based on patterns, add new skills, or remove stale ones. Jonathan, you mentioned that outcomes are a very interesting thing. Could you say more on that?

Yeah, so here I think, um, you know, you have— we've been talking about agents and agents getting to closed loops. And so, you know, the idea that you can just specify an outcome and then use a subagent as an LLM judge to make sure that like the agent is producing outputs that are in line towards this outcome is really powerful. And I think it's not magic, right? Because you still have to define a really good rubric and you need to do that outside of what the tools that Anthropic gives you here, which is to, you know, this is what, you know, we've been taught and like this is how we work to build datasets, right? To use human expertise, open coding, to define the actual axial coding and then the rubric of different, you know, aspects that a given agent should conform to. So you can do that outside of the system, but then once you have a good rubric, you can just ask your agent to create the outcome and use LLM as a judge to do that. And so it takes away a lot of the machine learning work and a lot of the engineering work, and it makes it simpler to create agent-based outcomes. I think, so I think a lot of, uh, Cloud Code users might look at this as like, like, oh, you know, I'll be able to use this on all of my agents. And, uh, it's, it's not like a beginner's tool in my mind. It's, it's more like, um, once you know how to create good rubrics, then you've got a really powerful way to use agents with LLM judges to create outcomes. And I think that's something that's very empowering.

Yeah, I think the implication for AI builders, uh, especially ones building for businesses, right? If you're in an enterprise or you're building for an enterprise client as an AI builder, I totally see this as part of the bigger strategy shift around OpenAI and Anthropic competing for the enterprise market. They're trying to ship features that make it easier for the AI common citizen AI developer or builder to build closed-loop agents. That's really like me and Jonathan, talk about this a lot for our own company strategy, like what are the moats in the future? It's clear to us that the world is going to move to autonomous closed-loop agents around economically valuable workflows, full stop. Like that's the world, that's the way the world is going. And in this, like, I could totally see just them creating this makes that AI builder's life easier to go install a closed-loop autonomous agent around some economically valuable workflow at an enterprise company, right? Because how— the question kind of down to the details becomes, well, if the rubric is the gold, where you need to get a good rubric for a particular domain for that workflow, say here they're using the example of revenue projections and some financial model case, that's something every company has to do is close their books and have forecasts and reporting, right? Every company's got to do that. But every company's, revenue recognition and their accounting will look different depending on the maturity of the company, depending on the industry, uh, depending on how they want to position for further investment or an IPO or whatever, right? Uh, so obviously the rubric is going to be a bit different for a financial agent closing books at Company A versus Company B. So how does a developer who doesn't understand accounting do that? Tactically, what they would do is go talk to the CFO and their accounting team and go translate all the open coding, all the datasets, look at traces, etc., get into a failure taxonomy, get to a rubric. What Anthropic did with this move is just make that particular process a lot easier on that developer or builder, which I think is a great thing for their enterprise adoption story. Any thoughts on that, Jonathan, like this intersecting with the whole enterprise adoption game that they're putting so much money into?

Yeah, I thought, um, did you see the, the Harvey example they gave for, oh, release this?

Yeah, say more on that.

Well, um, you know, they, they showed, uh, how Harvey used its rubric that it had annotated itself using domain experts from legal teams, right? And how it was able to, like, use the LLM as a judge and improve the, the agent itself, improve efficacy from, like, 44% to 88%, just, um,, you know, through this, uh, through this outcome, um, based feature and, uh, the sub-agent that works as an LLM judge. Um, so, you know, it's, it's the type of thing where the domain expertise in the enterprise is, is like still the valuable part, but then you can, you can like Claude's making it or Anthropic's making it so much easier to just like bring it into an agent and have the agent work. And that's, that's the, um, I think like back and forth we'll probably see as the harness evolves.

Yeah, that's interesting. Yeah, here, uh, is an article just on that. Anthropic said teams are already using tools to build agents that verify their own work, learn across sessions, and handle complex workflows in parallel. Harvey, the legal tech, like build an AI lawyer company, uh, they use Dreaming to help legal agents remember file type workarounds and tool-specific patterns with completion rates rising about 6 times in tests. Another case in real life, Netflix's platform team used multi-agent orchestration to analyze logs from hundreds of builds and identify recurring issues across applications. That's a great internal software delivery use case, actually. Yeah. For those in enterprise, just go do that. Just go prompt your agent right now to go look at all your logs and then learn from it. That makes a lot of sense. Um, and what do you think of the whole—

yeah, go ahead, Jonathan. One thing, I mean, the log example and the Harvey example, um, you know, it puts like, what should we work on as developers? And this tells me the direction the tools are getting better in, just making really good rubrics and training, you know, really good rubrics is gonna like pay off like, you know, for probably generations of models to come. So if we made like, you know, we invest in Rubrics to analyze our logs across our live servers and autocorrect, you know, if we really invested in, um, in making those Rubrics strong so that we had a good LLM judge, we have a closed-loop self-improving system. And like that, it's clear where I can invest my time as a builder and as an engineer. Yeah, definitely.

And a very simple case that we always talk about on, uh, the pod and our team meetings is In the world of determinism, it's 2 2 4. How do you verify 2 2 is 4? That's math and physics, right? In the world of non-determinism, your agent has to answer questions like, what's for dinner, kind of thing, right? So a great example I always use for enterprise customers and prospects is just think about it like this. If rubrics and eval sounds so complicated, it just comes down to can I make a rubric against a very subjective user query to grade the output? And, um, it kind of comes down to what's for dinner. Okay, well, let's break that down. What would one thing we grade on is, does this meet the person's dietary preferences? One, if it doesn't, and I'm vegan, and, uh, if, if it recommends me meat, that's already bad. That'd be a 0 out of 10. Like, you know, very binary, like you failed. What if it recommends something that's out of budget? That's another heuristic that's quite subjective. What if it recommends something where I don't have the ingredients on hand and I specified I really, I have these ingredients and then it goes out of scope? So there's these subjective criteria that as a developer, that's really the domain expertise. So for Harvey, you know, building lawyer AI, lawyer as an AI service, you would think about those subjective criteria, but for things like what are the toolchain calls? Like what did this cite from this particular resource for the RAG function, right? Those are the types of things that you would start creating rubrics around and the rubrics are what you version and the rubric ends up being your harness engineering, right? At a very high level. So this is where the work compounds, is in capturing that domain expertise. And Anthropic is obviously seeing that and they want to make AI builders' lives easier in capturing that domain expertise, versioning it, and iterating on it over time. So the harness engineering becomes easier. Is that your thought too on the harness engineering kind of intersection? Jonathan? Yeah, yeah. That's how I'm thinking about it too. Cool. Sweet. Let's talk about the dreaming thing real quick before we wrap. Like, I just wanted to say, like, the dreaming thing is just a cool concept because from a product perspective and design perspective, you're usually trying to meet the user of your product or tool in their world. Like, their subjective experience. We're all humans. We all know what it means to dream. So, there's a bit of anthropomorphizing there when talking about agents dreaming. But there's a— I just wanted to comment on the elegance of that naming. It makes a lot of sense. As a human, I'll grok, oh, yeah, dreaming to remember stuff, REM sleep, that kind of thing. So I love— I actually just love the name of the feature, Dreaming. Let's talk about persistent memory and dreaming. So dreams let Claude reflect on past sessions to curate an agent's memory and surface new insights. Agents write to their memory stores as they work, but these writes are local and incremental. Over many sessions, a memory store accumulates duplicates, contradictions, and stale entries. Dreams let Claude clean that up. A dream reads an existing memory store alongside past session transcripts, then produces a new reorganized memory store. Duplicates merge, stale or contradicted entries replace with the latest value, and new insights surface. So a dream is simply an asynchronous job that takes a pre-existing memory store, The store Claude verifies, deduplicates, and reorganizes, and optionally up to 100 sessions, past transcripts Claude mines for patterns and insights to fold into the output. The dream then produces another output memory store separate from the input. The output store ID appears in the dream's outputs array once it starts running. And I, I love this, like in here they have like a little developer experience for creating dreams. Uh, we all, we all got dreams, like I think that's just cool, uh, marketing and, and developer experience work. Well done by Anthropic. Anything on your end from your CTO perspective, Jonathan, on Dreams?

I think it's very useful, and I think we've all had these experiences where, you know, like we've done something in one session that we want to represent another, and then to have like a general mechanism where it can kind of, you know, make, uh, inter-session memories and profile of you. You know, that's great. One thing I found, I actually heard the Claude Code team talking about this in a podcast in October. And so they had sort of like ideated and were starting to play with the idea of dreams in Claude Code. And I heard this on Every's AI and I podcast with Dan Shipper. They had a couple of the Claude Code team members. So for the team to incubate it, and then I guess it took about, what, 5 months before they're releasing it, launching it at a conference, was kind of like an interesting footnote in this feature for me to hear like, yeah, we're kind of playing around with some ideas to like, oh, this is a feature in our API and we're launching it at our developer conference. That's how long it takes a world-class team to output something like this.

Yeah, that, that is a good point. I also read something from Bored Ape Yacht Club that they're really trying to dogfood their own product and they see themselves as a platform. So they need to be using the same tools as the developers they release to. Obviously, they're— I think any, you know, DX team internally, they're going to have more like cooler stuff ahead of time, because they got to use it, dogfood it, make sure it works for the rest of the world, then they release it. That's That's always going to happen, some sort of alpha, beta, like internal alpha, beta, and then GA, general availability release. And yeah, I think we can all agree with what you said, which is everybody's dealt with this, where you have had this sick cloud code session around some context. And I don't know about you, but I'll just leave that terminal open for days. I'll just be using it. Because I feel kind of scared that if I lose this, oh, I got to then /resume and go find this. And that's kind of a bad experience. I don't know. It's never been easy for me to find all my sessions that— to find the right session with the right context when I'm ready to work on it. So, I just kind of, like, pop up in different terminal tabs. And then it steals my memory. So, this is cool. We could just then use Dreams to— Well, one, as agent builders working on memory for the agents we deploy, obviously, we could use Dreams to make the memory better and help the user across sessions. But then as a user of Cloud Code myself and coding agents myself, I think this is such a great piece of functionality that will help the developer experience. Yeah.

Yeah. Did you see if it's— is it enabled by default on Cloud Code now?

I don't know.

It's in research preview, so it might be a flag you can turn on. Or, uh, I mean, this is the API we're looking at to build your own agents, but, um, yeah, yeah, I don't know if it is used in Claude Code by default.

Hang on, Claude Code dream feature. Let's take a look, dude.

This would be a really good, uh, feature for Mystica. Like, you could even, like, say, you know, dream from your sessions. Here's some dreams from your sessions and just, like, send them, the user, an email or something like that.

Yeah. Yeah, that's really cool. So, when it dreams, it's just from a software perspective and product perspective and I guess engineering perspective, that is a point in time where— and for folks on the pod, Mystica is one of our customers. Where we're working on hyper-personalization for them from an AI perspective. And so we're thinking of all sorts of ways of how to engage their users more and help them retain better to maximize customer lifetime value, essentially from a business perspective. They're a consumer application. We've been able to help them grow their monthly revenue with our personalization approach and software with Clarity.ai. And so we're always trying to bring in best practices to service our customers and clients. And so one of the use cases Jonathan is talking about is they're very heavy on email campaign engagement. In fact, they could see on some email campaigns like click rates, open rates, like I should say open rates over 80%, which is wild to me because I run a newsletter and like I'm getting nowhere near 80%. So they have a very highly engaged user base where the emails are sort of a digital product for them, right? So what Jonathan is saying is, hey, if we could track the individual users with AI to personalize across their marketing funnel all the way through their product and what they do in the service area of their application with analytics, like through Hiros and through PostHog ingestion of data, event data, then we could see like, what does a user care about, and then use Dreams to sort of go across their sessions. And then boom, that's just another job that could inject context into the email chain. To then further engagement and retention.

And even just the chat sessions, like, you know, like what could we infer between chat sessions, be interesting insights.

Mm-hmm. Yeah. And so they have a chat feature where they're chatting with digital tarot readers, like that's the product, the core like product. And so what we're talking about here is how do we make that experience better and better? One of the key jobs to be done is remember me across sessions, like users really want that, customers usually want that. Because that's a very emotional social job to be done. So I could totally see how this would help with that problem, actually. So yeah, more to think on there, but immediate value to AI builders that are thinking about personalization, especially at the intersection of agents, whether your product is, is an agent and the customer's experience of it is an agent, or you're using agents to help with personalization on the backend., for AI-powered experiences. Super sick. Um, okay. I think that's enough for the pod today. We like have way more to go over, but, uh, we're running out of time here and me and Jonathan got to meet up and talk strategy for the business. So I guess we'll leave with this. Um, today we went over, you know, our 3 segments. I'd love to get feedback from the community. If you have other things that you want our takes on from an AI builder lens, building on the front lines, serving real customers in startups and enterprise, and from the lens of quantifying subjectivity and hyper-personalization with sort of our neuroscientific approach at the intersection of works by Michael Levin and Karl Frischner. If any of that stuff is interesting to you, hit us up. We'll have all of our contacts. I'm on social media. Hit me up. I'd love to hear from you. I'd love to make the podcast better for you. And if you have memes to drop in as well, hit us up at podcast@epistemicme.ai. I'll drop that in the pod notes, the show notes. And we'd love to get your memes featured because I think more community around just laughing at the absurdity of how things are going is needed. We got to stay glass half full and comedy and humor is definitely needed in the AI space. So, um, Jonathan, thanks so much for being our awesome co-founder and doing this pod with me. And this has been your weekly AI Clarity. We'll see you next time, folks. Take care.