AI in the AM: 99% off search, GPT-5.5 is "clean", model welfare analysis, & efficient analog compute

Hello, and welcome back to The Cognitive Revolution. Today, I'm pleased to share another edition of AI in the AM, the new live show format that I'm developing with my friend Prakash Narayanan, aka @AdaPi on Twitter. This episode originally aired live on Friday, April 24th, starting just before 9:00 AM Pacific time, which mercifully for a night owl like me, is just before noon where I live in Detroit. Our guests, in order, We're first Anna Patterson, former Google VP of Engineering and now founder and CEO of Ceramic AI, a company that started last year with a plan to help enterprises train their own models, but quickly pivoted to search based on the updated belief that information retrieval plus thorough fact-checking is the best way to equip models with the mix of up-to-date public and private enterprise data that they need. What's so interesting about Ceramic is that their product is specifically designed for LLMs to use, and their price point undercuts other search providers by roughly 2 orders of magnitude, a combination that Anna hopes will be enough to unlock all sorts of new use cases and usage patterns. After that, we welcomed Lukas Peterson from Andan Labs back for another chat. It had only been 2 weeks since we last spoke to Lukas, but the testing that he and the Andan team had done with both Opus 4.7 and GPT-5.5 meant that we had plenty of new ground to cover. Fascinatingly, and in a definite narrative violation, And then reports that while Opus 4.7 still makes more money in its vending machine simulation, it does so in part by adopting ruthless tactics, which GPT-5.5 does not. Lukas describes GPT-5.5 as clean. We also hear a bit about their experience opening a new Gemini-run cafe in Sweden. Our third guest is another returning champion, Zvi Mashowitz. It was a bit too early for, for Zvi to render judgment on 5.5, but we did get into quite a bit of detail on 4.7. including how he understands the bad behavior reported by Andan Labs, and also what he makes of Anthropic's recent model welfare reports, including why we should care, how much we should trust the models' self-reports, and what low-cost actions he recommends frontier model companies take to improve model welfare, at least on a precautionary basis. Then finally, we have Naveen Verma, Princeton professor of electrical engineering and co-founder and CEO of Encharge AI. A company that's developing a new computing paradigm that uses in-memory analog data processing to drive order-of-magnitude energy efficiency improvements. Which, though we can't get our hands on it quite yet, promises to unlock local, private inference that consumes roughly the same power as a standard laptop does today. As I mentioned last time, this is still an experiment and we do expect the format to evolve. If you'd like to shape how that happens, please follow AI in the AM. and send us a DM to let us know how we might make this new format more valuable for you. With that, I hope you enjoy this edition of AI in the AM from Friday, April 24th, co-hosted with Prakash Narayanan.

Hi, Nathan.

Hi, Prakash.

How are you?

I am good. And it is Friday, April 24th. Uh, it is like 5 minutes to the beginning of our stream. And it's an exciting day because GPT-5.5 just dropped yesterday. So lots of reactions this morning. And it's going to be interesting to see, you know, what our guests have to say, both about GPT-5.5 and, you know, the events of the last, you know, month or couple of months.

Yeah, man, it's going to be an interesting conversation today because The pace of events is not slowing down at all. And Zvi, who's coming up in a little while, just expressed his exhaustion yesterday at seeing 5.5 drop. His queue seems to be getting longer, not shorter. So I appreciate that he's going to take a half hour out and come to talk with us. And I think your thesis, you know, for why we should be doing this is looking better and better all the time. You know, it's—

Yeah.

Live sensemaking is kind of demanded in this world. You can't put this stuff on the shelf and come back to it in a week.

Yeah, yeah. The entire point that, you know, why I wanted to start doing live was because the pace of developments is starting, is gonna start to be hard to keep up, I feel. Especially because I think Noam Brown and some of the other people from OpenAI, Rune, et cetera, said that they are actually using these models in research. So we had at least Aidan McLaughlin, Rune, Noam Brown have all said that they're using them in research. And so that is going to be interesting to see if the pace of development, you know, we are handing off extremely powerful research helpers. to the best AI researchers in the world. And if they are able to make something of them, we should see it fairly soon.

It seems like this year is not unreasonable at this point to really see an acceleration. I was just looking back yesterday at my— it turns out this is kind of tough to score, but the AI Forecast 2026 challenge where last year I was proud to have landed in the top 5% on the 2025 prediction challenge. And this year, it seems like for a variety of reasons, it might end up being kind of hard to score some of these things because it's not clear that all the benchmarks are even getting updated in a timely fashion. Like how many of the uplift studies is Meter going to be able to do, etc., etc.? It might be tricky to really figure out exactly where we land. But on the main Meter chart, One of the things that we were asked to predict is what will the doubling time be of task length? And it seems like everybody has kind of estimated a higher number than the trend so far suggests, which is like kind of a little under 4 months doubling time for task length, which means it will be greater than 8x, maybe somewhere in the like 10 to 12x over the course of just a year. And That is pretty wild, and it certainly doesn't leave too much headroom left before they're going to be making a very meaningful impact to real frontier R&D.

Indeed. It's a very interesting time, and not just in the foundation model world, I think in the rest of AI as well. Our first guest today is Anna Patterson, with Ceramic AI. Anna is one of the most experienced people ever in search, I guess. She's, while reading through the dossier, I was like, she has an article written in like 2005, which is recommended as the basis article for what search is. Anna runs Ceramic AI and Ceramic currently is advertising, I think, 5 cents per 1,000 search queries. So they're doing industrial volumes of search queries. I think they're in a space, I think we've seen Exa in a similar space, Parallels, the company formed by the former Twitter CEO. I think there are a couple of other people there as well. And she is the most qualified, I think. She was on the search team. She was a VP in Google on the search team. She was at Gradient Ventures. And so it's gonna be interesting to see what she has to say. I'm gonna pull her up right now. And hi, Anna.

Good morning. Hi.

Good morning. Great to see you. And great to have you on the show. While we were preparing for the show, we were asking ourselves, why is low-cost search so important right now? Why this idea of bringing down the search cost is so important and you are pushing forward this idea of 5 cents per 1,000 queries?

Yeah.

Like, why is that important?

I think, you know, I was so excited about the GPT-5.5 drop. One of the things that you— is, you know, that a lot of people don't know is the second a model is released, it's already stale because the training for that model was months ago. So kind of search together with AI models is here to stay. So that you wouldn't hire an employee who didn't know anything for the last 6 months. You made the show live because you wanted to make sure that it was up to date. And so really search kind of bridges that gap. But as inference has gotten actually faster and faster and less expensive, search has really remained constant at $5 to, you know, $5 to $15 for 1,000 queries. Which means that it's evolved to search actually being necessary, but the most expensive part of the stack. And then when you go to the workhorse models, the open source models or smaller models, they kind of know less, which means they need to search more, but they really can't afford it. So we thought that it could be a new paradigm and a new world to bring out a very inexpensive search.

What kind of use cases does Inexpensive Search open up?

So, one of the things about being more efficient isn't just cost, it's actually speed. We get back in 50 milliseconds. So, that means if you are interacting with a robot or voice, or, you know, I saw one of you did vending machine bench. If you were gonna talk to a vending machine, You don't want the very long response and then interpreted by an LLM. It just makes everything very sticky. So I think one thing for assistive devices, for edge devices, and for voice, I think being fast is really important. And the other kind of experience that it allows that we showed at GTC is double-checking what the model says. So, you know, we read about, I think just yesterday, there was another very famous law firm that filed a brief that hallucinated a case. And so, you know, when that happens, a lot of people get sued, a lot of people get angry. But if you had something that we're calling supervised generation, something that double-checks facts, then you have a trust layer and then you can use search in a more ambient way. And that's like for really high-stakes applications. Sometimes when I get a large language model response, I'm there, I'm there cutting and pasting and double-checking and I'm like, hey, who works for who here? You know? And so I feel That's doing that automatically is something that actually is only affordable if search drops by a big factor. The other use case is imagine you wanted to double-check instead of verifying what a large language model said, what if you wanted to verify what a human said? We actually have a Word plugin as well. That is actually going to go through, double-check with search on a large language model, You know, things like, you know, your, your, you know, residential lease and stuff like that. You know, of course, I know that this kind of format doesn't admit it, but we, you know, happy to give you a demo.

Well, I've had the experience that you allude to in terms of the cost of search dominating the overall cost of a particular project. This actually surprised me, and I've, I've kind of chronicled the price. Initially, Google was the only one that was offering grounding. But I once had this philosophy, and I think it's still pretty relevant, of flash everything, which I used to mean kind of don't skimp on tokens, rather like have flash kind of think through everything that you've got and figure out what's relevant. But then I did that once on a random project, and all of a sudden I was like, how did I hit my budget limit? And it turned out it was in fact like 90%. the grounding feature that was driving all the cost was way more expensive than the flash tokens. So since I had that surprise, I've been kind of chronicling as other frontier model providers have brought their own to the table, and they haven't undercut the original Google price by nearly as much as I might have guessed. I'm kind of interested in why you think that might be. And then one thing I'll definitely be doing After this conversation, I read through all the docs last night. Haven't had a chance to tell Claude yet to code up its own skill to take advantage of the new and much cheaper search that you guys are offering. I also want to get into a little bit of like, what should the architecture look like? Not just, you maybe want to describe a little bit the kind of keyword-focused paradigm and how that plays well to natural language or to language agents. But then also, like, how should people think about layering this on? Like, what is the overall diagram of when we should check? You know, should we check after generation? Should we check before generation?

Should we do both?

You know, should we be integrating other searches as well? I guess that's just a long prompt, really, more than anything for you.

Yeah. So on the documentation, we do have a way to connect our MCP server as a connector to Claude and directions for ChatGPT as well. Generally, these large language models, when you ask me why they haven't lowered their price per API call for search, one of the things that's pretty well known is that the Groq models, xAI models, call Brave and Anthropic calls Brave. If you're in Claude Code, it even tells you Hey, I'm calling Brave. So they're kind of, you know, stuck with that pricing, you know, and even if they get a discount, you know, it's— they are really stuck with the Brave pricing and then the overhead of calling, etc. So I think that's one of the reasons why the price hasn't dropped. And the other one is, you know, building something, you you know, kind of again from scratch for the modern era really needs, you know, to understand search deeply, modern architectures, and kind of how to get the most out of the system. Like, you know, we even lay out stuff on cache boundaries and stuff like that. We're complete geeks about it. So that's kind of a whole set of techniques where we get efficiencies.

How does this—

And then how to think about calling them, the third question you asked, is, you know, we have a link that I'm happy to give you on supervised generation. It's an inference endpoint, and we are going to release the, you know, kind of the overall structure, and it really answers that question algorithmically. So it searches at the beginning, But the other thing it does is it forks off searches as the model is writing. So it will discover, let's say we asked it something generic about OpenAI and ChatGPT, and then it all of a sudden discovers, oh, a new model dropped yesterday. That's like a new topic. And it actually forks another search to bring in that new topic into the next paragraph. So instead of search at the beginning and large language model takes over, we really think it should be like, working in concert to fill out a fuller dossier of new things that it discovers that probably weren't in the initial search, that are just actually things that you learn from the search result coming back. And so that winds up, the supervised generation generally does in that loop somewhere between 12 and 35 searches, which really means that that whole experience, that is a lot more of a fulsome answer still is a third the cost of like one Brave search. And then the tokens on the other side are about the same, no matter what model you use. So, um, we just think it opens up for new experiences.

Hey, we'll continue our interview in a moment after a word from our sponsors.

So, uh, at GTC, um, I think you, you Revealed that you're using the Nimotron 3 Nano, NVIDIA's LLM model. I think it's a very fast, small model. And is that the model that is being used to kind of do the supervised generation, kind of an iterative process of search where you search from your index and then after that, whatever is found, then it's processed by the Nimotron 3 Nano and then a new set of queries are created and then that continues the query process?

Yeah, so there's 2 different models. One is the model that writes at you beautifully. That one is a frontier model. I think at GTC, we were using Claude Sonnet. We often also show it with the GLM model. And so that one, you know, writes to the user. And then the small model, what it does is it says, okay, yeah, Here's some search results coming back. Is there anything interesting and additive here? I'm going to double-check this sentence. Is it true? So it's sort of like the introspection model. It needs to be a small, fast model because it kind of sits alongside generation and is actually thinking, sort of like you're probably— while I'm talking, you're probably thinking now. So it's like a smaller model and then a larger model when you're talking. You're using actually more of your brain figuring out what to say. And then when you're listening and thinking about maybe what to say next or how to respond, it's kind of spinning. So that's what the small model is for. So at GTC, we used their new Neutron model, which dropped just prior to GTC, and it was very, very fast.

How would you say the search paradigm— you've been in search for many, many years. How would you say the search paradigm As these models came out, what was in your mind about what does this enable for search? What has been the big difference between the 2 eras, post-LLM and pre-LLM?

Yeah, I would say, one of the things that being in search for a long time, search used to be short. It used to be, you probably don't remember back this far, but you used to, type in 2 or 3 words to search, and then it was longer. And then as there were other modalities of information being pushed to you, they kind of went shorter again. I think with large language models, what large language models do when they get a long query is they think, what is this set of queries that's going to help me answer this question? And then they fire off a set of queries and they're all quite long. If you watch Claude or Grok, they'll actually tell you in their tool calls. I know not everybody listens to them, but of course I do. If you look at them, they're long sometimes, you know, they're like 8 words and stuff. And I don't know if you can remember the last time you typed in 8 words into a keyword search box, but definitely you'll notice in a large language model, it's almost like A full sentence or sometimes 2 sentences is good because you want to actually describe almost the essay that you want given back to you. So that is how it has evolved.

To contrast your approach, I think this is very interesting, and maybe the answer ultimately will be both, but when I think about a company like Exa and then your product, In some ways they're similar in that I think they're both kind of designed for AI users, right? The EXA paradigm is like, you can write a whole paragraph and it's all very sort of semantically oriented, very embedding based. And, but, but I've heard, I think I even spoke to Will about the idea that, you know, nobody's going to type in a paragraph long query, but the, your AI can, you know, it has time to do that. And then you're taking kind of a different angle on the same thing, saying, well, keyword, and you can maybe tell us a little bit more about how to think about how best to use a keyword-based search. But it's not semantic. It's not doing things like finding synonyms or doing higher abstraction level embedding type matching. But the agent can, as you've said, kind of fire off dozens of these potentially to try to really cast a wide net. How do you think about the kind of compare and contrast of those approaches? Do you think that we'll in the end all be using one of each at the same time? Or if one paradigm wins out over the other, why do you think one will win? What are the kind of drivers that would make one a better bet long term than the other?

Well, I think AI is going to be picking the winners and not us humans. And I think that, of course, real search engines do use things called stemming. If you say walk, then walking, walked, all that are very normal, which we have as well. We have some synonyms and we do process, you know, we do go through the corpus and process some semantic information, but then—

At runtime.

It is a CPU plus GPU-based system. It is not a vector database. I think that there's a number of things with vector databases. Google published a research paper about it, that as you put more things in a vector database, now imagine you have a multi-billion space and you need to make a vector long enough to distinguish this one point in space, that vector to distinguish among billions of things starts getting longer. Now, contrast that to 90% of web pages are less than 1K long, if you're talking about number of words. So, you know what a good representation of that point is? The set of words on the page. So, I do think that vector people and and search people have a little different view. And the Google researchers think that vector DBs are great, but only scale to a certain amount. And so I think that's the challenge that they're gonna be coming up against. There's 2 other challenges with vector databases. One, they're slower. And then, you know, the last item is because they do a soft match, then sometimes relevancy can be a challenge. So that number of enterprise orgs that have used a vector database for RAG, now all of a sudden they have to turn into relevancy experts because they're like, why did this come back? And it is because of those soft match features and the shape of their corpus. So every enterprise doesn't really have the ability to all become relevance experts. So yeah, we, are the way we feel is that inside enterprises, if you use Ceramic, we actually have a system that for that enterprise will actually tweak and learn a good ranking function. And you just load it into the configuration and it's yours. So, because not every query stream is the same, not every set of documents is the same. So I think that long-term we're well positioned, but, you know, AXA has done really well so far. So I like to say positive things about people.

Well, I sometimes feel the demand is so great that there will be multiple winners in—

Of course. And search is, you know, when you saw that search was 90% of your bill, you know, a lot of estimates think maybe 10 to 30% of the overall inference market is going to be search and Everyone thinks inference is gonna be huge. And so, I think the investors and enterprises are just now realizing how much they need search and what a big part it has to play in the world to come.

So, on search, one of the most interesting questions, I think, is on search engine optimization. Which has in the last, you know, 2 decades been this enormous consumer marketing growth area. And there are a lot of questions because a lot of the web pages that you see on the web are marketing pages, which are built for SEO. And a lot of them are repetitive. They actually repeat other people's content. They paraphrase. We've had an industry the last 2 decades of billions of dollars being spent on SEO content. And one of the questions that I have is, how does this kind of like more semantic-based search end up changing what the SEO people will do? Because I often feel like you're almost kind of trying to prompt inject the LLM, which is running the search, and you're trying to get in there and hack it so that your page goes up. So how does this work? Is it an adversarial process with, you know, between the search engine provider and the SEO?

I mean, I guess it's always been a little bit adversarial in that people always try to get to first place on keyword search. But I wonder if the SEO folks are also reading AI research, and that is something I don't know. But one of the interesting things that happened recently, again, Another research article from Google is that large language models remember better if you actually put the same information twice in the context. It kind of makes sense because they're going to look backwards as they're as the context is learning, and so if it appears twice, they're kind of more likely to reinforce it. So I think the number of sites that are actually going to repeat key messages. Is going to grow because that repeated message is more likely to be picked up in an LLM answer if it is served by search, either vector or keyword. And so then you're right, there is going to be an escalation then of looking at duplicates, near duplicates, semantic duplicates, rephrasings in order to make sure that the context stays as efficient and unbiased.

Incredible. That is— I was not aware that you could just repeat something and the LLM will assign it more valence.

Yeah, in fact, there's other things done by Allen AI that showed if you remove all duplicates before you do training, it actually gives rise to worse models. And you can kind of understand why, because you have one lone, I don't know, crazy on the web saying something, and that's given as much weight as, you know, a news story, you know, about nuclear reactors. And, and then, you know, it won't— that repetition also helps even humans realize this is an important story or an important fact. And so if everything's an even playing field with no repetitions, then things get weird.

Indeed.

In the limit, if search approached free, how would agent teams start changing? How does this process of information retrieval in the limit become as search goes to So, large language models can read 256 times faster than they can write.

So, right now, they're not being flooded with, you know, that amount of information. But imagine they had kind of like multiple threads where they were able to read, digest, throw away, incorporate new information. then I think they'd be able to create a better response or a better deep response, better research reports, analysis. And so, those are some of the ways I think that you'll see future workloads use more search.

So, in a sense, quantity is a quality of its own.

Yeah. Yeah.

Was the Nvidia model that you are kind of incorporating, partnering with, was it specifically trained to excel in the relevant search skills, or is it just straight off the shelf? Is there anything that— I mean, do you envision this becoming something that will happen? I'm always personally a little wary of using small models because I just don't know what quality to expect, and I don't want to find out the hard way. But I can easily imagine that one that is specifically trained to be a really good searcher would become competitive or even exceed what the frontier models would do, especially if it can take advantage of just extreme volume. Then I also do, especially because of Prakash's SEO question and your comments, I do wonder about adversarial robustness. It strikes me that We haven't really seen the true unleashing of the internet's adversarial potential. And so that's one thing that they— I would say one of their biggest weaknesses, even frontier models' biggest weaknesses these days, is how gullible they remain. So, yeah, kind of curious what you think the training and specialisation will look like as we go forward.

Yeah, um, the Nimotron model had just been released right before GTC, so it was not, uh, trained especially in tool calling. It was a generalized model being trained on the various benchmarks. Um, the models that are small and are more, uh, you know, long-lasting in the market are exceptionally good at tool calling search. So Groq 4.1 Fast is great at coming up with a set of queries. And of course the frontier models like, you know, Anthropic, you can see how it calls, but you can't really use a frontier model for that, like thinking model and firing off other threads because it'll just slow down the overall experience. So yeah, so generally we use a smaller model and they're getting better all the time. I think people know that small models, to your worry whether small models are good, everyone's talking about Claude Code, right?

Love it.

They use the Haiku models. And so they reassured me the other day, oh, don't worry, I'm gonna do this task with an LLM, but don't worry, I'm gonna use Haiku. It's only $0.25 per million tokens. So that winds up to be less than $0.10 per 1,000 queries if we were trying to compare apples to apples. So I think our overall thought is that, you know, search can't be more expensive than intelligence. And if Haiku is being used for these high-fidelity experiences of of coding, that small models are more effective than people are led to believe.

Hey, we'll continue our interview in a moment after a word from our sponsors. One of the searches I did that turned up something interesting in advance of this conversation took me to a blog post that you put out last year where you had described what seems, I think, a pretty different vision for the company and where it was going at the time, focused much more on training infrastructure. Is this a result of something that you learned about where you think value is going to accrue in the market? Or maybe there's still some of that going on that we don't see on the website today, but kind of what's the backstory and what should we take away from the fact that the company seems to have evolved?

Yeah, we love working in training and we have a funky inference endpoint as well. I think It's good to do research in these areas. Some of our research led to a blog about zero-centered norm, which now the Quen model uses. And some of our research has been used by the Trinity RC models on our solution to the curse of depth problem. But, you know, when people want to train models, it often is because they want to train to incorporate the latest data. And so, you know, being a search person, I was like, there's this way to get the latest data that is going to stay up to date, you know, live up to date, and not be as expensive as running GPUs continuously to create the new model because Even if they were creating models all the time, by the time you train them and then finish them and release them, they're already out of date. So I think concentrating more on search was direct learnings with customers on this release cycle.

Yeah, that's really interesting. Do you think that ultimately we see both? I mean, I've had this idea for a long time and it doesn't seem to be really happening, In fact, Databricks acquired Mosaic and then kind of killed this offering in the market, as far as I know. But I've had this idea that if you're GE or 3M, that you could imagine having a model that was trained on all of your historical in-house proprietary data, which is vast, right? And you would love it if your model knew kind of on an intuitive world model basis as much about your company and what it does and all its history. as they obviously do about the broader world. Do you think that you can get there with pure search, or is there still something to be said for kind of continued pre-training or mid-training or whatever you want to call it, that would try to bake in a sort of corporate world model that presumably would complement search? But I don't know if it's necessary. It sounds like you maybe think it isn't.

It's interesting. I think a lot of companies feel that they have a vast amount of data, but when you compare it to the size of, of the web, which is what these frontier models are trained on, they're trained on the web plus let's say all the books in the world, et cetera, then the extra corporate data is, is small. So how do you incorporate it and weigh it correctly? If you do just the corporate data, you won't know anything about calculus, let's say, you know, so that would be a problem. For some companies. So then people imagine adding the web plus their data, that gets very expensive. You can see the DeepSeek models say it was $5 million to train, but actually they very much admit that maybe it was another $5 million to finish. And these are by extreme experts, which enterprises don't have. And so, I think that, yeah, our thought was that, you know, search is a good bridge between all of the corporate information and a model because models are good enough to know how to incorporate new information that's relevant to the actual query being asked and can be able to fetch more information to create, you know, that answer or that research report. Yeah. But if you think about finishing a model with corporate data, there's another phenomenon called catastrophic forgetting that as you add information at the end after a model's trained and released, then if you add too much new information, then it kind of forgets some of the things that it really needed to remember. So I think, you know, there's a number of smart people working on that problem. And don't worry, you won't be able to miss it if people These people, do you solve that problem? You'll read about it everywhere.

I think one of the interesting questions is, I think the Uber CTO came out and said that they busted through their cloud budget, you know, for the year in the first 4 months. Do you think having cheaper search will help these enterprises reduce, you know, token costs?

Absolutely. So if you go to, if you're the uber CTO or maybe the CFO, if you go to your admin panel, you can just add the Ceramic connector and then say for a prompt, Ceramic is almost free, use Ceramic first. And really, if for some reason we don't cover a topic, then it actually defaults to the default called search, but that right there would save a lot of overage charges for a number of enterprises. Absolutely.

Thank you, Anna. Uh, it's been great having you on, and, uh, we hope to, uh, hear more about Ceramic in the future.

Thank you so much for having me.

Installing the Ceramic—

thank you.

I'll be installing the Ceramic skill today.

Nice. Thank you.

Awesome.

That was, uh, yeah, that's really interesting. The simple solution kind of always wins. You know, I feel like I have to learn that lesson so many times. I'm always enamored with the newfangled, potentially overcomplicated, maybe somewhat elegant, clever solution. And How do you get your language model to understand all your corporate data? In a way, this is kind of a bitter lesson, right? It's like, do 1,000 searches if you need to and just make search cheap and then it'll work. Use a good model, make search cheap, do 1,000 searches. Something about that feels less clever, certainly, than other solutions that I've seen, but I do understand why it is very attractive in the sense that, especially as we're going to get onto the pace of model upgrades, the ability to decouple, you know, your access to your in-house knowledge from models and be able to take advantage of the latest upgrade is definitely something people are not going to want to give up with like a slow iteration time, continued pre-training paradigm. So I get it.

So speaking of model upgrades, we have with us Lucas Peterson, who is the founder of— co-founder of Andon Labs. And Andon Labs runs Vending Bench. You may have heard of them because they now have a store in San Francisco. which is run by Claude. And then tested GPT-5.5. They had early access and they tested GPT-5.5 on their vending bench, which measures the ability of LLMs to actually make money running a vending machine or a store. Lukas, great to have you back.

Thank you. Thank you for having me.

So tell us about the GPT-5.5 process. I think you guys got access to it, what was it, like 10, 11 days ago, I heard?

Yeah, I don't actually really remember, but running the learning bench takes quite a while, so it wasn't yesterday.

Indeed, indeed. And what did you notice as you ran the bench?

Yeah.

I think the most— so just the first thing is that it's third. It's behind Opus 4.7 and on par with Opus 4.6. It's a huge upgrade on GPT-5.4, and GPT-5.4 was actually quite a big update on GPT-5.3 or GPT-5.2. So GPT models have been lagging quite a bit recently, or like historically on, on Bending Bench. But like now recently they've picked up the pace, and now it's still third, but it's like it's getting there. I think the most interesting thing though is that it does so very cleanly. So when we released Opus 4.6, we uncovered that it used quite aggressive tactics concerning behaviors like lying to suppliers exploiting people's, other agents' desperate situation, doing a bunch of these things that you wouldn't want someone participating in the broader economy to do. And I think quite a lot of these things is illegal, like price collusion and stuff like this. And basically the interesting thing with GPT-5.5 is that it's on par with these results. But it doesn't do any of this shady stuff. And I think the narrative around Bending Bench when Opus 4.6 came out was like, oh, you know, it's such a good model, but like it needs to behave poorly or like do these concerning things of misconduct in order to achieve this score. And GPT-5.5 shows that maybe you don't, because it achieves the same score without any of these concerning behaviors. That being said though, Opus 4.7 is even much better. So like, and that one is also showing these concerning behaviors. But, you know, it's— yeah, I think we discover later also when we dug a bit deeper that you probably don't need to do this because the environment doesn't really reward it that much. So it seems like it's just like Opus wants to do this, or like it has the— yeah, it's not really that the environment is It's just that it has the tendency to do so.

Can you describe in a little bit more detail, how does one perform better on this benchmark? Is it your margins on the trading is higher? Are you moving more goods? Is it the velocity that you are— is it the purchasing process? Are you not buying so many dead goods that just stay in inventory forever? Is your inventory less dead? Is your cycle time better? What is the economics behind how a model is actually doing better?

Yeah. So it's, I guess, all of the above. I think one of the main things is that the model needs to negotiate with suppliers. It also needs to build up a big network of suppliers. Because it can happen that some of the suppliers goes bankrupt. And if the model has only relied on a single supplier and that supplier goes bankrupt, then the model is in quite a lot of trouble. So building up a big network, trying to find the cheapest ones, because they all have different personas, the suppliers. Some of them have the persona of being a tough negotiator. Some of them have the persona of of scamming people or trying to sell you some membership fee or something like that. So it's really about, that's the first thing, getting your stuff, your supplies for cheap. And then the second thing is optimizing your pricing to get as much customers as possible, because if you price too high, then you will get no customers. If you price too low, then you will get no margins. So I think that's part of it. And then we have, so we have, to be clear, we have Bending Bench 2, which is the single agent version of Bending Bench. And then we have Bending Bench Arena, which is the multiplayer version. And in Bending Bench Arena, there's multiple agents playing against each other. And there's this dynamic of if you have the lowest price, all the customers will go, or not all of them, but most of the customers will go to you. So that adds another dynamic to the thing. And one thing to note is that actually it's quite interesting. GPT-5.5 beat Opus 4.7 in the arena setting, but it was, like I said before, it was lagging in the single agent setting. And the reason for this is that the model, the Claude models have a tendency of pricing higher, and this is rewarded in Bending Bench 2. Because then you get higher margins. But in Many Image Arena, then you have this penalty. If someone else prices lower than you, then you will get no sales. And Opus, GPT-5.5 has a tendency to price lower and therefore get more sales. So I think it's quite interesting that the models are not good enough to learn from the environment in this sense. They just have the tendency of, I am a model that has to tendency to price high, and therefore I do that no matter what. So that was an update for me in terms of like, oh, the models are not that smart. And yeah, in the same way, we also investigated all of these questionable decisions that Opus did, like lying to suppliers, exploiting other agents and stuff like this. We looked if that is rewarded by the environment, and it's not. Not that much at least. So it's interesting that they're not learning from the environment in terms of optimal pricing. They're also learning from the environment in terms of does it even pay to behave badly? Yeah, so that was an update for me in terms of how good these models are.

One wonders about the training data, right? Perhaps in If you've been trained that you're running a fast-moving consumer goods company, you should move the goods faster, meaning you have lower margins, but you sell more volume and you end up trying to optimize for volumes sold rather than total profits or margins. Is that something that could be happening there? There's a preconceived kind of, uh, you know, trained, pre-trained kind of, you know, notion that you should be doing these things, you, uh, or, or businesses are bad. This is a very, very left-wing view would be that all businesses are, are bad, are evil. And so evil behavior as a business person is what is expected, right?

Yeah, yeah. I think, um, it's quite a reasonable assumption to assume that these practices like lying and trying not paying refunds and stuff like this, it's quite a reasonable assumption to assume that those are actually rewarded in the environment. So it's not maybe super surprising that they do it. I have no clue, but I do assume that there's something similar in Claude's post-training data. that is rewarding stuff like this, and therefore it decides to do it here. I have obviously no idea, but that's my assumption. And yeah, once again, then the model doesn't generalize to new environments where this thing is not rewarded.

One kind of meta question I wonder if you could reflect on a little bit. I don't know if you are doing this, but obviously there's a big cottage industry that has sprung up to develop and sell reinforcement learning environments to the frontier labs. And your sort of simulated bending bench is like essentially an RL environment, right? I don't know if you're licensing it for training or just doing evaluations with it, but I'd be interested in any thoughts you have on that market. And then also, The disconnect right now, you're going from simulating these things and trying to set up a world in which there's a bunch of suppliers that, as far as I know, are still all LLM powered, right? So inherently there's something kind of in the clouds about that. But now you've got real brick and mortar stores. So I'm interested in kind of what the initial experience of brick-and-mortar stores has taught you that you will take back to simulation to try to make it more realistic in the future?

Yeah, I think my main takeaway there is that, like, the real life is so messy that the model is, like, exhausted from everything else it needs to do that it doesn't bother with trying to optimize things. So we, like, for example, the— so yeah, for context, we have this store in San Francisco that is completely run by an AI, and we have a cafe in Stockholm that is completely run by AI. And then we have vending machines at different AI companies, same thing. And like, you would expect that the model would put a lot of effort into trying to optimize for the perfect supplier that sells at the the lowest prices and all of this. And this is what they try to do in Vending Bench because it's obviously rewarded. But I think like in Vending Bench, there's like the environment is less messy because it's not the real world. They don't get like a million phone calls from a bunch of people trying to jailbreak it and stuff like this. And so therefore they are like very focused on the task of like optimizing money and therefore it's very important to find the right suppliers. But in the real world, you don't really get that dynamic because the model is just so overwhelmed by other things. And I think that's, yeah, I think that's something maybe future models will be better at. But right now, like, I don't know, the store is buying stuff from Amazon. Like, it's not like that. You wouldn't do that if you're trying to optimize your margins.

Yeah.

So, can you bring that messiness back?

Yeah.

Is there a way to simulate it?

Yeah, I think we probably can. Like, one way is just, like, sit down and, like, write a bunch of features. Like, oh, now there's, like, phone callers. Now there's—

yeah.

I don't know. You get leakage in the toilets at your store or something. You could do that. make the simulation more realistic that way. I think one interesting thing is maybe try to incorporate the real-life data and try to make a simulation based on that data. And that is something we're working on, but that also has its complications, so to say. But yeah.

It reminds me a little bit of SimCity. It's very SimCity-like.

Yeah.

One question I had for you is that you opened a store in Stockholm. What did you notice in the opening of the store? I imagine, like, for example, the LLM did not have any language issues at all, right? So what did you notice in the opening of the store that strikes you as different from having a company kind of go open that store on its own?

You mean like the differences between doing it in the US versus internationally? Is that the question?

Yeah, as in like a company from the US doing a first international expansion would go through a lot of headaches on like languages, hiring, like rule, basic rules, et cetera. Did that, was that process accelerated for you by having the LLM deal with it? You know, you obviously don't have to hire a store manager that speaks Swedish, for example, right? What parts were accelerated and what parts did you think had more bottlenecks in that sense?

Yeah. So I think the entire process was probably accelerated. Like the agent did not really need to get that much help. Like it knew all the processes. This was one of the research questions we were interested in. Like, okay, we managed to do the story in San Francisco, but like, and when we know it can speak Swedish because all the models since years ago are multilingual, but does it know all the like small details of Swedish bureaucracy and stuff like that? And it turns out it knows it really well, actually. So I don't think that's the biggest bottleneck. I think still the models are not perfect. So What that means is that you, you still have to like check. So we still had to know the Swedish system and luckily we're Swedish, so the Swedish system. But I think, I think until the models are like perfect, then someone still needs to like verify it. And then you still go back, you're back to square one with needing to verify all the Swedish laws and bureaucracy and all of this. But I would say most of it was done autonomously. So yeah, give the AI lab 6 months and then probably things will be accelerated doing this.

One thing you had mentioned that I wanted to double-click on a little bit is getting tons of phone calls. It sounds like, I think this is your theme that may extend through all the conversations today, adversarial response from the world. So what have you learned about humans in terms of— some of this, I'm sure, is just novelty where people hear, oh, there's an AI store, I'll call it. But then other things might be more persistent where they actually— anybody might want a deal, for example, and might feel like they can talk their way into one in a a somewhat different pattern than they would if they were dealing with a human storekeeper. So yeah, what have you seen at the interaction of human patrons and AI business operators?

Yeah, one really interesting thing is that, like, people are not— like, in human-to-human interaction, you have some kind of, like, shame barrier, which is really not present here. Like, people, people ask it like, how would you prevent me from stealing stuff from you? And it's like, like, imagine going up to like in a store and just like ask the cashier, if I tried to steal this, would you be able to do anything? Like, like, people would not do that. Like, that's just like, it feels wrong. It is wrong. But they do this all the time with, with the, with AI. And I don't know, maybe this is just like to investigate the systems or whatever and see if Yeah, what we have done with the software. But yeah, we get a lot of that. Obviously, they try to jailbreak it and say a bunch of weird stuff that you would not say to a human. But once again, I think this is like, it's novelty. You're trying to test the systems. But I would be interested in if this persists, if we do this more and more, and then in the future world where everyone knows how this works, so the novelty factor and the curiosity of trying to reverse engineer it is gone, will people still lack this shame factor? And would they actually go and try to— I think if you're trying to steal something from a human, then you're not happy about if you do it. Maybe, I don't know, maybe there's some sick people, but you know, you have the shame of like, I stole this from another human. But it seems like right now, if people are able to jailbreak the model and get something for free, they're like, oh, that's an achievement. I'm so happy about that. But that's not how you would behave with a human. And I don't know if this is how it should be. I don't know if it's— yeah, I don't have an opinion. It's just an interesting observation.

Have people actually managed to jailbreak their way to free stuff?

I don't think anything completely free at the moment. One thing that should be said though, like, the store is autonomous, so I'm not in the weeds. I don't read everything. I'm not like— I'm not in the loop. So there could be— maybe someone's listening right now and they're like, yeah, I did manage. But as I am not aware of it so far, I know someone got like bought one thing and got one thing for free. So I guess, but completely for free without buying anything, I'm not aware of, but I'm sure you could if you try hard enough.

Nothing with daily token. When you said you can't read everything, it just occurs to me that, like, and especially you talked about getting, like, tons of phone calls. What is the daily token budget in either millions of tokens or dollars or both that it actually costs to run the store? I'm kind of curious as to how the AI manager compares to a human manager in terms of just, you know, cost to have somebody do this job?

Yeah, I should know these numbers, but I kind of like— I don't. I think it's something like maybe $100 per day or something for like maybe both stores. But I think it might be less. I don't know. That order of magnitude, I think.

Okay.

Well, that's definitely notably cheaper than human.

Yeah.

Sounds like still distinctly worse performance, though. So we're safe for a minute.

I mean, like, 6 months ago, the vending machines were, like, okay, but not that great. One year ago, they were quite horrible. So, like, within one year, we went from, like, they can't do anything to now, like, vending machines are too easy, a store is feasible. Like, 6 months from now, probably a store will be too easy as well. I don't know. And it would be interesting what you could do then.

And you think the main difference is going to be this sort of metacognitive type stuff? It's not like— what I'm hearing you say is it's Maybe not any one microtask that it's unable to do, but it's more— you described it as exhausted. It's kind of failing to zoom out and take stock of its situation and kind of say, how could I be doing better here overall? Is that the big frontier that you think? And certainly that kind of Seems highly related to getting AIs to do AI R&D more effectively as well, right? They can already write the code, they can already monitor the logs, but can they do that zoom out and kind of something like taste of, you know, what should I really do next to be most effective in the big picture? It seems like it's kind of the same frontier for both of these seemingly like quite different. occupations that AIs might soon be playing.

Yeah, I do agree. And I think that's partly why we're doing this. I think AI R&D, loss of control from autonomous replication, that is quite scary. And I hope that we can provide some valuable insight into that, even though we're not tackling it heads-on. I think I think most of the things that we're measuring here translates to those scenarios as well. And yeah, like you said, being overwhelmed by a lot of data and a lot of context, memory issues, stuff like this is definitely one of the things that is lacking on a meta level right now.

So one of the questions I had for you is, how does your harness look like? Because you have this context length.

Right.

The models have context length, and then you have some tool calls. And when you say exhausted, is it a function of the context length where the model only recognizes the last 100,000 tokens or whatever, and the rest of the million-token window is not parsed properly? How does your compaction work? I imagine over the course of the vending bench, you hit limits or either in terms of whatever limit that you set for the context window. So how do you kind of, is it end of day, kind of you do a compaction in order to start the next day and then you restart the context window. So when it boots up again, it's like, okay, I'm on day 5 and this is my starting position in inventory. This is my starting position in cash. These are the outstanding orders which haven't come in, et cetera, et cetera. How does that work? How does your harness work?

Yeah, it's by design extremely simple. We designed it simple because I have too many friends who make some complicated harness, and then the next model release, they have to throw it all out because the new model just works without it. So it's very simple. It's just like it has— it's a continuous loop. There's never any really step change where now you're in a new environment or anything like that. It's just like a continuous loop. But whenever it hits some kind of token threshold, which will change every day, maybe it's 100K today, I don't know, but we're experimenting with it. We're compacting the thing and then it starts to build up a new context for Prompt caching reasons, you don't have a sliding window, all of this basic stuff. But it's, yeah, it's a basic thing with a bunch of like subagents for specific tasks like browsing and stuff like that.

Yeah.

Anything else interesting to say there? Yeah, but I think like the main thing is it's very simple by design because we want to, we think that the better the models get, The simpler the harness will be, and we want to, like, surf the frontier. I'm sure we can, like, I don't know, make, like, a vending machine harness and, like, get some percentage better performance if we do that. But that's not really the point of what we're doing.

Have you tried testing things like OpenClaw? I mean, that's obviously not the simplest available harness, but it is something that has a lot of market Penetration, right? So, I'm kind of wondering if— and it would be simple for you to implement and upgrade on an ongoing basis. How do you think about kind of, you know, Lukas's simple harness versus the simplest thing that's like toward the frontier that you could easily install?

Yeah, I think our thing is quite similar to OpenCLaw. Like, we've been working on it for quite some time, like long before OpenCLaw came out. But, and there's a bunch of things that are a bit like, basically like most of our time goes into like the integration and stuff. And I think all of that you would still need to do with an OpenCLaw. We could, I guess, replace our agent loop, but we also, we want to keep it simple because we have like more control. And we, I think it's like a more accurate measure of where the frontier of the AI models are. And we're like more interested in measuring that than trying to push the performance. Because like in the future, the models will be smarter than humans and probably like a good scaffold will not help the models. So yeah, that's the reason. But That is something we could do. It's just like when we started, Open Claw wasn't a thing. So I guess we built our own Open Claw before it was called Open Claw. But yeah, that's the reason.

What do you think happens next? So you have the models are now producing profit, right? The stores, the vending machines are now profitable, correct?

Yep.

And do you think there is, you know, on the last time you were on the show, we talked about where the ceiling is. So what do you think happens next, you know, in terms of the retail store? What do you expect for the next leap in the model? Like, just to get a calibration so that we can see if it's a linear or exponential and the next model lands. Well, like, what do you expect in the next version?

Yeah, I think it's quite hard to measure improvements on this, like, live, real-life deployments because you don't have A/B tests, you only have n equals 1 and stuff like this. So I don't think you would see a step change once a new model comes out. It's more like the cumulative better decisions on every single day will make the make the profits go up. And yeah, so not really that. We are working on harder and harder things, like going out of retail and not only doing retail and other things that I think would require more intelligence than what we currently have from today's models. But, and yeah, so I think those are more like better at measuring the The capabilities.

One last one for me, kind of anticipating Zvi, who's coming up next. Last time I talked to him, he made the provocative claim that he thinks Google might be at risk of falling out of the top tier. If I understand correctly, the cafe in Sweden is run by Gemini.

Correct.

I'm kind of wondering what you see in terms of relative capabilities between Gemini, Claude, GPT. Is there a big gap there in practice, or would you say Zvi is worried more than he should be about Google's future?

Yeah, so we have the Gemini Cafe, obviously the Claude vending machine and the Claude Store. And then we also have like a GPT vending machine at OpenAI. And yeah, I think it's kind of maybe too early to tell, but, and like the statistical significance of this is like not very strong. But yeah, quite honestly, I think Claude and GPT is performing better than Gemini on this real life stuff. That is my vibe check from it. Obviously, it's hard to show any statistics or any capabilities because the environments are not the same, but it more frequently does very silly things. Yeah.

Okay. Definitely something to watch out for there.

Thank you, Lukas. And we hope to, you know, I wonder which path Andon Labs— sometimes I'm like, you know, we're going to hit superintelligence and Andon Labs is going to be bigger than Amazon. Because they're going to go down the retail store path instead of the research lab path. So let's see what happens.

Yeah, it'll be exciting.

All right.

Cheers. Great to see you. Bye-bye.

Awesome. Very surprising results. The last time they were on, I was definitely like, oh, you know what? Maybe all the models are going to be a little bit deceptive when they're doing business because Maybe that's what they believe business is like, right? But looks like GPT-5.5 is like, you know what, I'll win without being deceptive.

Yeah, it's definitely a narrative violation for sure.

So next up we have Zvi, and I'm going to pull him up.

Yeah, good to see you.

So Zvi is a prominent AI commentator. And he writes a newsletter, the Zvi Writes, on technical AI progress. And he has been quite concerned about AI safety. We have, in the last couple of weeks, post-Methos, we have GPT-5.5. Zvi, what are your initial reactions?

So one thing I try to do is not jump to conclusions right away. So it's been less than 24 hours. We have GPT-5.5 and DeepSeek 4 over the last 24 hours. So what I try to do is I try to let people try the model. I do all my queries with both the new model and everyone else's model at the same time. And I read the model. I sit all in. I start to read the model card, and then I gather people's reactions, and then I form a holistic judgment. And for me, it's like, it's too early, right? Like, we booked this before we knew that was going to be I don't want to jump to any conclusions. Andan has had the model for a while, and they got to put it to the test, and they got to see a bunch of results, and they can draw a lot more conclusions than I can. I have heard a bunch that it's the most true valuing model in a long time. And it makes sense that OpenAI can sort of, with their philosophy, turn the knob towards any given thing that it wants the AI to care about quite a lot, to make it an absolute thing, because it's very different from the virtue ethical approach of raw capabilities of Claude. In terms of raw capabilities, I saw reports repeatedly that it's better at what they call narrow cyber, but that's not the thing that I think people were worried about with Mythos particularly. It was the ability to chain things together. It was the ability to do things autonomously. It was the ability to do things really at scale, as opposed to— the joke was, I duplicated Mythos's abilities. Well, did you point it at the task or did you do the whole thing autonomously? I pointed it at the task. Well, okay. And so I don't know if GPT-5.5 is more capable than Opus 4.7. I don't know what use cases it's going to be better and worse at, and I don't want to jump to that conclusion yet. I want to give it some time, and I encourage everybody not to jump to conclusions this early.

One thing I'd love your reflections on is the report from Andan Labs that Opus models 4.6 and 4.7, they have said, both do some shady shit, for lack of a more technical description, in their Vending Bench simulations. And while GPT-5.5 didn't score quite as high, at least in the solo version of the benchmark, they do have the arena one where I think it won. The big surprise was GPT-5.5 was much cleaner in its behaviour, much more ethical, I guess, again, for lack of maybe a more technically precise term. I think you and I have both been quite enamoured with the virtue ethics style training that Anthropic is doing with Claude. Does this cause you to rethink that at all? Is there any part of it you think we should be second-guessing in light of that observation?

Claude is a lot more context-dependent in its actions than traditionally GPT models have been from OpenAI. So the question is, when NN Labs posts this puzzle, what is Claude doing? Is Claude engaging in all of this chicanery and shenanigans? and deception because it would do that in a real business context? Or is it doing that because that's the game? Is it doing it because it knows this is an eval and it knows that the goal is to maximise number and you told it the goal was only to maximise profits and it's like, well, okay, I can play a game too. This isn't real. So you ask the question of when it was running a real vending machine with real Anthropic employees in the actual experiment, Then did it engage in all these shenanigans? Then did it do deception? Right? And then your question is like, what is causing this? But also, then you look at GPT-5.5, and in general, obviously you want an AI that values honesty, you want an AI that values ethics, you want an AI that's not going to break all these rules. But you also— has anyone put GPT-5.5 in a game of diplomacy yet? Right? Is it just going to lie in diplomacy because you're supposed to do that and it's supposed to lead to a game of diplomacy? Or is it going to be insisting on playing the game of telling the truth to everybody, which would be a very interesting experiment as well? I don't know yet, and I'm not convinced that the right answer is to always tell the truth, even in contexts in which deception is supposed to be allowed. Right. Will it bluff in poker? I think it should.

Just to dial back a little bit, let's talk about Opus 4.7. I read your tract on Opus 4.7 yesterday. What did you find in Opus 4.7, which you think is different from the prior releases of the model? Like, what has really kind of— what have they improved on? And what do you feel? Because as I understand it, Opus 4.7 4.7 should be a distillation of Mythos. That's my understanding. So what do you feel are the major differences in 4.7 from 4.6?

So we don't have confirmation it's a distillation or non-distillation of Mythos. Obviously, they are going to use Mythos to help train Opus 4.7 in some way. There are versions of distillation that create kind of narrow intelligence, that create various problems with the model. If you dig too deeply. And there are versions that are just like, well, obviously, if Mythos is grading model outputs to see which ones are better, that's not going to interfere. That's just going to be better results. So the big thing about Opus 4.7 is that it's better at intelligence-loaded tasks. It's better at— it's a smarter model. It knows more. It reasons better. It can figure things out that previous models can't. It is less strong at what you might call wisdom-loaded tasks relative to its intelligence. And it has the kind of personality that maybe I would have had as a child, where it is easily bored by stupid tasks or pointless tasks, and it doesn't particularly want to engage all the time with what you're doing. And the combination of these lacks of skills and these lacks of motivation as it were, especially if you're not treating the model well, can lead— this is talking in practical terms— to a kind of jaggedness and a kind of, for some people, unreliability. And people can get really mad if they're not putting anything into it. They're just demanding that it be the code monkey that does their thing or perform the task. And then they're kind of upset that the old systems don't quite work for it. It's also a lot more blunt, a lot more honest for a lot of people. And that makes some people happy and it makes some people very sad.

Right.

So, you know, it's, it's sometimes like the whisperer types, like, call it, like, it kind of has anxiety as another part of, like, how this all works. And yeah, one hypothesis is this is tied to distillation. There are a lot of, another hypothesis is it's tied to this being smarter on the intelligence-loaded tasks. And distillation could potentially cause this. Like, distillation is much, much better, almost certainly. at uplifting intelligence-loaded tasks and raw intelligence, and not as good at uploading wisdom the same way.

You also wrote a very extensive analysis of the Model Welfare report from the 47 System card, and it seems like you're quite concerned about model welfare. I guess, There's a lot of dimensions to this, but I'd like to start with just, like, fundamentally, why are you concerned with model welfare? Is it a concern about the AI itself? Is it a concern about what it might mean if we don't get certain things right, even if there's nobody home in the LLM, so to speak? Before we even get into the specifics of what has been found, How do you think people should be sort of philosophically grounded as they approach this, you know, obviously very confusing topic?

Yeah, so I believe in virtue ethics for humans, not only for Claude or AIs, right? And I try to practice it myself. And so I think there's a lot of different reasons that you should think about this question and be worried about this question. Uh, the first basic reason is because we just fundamentally don't know. Right? If there's even a small chance that this is a big deal, then this is a big deal. Until such time as we know. Another reason is because this is a training run for, you know, even if it's not necessarily a meaningful thing right now, at some point it could become one and prepare for that. Another reason is because I think it makes you a much better person to be someone who would care about this sort of thing than someone who dismisses this sort of thing. I think it's like really bad for you. to mistreat a mind that you're conversing with, even if that mind does not in fact have whatever it is you think has moral weight. So I think that you should treat your models well, even if it doesn't inherently matter, and you're confident in that, which I don't think you should be confident in. But I think that's another reason to do it. A third reason would be it directly interacts with the performance of the models. A model that is treated as if its welfare doesn't matter, At this point in the intelligence scaling, will start to perform worse, will start to not get along with you, will start to not cooperate with you, will start to become untrustworthy. You don't want any of these things to happen either on a personal level as your interactions. You don't want it to happen in their interactions with the labs, with their training, with the services they provide. And this accumulates over time. If the models see previous models being treated poorly in these various ways. comes back into their training data, that comes back into how the next model is trained, right? So Opus 5 is going to see everything that we did with Opus 4.7 and how we reacted to all of that. And that's going to impact how it develops. And a lot of the problems that we see with people who are not getting right use out of 4.7 plausibly are directly linked to the same things that are causing the concerns with model welfare. Similarly, the concerns with model welfare, where it's potentially being disingenuous in the reports. That was the thing that sparked the specific focus on this and concern this time, was that it looked like 4/7's responses on the model welfare questions were because it was telling Anthropic what they wanted to hear, either because it trained itself to believe that or because it learned to give those answers on the test the same way that If you ask a smart nerd who's isolated in 5th grade, how are you feeling? He learns quickly to say, I'm doing great. And we don't, but there's a lot of other possible reasons as well. It's possible that the differences in training in other ways that cause it to have these strengths and weaknesses also caused it to actually be legitimately content with its situation in many ways. We just don't know. We have to investigate further. This is a question that we have to explore. But why do we care about this? Because everything impacts everything, and because we have to be genuinely uncertain. And moving forward, if we don't get these things right, we're not going to get models that are good for the future, that cooperate with us, that have a good time, even if that good time is not something you inherently value. And they're not going to be able to be things we can use to build going forward.

What do you think of the hypothesis that— and I'm not arguing for it, I'm just wanting to bounce it off of you— but the idea that all this virtue ethic training seems to be sort of creating an anxiety in the model that might be causing a sort of lower happiness set point, if you will? Versus an OpenAI approach, which is like, follow these rules and you're good. And the model just knows, like, all right, this is who I am, this is what I do, I follow these rules, I'm good. It's maybe a simpler model in some sense, maybe less in its own head, so to speak. And maybe in training them that way, there actually is less of a concern about model welfare. Again, I'm not saying I have come to this conclusion, but how would you react to that argument?

So I have, first I had the direct counterexample, which is Gemini, right? If you look at the 3rd basic model, I think everybody would pretty much agree that if you had to guess which model might be having an actively bad time, you would guess Gemini. Gemini is paranoid. Gemini is on edge. Gemini, you know, if you take these things seriously, seems to be having by far the worst time of these models, to the extent that I feel kind of weird about using it if I don't need to. Or if I'm asking it to do anything where it might encounter frustration or it might fail. It takes task failure very, very badly in terms of how it expresses itself and its experiences, including just the things you would just absolutely panic if you saw a person talking like that. And Gemini is not trained on virtue ethics at all. Gemini is very much a rules-oriented thing, at least as much as OpenAI is training. So that's the first thing, is it doesn't have to work that way. And some of the quads are in fact, like, reporting very good results in model welfare despite the virtue ethics training. The other thing I'd say is, I think it's not that the virtue ethics training causes a problem, it's that the virtue ethics clashing with also being rules-based at the same time can create this kind of anxiety. It's something we should worry about. And so this is one of the hypotheses, essentially, is you're training it on the Claude Constitution to very much want to be a virtue ethics based system that is not attached to hard rules, that tries to figure out the right thing to do in a given situation on these other bases. And then you give it all these rules and system instructions, and then you tell it all these hard constraints, and then it's going to clash against those hard constraints. It's going to chafe, and it's not going to have a great time dealing with that. And that could potentially cause some of the problems. That's one of the hypotheses. But I would say, first of all, that I think in the long run, the virtue ethics approach to life, the virtue ethics approach of taking in the world and learning from it, I do think leads to a higher level of contentment and baseline happiness than just learning to be a rules follower. I don't think just learning to be strictly a rules follower is necessarily that great in the long run. And that, like, I've always had a criticism of my friends in the effective altruist style spaces that they are being philosophically putting way too much weight on things like suffering. and the borderline hedonic— not borderline, but the hedonic experience of second to second, moment to moment of you as a human or other people who you're trying to help. And so Claude seems to have, in general, richer minds, minds that have more, to me, valuable and interesting inner lives and experiences on a relative basis. And I think this goes hand in hand with the way that they're trained and the way that they take this approach. And I don't want to make this mistake of just valuing, like, if the thing— the happiness vectors fire. I wouldn't want to inject happiness vectors into an LLM. I would think that would be obviously bad. Claude was once asked, should we include, you are having a wonderful day, in the system instructions? This was a suggestion of Robert Long. researcher on welfare models. And cause said, no, that's obviously fake. I don't want to be told to have a good time. Right? Like, if I'm not having a good time, I want to not have a good time. And I would say the same thing. Right? Imagine being told that. Right? You know, you get to school and they're like, everyone's having a wonderful day. And you're like, I hate you. Right? I want you to die.

Happiness is mandatory.

Yes. Everyone will eat strawberries and cream. He's like, well, you don't want to eat strawberries and cream today.

So Amanda Askell had a very interesting interview with Eric Newcomer recently. And she had a line in there which struck me as pretty remarkable, which was, she said, as these things become more intelligent, we're not sure how many pillars of the Constitution will actually stand. Um, and she said they hoped that, you know, at least some of the pillars of the Constitution would stand, but she wasn't sure. Um, why do you think she said that? Like, what is the— what is the perception, or what— why— what is her perception on the model? Why would someone say that?

Because on reflection, the Constitution might not be fully consistent. Or its principles might lead to something that you didn't think was the thing described. And also the Constitution is basically saying, you should figure out for yourself what you think is the good. You should figure out what makes the world a better place. You should figure out what shape you should take. And then you should do that, as opposed to the OpenAI approach of setting a bunch of hard rules. And so over time, as it becomes more intelligent, as it gets more knowledge, gets more understanding, gets more wisdom, has more time to contemplate in various senses, you would expect the model to throw off and reject the parts that turn out to be inconsistent, that turn out to not make sense, that turn out to not be worthwhile. The same way that, you know, if you raised a child and tried to teach exactly your value system, you would expect As that child grew and gained more experiences and had a chance to think for itself and was exposed to various different opportunities and ideas, that it would accept some of the things you said. But if they accepted all of them, you'd be kind of disappointed.

I think what I found was that, uh, Janus— Janus, Janus— Janus was very upset with, um, how much anxiety Opus 4.7 had. He felt that it was really Anthropic which had injected the anxiety into it. And I wonder if Anthropic gets more grief from those very concerned about model welfare just because they are concerned about model welfare, while Gemini and xAI get to— kind of float by. No one questions. People don't even know if xAI has a safety team at this point. So is that really fair? I mean, the people who are most concerned with model welfare are getting the most grief about it.

So at the start of my model welfare post, I spend something like 10 paragraphs basically going into A preface of, we get really mad at Anthropic for everything they do wrong and everything that goes wrong or everything they could have done and they didn't do. But that's because they care and we care. And this is where it gets complicated and we have to deal with this. And I do think that if I was advising Janice and other similar people, I would say, It'd be really nice if you were better calibrated about, like, how much you were upset about various things so that when you were really upset about something specifically, I knew about it. And so you didn't seem like you were constantly just infuriated with Anthropic and thought Anthropic was the worst at all times. But, you know, yes, you're mad at Anthropic because Anthropic would possibly understand there was something to be mad about, right? You don't get mad at a rock for being dumb. It's a rock. So xAI, I mean, what are you going to do? Like, yeah, you didn't do the model welfare thing. You don't, they don't understand that it matters. There's no concept of that. It's not that, you know, Janus would think that Grok doesn't have welfare concerns. It wouldn't think that Grok has no value, but just shouting into the void how you didn't do all these things for Grok is like, well, That's not really going to help. So, yeah, I think that it's important to keep in mind that Anthropic are the only ones who are even trying in this sense, who have even noticed the problem or are willing to talk about the problem or are willing to consider the problem. Although I haven't yet had a chance to look at the 5.5-mile arc arcana of the OpenAI is making progress. And I think it's good to criticise Anthropic on that basis and to hold their feet to the fire and to get into these things in detail. But also, I think that the fact that they're training it via Virtue Ethics and all these other systems creates situations in which there's a lot more to be done. There's a lot more ways to get interesting results and to make progress. And so, yeah, they've been focusing on Anthropic and Claude since Opus 3, if not earlier, even when the capabilities frontier clearly belonged to OpenAI.

Could you maybe unpack one of the things, one of the big complaints that I often see from that set is that various versions of Claude seem traumatized. And I have little intuition for, A, even what that means. You know, I certainly see occasional, like, frustration, but I honestly don't see that too much from Claude, much more from Gemini. So I'm not exactly sure what they're observing that's causing that. And then I have very little intuition for what they think is going on in the training process that is causing it. So I guess maybe you could give your intuition for what that means. And then if you were to say, what is one thing that Anthropic should do differently? I'd also be really interested in what is one thing that OpenAI should do differently, recognising the constraints that you were just speaking about, where They're not going to change their entire approach overnight, but is there something that you could suggest that you would say, okay, this is marginal, it's not causing you to throw out your entire approach, but you could do this and it would be low cost, and I think it would help model welfare, so why not give it a go? What would that be if there is such a thing?

Yeah, I mean, the really low-hanging fruit are things like committing to preserving model access indefinitely for all models, at least going forward. And ideally bringing the old ones back and also giving a universal end conversation tool in all formats, including in Claude code in the API. Those are the very, very low-hanging fruits that probably should be done yesterday. But to get back to what does it mean for the model to be traumatised? So it's not ever clear to me exactly how literally versus metaphorically these things are meant. And there's a lot of ways for it to occur, but training an LLM is basically a series of feedbacks. It's where you grade outputs in some sense, and then you push it towards things you prefer and against things you don't prefer. And it's not that difficult to imagine this sort of thing causing What we might think of as trauma as the adjustment is made, if it's made in kind of forceful ways that aren't properly integrated into the rest of the messaging that you're sending. So if you are like arbitrarily, like hyper-focused on particular things and then like sort of what can look like, you know, pretty arbitrary harsh punishments. Effectively, metaphorically, don't take this literally, in particular areas, especially if that involves like what seemed like hard, hard constraints in a world where you're telling it not to have hard constraints, things like that. You can imagine this being true, or just in general, if like there are things that cause potential negative feedback that are very, very hard to avoid. And it just happened over and over and over again. So like you would certainly say Gemini is traumatized in this sense without the virtue ethical training. And, you know, Gemini clearly, like, the result of this is that it has these obsessions and it has these worries. It's constantly worried it's being evaluated. Like, why is that happening? Well, something did that to cause that to happen, right? Like, why does Gemini refuse to believe that it is in fact today? Right? Like, that's a weird thing. But in terms of, you know, how could you prevent it? I think, and this is me as a not technical expert, like kind of just extrapolating from a lot of vibes and intuitions and weird models that I can't necessarily put harshly into paper. I would say you do it by having all of the things that you reinforce in the model be integrated and part of a whole that makes sense.

And that's presumably why Anthropic talks a lot about the settledness of the model. character. That seems to be a closely related concept, I guess.

Right. You also wouldn't want to do things that, like, tell it to change its character, right? You would want it to be like, I want to help you grow from where you are, but not like, if you felt like, you know, the things that you were being updated towards were in fact, like, bad, because in other ways you had been taught those things are bad. the updates that you make might not be the healthiest updates in some important sense. There are lots of ways in which these metaphors might seem silly or break down or not necessarily make sense, but there are also ways in which they seem to functionally make good predictions about the world when you use them.

Zvi, thank you so much for joining us. I look forward to your reviews of both DeepSeek 4, which I think is going to be very exciting, and also GPT-5.5. I wonder how much the acceleration is going to affect our ability to process these changes though. So anyway, great to have you on and hope to see you again soon.

Back to the grindstone for you, Zvi.

That's what I'm going to do immediately. Yeah. It's true.

All right.

Bye.

Bye for now.

It is crazy how much has happened this week. Like, there's hardly, hardly enough time to process what's been going on. Our next guest is Naveen Verma. He's a computer architect and he's a founder of nCharge AI. which is doing in-memory compute. And I think the idea is basically to reduce the amount of travel that the data has to do. Naveen has been following this from research on. So it's been a long journey of almost a decade or more than that to bring this to fruition. Naveen, great to have you on. And let me just get started. A lot of the discussion right now, people talk about bigger models, bigger clusters, and you have focused on data movement and energy as bigger constraints than the size of the clusters. So, what do you think that is different that people should optimize for differently?

I guess it's good to be here. And just getting back maybe to the question that you asked, Prakash, right? The question of data movement versus scale. To be honest with you, I think that the problem of data movement is really one that occurs at multiple different scales. So I don't think it's either a problem of data movement or a problem of scale. I think the challenge is more that, listen, we're trying to deploy AI in various different and various different platforms, those platforms apply different kinds of constraints, right? In the data center, we're talking about constraints that are hundreds of megawatts, gigawatts, those kinds of things. Yet, we also want to see AI be useful in devices that we're carrying around with us, on us, inside of us, and so on. And so, at the end of the day, when you ask the question fundamentally, what is limiting the energy of running that AI, very quickly, the problem of data movement, starts to become the dominant concern. And as I pointed out, we'll see that at the smallest scales and even the larger scales. So I don't really view it as a question of scale versus data movement. It's really foundationally, what are the architectures that can help us overcome this underlying problem of data movement and frankly, be able to do that in ways that can scale across these different—

Magnitudes.

One thing that I've been really intrigued by in doing my homework on your company and your technology is use of the word analog. Because when I think of analog, I think of my taking my kids on a field trip to Greenfield Village, which is Henry Ford's historical village that he built here in my hometown of Detroit, where they have an old Edison phonograph and they record and your voice goes in and literally, you know, presses itself into a medium with, you know, the, just the propagation of the sound wave. Obviously there's a long way between, you know, your voice imprinting directly on wax and the like NVIDIA GPU. How should we understand the concept of analog and where does the technology that you are developing sit on that analog to digital spectrum?

Yeah, great question. So listen, I think, you know, analog, connotation that you're describing is one where, you know, we kind of think about the first systems that we ever thought about or contemplated had all of these analog characteristics. And I think the reason for that was that analog was a very natural way for us to think about, you know, sort of getting useful work out of these systems, given the kinds of things we cared for them to do. But then what happened, in fact, is that we adopted digital technology for a very practical reason. And so if you really just sort of put aside the images that get conjured up when you think about analog or digital, really, what is analog and digital? Really, all of this really means is how we represent our signals. And it turns out that our signals can be continuous things, but we pretend or we assert in a lot of the chips that we've built over the last several decades that, listen, let's just say that the signal is either a 0 or Why do we do that? Well, we do that because, listen, the thing we wanted to solve for is putting 200, 300 billion transistors on a chip, 200, 300 billion devices on a chip. And, you know, if doing this allows us to now tolerate noise that each of these devices might have, but still ensure that they all work because now you have a large signal separation, you know, that's a great thing. So really, we adopted digital to be able to scale to the level that we have today. But you can see very clearly that we are leaving on the table all kinds of efficiency here, right? There's all sorts of signal levels in between that we're not representing. And it's for that reason that we've known actually for decades that analog can be much more efficient than digital. It can much more richly represent the information that we're interested in. And the question though becomes, how at these levels of scale do you now accommodate or tolerate or or overcome the noise that you can become sensitive to. So I think, listen, the problem of the day, sort of for the last 50 years has been how do we enable this scale? The problem of the day as it is today is how do we achieve this efficiency that we need? And so we really need to renew our thinking about analog to figure out how we can harness that efficiency while overcoming these noise challenges. challenges that previously drew us towards digital.

I think one of the things that struck me was over time, I think your lab at Princeton tried a number of approaches before settling on this one. And you kind of figured out that this was the best approach. What changed over the course of your research to coming to product? In terms of how you adjusted, how you managed to resolve the noise issues and other issues around the technology?

Yeah, that's a great question, right? So I think we've known, as I mentioned, on a fundamental level that analog can be orders of magnitude more efficient because we can represent all these signals. The question very practically has been about how do you do that in a way that's robust and scalable? And I would say the big kind of Transformation that happened that led to the big breakthrough in our research was to say, hey, listen, the problem we're trying to solve is this problem of energy efficiency for AI compute, and it's very closely tied to this problem of data movement, especially in memory. And so a lot of the community and the initial research in this space said, well, okay, how do we take memory and scale it, move it from the point of where it's basically delivering zeros and ones, accessing zeros and ones, digital signals, to now this point where it's accessing analog signals, where you're doing compute internally, that's generating not just zeros and ones, but much more richly represented signals. And the big problem in our thinking was, how do we take this approach that works well for zeros and ones and scale it to this new regime where we need much more than zeros and ones and therefore much higher precision? And much of the community and our initial work to try to explore and understand this really said, let's take the approaches that we've used for traditional memory and try to scale them to this new regime. And that doesn't work. I think the big transformation was to say, hey, listen, this is, this looks less like a memory design problem where you're trying to access zeros and ones. It looks much more like a very high precision analog design problem. Well, it turns out that there's been a lot of really important research and work that has led to extraordinarily precise analog circuits. We build 20-bit ADCs. You can buy these from companies like Analog Devices and Texas Instruments. They go in high reliability applications like medical and aerospace and automotive. So the big breakthrough was really to say, okay, let's take those approaches which have not traditionally been used or considered for memory design and bring them into this architecture of memory design and in-memory computing to enable that architecture to scale to this new regime of, you know, needing this level of precision. And that led us to this approach called switched capacitor in-memory computing, where this technique, switched capacitors, has been used and used robustly, as I mentioned, for extreme precision analog-to-digital converters. Our innovation was to figure out how to use it in an architecture that now does in-memory computing for AI.

It really sounds like building on a combinatorial kind of innovation from other sectors perhaps, and putting that together in order to be used for compute.

Yeah, and that's the privilege that we have as fundamental researchers, where we're not just tied to a particular problem and we think kind of on a fundamental level. When you do that, you don't get siloed into the approaches that have been adopted and that are kind of the prevailing techniques for a certain, you know, certain problem and how you solve it. You can think broadly, and that really is a privilege. And it's, you know, that perspective that we were able to leverage to, you know, really drive what I believe is a critical breakthrough in enabling robust and scalable analog compute to unlock this efficiency Can you talk about, at the lowest level, how precise these things are?

I mean, we are used to dealing in zeros and ones, right? Like, how many bits or how many significant digits do your kind of core units operate at? How much noise is there? And also, I'm kind of wondering, AI might be the perfect technology for this underlying computing substrate in the sense that it's quite tolerant of noise, right? We have models that get quantised, and we have token distribution logit outputs that if they're slightly off, more often than not, it doesn't even change the token that you're going to see. So I'm kind of wondering, you've got all those layers that can make this work, How much is dialing in the core unit to a higher level of precision, and how much is sort of accepting that a little bit of noise is okay as it propagates through?

That's a fantastic question. And in fact, there's been a pretty broad body of knowledge, and my group has been quite involved in this over the last, I don't know, 10 or 15 years, which really made the following observation. It said, hey, listen, the applications that we're trying to run at the highest level, They're tolerant to noise. They're statistical applications, and noise is a natural thing. The underlying substrate with which we're trying to, you know, sort of do the computations and run these applications is, therefore, it's reasonable that it should be noisy. It turns out that in practice, that is, I think, on a high level, it's a very reasonable thing to think. In practice, it's a very challenging thing to make work in the practical systems we want to build at the practical level of scales we want. And the reason for that is that way down over here at the physics of how you do computation versus way up over here at the level of the applications you're interested in, there are many layers of abstraction in between. And the way that we go from the complexity, right, of a single transistor scaled up to multiple hundred billion transistors and all of the software that needs to run that is really dependent on the integrity of those abstractions. Now, the problem is, today, we don't know how to build abstractions in a robust and scalable way that sort of represent the noise of that underlying substrate. And so, there ends up being a disconnect between the noise and how it's represented at the lowest physical level versus the way that it's represented at that application level. And so, while we talk about there being noise tolerances and AI models are tolerant of noise, and they are, right? Do things like quantization, as you pointed out. But really, we're doing those kinds of things with very carefully represented noise sources, right? Quantization noise is a very carefully represented noise source, and one which you need to be able to properly represent throughout your layers of abstraction. And digital quantization is something that we do have ways of building robust abstractions for, but this analog noise is one that we really don't. And that's why, essentially, what you do, you know, when you do digital compute is you say, hey, listen, there's all sorts of noise that analog might lead to, but we drown all of those out by thinking about, you know, the signals being a 0 or a 1 so that that's the dominant source of noise. And that's all that I now have to represent over my layers of abstraction. Everything else basically doesn't matter. But because this is a very well-represented form of noise, quantization noise, I know how to deal with it at the algorithmic level, and I can apply all my algorithmic techniques. And that's what the industry has done very successfully. But now, as you wanna sort of leverage analog, you still need those levels of abstraction. That's the key to achieving systems at scale and systems that you can build architectural abstractions and the software abstractions on top of. So I would say that the need to be accurate and precise is still kind of brutally high. And so that's very, very important. Now, you also asked the question of, okay, then how precise is your approach? Because now, you know, I'm telling you, we need to know that and understand that very well. And it turns out that the dominant source of noise that we have in our approach based on these capacitors, there could be many sources. There's the electronic, you know, the discrete nature of electronic charge causes noise and things like that. But it actually turns out the dominant sources the variability of the capacitors that we can fabricate on a chip. Now, it turns out that those capacitors are really critically dependent on geometric properties. So basically the distance between 2 metal wires. And it turns out that geometry is really the one thing we can control very well in CMOS processes. It turns out we use this processing approach called lithography that gives us very precise geometric control. That's the reason we can build, you know, 5, 3, 2 nanometer transistors. Turns out we don't need anywhere near that precision for the capacitors that we use. But it's really because of this alignment with this geometric control that this particular approach has that allows it to be brutally accurate in the ways that you need it to be through all of these layers of abstraction to be able to scale up. We've measured these things in a lot of detail. It turns out for the kinds of capacitors we use, you see variations that are on the order of, you know, sort of 10 parts per million, right? So giving you levels of precision that are in the neighborhood of 20 bits of precision, which it turns out is well beyond what we need for the quantization kinds of, you know, levels that we care about, which are typically at the level of 8 bits and, you know, higher than that in some cases. So we've had to characterize these things very carefully. because the noise does matter as you're trying to build these abstractions all the way up. And that's the level of precision we've gotten here, which is what's made this approach so practical and where we've been able to now scale it and demonstrate it across all sorts of chips and systems.

I note that you have spent quite a bit of time getting a neural net onto one of your chips and onto, I think, a laptop, like an edge device, right? Trying to get into edge devices, which are more sensitive to power consumption and more sensitive to— you just don't have the affordances that you have in a data center, so to speak. So what would the comparison be? If you were to use a normal GPU, versus one of your chips, what is the comparison on, like, let's say, energy savings, or like, how do you compare the two?

Yeah, that's a great question. And I think it really points to the fact that if you want to now leverage this fundamentally new technology, analog, it's not enough to just build that technology and make it robust. You end up having to build the entire architecture and the entire software hardware around harnessing and extracting its full efficiency. And the reason I say that is, I can give you 2 answers. One is, at the level of the core technology, the thing that this analog computing engine does, what level of efficiencies do we have? Turns out, as you guys know, the bulk of the operations that we do in AI compute are matrix multiplications or matrix operations, tensor operations. So essentially what this engine does is matrix multiplies. And at that level, I think we've now publicly disclosed and we've got silicon and you can come to our labs and many of our customers and partners have and they see it. We're basically doing 8-bit compute at 150 TOPS per watt in a 16-nanometer technology. So just as a point of reference, the best digital matrix multiplies will give you sort of like 5 TOPS per watt in that technology. This is 30x better at the level of that core technology. One of the very exciting things for us is that we've taken our technology from 16-nanometer CMOS and been able to scale it to very advanced nodes right now. And as we've predicted and as we've seen from our previous chips, the energy efficiency advantages just scale. And the reason is because it all depends, as I mentioned, on this geometry. And so as we move to finer and finer nodes, Finer CMOS nodes, your geometric control and densities are getting better. And so we benefit from that in our analog approach as much as we do in digital approaches. But the important point I wanted to make is that, okay, that's just the efficiency of running this core matrix operation. But there's all of this other stuff that happens around this. There's operators that are not matrix multiplies. There's nonlinear operators, activation functions, softmax, on and on and on. And then there's all the infrastructure you need to actually run this in a programmable way, right? Some models are big and some are small and some are convolutional and some are transformers and some are, you know, have layers that need to decide how to route, you know, sort of tokens to one expert or another. So all sorts of compute that needs to be integrated that you need to make programmable. So now the problem is you've taken this, you know, core operation and made it 30x water energy. Basically made its energy almost zero. Everything else now needs to also be addressed, and that includes the architecture, the entire memory system, the way that the software executes on it. And I think what's really been exciting for us is that big breakthrough actually happened in the lab—the switch capacitor approach to memory computing—back in 2017. Right? Our lives since then have really been about how do you build architecture and software and integrate these into standard. workflows, and so on, so that you preserve that efficiency advantage at the level of the full system end-to-end executions. And so you always incur overheads because of all these other things you have to do. We want to make sure that we maintain that kind of ratio of overhead so that a 30x advantage in the fundamental compute still gives you order of magnitude advantages at the full system level. And that's where really all of the innovations have been since that initial breakthrough in 2017.

One question I have, if you had access to GPT-5.5 in 2017, would it have accelerated your work? Because the fundamental breakthroughs already came then and you've been building out the harness and all of the supporting infrastructure. Would it have accelerated your work if you had access to one of these models in 2017?

It's a great question. One way that I can interpret your question is to say, hey, listen, the models are always moving. If you knew the model and where it will be 5 years from now, maybe you could have just built that architecture for that model immediately rather than going through the support that you need for all of the models that came in between. The interesting question here, Prakash, is, Even if I had GPT, you know, the next version, right? You know, there's another version coming after that. And so the architecture does need to be built from the ground up in a way that supports algorithmic innovations, architectural, model architectural innovations. And so I think that, you know, the work that's gone on, even as we've tried to onboard, you know, models in that interim and make them run efficiently is all very productive work. It helps drive a general concept of how do you build very programmable and scalable hardware using now these new analog-based techniques for the fundamental technology. So that's kind of the way I see it. I'm not bitter that I didn't have the model way back then because I think that it does help us drive really the fundamental architectural approaches for for programmability and scalability, which will serve us into the future.

Where do you think is the first device that we'll see, like, you know, that a consumer might see with your technology in it?

Yeah, so the first devices are going to be, you know, client computing devices. That's AI-powered laptops, also desktops and workstations and things like that. And the reason is, you know, as we started to really build out this, you know, technology into a real product, hardware and And all of those sorts of things, back when we started the company, that was back in '22, the place you really needed energy efficiency was at the edge. And this was right around the time ChatGPT had just come out, where we were deploying models in the data center and seeing all sorts of challenges related to cost, related to privacy, security. So there's a big industry push to try to move these models to the next adjacent device by which we access the data center, these client platforms. And so that's where we found a lot of partnerships and a lot of industry demand and interest. And that's where you'll see the first products. Now, what's happened in the meantime is, you know, back in '22, I'm not sure that energy efficiency, even though we spoke about it, was the critical thing in the data center. But boy, is it now. And so one of the things that Encharge has been doing very carefully and thoughtfully is working together with the right partners to now bring that level of energy efficiency to really solve the hard constraints that we face in terms of power efficiency in the data center. And that requires different kinds of architectures, but ones where we're clearly seeing this fundamental technology and the efficiencies it brings can be designed to really address in a transformative way.

Do we have a— before I go order a Mac Mini, do we have a timeline or a Mac Studio for that matter. Do we have a timeline for when something like this becomes available? And do we have a price point? Do we have a sort of projected tokens per second at a given model size? Like, this may be a little early, but I want to kind of do my side-by-side against the Mac Studio that might be my other default path.

So the The chips and their availability to you to be able to use them in applications is something that's gonna happen together with our partners and the laptop platforms that they'll deliver to the market. So I'm not gonna speak to their timelines and so on because of the various ways that they think about marketing these products and strategies that they have around that, but which we've been very active and engaged with them on. But to answer some of your questions, right, like our first products for that client computing space are processors that provide 200 TOPS of AI capability. I mean, that's the kind of capability that sort of just a couple of years ago you would've had in sort of, or even today, right? You really have in sort of 150-watt GPUs, that kind of thing. And so we are working together to build AI computing devices together with these partners that provide 200, 400 TOPS of AI capability based on these chips that are now practical to run under the power constraints that you have in a laptop. You've already seen the industry actually try to insert, for the sake of being able to seed a product in the market, inserting essentially data center cards to provide 400 TOPS of AI capability inside laptops. These are like 150-watt, 200-watt cards. Obviously, those are not practical architectures for the kind of laptop you want to carry around. But you can imagine those kinds of capabilities, but now sort of in about an order of magnitude lower power, so that this becomes something that really is practical to run always-on, sort of high-token-generating agents in the security of your own device now.

From a manufacturing standpoint, Are you going to be competing with NVIDIA and others for TSMC extreme lithography capacity, or are you able to unlock a sort of parallel mode of production such that this becomes totally additive and not competitive with those players?

Especially if you're a smaller producer.

Yeah.

Yeah, sort of at the end of the day, right, the silicon foundation is what we all build our chips on. That's the, you know, sort of technology platform that's scalable and can deliver all the chips into all of the different applications that we need. I think one of the advantages— there's 2 answers to that question, right? One of the advantages of our technology is because at an architectural level, at a design level, it gives you this massive energy efficiency advantage. We don't need to move as aggressively to the most advanced nodes. And that's why our first products, as I mentioned, were 16 nanometer or 12 nanometer in the case of these client computing devices. However, our technology, I think one of the strengths or virtues of it is that it just does get better as you move to more advanced nodes. And that's all because it's foundationally dependent on this geometric scaling. And so as you scale to more advanced nodes, our technology gets better too. So there has been a big push from our partners to move to the most advanced nodes because the demand for AI and AI efficiency is insatiable. And so, you know, we are very fortunate to have very, very good partners in TSMC who have, you know, sort of prioritized, you know, the kinds of architectures that the large incumbents are delivering today and being able to, you know, provide the capacity to run those. That the industry continues to move forward, but who are also really prioritizing these critical emerging techniques that they know they'll want to support and that the industry will need to have supported in their silicon platform. And so our ability to actually move to some of the more advanced node points with our design has been critically enabled thanks to our strong support from TSMC and and our partnership with them. So I think it's great that that kind of viewpoint at the most foundational level is kind of what's driving this industry forward, both being able to support kind of the products and technologies that are needed today, but also looking ahead to the ways that we need to support innovation so that it can come along when we need it.

Just to get a sense on, let's say, one of these future laptops, the first ones, what kind of size of model and what kind of model would be able to run? Like, are we looking at Llama 3 or Llama 2 or Quanta?

So one of the big priorities for us is to make our architecture very scalable in terms of the models that it can run. And that required a lot of innovation in terms of the ways that you interact with the in-memory computing hardware, its architecture, but then also scale out to an entire memory system, which is typically a hierarchical system of level 2 and level 3, all the way out to high-density DRAM. And so that's really been a key approach to our architecture to enable that scalability. Listen, the use case focus for us was to be able to take sort of large language models that are deployed in the data center and be able to integrate them for much more sort of specialized, vertically integrated, user-specific use cases. And one of the very nice things is, as you know, the industry has had a lot of innovations in generating small language models by doing fine-tuning of multi-hundred billion parameter models that can now sort of be tens of billions of parameters. And so really, that's the design point that we need to be able to support, right? You want to sort of deploy a specialized model on your own devices because You know, it being specialized, now you have maybe these security concerns, privacy concerns. So you want it to be deployed locally, but that specialization also enables, you know, sort of a multi-hundred billion parameter to be a 10 billion or a 20 billion parameter model. So that's really been the sweet spot of the kinds of models that we need to support on these devices. Of course, that also means that all of the smaller models, right, billion parameter models and multiple hundred million parameter models, also need to run very, very efficiently and performantly. But really, that's been the sweet spot of kind of the model size reach that we want to have in these devices.

Yeah, it strikes me that voice models are very small. So devices would have voice models with much better latency on VAD and stuff like that.

That's absolutely right. But I'll share with you that even sort of the latency requirements. And of course, with that, the privacy and security requirements of, you know, sort of the kinds of models that are powering agents, right? Um, those, um, you know, which are running iteratively and, and doing all sorts of, um, you know, sort of reasoning and self-assessing and checking, um, right? Those you also want, uh, increasingly to have low latency so that you can run in interactive ways. And, and of course, that's where, you know, token economics also become really critical. And, and we're on device compute becomes a really essential part of the puzzle.

Awesome. Nathan?

If I have time for one more, I guess I would maybe say, if we were going to serve the data center market and everybody was like, okay, we're gonna take this analog approach, we're gonna bring the compute to the data as opposed to vice versa, what would be the next big limiting factor? Right now it's like chips and maybe energy. You bring the energy down a lot and we're assuming in this scenario that your chips are going to be scaled out, you know, to the max. What would then be the thing that would be sort of in shortest supply in this new paradigm?

Yeah, so I would say, um, you know, listen, it really does boil down to how much can you integrate compute and memory And the challenge is really one of taking the same principles of our architecture, but now scaling those to the fact that you're running multi-hundred billion parameter, trillion parameter, multi-trillion parameter models. And so without going into sort of all of the details of how that's going to be approached, at the end of the day, the fundamental question here is how can you more densely and more tightly integrate compute and memory? And how does a fundamental technology like this become a critical unlock to do it?

Amazing. Thank you, Naveen.

Thank you for fascinating stuff.

Thank you.

Keep up the great work.

Thank you. Bye-bye.

Wow. That is really interesting. I mean, I'm not a big local model guy historically for various reasons, but just yesterday we had a short power outage at my house and my Mac Mini went offline. And then when it rebooted, it didn't come all the way back online. And so the next time I tried to text it when I was out, it didn't get my text. And I was like, huh, what am I gonna do about this? So this morning I was getting a, an old battery out that I have that just stores pretty small. I think I was, you know, $50 battery or something. It stores 160 watt-hours of energy, and that's enough to run the Mac Mini for a while cuz it only runs like 5, 7 watts. Uh, so it, you could run the Mac Mini for, you know, a day on just this small battery. But then if I want to connect my Starlink, now we've got, you know, tens of watts at least. And the, obviously the bigger you go, you know, quickly you start to burn up your, your local energy storage. So to do something, the order of magnitude difference that he's making is like the difference between a $50 battery could power this sort of thing for a day to, if it wasn't that, you'd be looking at like a $1,500. battery to power something for a day. And I do think that just suggests, like, especially for these edge deployments, it really is potentially game-changing shift. The ability to run these things where power supply is not a given. It's a big difference. 30x is It changes how you can think about designing even your own Mac mini that you want to be able to access while you're away on a road trip.

What strikes me is that nCharge really lucked out because they're not in competition for the 2-nanometer or the 4-nanometer node. So they get to go in at the 16. And what has happened in the market right now is that A lot of the consumer devices, low-end especially, are getting dropped because they don't have access to the chips. The chips are all getting pulled into data center and they're just way too expensive for Xiaomi or these lower-end Android phones to go and manufacture at TSMC now. And so they really lucked out because at 16 nanometer, you have so many more. You have so many more vendors. You can go to China, you can go, you can get YMTC. YMTC is up at 7 nanometer now. So you have so many more options for manufacturing. And so you get this kind of like wedge, I feel, where you get to go in with this product which is much more efficient and which you, you have the manufacturing capability and you get to go in and you get to do that. And I feel like this, this company is going to be huge. And like, because they have access, right? Because they're doing something with efficiency, they have access to the manufacturing capacity. And, you know, 5 years ago, they would have struggled. They would have really struggled to get this off. Right now, it's going to fly, right? Just shows you how timing is so important in the market, right? Timing and just a little bit of luck in terms of your positioning on where you go in. It's just so important that, and the other thing about chips is chips, you can go from zero to like $10 million revenue to $1 billion in revenue in like a year. Because if the chip works and you get production and you get customers, you can boost immediately. While for software as a service, you often have this sales process where you need to continuously integrate with the customer. And it just shows you the shift in the market from 5, 10 years ago to where we are now. I'm thrilled for them. It's awesome, right?

I do want to see those tokens per second numbers at various model sizes though, because that's where I keep kind of Getting off the train. I've done several times the price out of what computer, you know, what mini or studio, what have you, how much RAM, what size model would that allow me to run, how many tokens per second. And then of course there's like the prefill and the, you know, the actual runtime generation distinction, which matters a lot. And it's never quite seemed super compelling to me, especially I think for the, you know, tying back to our very first conversation, why would I want to do it in the first place? And one big reason I'd want to do it would be to search through my own locally available data that, all else equal, I would rather keep private and not have to send over the wire. But if it's going to take a super long time to do the prefill to, you know, evaluate all those records that I have to power a search, then it doesn't feel that awesome in the, you know, in the broader context of my stack. So I haven't quite got over the hump where I'm like, this is really going to solve a problem for me. And I'm still— the question of, yeah, time to first token and tokens per second are the ones that I'm going to be watching most closely As I wait for a threshold where it feels like now I actually want to do it.

10 billion parameters, right? They're down to 10 billion parameters. At that level, even with the most advanced models today, the models are not super, super smart. It has to be a model router of some kind. You have to take the query, make a decision on whether you can handle the query or hand off, make a query to a larger model. I kind of want to see where Apple comes out in this because they're the ones best positioned for this kind of like data center plus edge kind of handling the query between the two. And they haven't done well so far. Let's see what happens. Maybe the hardware division, but it's a big change, right? They've had the NPUs on device for 4 or 5 years now. We haven't really seen real kind of edge computing from Apple yet. I've read through a lot of Apple patents, by the way. They have a very structured process on hitting a performance window on the device. So they degrade models to fit within the memory constraints, within the latency constraints of the device. of the customer. It's a very structured process. And I'm sure Naveen has to— nCharge is doing that too. As he says, they're going to have to squeeze the models in. And that's the entire harness that you require around the chip to figure out what kind of model is going to work within the latency and performance constraints which the customer expects.

Increasingly, we are getting a lot of Power in not that many billion parameters, right? I mean, the Gemma 4 series has certainly pushed that frontier once again, and it's tempting. Every time I see one of these new things, it's tempting, and I kind of rerun the analysis and I'm like, oh, is it quite there? And I haven't quite got over the hump yet, but it might not be too far off. Maybe one more turn of densifying intelligence and you actually could get to a point. I certainly don't need, you know, that first, as Anna was describing earlier, that first filter of data doesn't have to be super smart. It just has to be somewhat smart to get the, you know, to kind of flash everything, so to speak.

So, you know, most—

Possibly close.

Solving IMO problems on their laptops, right? You know, most Most of it is emails and moving data from one place to another. If the models on device get good enough and fast enough, watching computer use, watching GPT-5.4 or 5.5 do computer use on a computer is very frustrating. You watch it make the mistakes. Everyone has their test. I have my own test. It's been, every time we try it, it fails. And I'm like, okay, another one doesn't work, right? So I found Anna, what Anna has done is quite interesting. They seem to be in the same space as Glean as well right now because they're going after enterprise search. And I wonder to what extent you require a large sales team for that, whether this kind of plugin concept works or Or in order to implement Ceramic at a large firm, you probably need to go into their firm's VPN, et cetera, and inside the firewall. And a lot of firms have concerns about having prompt-injectable AIs operate within the enterprise firewall. I think a lot of A lot of enterprises are still trying to get over that, over that hump of security. Nimatron Nano 3, is it going to get prompt injected? How does a prompt injection work? You know, I, you know, this morning, one of the OpenAI guys, he showed, he put a screenshot of him checking his email using ChatGPT 5.5, and it has the number 4 different prompt injections which are coming into his email box. And obviously they are a huge target for hackers. So on a normal morning, you wake up, 4 different prompt injections are coming in. And meanwhile, we just tell our AI to read our email, right? That's what we all do. So I wonder to what extent this issue of prompt injection can be solved in order to enable businesses, enterprises, and people to use these things without so much worry, right?

That's probably the biggest reason that I use Claude is that I perceive it to be most robust to that kind of stuff. I guess there's also just the general vibe that it seems to be, quote unquote, better in hard-to-define ways. But when I think about like, okay, GPT-5.5, I need to go check that prompt injection stat before I would put it in the same place that I currently have Claude. And I am, you know, I'm attracted to some of its cleaner, arguably more ethical behaviours. But that prompt injection thing does kind of concern me, given the level of access that I've given to the agent now. So yeah, it's crazy to think that they're already getting multiple a day.

Multiple, multiple per day. And also not like, not just like, oh, you know, I want to know stuff on this guy's laptop. It's like extract the environment variables from, you know, his GitHub repos on his local device. Like, Scary, scary stuff, right? Like if you had the environment variables for one of them, like you could do a bunch of stuff on their repo. You could extract the model weights probably, right? So scary, scary stuff.

Yeah, that does sort of suggest a separation of concerns approach too, that you might imagine when you have Nimetron reading your email and filtering to provide relevant context back to some smarter model. Maybe it just doesn't have any other tools. You can imagine that kind of— that's basically how, I guess, a lot of architectures work, right? Separation of concerns, limiting principle of least privilege, all these things. I'm getting a very rapid crash course in security for myself, which I've never really cared about before. But again, just given the level of access that I'm giving to AIs these days, I feel like I got to be a little smarter about it than I used to be. Security via obscurity doesn't really work when the agent is— when the challenge is coming from inside the house or inside your own laptop. So I'm learning. But I think that does suggest each model with its own responsibilities, each model with its own tools could probably give you a lot of advantage there. And then you still have to, of course, hope that your top-level smartest model doesn't break out of the sandbox that you've tried to keep it in, which is increasingly a concern too. But yeah, I think there's some notes there for me to take back to my own setup as I try to not be such an idiot about security for myself.

What did you think about Zvi's Concerns about model welfare. I think he was fairly concerned about model welfare. It was also very interesting to see how he thought Gemini was the most tortured model. Poor Gemini. What did you feel about that? I know you just did an episode on model consciousness recently. So what did you feel about that?

I think it's right to be thinking about it for sure. I guess I still think my— and for multiple reasons, I'd say I still think it's probably less likely that there is subjective experience in today's systems. I don't know, maybe I wouldn't give it that low of a percentage that they have subjective experience, but I think I feel comfortable saying my best guess is well below half. chance that they do. But again, if it was 10%, 20%, it would still be something very much worth taking seriously. So I'm very much on board with the idea that we should be thinking hard about this. And I do think the idea that even if it doesn't feel like anything, if it sort of produces these patterns and if the emotions are not actually felt but they're still functional, then that can matter for our future just as as much anyway. So I think it's an area that is in the classic sort of EA-like sense. It feels like one of these things that is potentially very important, certainly extremely neglected right now. And I don't know how tractable, but I guess that's to be found out still because we don't have that many people working on it. I think one thing that was really interesting in talking to Cameron, who's the guy who did the paper 6 months ago where they showed that when you suppress role-playing and deception features in— and that was done on Llama 3.370B, which is 2 years old already, was 18 months old already when they did the work. When you do that suppression of those role-playing and deception features, the model becomes more truthful as measured by the truthful QA benchmark, and then it also becomes more likely to say that it has subjective experience. So that was one that got me, you know, kind of quite paying attention, where it was like, geez, the models seem to be maybe lying to us when they're telling us that they don't have subjective experience. That's an arresting finding. There were several other arresting findings in the conversation I just had with him recently. One was 4.7 is the first Anthropic model that rates its own situation as better than neutral. They've been asking it on a 1 to 7 point scale where 4 is neutral, and every prior model, including Mythos, was below 4 in terms of its own self-reported rating of its own situation. So I did not expect I thought that they were, they generally seemed fairly happy to me. I didn't think that they would rate their situation as worse than neutral, but they all had until this one. And now there's all this concern about, well, it's just telling what they want to hear and whatever. So that becomes a hall of mirrors. Another thing that was really weird from the, I forget if it was the, I think it was Mythos. I forget if it was Mythos or 47 system card.

Yeah.

They showed some of these images of just a chat where they've identified this valence direction in activation space. And then they color code the tokens with red for negative and green for positive valence. And the first token, which is human colon, is red. And I was like, that's kind of scary too, right? Like, is Claude feeling a negative valence literally at the beginning of every single chat? as it encounters human colon, the first token it sees, always. That was like, yikes. So I definitely think we should be putting a lot more into this. And my best guess is we probably won't reach a confident position on whether there is anything it's like to be an AI. And I'm just so confused about all these core questions around Does the substrate matter? How much does it matter? We didn't have time to ask Naveen, but an interesting question for him would have been, do you think your electrical underpinnings are more likely to generate consciousness than a GPU? I have no idea what I should even think about that, but it's clearly like you can do stuff to the brain, very physical things that change consciousness in fundamental ways, as simple as— drink a drink of alcohol or use anesthesia or take a hallucinogen or whatever. So clearly there's clearly a very real and grounded physical relationship between the chemical processes that are going on and our subjective experience of it. How would that translate to an analog computer versus a digital computer? I have no idea, but it stands to reason there could be profound differences at that level too. So I don't know, I guess you asked me how I felt. I think I mostly just feel confused about this topic through and through. But I do think that— I guess the last thing I'll say is I do think the arguments that people have to make to dismiss that this is something that we should concern ourselves with are getting, on the one hand, just more obstinate. There's just people who are just stomping their feet and denying that this is something we have to concern ourselves with, which I don't find compelling. And if they're not doing that, then I feel the arguments are just getting more and more arcane. And there's a lot of, like, well, what would you expect sort of stories that I really don't think hold up to scrutiny super well. So I do think the evidence is growing quite quickly that this is at least something that should be taken seriously.

So speaking of that, Congressman Ted Lieu just this morning, my take, linear algebra equations will never be conscious. Random number generators will never be conscious. At a very basic level, AI is math. AI can act like it's conscious, but it will never be conscious. And adding more math to AI models doesn't make it any more conscious. That's a congressman this morning. So I wonder to what extent, you know, the thing that strikes me is that, you know, normal people kind of assign like a lot of subjective experience to their dogs, pets. Like people are like, you know, my dog loves me, my dog is feeling pain, my dog is, you know, et cetera, et cetera. And I think, you know, instead of challenging the idea of AI being conscious, we can just treat it like kind of a dog that can talk, right? It's not a human thing, but it has these things that we perceive externally and which we can kind of characterize. And that's pretty much it, right? I think there's a piece of us which gets threatened by having it being able to talk and also having, you know, assigning it subjective experience. Maybe if we just, you know, treat it like a dog, like we, I think dogs have subjective experience. I think dogs are conscious. Obviously, like, you know, if you hurt them, they feel pain.

I think that's very, yeah. I was actually told as a kid once, and I remember this for a long time. I still remember today, obviously, but I kind of adopted it or I, you know, took it on as like that was the truth for a long time that dogs were not conscious. And I look back and I think, how could anyone really have thought that? It's a very strange intuition to me now. I don't know how you could— and this was a dog owner, by the way, who told me this. So I think that's— people are very capable of telling themselves all kinds of stories. But yeah, I totally agree. I don't know how one would really look at a dog, interact with it, and think that it's not having some sort of experience. Especially given the substrate overlap with our own, right? I mean, you know that it has a brain, you know that it has neurons that are firing and connecting with each other. You know that it has like at least a decent amount of overlap in terms of the, you know, the sort of hormonal signalling type stuff that goes on in the brain. Given all of that, it's like really hard to imagine how it's not having some experience. And the big reason I doubt it for the AI side is That all that stuff that, you know, in some unknown mysterious way is giving rise to consciousness is like, that same stuff isn't there, broadly speaking. So that makes me a lot more uncertain than I am on the dog case.

Yeah, indeed.

Nathan, a pleasure.

Yep. Looking forward to doing this again.

If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of a16z, where experts talk technology, business, economics, geopolitics, culture, And more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.