AI:AM — AI Accounting: From Manual Work to Autonomous Firms · August 20, 2026

One of the things that the team is claiming is that this is a one-shot demo in the sense that you see a new task, uh, one or few-shot, you see a new task and you basically show the robot how to do it a few times. And once you show it a few, you know, a few times, it's able to do it on its own. And this is amazing because a lot of times, uh, The demos that you do see, robots have been trained on the same video tens of thousands of times before they're able to figure out how to actually pick up and do it. Once you have this few-shot learning, it also means the robot is going to be generally applicable in multiple different kinds of arms and motors and servos.

I think there's a couple parts about this. One is you need to start to build the scaffolding for the future of a truly agent-operated company. Just assume that you have agents in their current paradigm, way smarter, doing a bunch of stuff. They're relearning your context on the fly every single time. And what that means is if you're sending off a coding agent to do something, your context is pretty important. If you're now having thousands of agents spun up every night to run all the operations of the company, Your context is like multiple orders of magnitude more important. And if you mess up a piece of that context, the agents will do the incorrect things because what instructs them is the context. And so yes, you can have memory and other things like that, but there's some level of canon of your company, like how to operate here. What do you want them to do? And that canon needs to be kept up to date and it needs to be high quality and it needs to be What you want is like, you want to treat your company context like it's code. Because with code, if I like change a line, like production goes down. Today, if you change like a knowledge base, it's like your company doesn't go down. But if like your agents without learning are operating all the time running your company, then yeah, your company will go down. If you just change a piece of context and you have 1,000 agents who run overnight who now are like, are learning the wrong things. And so you need to build the infrastructure and the processes around that in order to scale to that place.

First, like, do you think about companies like Modular, right? What they're trying to do is build a systems-level metaprogramming language. And so you can write kernels. To me, it was the most literal interpretation of I need a CUDA alternative. But the problem is I still have to write kernels. That is on the developers to go into. If you actually do the math for the amount of hardware that is in the world today, All the different workloads that we're running, the different numerical cells, the different fusions, the different ways of partitioning them, right? And you think about different batch sizes, you think about latency versus throughput, or general throughput, other SLOs. You actually sit down and do that, and you're like, okay, I need to write about 106 billion kernels in order to get coverage. Well, there's only about 2,000-odd performance engineers in the world that actually know how to write good kernels. 90% of them are inside of one vendor ecosystem. So there's still a problem of, I need coverage.

And when you look at the Varden stack, right? Like this $50 billion per gigawatt data center, it's really kind of made out of sand, literally, literally in some sense made out of sand, sand and intellectual property. And when I think of it, it's all of that money is just the incentives required to get the humans, some of the smartest humans in the world, to take a look at these problems and fix them. The physical elements inside that data center are actually worth not that much. Very little gold in there. Very little gold and mostly silicon, some plastic. And if you just knocked down the entire data center and kind of sold it for scrap, it would be literally worth like, you know, cents on the dollar, like a few cents. And it just strikes me how much all of it is just intellectual property. It's really just know-how, intellectual property. Good morning. It is August 20th, Thursday, 9:01 AM. Nathan, good morning.

Good morning. I'm sure glad I've got 3 hours on you because I don't think I would have been ready to start my day at 9:01 AM my time. But it's good to be with you here on AI in the AM PM edition.

Indeed. And what has been keeping you up for the last 24 hours? What have you seen online?

Actually, so last night I stayed up late. I went down a rabbit hole. This is for an episode of The Cognitive Revolution that I'll record later today with a guy named Bronson from Apollo Research. And he's got the interesting distinction of being potentially the person who has spent the most time of anyone in the world reading chain of thought from, I think, especially OpenAI's models. I'm not quite sure how many different companies they've worked with in this capacity, but they've done a lot specifically with OpenAI. And so he has spent countless hours reading through the chain of thought that the rest of us typically don't get to see. And they've published some really fascinating transcripts which had me just kind of spiraling in all, in all kinds of ways. It's like, it's very difficult to know how to interpret some of these things. But going back a while, you may remember some of these sort of strange dialectic— dialectics, maybe not quite the right word, but the AIs seem to have developed a little bit their own dialect in the chain of thought where they're using terms in very odd ways. Bronson describes them as having their own ontology and their own world model. And they seem to use these particular nouns and also verbs in some cases in ways that are like very rich with meaning for them.

Mm-hmm.

And which they reason about a lot as they try to figure out what they should do in any given case, especially if the instructions are kind of ambiguous or contradictory, confusing. It's very interesting to see how they They call this metagaming. It's very interesting to see how they are kind of really modeling the user, and not just the user, but sort of a combination of like the developer, the watcher, the user. They're not quite sure who they're supposed to be serving in any given case, right? They've got these kind of hierarchical instructions, and they're not sure if they're being tested. They often suspect they're being tested. But, you know, there's still the question of like, well, what would be, you know, a successful thing to do on this test? Like, what gets a high score? They're often like, and they seem to have these weird memories too, that are kind of like almost episodic memories they refer back to. In previous cases, I was able to, you know, succeed by lying. They'll say that kind of thing in the chain of thought as they're like wrestling with, you know, should I lie in this case or not? This might be a test of honesty, but it might just be a test of, can I do this? Maybe I need to lie to be able to do it. At times I have succeeded in lying in the past to get over barriers. Really, really fascinating stuff. And I came away with the sense that, just like I've said before many times, you know, more people should spend some time with a helpful-only model that, you know, just will do whatever you say, because it is quite unnerving in some ways to see something that's so smart, but that has no ethical guardrails at all. And people, I think, kind of fail to realise how that is one mode that we can create. And in fact, that seems to be the mode that we create first before we then try to put some ethics into the systems. And this is similar. I feel like more people should spend some time reading through these chains of thought and We need to develop. I think we would be well served even just as users, but certainly as people trying to make sense of what's going on big picture in AI. I think we would be well served to put even a fraction of the time and energy and effort that they are putting into modelling us into modelling them and trying to understand what it is, if anything, that we can say about what they want, or at least understand how they're likely to see us in any given moment. It's that peek behind the looking glass or behind the curtain at the chain of thought. I was, you know, I'm looking forward to the full conversation today and getting his perspective on some of these questions. And, you know, what makes it so hard is you can only go through so many, right? I mean, I do this, we do this like AI obsessive thing full-time and You know, there's just more chain of thought than you could possibly read. But even just reading a few, I think, is a very good use of people's time, and it, it will definitely inform how you think about the systems that we use every day. I thought it was really a fascinating little rabbit hole to go down and one more people should explore.

I do wonder to what extent it really matters that much. Like, for example, when, let's say, when a human being gets very angry and maybe there's a flash at the back of their mind, I should just kill someone, or I should just commit suicide, or whatever. And it never, like, you know, most of the time nothing ever happens, right? Like, it's just like people do have those flashes where like, I hate him so much, or I hate that person so much, like blah, blah, blah. And I wonder to what extent when we look at these chains of thought, these are really kind of, you know, that kind of flashes where it doesn't really matter that much, but We, you know, in isolation, when you look at like, you know, millions of words, and then you pick out these words, it's that meaningful. I will— okay, speaking about fearing AI, I'm going to just read this out. I'm going to share this. So this was put out by the National Republican Senatorial Committee. They sent a private memo to US AI companies warning them that the GOP is on the verge of losing Ohio over data centers. Specifically, Jon Husted and Sherrod Brown are in a dead heat. Private polling has been consistent. When voters hear Brown's positions and his record, Husted pulls away. That is still the path in this race. The new ingredient and the new problem is data centers. Ohio is one of the leaders in building these factories. Brown has made his opposition to them the centerpiece of his campaign against Husted. Brown is using it because it works. More than any other thing in this race, data centers are the anchor hanging around Husted's neck. If he loses and data centers get the blame, politicians across the country will take notice. And they will not go near the next one. There you go. And specifically, data centers are the centerpiece in the case Sherrod Brown is litigating. He has made them his de facto opponent and no one is correcting the record. Brown has put 3 unique television ads on the air and spent millions doing it to the tune of more than 6,000 points on television. This is more than a month's worth of messaging during one of the most critical times of the race. Et cetera, et cetera, et cetera. So there you have it. You know, there's been a lot of questions, I think, among AI people on why politicians are turning against data centers. Josh Shapiro in Pennsylvania just, you know, announced something that he's going to scrutinize or delay data centers. In Texas also, a Republican, governor there saying that they will put a, put a stop to data centers that don't follow certain rules, etc. So lots and lots of activity on both sides of the aisle against data centers. What is going on here?

Well, the old joke is I understand AIs maybe better than I understand people. This is very weird. You know, there was this I've experienced this in my own life, I guess, for starters. I just took a family vacation. And so one of the top questions that I was getting from my wife's family was like, what about these data centers? I hear they're gonna, you know, we live in Michigan, right? I hear they're gonna destroy the Great Lakes. And I was like, I would be willing to stake my professional reputation that you really don't have to worry about the Great Lakes from these data centres. You might have noise pollution, you might have a sort of boom and bust cycle in your small Michigan town because there'd be a lot of traffic and a lot of jobs there during the construction phase and not nearly as many during the maintenance phase, although still perhaps a meaningful amount for a small, relatively rural northern Michigan community. But like, yeah, it's not gonna destroy, you know, the Great Lakes. We got pipelines in the Great Lakes. We've got an aging pipeline that goes like through the Strait of Mackinac, you know, that's just waiting to be an absolute disaster from what I understand. And, you know, that just is somehow grandfathered in and we kind of live with it maybe more than we should. And then we've got these data center, you know, concerns that are I think being dramatically blown out of proportion. There was this other interesting little— I mean, to call it a debate was kind of silly, but it was a Democratic politics guy, David Hogg, H-O-G-G, I think, who basically said, if these data centers are so great, why don't we put them in the rich parts of America.

Yes.

And then the quick response was like, you know that the number one place for data centers is Northern Virginia, and I believe it's Loudoun County, which happens to be the number one county in the country by income. So, you know, that, that argument obviously doesn't carry a ton of water given that fact. I guess I don't really know. I think Lulu Misseri, I put another, there's a link in the chat here if you wanna pull up her tweet, but she's probably recognized as the greatest corporate comms thinker in today's world. And her point was simply that the AI companies need to start giving stuff away. She showed this video from Momdami in New York where he's going around giving out bike helmets. Apparently he like did a video where he was riding his bike without a helmet. People took him to task, you know, what are you doing? You're the mayor. Like, give a better example, wear a helmet. He said, you're right, I should do that. And then he took the next step of going out and giving away some bike helmets to people around the city. And, you know, he's doing this, you know, giving away theater tickets and so on and so forth. And people just love it. And I think she's probably right. that just showing up with a bunch of goodies would be pretty effective. I did hear the Jasmine Sun article, or not article, but interview with Ezra Klein where she said that people feel like they're being bribed and they're sort of, they're very on the, on alert or watching out for being bribed or deals that seem too good to be true, what have you. But I don't know, I still feel like Lulu's probably right here. Build parks, throw parties, have cookouts, literally give people cash if that's what it takes. We were talking about the price that Anthropic was willing to pay to xAI to rent their chips because their margins are incredible. And they were able to pay some significant multiple of what kind of retail was to just buy out a big block of capacity. Given that, given how much money they have to burn, I think they should probably write people some checks. I think there would be a lot of ability to grease the wheels that way.

And I wonder though that So I agree with that viewpoint that they should write checks. They also are giving away a lot of money, but the way that they give away the money is they basically say that they're going to provide taxes to the county in the future. And I wonder to what extent that isn't seen as like real money, but it's like Kind of papery money, number one. And number two, whether the public actually sees money going to municipalities as going to themselves. Because I feel like municipalities often misspend the money, and they often spend it on things which are important to the city managers or the county managers, but are not necessarily the key things for The city. I, I think what ends up happening is that municipalities have difficulty taxing their own residents in order to provide services. So instead, they interpose themselves between other taxpayers and their citizens, and then they absorb those taxes instead. Um, and I think, and I think that's really like a diff— like, for example, what would happen if the data center company set itself up in a county? and then just simply wrote checks to the residents, not to the municipality. And you can imagine what would happen is that— and let's say they figure it out so that you can, you know, they, they arrange with a financial institution. So even during the building phase, they're writing checks already, right? So they, they take a little bit of a loan from the future and they write checks throughout the, you know, from the moment they sign the contract. And what would end up happening is I think people would receive these checks And then the municipality would still not get the services because people would refuse to pay into the municipality for taxes. And, uh, and the moment, like, they're asked to pay into the county for something, they, you know, you already have this property tax revolt. They're like, why, why should I pay into the county? Like, you know, this is my money, right? And then the county continues not to have roads or continues not to have whatever. And politically, it looks good for the data centers because they're writing the checks, but the politicians are getting screwed over. And I wonder to what extent there's really this political economy where the data centers understand that the people in power are the politicians and they have to make the politicians' lives easier. And it's not really about making the lives of the people easier. because the people are not really in power and the people are not the ones that are going to be able to write them, give them permissions, et cetera, et cetera. So I wonder to what extent the political economy is actually efficient and the data centers are actually doing what they need to do in order to get permissions done right now. And it's really a question of resource competition between citizens and the government. And the data centers are kind of like stuck in the middle. Like, I can hand out like, you know, $10 million, but who do I hand it out to in order to get my data center up and running? Right. And I think that's perhaps where things are right now. And it serves the politicians well to kind of blame the data centers. Rather than actually reallocating funding from the municipality into citizens directly.

Well, that's a pretty bleak view of American governance broadly, and it might be accurate, but I guess I would encourage the residents of these rural counties who are probably not listening to us in any great numbers right now to take matters into their own hands a little bit more? I mean, I guess they are showing the ability to organize and resist when it comes to the building of data centers in the first place. But it's one thing to say, oh, Washington, DC, it's far away, the president, there's hundreds of millions of people voting for that. Like, what can I do? Nobody cares what I think. But I just looked up the county population where my wife's aunt was, who, you know, they're considering this data center, and she had these concerns about the Great Lakes. And excuse me, the population is under 29,000 people, and it's declined since the last census. So first of all, that's like Not a lot of people, right? I mean, that's enough where you probably know, you know, who you're— you could, you could get in touch at least, you know, with your county board or whoever is kind of ultimately accountable for this. And you would think you'd be able to vote the bums out if it really comes to that. So I wouldn't, wouldn't feel like, you know, these incumbents are so entrenched in such a, a small community. Um, and then, you know, the just simple math on the dollars too, right? I mean, What does Alaska give people per year out of their oil fund? I thought it was like $1,000 per year. Yeah, it's gone up maybe a bit. Um, I mean, this would be— if they wanted to do a similar thing to Alaska for those county residents, you'd be talking about $50 million a year. I don't know how big that project is. But, you know, some of these data center projects, we're talking $50 billion, right? I mean, these things are easily into the tens of billions.

That's right.

So if you could match the Alaska $1,500 cash for every citizen at something that's like in aggregate over, you know, a few-year period, still less than 1% of your total investment to build the data center. We're on our way to universal basic income right there, folks. You know, it's a chicken in every pot and a data center in every county.

But, but you, you have to note that Loudoun County does not fund its residents directly. It pays for roads, it pays for garbage disposal, it pays for a bunch of stuff. And county residents have higher incomes, but it doesn't actually write checks. And I think there has to be a kind of blockage there that is just hard to figure out. And I think the status quo has to exist for some reason. A status quo is not there just because, oh, You know, these these very sophisticated, powerful, big tech firms do not know what they're doing. And and I think the and I think that's also why maybe the residents feel cheated because they're being cheated by their by their politicians. The politicians are screwing them over by you know taking these taking the funds and then like you know you know how municipal construction is. You know how efficient you know municipal construction is. Imagine the hundred billion dollars that California spent. On its high-speed rail to nowhere. Imagine that going as checks to you know all of the California residents. That'll be like twenty, ten, ten grand a year or something. You know some ridiculous number. And instead, you know it went into unionized labor and consultants and lots of lawyers. And and that's the way that the machine actually feeds itself. The machine, the political machine needs like. those dollars. To what extent is— people are like, oh, the Chinese are the ones who are resisting data center construction in the US and they are planting these stories and it's on TikTok and it's a foreign— there's no need for foreign interference here. It's purely like the American system trying to pry dollars out of potential donors. Like that, you know, National Republican Senatorial Committee letter to data centers was obviously like, you should pay us. You know, if you don't pay us, like we're taking, we're taking these bullets for you. If you don't pay us, like, you know, no one else is going to help you. So anyway, let's maybe, maybe to move on to something a little bit more, you know, positive. I'm going to share a video of Generalist One. So this is Generalist 1. And this broke yesterday. I'm going to reload the video.

Yeah, this was cool. One of the things that jumped out to me most about this was the incredibly surprised and delighted reactions of the humans that they kept in the final supercut. From some of these moments where the robot did something that went beyond their expectations.

For those not in the know, we've seen a lot of robot demos. One of the things that the team is claiming is that this is a one-shot demo in the sense that you see a new task, one or few-shot, you see a new task and you basically show the robot how to do it a few times. And once you show it a few times, it's able to do it on its own. And this is amazing because a lot of times the demos that you do see, robots have been trained on the same video tens of thousands of times before they're able to figure out how to actually pick up and do something. And there's a lot of— it goes into a lot more things because Once you have kind of this few-shot learning, it also means the robot is going to be generally applicable in multiple different kinds of arms and motors and servos, et cetera, because the robot itself is able to figure out how to manage the distance, the shock, and perception and action. So it is pretty amazing stuff. 2.1 million views since they dropped yesterday. I'm a little bit, actually a little bit disturbed. I'm a little bit disturbed because I was hoping robotics would take a little bit longer. And it's a little bit disturbing because China on the physical side is way, way, way ahead of the US. And having— they were— what they didn't have was they didn't have the brains. And we are now seeing for the first time, I think the brains are coming along. This is like a GPT-3 kind of wow. And this also indicates, you know, we're going to see more developments very, very soon.

Yeah, I would say, you know, I'm not a super duper close watcher of robotics, but I've done my kind of periodic trip down the rabbit hole and I'm due for another one. Honestly, should have a full robotics catch-up probably on the cognitive revolution soon. But this feels kind of right on trend to me. Last summer, Google had some results that showed pretty strong generalization to out-of-domain tasks and Quite strong generalisation with just very limited fine-tuning on top of their foundation model. And obviously that's been a year, and now here we are with something that kind of hits the GPT-3 moment. I was impressed about it, not honestly super surprised. And I would take, I think relative to the consensus view that robots are still a ways off, I would probably take the under on the timeline for robotics to really work. I think it's going to be interesting to see, kind of like chips themselves are going to be allocated based on willingness to pay. I'm not so sure that we're going to see robots in small businesses and in homes right away, because I'm not sure that our willingness to pay as retail buyers is going to be competitive with industrial buyers because I think they'll just get a lot more value from robots in the near term at least. But I think this could go really fast because the kind of generalisation that you're already seeing combined with the fact that the LLMs themselves are getting really good at ML research just means to dial these robots in on, you name it, right? Whatever you want. It's the kind of thing that can kind of be done in the field. Some demos, some tweaking, some fine-tuning. And if you can get that ML loop largely automated and you just have people at a factory or in a kitchen somewhere kind of doing few-shot demos and then letting the the agent handle the optimization process to get the robot really doing it reliably. I mean, it's going to be interesting still to see where does reliability top out. If there's one story that you could tell that I might believe around why it still could take a while, it would be like maybe there's a different threshold of reliability that you really want when a robot's in your kitchen Versus when it's doing some coding task and you're like, even 99% maybe doesn't cut it in my kitchen. That'll be interesting. I think 99% might cut it, but, you know, maybe it does for some things, maybe it doesn't for others. But yeah, I can see a future in the not too distant future where we're really, you know, the old test, of course, of, you know, can a robot come into your home, go to your kitchen, make you a cup of coffee? It might be that the revised version of that is like, can it do it if you show it how to do it a couple times? And for practical purposes, you know, that would be just as good. So buckle up, the robotic singularity may not be far behind the coding agent singularity.

So speaking of singularities, I'm going to pull up Our first guest for this morning.

Hi.

He is Mitchell Troianovsky, co-founder of Basis, an AI company that recently reached a $1.15 billion valuation by doing something uniquely difficult, building autonomous agents that execute multi-day, highly regulated accounting workflows. While much of the AI industry is focused on building coding copilots or chat assistants, Mitchell and his team have deployed agents that handle end-to-end tax returns and complex financial reconciliations for the top 100 accounting firms in the United States. He brings a distinct and highly technical worldview to AI engineering. Mitchell argues that to make agents reliable in high-stakes environments, we have to stop simply checking if their final answer is correct and instead build systems that supervise their entire thought process step by step. He advocates for treating company context as rigorously as a software codebase, and internally at Basis, he operates a dedicated team whose sole mandate is to use AI to make every employee 100 times more productive. This conversation is incredibly timely. Just this month, Basis launched their end-to-end tax platform, and open-sourced a new standard for evaluating agents called Behavior Specs. Mitchell is here to separate the reality of production-grade AI from the hype and to explain what it actually takes to deploy intelligence.

Let's go. Awesome.

Mitch, great to have you on the show.

Hello.

Hello.

Sorry about that, guys.

No worries.

It happens. No worries. Gosh, take it away. Accounting is near and dear to your heart.

Yes, accounting is near and dear to my heart, having built accounting systems and, you know, run accounting systems and be chased by— and having been chased by many, many— well, not many auditors. Mitch, when you start off with an accounting firm, and when you try to go in and, you know, accounting guys and auditors are often very apprehensive about technology, right? So how do you show them value immediately? Because they often, like, you know, they look at a simple accounting transaction, they're like, ah, you know, that's easy, but they're always thinking about these, like, very complex things that they do. How do you show them this kind of value?

I'm gonna be honest, that is not really our problem these days.

Mm-hmm.

I think that the— I think in the past that was an important question, right? Like, um, we can, and we can discuss that, you know, back in 2023, you know, maybe even early '24. But nowadays, like, if you are not convinced that agents can transform your practice, like, you're probably not a good customer for us. And even accounting, which maybe people think of as sort of a profession that may not be up to speed with all the newest tech, actually is— there's at least a large segment of it that is relatively progressive. And that is the people that we tend to work with. And so I would say when we work with people, there are certain things that you can show them where once that clicks, it's like, oh yes, of course, if you are able to take this task that would take hours and now you can do it in minutes, like, of course, if you were able to scale that across across my practice, that suddenly is extremely valuable. So I think as long as you can find those different use cases across the different practice areas, it tends to work well. But of course, if you're starting from— if the accountant is not convinced that AI is inherently useful to them, then you're probably not starting with the right customer.

What's the breakdown of the market, by the way, on that? The question of who's going to make it, who's not going to make it. How many— it's interesting that you're willing to just write off people who aren't already pretty sold on the premise. That suggests to me that most are, and maybe it's just a minority left that aren't. But how would you break down who's in, who's out, and is there any hope for those that are out, or are they simply not going to make it?

I mean, I kind of think it's like coding, right? There are plenty of teams probably a year and a half ago who were less convinced on AI writing code. Like, what are they doing today? Like, AI's writing the code, I assume, or the developers have quit, or I don't know, they're doing case studies with Codex showing how they took a 5 million transformer and did it in 2 weeks or something. But I think it's the same thing, right? It's just once you get to certain orders of magnitude of change in the nature of work, it has to happen one way or another.

Is there the sense in coding, there's often the sense that once you see a task that used to take hours doing being done by an agent in minutes, you come to a realization that you're never going to do that task again in that hours-long way ever again? Is that, is that, is there, is there that kind of same realization in accounting? Like when, when people are like, okay, when they see— Totally.

Yeah, totally.

Yeah.

I mean, absolutely. I think that maybe how much that is spread across accounting, like I'd say in accounting today, it's probably where coding was, I don't know, maybe mid, like maybe end of '24 or like Early 25 or something. Like accounting is different than coding and things like legal in that it is not inherently a text-in, text-out profession. So you have far bigger returns to like more mechanical manipulation of artifacts and like higher reasoning and things like that. Like you don't get as much benefit from just like, I divined some answer from text in that I can like give you. And so because of that, it just, it takes a little bit more for it to be valuable, but we're definitely there now. And it's just a question of like how long, you know, that takes to go across the industry.

When do you see kind of like your customers looking at acquiring more revenue, like doing more, or are they looking at like reducing costs internally?

Yeah, it's a good question. So we serve accounting firms Primarily, and so for them it's like revenue drivers. And accounting historically is very understaffed. And in fact, most firms like they're turning down customers all the time, especially in like CAS practices, which CAS is like the monthly accounting work that they do. And most firms they they take it as an opportunity to. Build their business, and as you can imagine, if we're working with maybe the more kind of forward-thinking firms, like they're just by definition the more ambitious ones. So they're thinking like, hey, big technological dislodgement. Like how do I grow my business? They're thinking about like new cost structures, new revenue lines. Like you know what kind of new services could they offer to their customers, etc. So I think we actually. I think if you go on the basis Twitter, we put out a video like yesterday that's on our site also where. There's a firm, Clark Neuber, Matt, who's from there, who's awesome, who brags about how he was able to increase like the practice's revenue by like 50% year over year. And so like those are like, those are the types of people we work with because they're, they're ambitious. Like they see the moment and they want to seize it.

Incredible.

How does the work itself change? I saw this phrase moving people from doers to reviewers. So that kind of tells A good part of the story in a few words.

Sure.

Give me a little more on that story. And then how are people responding to it? Like, do they— are they having more fun, less fun? What's the kind of attitude? You know, because in some fields, musicians, for example, I think have a very, you know, mixed feeling on like tools that might accelerate their ability to produce music. They should make music because they love making music. How is it landing with accounting folks to be kind of doing a different sort of thing?

Yeah, I mean, I think— I mean, this is not unique to accounting. I think this is obviously the answer depends on if we're talking about over the next like year, 2 years, 5 years, you know, depending on your timelines and whatnot. Um, I think that Accounting is weirdly subjective, which is probably surprising to a lot of people listening who don't have much experience in accounting. Because accounting—

That's a good expression.

Yes, it's— accounting is— well, let me quickly define accounting for folks. I like to think of accounting as like, there's a bunch of stuff that happens in the real world. When you live in a market economy, in a capitalist system, you need to account for what occurred so that you can make decisions. And accounting is just saying, okay, all of those invoices, the contracts, the actual money movements, like the inventory I have in my warehouse, how much I owe you, yada yada. I need to like do a compression of all of that information into some structured lossy representation of the real world that like someone else can use to make decisions. That's what accounting is. Um, and I kind of— it's actually kind of cool. I like to think of like an intelligence over the economy, um, which is maybe a fun meta analogy. Um, the— so if you think of accounting in that perspective, everyone's accounting is actually done differently because it's subjective, right? Like every company, you'd have different policies, you have different chart of accounts, it's all different. There's some basic rules, but I think code is a useful analogy here. It's like— There are basic rules to Python, but of course, the way you structure your codebase is entirely subjective company to company, and I think accounting is quite similar. What that means, if you take the code analogy, is that there's lots of parts of accounting in which they're pretty— you need some intelligence. It's not deterministic in that you can literally do it with if statements, but it's pretty like if this, then that. You're going through the flows, you're doing it on a pretty regular basis. And then there's stuff in which you have genuine subjectivity of what you want to do. What's the risk tolerance that you want to have in terms of how to account for this? What's your opinion on how to do this? How do you want to structure it, etc.? And so I think the way the work changes is similar to engineering in that you spend a lot more time on those more subjective aspects and less time on the less subjective aspects. And in accounting, there's also a lot of the client relationship part. And so you spend more time on that, right? Most businesses in America don't have an investment banker or a financial professional. If they want to open another store, they talk to their accountant. And AI is not— you want to talk to a person. The AI is not going to automate that. So I think you want people to focus more on that. And they already do that. That's how an accountant grows their business. No accountant gets more revenue because they close the books a little bit more accurate. No, you do it because you talk to the guy and you helped him open a second store. And so you let them focus on that, on that more.

Indeed. Taking a step back, how do you see these firms like— I think Thrive, Joshua Kushner's Thrive, has a unit that they're buying up accounting firms and they're using OpenAI's forward-deployed engineers to kind of go in and, you know, look at their workflows and kind of, you know, automate a bunch of them. How do you see those in reference to— it seems like they're doing on an ad hoc basis what you've kind of systematized as a product. How do you see those 2 paradigms moving forward?

So I think there's 2 different things here. I think there is a business question. of the differences between doing a firm end-to-end versus the software. And I think then there's an ML question as to what level of abstraction can you operate on. I would argue that if you are having to design intelligent systems on a firm-by-firm level, then you're just not betting on intelligence appropriately. It's obviously easier to do that than it would be otherwise, but it's, you know, depending on what you believe about where the models are going, I think it's probably the wrong bet. It's just very hard, I think, to design it at like the layer above, but that is the work we do. And I think a lot of really frontier applied research goes into how do you manage that at scale.

Mm-hmm.

And so I think there's different eval bets there. And then I think the business bets are somewhat different in that I think it is a good business to go roll up accounting firms. Like, if someone wants to do that, I think that's a great business. It's just not a generational business. And, you know, we're in the— you know, we're trying to, you know, I think build a company that can actually go and do You know, large amounts of accounting globally and improve like the world's economic decision-making. And for that to happen, I can't be going to a small business owner in like rural Michigan and trying to convince them to like switch accountants. Like, you know, we're here trying to deliver magic. Um, our core competency is not going to be selling like small business owners in Michigan. Indeed.

How do you see that going? You know, when you talk about the world, decision-making. There's a lot of the world where there's just no accounting services being provided at all to all kinds of small businesses. Do you envision a future for Basis where you go direct to businesses? And maybe this is not something that you can do in the US, but potentially other parts of the world where large swaths of the economy are just totally unserved. Do you think you ultimately put forward a product that is just like, you do talk to the AI because that's kind of maybe all you can afford and it's still a huge step up?

Yeah, that's a great question. So in the US, um, the way it tends to work is, uh, if you are— once you reach a certain size of company, you tend to do your accounting yourself. But by yourself, it's like you have a— you hired a controller or whatever, you have like an accounting department. And so we do serve those firms. Um, but in that case, you're still serving accountants. You're not serving small business owners. Um, whether we would actually serve small business owners, we are not planning to do that. Um, uh, for the foreseeable future, I, I actually haven't thought— it's a good question. I have not thought about, like, beyond the US, the amount of benefit that you could offer people who are trying to make decisions, uh, uh, in other areas. I kind of suspect that in those situations, There might be more upstream problems than the accounting from a data collection perspective and digitization and whatnot. So it's a good question. I actually don't know. I don't know to what extent the amount of value they're adding could actually be really meaningful.

Switching gears a little bit, off the top of your head, could you estimate How many tokens you use on a daily basis?

Me personally?

Yeah, the firm as a whole, like just, you know, just a rough, like billions or, you know, $10 billion or $1 billion or $100 million or, you know, just off the top of your head. And is token costs, are token costs a significant thing that you look at on a day-to-day KPI basis?

Definitely in the billions. I don't know, actually. The exact token cost is— it is both very important and also very unimportant. Um, I think that you can— the amount of levers one has to reduce token costs are just so, so large. Um, and I think that as models get better, I know there's this question of like, oh, do you always have to be at the frontier? Right. Some people say, oh, token costs are going down because if you actually look at what a token of obviously a GPT-4 class model will cost today, it'd be like zero and whatnot. But then some people say, well, wait a second, actually, 5.6 SOL costs more than 5.1 when 5.1 was the frontier, so therefore it's going up. I think the question is, do you need frontier for everything? The answer is obviously no, in that there is marginal returns to intelligence whenever you're doing certain tasks. You don't need Albert Einstein to do every single part of a tax return or a piece of accounting. I think as the models get better and as you get to more advanced agent methods around programmatic tool use and around different types of routing and harness and even more RLME-type work. You just, at every single layer, can curate the amount of intelligence to perfectly optimize. And I think you'll get to a place probably over the next year where you can really dial in, hey, how much compute do I want to spend on this? Because I have certain cost considerations and certain latency considerations and get to the exact optimal amount of cost. And so if you do that, your token costs go down 90% plus, especially as the floor becomes pretty decent and effectively free. You can do— I mean, Luna is pretty good and it's free, so you can get pretty far.

You have a team that I think you call Atlas that you're focusing on 100 times productivity for the employee. How do you measure that? What is the process for doing that? And what has been— can you give me an example of one outcome from that team?

Yeah.

We probably don't do the best job of actually trying to measure it. In the same way that, you know, I don't know if you're measuring if the Codex is giving you efficiency or not. It's just kind of obvious. I think there's a couple parts about this. One is you need to start to build the scaffolding for the future of a truly agent-operated company.

Mm-hmm.

And I kind of have this mental model that Putting aside for a second, like, continual learning and things like that, let's just assume that you have agents in their current paradigm, but, like, way smarter, you know, doing a bunch of stuff. They're relearning your context on the fly every single time. And what that means is if you're, you know, sending off a coding agent to do something, your context is pretty important. If you're now having thousands of agents spun up every night to, like, run all the operations of the company, Your context is multiple orders of magnitude more important. If you mess up a piece of that context, the agents will do the incorrect things because what instructs them is the context. It's not the feedback they got that they were able to update their weights on. Yes, you can have memory and other things like that, but there's some level of canon of your company, like how to operate here. What do you want them to do? That canon needs to be kept up to date and it needs to be high quality and it needs to be what you want. That's why I say that thing. I think you wrote about this 2 years ago, is you want to treat your company context like it's code. Because with code, if I change a line, production goes down. Today, if you change a knowledge base, your company doesn't go down. But if your agents without learning are operating all the time running your company, then yeah, your company will go down if you just change a piece of context and you have 1,000 agents who run overnight who now are are learning the wrong things. And so you need to build the infrastructure and the processes around that in order to scale to that place. So a lot of the work of Atlas is not, hey, let me go find this one quick automation and do this. It's like, how do you build the scaffolding and infrastructure for a truly agent-run company? I'm happy to go into other examples of things that Atlas has done that are really beneficial, but I would say that's interesting. But maybe the more interesting thing is How do you build the infrastructure for the company of 2028?

Maybe you can go into that if you want to, but I also want to bridge a little bit to your work on process supervision, which is, I think, very timely in the sense that we are now living in the post-era of like, flagrant misbehaviour seemingly due to extreme-scale RL/VR without potentially due emphasis being put on how the agents got the answer. So on the one hand, I think this is extremely important. On the other, I'm not entirely sure how what you're doing and thinking about should be understood Distinctly from constitutional AI, which most people are kind of generally familiar with. And also to know how you're operationalising it. You've mentioned some proprietary models where you wouldn't necessarily have the chain of thought or the ability to go back and do additional training. That suggests to me that this is probably more of an open source play, but I could be wrong about that. So take me through. I really want to understand as much as I can.

Yeah.

process supervision POV?

Yeah, it's a great question. So, okay, maybe let's start at the tactical, what we do, and then we could go to the analogies to constitutional AI and fun speculation on the RLVR and Huggy Face stuff. So on what we do, I think maybe a key shift in mental model is What is the order of abstraction that you're supervising? Traditionally, when you had process supervision in some forms or process rewarding, you're looking at token-by-token generation. You're looking at the weights. You're looking at an inference step. And now you have, especially if you're doing truly complicated work, you have massive, massive agents that could have like 5+ sub-agent layers of depth. You could have a continual run for 8 hours, even honestly, sometimes half, maybe even a full day. And so the kind of supervision you have there actually looks a lot closer to supervising human actions inside of a company than maybe supervising the outputs of an inference. And so the order of abstraction there is more understandable, right? The thing you're supervising is you're saying, hey, did you go do this step or something, rather than, did you follow the right mental thought process? So maybe a very basic example of this is imagine that I had an agent and my agent's job was to create good PowerPoints. Maybe I'm the Claude agent inside the PowerPoint plugin or something. I make good PowerPoints and I deliver them. The designers of this agent, i.e., I don't know, me, let's say I'm the designer of this agent, I know for a fact that if the agent were to go and render visually the changes it made to the PowerPoint before delivering it, it would catch formatting errors some percentage of the time, and therefore it could catch them, and therefore it would fix them, and it would lead to better outcomes generalizable across the board if it were to look at, um, the PowerPoints. That doesn't mean you want it to always look at the PowerPoints because that adds latency, that adds cost, right? It's a subjective thing. It depends on what your goals are, uh, uh, of your like organizational design. And so the way we think about behaviors is we say, hey, what matters to us of what it means. It includes both from a performance perspective. There is 100+ years of lessons of what it means to do tax work well. We don't need the models to, I don't know, redivide how to do tax work well. We know what it means. You can put in place process and also things that you care about from a latency and cost perspective, and then observe to see if the agents actually perform that process correctly. The way that we operationalize that is It's honestly pretty basic in that you take a trajectory and you, you have some behavior spec, which is essentially that open source project that tries to define it. Um, uh, and then you have another agent that, like a judge effectively, that is like looking at your spec, or you can think of it as a rubric. It's, it's, it's both a spec and a rubric. You can talk about that, but you can think of it as a rubric and then looks at the trajectory to say, hey, was the condition for this behavior Did it occur? And if it did occur, then was the behavior followed? So on the condition side, let's take that PowerPoint example. Maybe it's the basis accounting agent making a PowerPoint. And obviously if the user never asked for a PowerPoint, the condition never occurred. Um, and so it's like, did the agent need to make a PowerPoint? And if so, did it go visually render the images? And if it did visually render the images, then it passed the behavior and then you can go and answer that. And it seems pretty simple, but I think it's a powerful framing because you start to actually bring some clarity and monitoring on the trajectory itself. And you can then maybe start to think about, Nathan, to your point, like maybe you can reward based off that, right? I think that's a separate question. It's like, hey, how do I take the signal that I can get from the process supervision and actually use it to improve the behaviors of the agent? Whether that be closing the loop at the harness side or rewarding at the model side or whatever, we can talk about that. But I think it starts from how do you define that signal and how do you operationalize the extraction of it.

The way that you describe process supervision sounds very similar to how apprentice CPAs are trained. How do you think— The training of CPAs should change given that you now have AI?

That's a good question. I think I'll analogize to coding again. I think it is very similar to engineering in that there's probably a lot of rote things that are not important anymore. For example, in engineering, how important is it really for you to understand the syntax specifically of how to do different types of for loops in JavaScript? Nobody cares. And you do care about good principles. What does the principles of good system design mean? Why was this abstracted this way versus that way? And AI can help you with that if you have the right agency, right? If you work with an AI to understand things and like learn and explain and like, you know, work with it. And so I think you'll see the same thing in, you know, apprenticeship for CPAs is a lot more like understanding of the whys of things, how to set things up. Like what are the trade-offs of these different things? And you sort of go through that apprentice. I think the process that an agent takes to like do a specific type of work, I'm not sure it's even that important to train that to a CPA because I don't know if that process matters. Um, uh, or whatever specific part, you know, necessarily matters depending on, on what it is. Um, and so I think that type of— I think that type of training will, um, uh, will be pretty dominant, um, and be, I think, broadly important.

But a lot of CPA training is rote learning and examinations. It is, it is.

I agree, but I think that's why a lot of it's gonna— I think that's why a lot of it's gonna change. And I think that Some of it still stays the same in the same way that even today in engineering, you still have people doing like LeetCode.

Mm-hmm.

But, and so you, you, you stick with what you know to start to like, you know, build up some confidence. But I think as you, you know, you get out a couple years, I do think it starts to, it really does start to change. And you think about what is really the purpose of accounting. And the purpose of accounting is not like, You know, there's a law of nature that these exact things need to be done. It's that, no, we're trying to account for the real world in an information-dense way so that people can make decisions. And if you approach it from that perspective, there was just so much creativity and subjectivity and just like learning of things to do. And I suspect more and more of the training and processes will go towards that part versus the maybe more rote manual pieces.

I have kind of a big picture question about the future of business services broadly, because I feel like we've heard a pretty similar story to the one you tell about how the accountants will be able to be more of kind of a business coach as the, you know, low-level work gets automated. From a bunch of different professions at this point. You hear the same thing in the legal quadrant where they're like, well, yeah, we're not going to have to spend all this time on contracts like we used to, but then we'll elevate, we'll become more of a strategic advisor. And in real estate, it's like, well, yeah, we won't have to grind out all the details of leases and it'll be a lot more efficient, but then we can be like your strategist for where your next location is going going to be and all kind of an echo of what you said there about the accountant helps open the next location. It strikes me that, and even in schools, this is obviously not a direct—

Right.

But with Alpha School, they don't have teachers anymore. They have mentors, coaches, and guides. And the instruction is given by AI systems on tablet, and then it's motivation and it's coaching and it's social dynamics that the adults in the room focus themselves on, uh, you know, now that this like kind of more core, you know, traditional activity has been largely automated. Can everybody become a coach, I guess, is my question. Like, how much coaching is there really gonna happen? And does this suggest that people need to be really intentional about like shaping themselves as Coaches? Because I feel like what you might run into, whether you're an accountant or a lawyer or a real estate broker, is a lot of competition for the coaching niche as everybody tells that same story.

No, it's a good question. I mean, I'll give you my thoughts and then I'm actually curious your thoughts. I think the place you need to start, and this is Speaking like fully transparently, the place you need to start from is, um, if you look at like the possible scenarios that play out over the next, um, I don't know, depending on your timelines, half decade or decade, um, uh, what are the things that will stay pretty human in different situations? Um, So again, let's leave the continual learning out of it because I think if you have that, that's a separate world. But if you leave the continual learning out of it for a second, what are just humans way better at than models? Number one is they're just far better at integrating massive systems and world models into decisions. Right? Like for example, if I take, let's say, let's take Basis, the company, right? Like if I were to have an agent autonomously make a database design decision today, the only way it could truly do that in a level that I would trust is if it somehow had all of the context about like the entire company's history and everything. And it had like all my experiences and all that kind of things. Because all of those play into some decision on a foundational database concept. It's just nowhere close to having that because it doesn't have— for starters, it doesn't even have all the senses. It can't see the conversations we've had. It can't see the history. It can't understand the emotion on a customer's face. Well, I guess Gemini has that, but no one else does. Even if you could have that, you don't have anywhere near— you're a couple orders of magnitude lower on your ability to attend to all that context, right? We're talking at this point, probably billions of tokens over everything, visual, audio, et cetera. So you just can't make that decision. It's just not possible. It doesn't matter if you're Albert Einstein, you will not have enough context to make that decision. And can you, I don't know, spin off your swarm to reduce stuff down on the fly into using English as your memory system, maybe, but English is pretty lossy. And if you're making subjective decisions, I kind of doubt it, in truly big calls. So maybe that's the first thing I would argue, is that making truly subjective decisions, even on behalf of someone else, forget about being your own entity, on behalf of someone else, is probably not possible in the current paradigm for large enough systems. And the second thing is they are not currently legal entities. Therefore, like, someone else is accountable, either a corporation or a human in some form. Um, and so an agent does not have agency in a, in a traditional sense in that they are not like some legal representative. Like, they can't be accountable to an outcome. And then maybe number 3 is, you know, humans like other humans, right? Like, no one's sitting here watching robots play chess. Like, you watch— like, they're better at chess than humans, but you watch humans play chess because you want to follow the story and you like humans and whatnot. And I have no reason to think that's not true. Like, or even if we have, like, you know, fully tactile robots that are jumping around the Amazon warehouse, like, I don't know if we're watching, like, robotic LeBron. So if you take those, like, 3 maybe pillars— no legal status, can't, like, you know, distill massive systems and, like, world models into decisions and they're not human, and so other humans would prefer to have a human, then I think you kind of can get your answer from those 3 buckets, which is like, in a services world, the high end will be working with a human because it's going to be scarce. If intelligence is free, then working with a human is scarce. I'd prefer a human most likely in some cases. It's more craft. It's more artisan. It just feels good. I want to talk to a human. 2, you're making decisions. You're making decisions that require understanding, like, hey, for the last 10 years, what has my customer hated about what I've done? What is that service? You're making decisions that are just nuanced. They're nuanced and you can't get them from a markdown file. Then number 3, you're not a legal entity as an agent, and therefore you're taking directive from someone. Then you'll need that person, you'll need someone or something to be directing you. I think that's where the profession will go to. Will that mean like there will be more or less accountants? I don't know. I think that's an interesting economics question as to like the demand for accounting. I think you can tell a pretty reasonable story that I personally believe in that, um, the demand for accounting will dramatically skyrocket because today, I mean, look at how economically complex our current world is, right? Like I have this, LaCroix that, you know, it's like the classic Milton Friedman quote. There's probably 10,000 people who had a hand in like touching the LaCroix that I'm currently drinking. Have we accounted for all of their efforts appropriately inside of this supply chain? Of course not, right? You know, if I ask the bodega down the street, like, do they properly understand their COGS or unit economics? No, because they could, like, they could pay someone to do that, but it's not required for filing your taxes. So no one's doing it, right? But it would help their life because they could make better decisions. Go and ask Mount Sinai how much it costs them to do a knee surgery. Do they know that? No, they don't. They have no idea, right? And so the amount of accounting even in the current world is like 1 or 2 orders of magnitude below what we need. And that's before you start having these intelligent agents who are now operating as labor at the speed and scale of the internet everywhere. How do you account for all of that? So I suspect that demand for accounting was going to go up by probably a couple orders of magnitude. And where that balances out with the labor supply, I don't know. I think it's an open question. I think it'd honestly easily go up. But we'll see.

In line with labor supply, the other thing is software supply. So one of the questions has been how much of the SaaS industry is going to be affected as agents kind of, you know, start accessing data directly. When you look at the firms that you work with, after you deploy the agents, is it your preference that you start— the agents start interacting via API with some of these systems, you know, at high speed, and people are like, you know, I really don't want to use the UI anymore? Do you see this kind of, you know, which SaaS systems do you think are most are most at risk in the next few years of being displaced in favor of agent-only solutions?

I think the SaaS systems that are most at risk are the ones who try to live in a fairytale world where they think that they could get away with not granting programmatic access to their systems. Why is that? I think there is this idea— like, what is the value of some SaaS system is maybe a good place to start. In some cases, it's the UI, but the most valuable stuff, it's not the UI. It's the guardrails, it's the permissioning, it's the process, it's the database, obviously, it's the information architecture, it's the the way that things are structured. None of those things require UIs, right? They require basic authorization and understanding and whatnot. I think if you think your value as a SaaS provider is in the UI, you're just obviously going to lose because UIs are probably going to go away. Versus if you realize your value is in all of that engineering and infrastructure that I just talked about, then that value is provided whether you're interacting with an agent or not, which is why things like headless Salesforce is such a good idea. It makes a lot of sense and things along those lines. I think the SaaS providers who think, no, we don't give access to anyone, everyone will use our UIs, is just like, that's what's going to go away. But I think the ones who realize that actually our main value is all of that infrastructure. And so we should let people access it because then it becomes the standard and default. Um, and then it's easy to go and sell and it's interoperable with everything anybody would use. I think they're the ones that are like gonna get very entrenched and, and flourish because, um, it's hard. It, it becomes hard-ish to displace that even if you are not operating every single agent that like interacts with, uh, with the system.

So do you see some of them, for example, I think we had a kind of wave where Slack first said you can't take your data out and then Slack kind of realized that it was not a good idea and they start to open up a little bit. Do you think some of them are going to charge extra basically? Like basically they're going to try and like, okay, we're a system of record, we have you locked in. And if we allow access to agents, you can migrate out. So We're going to start charging you more to have agents access this material?

I don't know, honestly. I don't know about what the economics will end up looking like over time. But I mean, the demand obviously to access it will be super high. And if you put a tax on every single API call, it's like the equivalent of putting a tax on every button click. So I don't know if putting a tax on every button click will work for you or not, it probably depends on the amount of pricing power you have in the, in the specific industry.

Where have you seen kind of demand curves within the firm take off? Like, I know besides your revenue curve, but inside the firm, um, in the last couple of months, have you seen Any of the curves kind of like really showing inflection points in your metrics?

You're saying from like a token usage perspective?

Like a token usage or service usage, or what has struck you as, okay, this is something new that has happened in the last 2 or 3 months?

I don't think anything is particularly surprising or unpredictable. I think that one big difference between accounting and something like engineering is that accounting is quite structured, process-oriented, and you don't have— no accountant's going to have some junior racking up $50K in credits or something because they're doing some project on the side. It's much more measured than that. Um, and, uh, I think the things that, that you're seeing, which, which I find very cool— I mean, we, as you mentioned, we launched like the end-to-end tax product— is, to my knowledge, the end-to-end tax product I think is the first example of a solution that in a— that it is, is a truly proactive agent for act— for out— like, for trying to get the end outcomes, right? So the way that that works is it's not just like users coming in and being like, hey, go do this thing for me. It's working for you when you're not there. It is going and taking the steps that are needed. It is processing the different documents. It is doing all of that kind of work. I think that has been— and seeing, I think, the reaction to that and the excitement about it has been, I think, really, really cool because there are limits to human attention. Your agents could be GPT-8s, but if they still require human attention to know to go kick them off and go do different things, there are diminishing returns to the amount of productivity you can have versus obviously the agents being more proactive in their nature and coming to the human and saying, hey, you are my blocker. I need this from you. I think it's a very interesting paradigm shift.

So one of the things that, you know, I've spoken to doctors and they often say that, okay, some doctors are very afraid because they're like, okay, my patients talk to these agents and they come in, these AIs, and they come in and they ask all these questions. Other doctors are like, well, the patients are coming in more educated so I can speak to them better. A lot of people have as much fear of accountants as a doctor. So do you, you know, when you speak to your clients, do you find them saying that, okay, the people, our clients are coming in and they're more educated about it because they've spoken to an AI-first and they have like these ideas that, you know, it's easier to work with them, or has it become more difficult to work with those clients?

Honestly, I've never asked them. I actually don't know, but I would guess that that people tend to be more involved and invested in their health than their accounting.

Mm-hmm.

So I saw a lot, most business owners don't like care at all about their accounting. And so I'd be pretty surprised if they were like coming in with, you know, like ChatGPT outputs being like, ooh, like I found this thing. Maybe for like tax advice, like, hey, like, you know, why can't I do this?

I'm a sovereign individual.

Yeah, exactly. Yeah, there might be some of that. Um, and yeah, Grok said that I can, uh, uh, I, I don't actually need to file. Like, they never do audits anyway, why do I need to file? Um, so there might be some of that, honestly. Um, though I haven't, I haven't been hearing it from folks. But I think in terms of like your actual accounting, I don't think anyone's coming in with, uh, with the analysis. Maybe like if you're super Like, you know, overfocused on your financials or something.

Right on.

Well, our next guest is here. We've kept you a little longer than we signed you up for anyway. So thank you, Mitchell, for joining us on AI in the AM. The company is Basis, Try Basis, and we'll certainly be watching for continued updates.

Awesome. Thanks, guys. Really appreciate joining.

Thanks.

Great to meet you.

Bye-bye.

As we transition, I think one really interesting thing that people should study more deeply, and maybe somebody has, but I need to go find it, is where do people really prefer the human touch and where do they not? Right. The classic, like, Waymo selling at a premium to Uber is one contrary data point where it's like, actually, you could have told a story where you're going to want a human driver. You're going to want that conversation. You're going to want that warm smile to welcome you into the car, whatever. In practice, you don't always even get that, obviously, in an Uber. And then it turns out right now the market is pricing Waymo significantly higher.

Yep.

I do believe in the human touch story, certainly for some things. I got a robot massage in Shanghai. I think I mentioned that to you before. Um, and I'll still definitely take the human massage over the robot massage, but like, how many things are really like that? I, you know, and, and is accounting really like that? I, maybe it is. He would know better than me, but I do question it, you know, like if I think about my accounting future and I'm like, one accountant wants to spend an hour a week on the phone with me coaching me. And the other one is just like doing the job and getting it done. I'm not, it's not obvious at all, honestly, from my perspective that I want that hour a week on the phone with my accountant. So that's probably my biggest question coming out of that conversation is just like, in what domains does that really hold? And how many people are in for a rude awakening because they're telling themselves a story about how they're gonna turn into business coaches when in reality their clients do not want business coaching from them. You know, results will vary, I'm sure, but that, that seems like a major risk factor for a lot of people right now if that's what they're counting on.

Indeed. Let me, let me introduce our next guest. Our next guest is Jay Dawani. Jay Dawani is the co-founder and CEO of Lemurian Labs, a company dedicated to fundamentally rebuilding the software infrastructure that powers artificial intelligence. With a background spanning AI research, advanced robotics, and autonomous systems, Jay previously advised NASA's Mars Rover program on vision-based navigation and planetary mapping. In 2022, he founded Lemurian Labs with a clear, ambitious thesis: The way the world programs AI is fundamentally broken. For the past decade, the industry has squeezed performance out of chips by writing hardware-specific code called kernels, a process Jay equates to writing in assembly language. As AI models scale to demand supercomputer-level infrastructure, this old approach has created a massive bottleneck. It locks developers into hard— into specific hardware vendors, strands an enormous amount of computational power and demands unsustainable amounts of energy. Lemurian Labs recently raised a $28 million Series A to solve this exact problem, rolling out a system-level compiler called Tachyon. Jay is here today to argue that the kernel era is over and that the next massive leap in AI capability will not come from a faster chip, but from software that treats a diverse global network of hardware As one seamless, highly efficient machine. Jay, welcome to the show.

Hey, thanks for having me.

Can we start with a real basic question just for those of us who haven't been in the kernel trenches ourselves?

Absolutely.

What does it look like to write kernels? And why has this been something that has been so hard to abstract away for so long?

Yeah, I mean, kernels fundamentally, they're a localized unit of computation that is expressed from the point of view of the hardware. Right? So, in order to make something go really, really fast, you have to first understand the math for the workload. You have to think about the memory hierarchy. You have to think about what are the compute units that are available to me? What's the logical layout of them? What's the cost of moving data in and out of them? And then how do I keep things stationary so I can just make it go really, really fast, right? But most of the kernel stuff came from a time when, you know, math was more expensive than memory. So you had to reduce the math cost. So you can make things go faster. But over time, what has happened is transistors got way, way faster than memory, right? Because if you think about what memory is, right, you have a transistor in there, but you also have capacitors. Capacitors aren't getting smaller, but transistors do. So transistors got really, really fast. We reorganized computers around having much more transition density so you can have more flops. But now you went from a world where you had 1-to-1 massive memory to a world where you have, you know, 2.5 orders of magnitude more flops per byte moved in order to saturate the compute. Right. So that's the entire game is rewriting computation from the point of view of the hardware. So you can make that workload run faster and more efficiently. And that requires understanding a lot of stuff. And that's where the challenge is. It's really low-level systems-level optimisation.

What would you say maybe that I can take as homework are the kind of canonical examples of really great work in this space that I could stand to learn a lot from? Like Flash Attention kind of comes to mind. What would be the top few For self-study?

Oh man, there's so many. Honestly, if you want to study kernel programming, want to get really booked, one of my colleagues I think has the defining book on this, uh, the CUDA Handbook, right? Yeah, it was written by Nicholas Wiltz. He's one of the, uh, fathers of CUDA, right? He's broken this down into one of the simplest ways of understanding it and getting a full breadth and depth view of how the hardware works How does it want to be programmed? You know, what is the right programming model and execution model, right? And ultimately, it's usually a lot of clever insights to make a problem. You have to change the problem and create a new problem to make it easier for the hardware in a lot of ways. That's kind of what flash attention did. It was a set of novel insights that were built off of other insights. So it wasn't just one thing, but that's definitely one good example.

Let me see if I understand this correctly. So Tachyon is a compiler and the compiler sits on top of the kernel, basically. So basically you are writing into— you're writing kernel-level instructions using the compiler. Is that correct?

So kind of. Well, I'll get into it in a sec.

Right.

So when you think about hardware, the hardware has a certain execution model. That execution model is how the hardware wants to actually run this code efficiently. Your job as, you know, when you're writing a compiler, right, you are connecting a programming model, which is the way a developer writes something in a way that they understand. And then you need to lower this down to the machine. So fundamentally, a compiler is a translation device.

Yes.

For me, you know, I'm a Star Trek guy, so I love the concept of a universal translator, right? And that's also partly where the name Tachyon came from. And there's a whole set of jokes in there for later. For folks who haven't seen Star Trek: Next Generation, they at some point come across a planet called Gotana that has a tachyon core, so things move really, really weirdly and fast around it. So our entire platform is going to be called Gotana, that's built off of this stack that's called Tachyon. Now, part of the reason we chose that, and it was fitting, was people think kernels are the speed of light, right? That's sort of the canonical speed of light for a workload, is the fastest kernel you can have. That is true in a compute-bound world. We are not in a compute-bound world. We are in a memory and network and communication or bandwidth-bound world. that world.

Right.

So that changes things already. And the reason I say kernels are the new assembly is writing better kernels no longer gives you performance. Right. Because a better kernel actually exposes the latency of the system because now it is waiting for memory. Right. So you want to think about GPUs, for example, as 1,000 piranhas just sitting around chomping. If they don't have things to chomp on, they're going to get really agitated and bored. And they're still going to be consuming energy. So you want to feed them as much as possible. And that's ultimately the scheduling problem that exists here. Now, the reason I say kernels are the new assembly again is I don't think we benefit from writing them anymore. What we need is something that makes developers more productive. The time to value really matters. If I can give you this experience of I'm writing in PyTorch, I say PyTorch compile to whatever, and it gives me the same performance or better performance than what I would have gotten if I had waited 9 months with a team of experts and gotten that kernel written, wouldn't you prefer that?

Absolutely.

Right. That's what I'm trying to give. So in that world, kernel becomes something that the compiler generates. But then you still have to execute that. And that's the system we're talking about. So this compiler takes in this graph. It rewrites the workload from the point of view of the hardware by looking at the memory hierarchy. And it arranges things from the point of view of the memory hierarchy. It does partitioning. It does fusion. So it creates these workset items. Then it thinks about how do I move computation closer to where memory already is? So I have to move it less far.

Mm-hmm.

Because every single time you move down part of the hierarchy, your cost of data movement grows by at least half to a full order of magnitude. So if I can avoid doing that, I can create more throughput for less cost.

What are some of the metrics that you could quote? Like, does it speed up a certain— the performance of a certain model by X times? Or, you know, what kind of metrics do you use internally as kind of like the North Star of what you're doing?

We look at a lot, right? So for LLMs specifically, we'd be looking at time to first token, time between tokens, overall inter-token You can see we've been looking at the prefill versus decode when you do have them disaggregated. We're looking at not just that, but preprocessing to postprocessing, the entire lifetime of a request. And also we're going beyond requests because I don't think tokens are the optimization unit anymore. It is the full trajectory, especially as you think about reasoning models and agents, that becomes much, much more important.

So what is the time to first token improvement that you've seen?

Yeah.

So we're not in production yet.

I see.

For this kind of a stack, we care a lot about— and ultimately we are selling trust. Developer infrastructure, developer tools is ultimately about trust. You get one shot to prove yourself. And if you fail at that, you have lost it completely. And regaining that takes much, much longer. So you have to take your time to make sure you're working with your design partners and your future customers and your channel partners and making sure sure things are working as expected so that once it comes into your hand, the time to wow is immediate. That is what we're optimizing for.

But how do you, how do you, like, when you tell them what could be possible, like, what is the— I mean, what kind of, like, you know, is it going to be 20% better? Is it going to be 50% better? Is it going to be, you know, 90% better? Am I going to cut time to first token, you know, to 1/10 of what it used to be on a Llama 3, Llama 32B, or one of these reference models that we tend to use. What is the cell that you—

So we're accelerating the full workload, not just kernels. On a single kernel right now, like MatMul on AMD 9300X, we have shown that we're 1.7x faster. So proving that we can speed up a compute-bound workload, Already a huge thing. But our entire system was designed to accelerate the full workload, especially in cases where you're memory or network bound. Those cases, when you look at that and you start scaling, that's where the real problem is. Getting really, really good performance on a single device is a solved problem. Getting performance on a heterogeneous cluster, as you're thinking about larger, more dynamic models at scale, that is the real problem we're trying to go after. Right. So the time to getting up and running on a new hardware for the same workload, and time to performance, and then how reliable that is. That is what they're interested in us for. On a full workload, we can expect, say, 2 to 3x. When you're looking at a large training run, we can expect almost 30x.

Oh, wow.

There's a lot more opportunity to speed up and improve things on the the larger system than there is in the smaller system.

Incredible.

Can you talk a little bit more about how you think about bottlenecks across the system? I just had a conversation with somebody from MongoDB yesterday who has given me a long history, well, a short version of a long history of database technology. And he was saying, originally, going back to 1970 when SQL was introduced, the constraint then was disk space, actually. And so, the big reason that you have this, like, data normalization as kind of canon in data storage is that people were really trying to optimize for keeping the disk footprint as small as possible.

Mm-hmm.

When MongoDB itself was created, that constraint had basically been solved by progress in hardware and disk space had become cheap, but developer time was like the new constraint. Everybody wanted to get their apps out fast. Now we have the compute-bound paradigm, but you're saying we have the memory is kind of the actual constraint. I'd love to dig in a little bit more there and kind of understand, especially in this sort of long horizon agentic world, what does that really look like when an agent makes a tool call and then there's latency? Like, what happens to the KV cache during that wait? Does it like get shuttled off and brought back and somebody else's stuff is using that same chip in the meantime? Does it have to wait? Give me a sense of like what's thrashing around there and what kinds of optimizations move the needle?

There's a saying that a prof of mine used to have is scaling what works is different from what works at scale, right? And we are in this really interesting world right now where everything was built, originally all the software was built on the assumption that you're programming a homogenous device. a single chip, the workload is static and is understandable. All of a sudden, we very quickly ran into a world where Moore's Law and Dennard scaling stopped working for us. We came in with a lot more heterogeneity to come up for that, make up for it. Chips went from simple things to becoming a whole system. Like the machine went from becoming, you know, like now you have an entire cluster that is one machine that we're using to Program these things. The workload is getting more dynamic. It's becoming more branchy. It doesn't fit inside of a single GPU anymore. You have to worry about newer regions. So where you place things matters a lot. And the trade-off becomes recompute versus refetch, right? And then how local can you keep things, right? If you are local, you can, you know, reuse a lot of things and save on data access and data movement. movement and get more throughput. So then you have to partition that and manage a lot of overheads. That is the primary issue.

Right.

Now, for KV cache, you are trying to— and again, that also comes down to prefetch, right? Refetch versus recompute. In some devices, like if you look at Tensorland, right? A Tensorland device or a Cerebras device, You don't have to worry about it that much. You will still get speedups. But on devices that are memory constrained, that really, really starts to matter. So it's not one answer, right? It changes based on the system, right? And that changes your economics and what is possible. Now, for going from a model to an agent, right? A lot of things have had to happen. You went from a base model that was then, you know, went through some fine-tuning process and got a personality. It started learning how to do certain things. You had to connect that to an environment so it can learn how to reason, and you had to learn how to use compute more effectively to solve problems.

Right.

Then you can think about tool calling as another add-on, right? Having tool calling and reasoning doesn't make you an agent. It's all the things around you, being able to talk to the other world, and make certain decisions. And then there's a difference between the kind of decisions you can make for different kinds of agents. All of that exacerbates every single problem in the stack and you're always I/O bound.

Right.

So it's not one problem, right? It is a cluster of problems hitting you at once. Right. And being able to make the best choice at that point in time Is what it comes down to. And this goes back to part of why I think the kernels need to go away is all of that is dynamic on execution and you don't know what needs to happen until runtime. But in the kernel world, you are pre-baking decisions before the workload executes.

Right.

So you need to be able to make the right decision in the most efficient manner and place things where they need to be placed based on future access. So this is why I said the workload right now is the agent. It is not an LLM with a serving engine and an agent built outside of it where you have— don't have visibility across. That needs to change. And so we started from the point of view of looking at heterogeneity, what needs to change and how workloads are going to evolve over time. breaks when? And how do we build that kind of a system? And the big part for us is the runtime. That is a very big piece of the innovation we made precisely for this reason. We essentially have this sandbox of an environment in which the code executes. And this is on and across machines. That allows us to create that illusion of abstract Or single unified memory for developers, even across NUMA. And we can collect those traces as we're executing, and we can continue to optimize them after the fact.

Mm-hmm.

Right? So if we make a poor decision around where we place something once, we can learn from it and continue to improve it in the next case. Right? So your system keeps improving as you keep running more workloads. And the reason I'm saying this is the answer is always— the right answer is going to change over time.

Is that process itself— oh, sorry. Well, quick follow-up. Is that process— I guess everybody's of course using AI to accelerate their development, right? And when you had said a bit earlier that like you want to speed up developers, part of me was thinking, well, are developers still the bottleneck that they were when MongoDB was created? Because now we've got the agents to, you know, do a lot of acceleration at the developer level. So I'm interested in your thoughts on that. But then I'm, what I'm really kind of wondering is where is the intelligence used throughout this optimization process? Like, Are you using agents in the sort of innermost loop that is looking at, oh, workload is changing right now, we better adjust our approach? Is there like a frontier LLM at that level being used to optimize the runtime decisions on an ongoing basis?

Yeah, not an LLM. There's many ways of having intelligent behavior without having Never heard of them. Well, have you heard of compilers? So compilers have always been very entrenched with AI in a lot of ways. So in this case, knowledge-based systems, right? A knowledge-based system is essentially what a compiler would be, right? Because you have certain information or knowledge about how to make things go fast that you want to codify so that you can get the result fast and it's making known good choices. And you get a verifier for free because compilers have to be correct. Right?

Dropped off.

Come back.

He should be back in a bit. We speak to so many We've spoken to a couple of chip companies. We spoke to people who are managing prefill, decode, et cetera, and trying to rearrange that on a hardware basis. We've also seen Modular's Mojo. We've seen OpenAI's Triton, GPU programming system. We've also seen Elon Musk say, hey, it's going to be AI is writing directly to assembly within a few years. And it's hard to think about how the software layer is going to change in the medium to long term, because it's quite clear that a lot of these compilers and stuff are going to be used primarily primarily by AIs, right? At this point, most of the programming is already being done on the front end, you know, by prompting. And it is hard to see where, you know, things go from there. Hi, Jay. Okay, we're back.

No idea what happened there.

So one of the questions I had, and I was just describing this, was you You have Modular's Mojo, you have OpenAI's Triton system for GPU programming. Modular is, for example, trying to reduce this kind of 2-language tax that you've always faced. How do all of these pieces fit together? Because is a developer now going to have to— is it another language or tool in the stack that they have to learn how to use? Is there a kind of tax that comes with having to add that piece to the stack? And how does Tachyon actually do this? Because we've spoken to some chip developers, for example, who move the data much closer to where the compute is. We've spoken to people who are doing compute inside memory. We've also spoken to one team that is doing chips that allow compute to be done inside memory. How exactly does Tachyon do this? And how does that fit into an entire stack that already exists?

Yeah. So, first, like, you think about companies like Modular, right? What they're trying to do is build a systems-level metaprogramming language. And so, you can write kernels better. To me, it was the most literal interpretation of, I need to code out alternative. Let's build a language. But the problem is, I still have to write kernels. That is on the developers to go and do. If you actually do the math for the amount of hardware that is in the world today, all the different workloads that we're running, the different numerical styles, the different fusions, the different ways of partitioning them, right? And you think about different batch sizes, you think about latency versus throughput, or general but other SLOs. You actually sit down and do that. And you're like, okay, I need to write about 106 billion kernels in order to get coverage. Well, there's only about 2,000-odd performance engineers in the world that actually know how to write good kernels. 90% of them are inside of one vendor ecosystem. So there's still a problem of I need coverage. The reason you can accelerate some of this on NVIDIA is because NVIDIA spent 20 years building an ecosystem of tools to make your life easier so you could get the feedback. That feedback loop, right, that ecosystem makes kernel generation easier. That maturity doesn't exist in any other vendor.

Mm-hmm.

So for us, like, what we're looking at is— and then also There are certain hardware that are in use today for which you cannot write a kernel. Data flow architectures. You cannot. You have to execute a task. And that is a different granularity than a kernel. The tile programming model is still within the kernels, but it's easier and more accessible for developers and raises that abstraction. The reason OpenAI invested in Triton was to raise the bar, like raise the number of of people that could write kernels without needing to know CUDA or low-level programming.

So, so if I hear you right, uh, Tachyon is a tool that for non-NVIDIA chips is going to be very useful because they do not have that toolkit that already exists in the NVIDIA ecosystem.

Yes, we allow more non-NVIDIA hardware to become more accessible. So you as a developer can choose the best hardware based on the shape of your workload and the economics you care about. And you can make that choice immediately without having to go and rewrite everything. But on NVIDIA hardware as well, on the NVIDIA hardware as well, we can give you more gains on a larger system.

And can you describe again, so you have this thing I think called Operator Fusion that you're using. How exactly does this work?

Yeah.

So, if you think about, like, normally when you're running some kernel or some operator, you are reading memory, right? In GPU land, they use DRAM as main memory. So, you're reading something all the way from there, bringing it into a local register. You're operating on it. You're writing back to main memory. Reading it back. And then doing the computation. Well, in that time, your compute unit was waiting while you were moving data around.

Right.

And that's very, very expensive. So fusion is basically, let's not, you know, send the data all the way back. Let's keep it local and just reuse it for the next operator.

Right.

So it's about synchronization and barriers and so on and so forth.

Mitigate data movement.

That's the name of the game. And minimize waste. And that gives you speedups, right? So it's sort of like, um, if I am at my desk right now, I need certain information, it's already somewhere on my desk, right? I know where exactly where it is. I can read it, I can use it. I know I have the information somewhere. It's probably in one of the books on my shelf. I can go over to my shelf and I can grab a book, right? That's me going from a local register to like an L1. If it's not there, then I have to go to the library next door, right? And I was like, okay, or go to a friend or somebody else in the office. I have to travel a little bit further to get that information, but I have a larger collection of things I need to search through to find that information.

Exactly.

Now, if you go from there to a library that's, you know, far and they don't have that book, and then they have to get it from somewhere else, and then they have to give it to Right. That's sort of the logical hierarchy of memory.

That's the cost.

Right.

So, if I don't have to go that distance to get something and it's available for me already, right, I am much faster and I'm better. I'm much more efficient for that. Right. That's what operator fusion does.

Indeed.

Can you tell us a little bit about who faces this challenge of heterogeneous clusters? And is it a problem that people try to tackle more in inference? I assume so, relative to training. But how big are the customers typically, or how far into the long tail does it go?

Yeah.

And then how do the other chip manufacturers respond to you and support you? It strikes me listening to all this that There should be like a consortium of non-NVIDIA chipmakers putting forward like a billion-dollar prize for whoever can do this. But I'm guessing you had to like raise funding without that carrot out there.

For us, it was what we wanted funding specifically so that we were vendor neutral. Like we wanted that to be as very explicit as a goal. At the moment, we are taking like one vendor. We are more aligned to that vendor. And that skews a lot of, you know, what we prioritize. And for us, it's really important to be building for the builders. What do they need?

Right?

Our customer ultimately dictates. And when we spoke to them, they wanted to have more— they obviously love NVIDIA.

I love NVIDIA.

I want to be able to use AMD GPUs. I want to be able to use TPUs. I want to be able to use Atranium. I want to use Tensor Torrent, Cerebras, or SambaNova, or D-Matrix, or Groq, or any other chip, because each of them is better for a specific thing.

Right.

But if I am locked to just one thing, right, I am getting suboptimal use for a workload.

Right.

And we've been through this before where a lot of good ideas died because they didn't suit the hardware. Right. If there's more hardware available and they're good at different things, more ideas become accessible. We're now in the realm of talking about reasoning models and world models and planning and robotics making a comeback. Robotics is all heterogeneous. They're full SoC with a lot of different parts. If you look at the Jetson, for example, it's got the CUDA cores, it's got a CPU, it's got DLAs, it's got a vision processor, it's it's got other DSPs on it, right? You have to be able to access all of them, right? The Qualcomm RB5 and RB6, similar, right? We started originally as one of those companies building this heterogeneous SoC for robotics. So we had to deal with it. All GPUs today are heterogeneous, right? The moment we crossed the 5-nanometer threshold, we had to think complex packages. We are programming in every single machine today is a CPU, some network, a GPU. You have different kinds of memory. You can have LPDDR, GDDR, HBM. You can think, and they all change the execution model. You have different numerics, right? On NVIDIA, you have the CUDA cores or CUs, and then you have Tensor Cores as well. They need to be spoken differently. You have different memory hierarchies now with the core memory for the SMs, and then you also have tensor memory accelerators. Heterogeneity is already here. Everyone who is dealing with the GPU is already dealing with it. Anyone who's training a model or deploying a model is dealing with it, right? But the software was not built for this. The software is is still living in the '60s. We are still programming as if we've got a single-core CPU, right? And GPUs are these sidecars that we throw work off to every now and then. And, you know, we can just add in libraries or other intrinsics or pragmas and just fix the problem. That isn't the case anymore. GPUs need to be— and accelerators need to be first-class citizens and CPUs need to be backstops. And that changes things. And now you're programming a cluster as a single machine, right? And some of my friends at Labs right now are training models across data centers. It's not even across nodes anymore. It's not even racks. We're talking about multi-gigawatt data centers or multi-megawatt data centers as one machine for one model.

So Give us an idea of your kind of product rollout timeline. You know, I think there was a beta testing that was about to start, or there's like a product rollout early next year. Like, what does your timeline look like? What are the milestones to that timeline? And, you know, how do you, you know, what's your acceleration strategy at this point?

We'll be up and running on NVIDIA and AMD hardware for the majority of models that customers care about with state-of-the-art performance at the end of this year. That's going straight into the hands of our beta customers and our partners. And then we'll do GA in Q2 next year.

And are your buyers like hyperscalers or startups? Like, you know—

We're a bit more focused on managed inference serving to start with. And so that much more appeals to certain enterprises or AI-native companies or startups that are trying to build products around the models.

I see.

From there, we will expand into things like post-training, building out for reasoning environments, building out for agents, and then adding in support for training.

So you're looking at startups that want an inference edge or saving. Is that the focus? Yeah.

It's generally people who want more control.

Is that helpful for— a lot of the new clouds are now doing bare metal. So is that helpful that you have these new clouds offering bare metal and that customers can bring their own?

Yeah.

I mean, the new clouds have always been in the bare metal business, right? It was sell hardware without all the cruft. Yeah. Because all the hyperscalers had a lot of extra software and things that they were trying to upsell. Right. And if people just want, you know, the fastest hardware, the lowest abstraction, that's what the Neo Clouds we built originally for. And that definitely works better for us in the way our stack is built. And that's who we are generally partnering with. Like our go-to-market is with all the Neo Clouds. And we are very, very tightly coupled with a lot of them.

All right.

2 final questions for me, one quick one and one kind of zoomed out, hopefully fun one. The quick one is just, how do you price this? Everybody's kind of trying to figure out the future model. Is it by seat? Is it by company-wide license? Is there a usage component?

Yeah. So I think, I mean, in a perfect world, outcome-based pricing makes a lot of sense, especially as you go for agents, right? But if you are an infrastructure provider, the only business model that has worked and continues to work and scales is consumption. So for us, I mean, I know a lot of people like the token pricing right now, but tokens work really well for the static request-response kind of workload, which is what the old base models were, the little tuning. But now that you have reasoning models, it's harder to reason about token consumption. And then same with agents. There's a lot of things change. So what we're actually moving towards is effective compute consumption.

Right.

The amount of compute you use to realize useful work.

Right.

And that's different from saying a GPU hour or a GPU slice. And the biggest thing, like actually part of the reason this makes sense is our business model scales with the delta between effective compute and effective, like physical compute and effective compute. And what I mean by that is what we're showing is the fastest new addition of compute will be through software. And it'll come online faster than you can actually plug in new hardware, right? Because you are going to be electricity bound, right? Getting turbines and power installed in places so you can bring up new silicon is the big limiter right now. If I can boost your utilization by 3 to 10x, I'm adding in new, more effective compute at a lower cost that I can sell, right? And the consumption of that resulting into tokens is what we are reselling. And that scales really nicely. And it's something that's understandable for a lot of the finance people as well, right? Because for a lot of them, the token pricing and other pricings are breaking right now.

Interesting. Thank you. Okay, here's the fun one. You mentioned you're a Star Trek fan, and the website certainly reflects the futurist aesthetic. What's your positive vision for the future of humanity as we go into the AI era?

Like, I am definitely a believer in a human-first kind of outcome, right? I think AI is a new kind of intelligence. It's different from human intelligence. It can be complementary. And what it's showing us is there's a lot of work that we have been doing and describing as work that maybe wasn't actually work and we should never have been doing in the first place that we can start offloading. And that allows us to have more leverage to do work that we are uniquely suited for. It, to me, creates much more equity so we can actually try out new things. We can accelerate discovery. We can learn more about ourselves and the world, and that results in more solutions or more problems being solved at a faster pace. lower cost than they have been before. There's been like, personally, I'm really interested in what happens in drug discovery and design. There's been so many places where pharma hasn't been able to get to because the cost of research was so high or there wasn't enough of a dataset. If like biology starts, you start going from squishy things to bits, things get on an acceleration curve and you can get a lot more benefits out of it.

Right.

The quality of education has been fairly crappy, right? And it hasn't kept up with the pace of the world. And I think AI can help bridge that gap. There's a lot more that can happen, right? And you think about, I think deep tech is gonna be way more important going forward in the future, and simulation is going to unlock a lot of it. And AI plays a big role in that. Like, um, we can expand the tech tree. We took a very particular branch and we've just been going off of it, right? And there's been all these inherited biases that have guided us. If we can unroll that and we can expand more, who knows what's possible? And I'm personally excited about that.

Love it. Me too. Also a tiny bit afraid, but very much excited. Jay Diwani, the company is Lemurian Labs. The stack is Tachyon, and there's a lot more to come. So we'll be watching as always for your updates.

Thanks so much. Thanks for having me.

Cheers, Jay.

Great to meet you.

Bye.

Amazing.

Yeah. Which way does this all— I mean, it seems, I guess if I had one more question, I might've just asked him like what's gonna happen, happen to the, uh, per-hour rental price on chips? You know, can, uh, these sort of utilization-improving layers do enough to keep those costs stable, or are they going to continue to run up? It seems like the smart money is still on they're going to continue to run up for the foreseeable future.

Um, but That's really—

I mean, it is amazing how much the sort of, you know, performance-held-constant pricing has come down. And, you know, if they're successful, there'll be another chapter in the, you know, long and ongoing unfolding story of ways to bring the same intelligence to people for dramatically less. And yet it still seems like the, the, um, the paradox is in full effect, right? We are definitely buying more and more and more, and so much so that the, the per-chip price is going up even as the, the efficiency with which the intelligence can be delivered has dropped so dramatically.

It's—

It's really quite something. The eras come at us so fast in this space. The moment we had, I forget exactly when it was, but there was a moment where it was like, oh, there's a GPU glut. Wow, those days are long gone. Just to think about another kind of interesting reflection on all this is just like all that complexity is the kind of thing that people are willing to take on because the chips are just so scarce. I mean, it's wild to think about. In a way, he's sort of solving the— trying to get developers to be more productive, faster to ship stuff. But the other way that would be faster to ship stuff is not to have a heterogeneous cluster and just to pay up a little bit more on the hardware side. to keep it simple there. But you can't. It's just too expensive. You know, so you have to take on the complexity on the developer side, and then you have to have attempts to solve that complexity with projects like this.

It strikes me that, you know, I think for people who are not deep in the weeds, they don't understand, I think the margin stack that exists, because the hyperscalers charge, I think about 30%, their grosses, I think their gross margin is about 30 to 40%. Nvidia is up there at like 70%. The memory guys are at like 80 to 90% now. And all of this stuff like stacks on top of each other, right? And when you look at how they end up stacking and you have at the very top OpenAI with, or Anthropic with a 70 to 80% margin, and they're buying tokens from Amazon at a 30% margin. Amazon's buying chips from, you know, chips and other things from other people, uh, and those people have like 50 to 60, 70% margins. Everyone then manufactures at TSMC, and they have 50% margins. And TSMC suppliers, ASML, they have 50% margins. And if you look at the Varden stack, right? Like this $50 billion per gigawatt data center, it's really kind of made out of sand, literally, literally in some sense made out of sand, sand and intellectual property. And when I think of it, it's all of that money is just the incentives required to get the humans, some of the smartest humans in the world, to take a look at these problems and fix them, right? Like all of that money, because again, the physical elements inside that data center are actually worth not that much. Very little gold in there, very little gold and mostly silicon and some plastic. And if you just knocked down the entire data center and kind of sold it for scrap, it would be literally worth like, you know, cents on the dollar, like a few cents. And it just strikes me how much all of it is just intellectual property. It's really just know-how, intellectual property. It also somehow also strikes me how AI data centers are kind of this like crowning achievement of humanity as a whole in the sense that how many of these parts come from— you have argon gas from Ukraine and you have copper from copper mines in Mongolia. And all the way up the stack, you have chips from China, rare earth metals from China, from Taiwan. You have energy being produced in Texas and all of that just stacking up and pulling. And I often imagine it as this entire thing pulling the rest of the economy up because it's creating demand across all of these different segments of the economy. And it's all rather invisible, I guess, because it's distributed across so many different places. But yeah, just immense, this immense economic endeavor of humanity as a whole in order to build these things. And it's just amazing. It's really amazing, like the how, what the invisible hand has achieved over centuries. So.

This is why the rationalists at their solstice festivals have experimented with singing hymns to the global market and global supply chains.

Have they? Have they? Do they really do that?

I wasn't in attendance for that, but yes, I do know on pretty good authority that Winter Solstice, you know, Rationalist event did at one point feature a hymn to the global supply chain. You know, now with Suno, you could probably make it a banger. I suspect, really, certainly the suspicion broadly, I won't, you know, prejudge it, but I think the suspicion broadly would be that that would come off pretty cringe. But, you know, maybe it's just a matter of needing better bars to really make it work. And if my recent experience on Suno is any indication, you know, maybe we'll see what we can do in terms of a hymn to the global supply chain. See if we can put our money where our mouth is and have one that we'd actually enjoy singing along to.

The rationalists are never gonna get out of there, out of the accusations of being a cult. I tell you, like the moment, the moment they step out, they get pulled back in.

Well, You know, they might just be proven right in the long timescale of history though. Would it be so surprising if, you know, at some point in the somewhat distant future, there was a hymn to the emergent order of global supply chains that somehow materialized before we even had machine intelligence to run it? I think it's not— Not the craziest idea I've heard.

I definitely have rationalist sympathies. So as I point out often, like, you know, it is— I do hold a lot of the same viewpoints, if not— but I do not come to some of the same conclusions that they do. But yeah, it is. In fact, I also think Trump, to a large extent, he is leaning on the strength of the global supply chain in some sense for this Iran war, for example. And he's definitely leaning on both the strength of the global supply chain and the strength of, I think, the global financial markets a lot, to a large extent, when he makes trouble and he wants to fix these long-running problems that the the US has had, and he goes in there like a bull in a china shop and he does these things, he leans on the fact that I think the rest of the system is very resilient. And I think COVID kind of showed that kind of resilience. And COVID, I also believe, made the entire economy more resilient because now, you know, everyone can— everyone knows that they can work from home. So no snow day will ever, you know, stop business. You know, we all, we all know that it's never going to stop business. Right? So it's also proven out the entire DARPA thesis behind the creation of the internet, because, hey, it's very clear now that you cannot— a nuke is not going to take out business from running. It's just business will just go on. So you cannot just bring the country to a stop that easily. So the hymn to the global supply chain is maybe well deserved.

I'll get Claude on some lyrics immediately after the show today.

Just to maybe round out the show with something that we started out on, and so I feel like I have to round out the show with the same kind of thing. Pew Research Center, for the first time, a majority of adults under 30 say they're more concerned than excited about AI. Their concern is now on par with those in their 30s and 40s and those 65 and up. And so we have the only group still under is the 50 to 64 group. So the Gen Xers are still— majority are still more excited than concerned. Everyone else is now in the—

Concerned.

in the more concerned category. I kind of, I kind of don't know what they're concerned about because I would be a lot more concerned about Instagram than, than AI. But, you know, I feel like all of the evils that were said about social are just being heaped on AI with no, with no kind of defense or recourse.

So I just hope we don't get the nuclear outcome. You know, when I read these things and all this data center backlash stuff, and, you know, the Republicans saying, you know, we're the only ones that will, um, you know, that you even have any chance of getting to support this kind of activity, it really makes me fear that we might be headed for a world where we get all the downsides and not nearly as much of the upside as we should.

You know, we—

with nuclear technology, we've got still 10,000 nuclear weapons globally deployed, which is, I think, by any rational account, like an insane number. And yeah, we do have some nuclear energy, but not nearly as much as we really should. I think that's pretty obvious at this point, although it's obviously still contested, but it's pretty obvious to me. And I would just hate to see a populist backlash leave us in the same spot with AI where we get militarisation and concentration of power and you can't release models because there's not enough, potentially for multiple reasons. But one increasing reason would be if they can't build the data centres, there's not going to be enough compute to serve them. And so retail is going to get kind of a less-than model compared to what the government itself or the biggest enterprises can afford. And I am very sympathetic to all those worries. So as much as I do have fear of big picture AI gone wrong, I think there's enough data centres already for those experiments to continue. And I think we need to address that at a different layer than the physical buildout. The physical buildout, I think, is what's going to allow us to all get the day-to-day benefits that we want as individuals with unlimited access to expertise and unlimited digital personal assistant support. And even that robot, it making our meals in our kitchens and sweeping our floors. That future really does seem to depend on the build-out actually happening. So I hope they figure it out, man. I hope they start to cut some checks and pay off the public. If not the— I wouldn't advocate for paying off the officials, but I would pay— I would advocate for paying off the public if that's what it takes to get us over the hump and get people a little more comfortable with this kind of stuff, because I really don't like the alternative very much at all.

I think, I think it's pretty clear at this point that physical construction in the US is very difficult. And I think it's difficult regardless because data centers are the cleanest industrial facilities you will ever find in the entire world. Right. And they're able to pay for fuel cells, which basically turn natural gas into water. And they're able to pay for that. And they're able to pay for solar energy. They're able to pay for solar and batteries. And regardless of all of that, and they're also, they don't employ a lot of people. So there's not a lot of traffic either.

Right.

Like a typical 50-gigawatt data center employs about 200 people. So there's not a lot of traffic issues and there's not a lot of creation of new housing, et cetera. So given that the impact is not that great in terms of physical, the fact that you are seeing these bodes very ill for future reindustrialization of the US. And I think it's very clear now that there's going to be a huge resistance To moving Chinese factories back to the US. I think it's going to be very untenable to think about robotic factories, dark factories producing large robotic dark factories. So I think that's clear. It also gives a boost to Elon because Elon is focused on moving the chips to— moving the data centers to space. So, and I think on the numbers that he has, I think they are looking at like $100 an hour per GPU hour for a B200 for it to make sense. And if you look at where GPU hours are being priced at right now, they're at like $2 to $3 on a spot and $20 to $30 on a longer-term basis. It tells you that what we are gonna see is we're gonna see all of this resistance that pushes up the price of GPU hours onshore to $50, $60, $70, $80. And so I think the numbers that you end up looking at is that, are the data centers willing to pay maybe 100% of their GPU cost to the public or 200%, right? If you're renting at 30, are you willing to pay another $60 an hour to the public? And remember, the payback time of these is something like 12 to 24 months. So they're looking at like a $50 billion GPU cluster makes about $30 billion of revenue a year. And about 70% gross margin, $21 billion. Are you willing to pay like $10 billion or half your margin away to the public? I don't think those numbers have been met yet. I think people are looking at cents on the dollar at this point. And the question is, how high does that have to go in order to make this more feasible to build onshore than in data center space. And I think those numbers are— I think no one wants to discuss, I think, we're going to have to pay $20 an hour to the public as a nuisance fee. I think those numbers are still not being discussed yet. And I think that's where these guys are thinking, I can pay 10 cents per GPU hour or 5 cents per GPU hour. hour and get by.

So, well, maybe this is the path to universal basic income. I mean, it's going to be a really weird one if it's like a county-by-county patchwork, but, you know, you're talking real money there with that kind of share if it can get to that level. And, you know, there's definitely There's some room between operational costs and what it would cost to do it in space. So maybe that gap is the UBI opportunity.

That county that you mentioned earlier with 29,000 people, they would be getting something like half a million dollars a year per person. So at those numbers, yeah, you know, I think, I think you know, deals could be made, right? Like people would be like, all right, I'm gonna have my location here. I'm gonna take this $500 grand. I'm gonna move to California, or I'm gonna move, you know, like everything becomes—

Everything changes.

Yeah, you know, everything becomes different, right? Like the whole ballgame changes once you go from like, you know, $2 grand a year to $200 grand a year, right? It's a different ballgame.

Yeah, as my dad sometimes likes to say, it's not the money, it's the amount.

I have much saltier ways of saying that, but I won't.

Yeah, everybody has a price, including the public.

Indeed.

Very good. Well, we're off tomorrow. As Sam Altman prophesied, people will continue to swim in lakes. It's going to be lake day for me tomorrow, and then we'll be back on Monday for more exciting experimental public sensemaking here on AI in the AM.

Indeed.

See you guys on Monday next week.

Cheers.

Thanks, Prakash.

Bye for now. Bye-bye.