AI:AM — AI Drug Discovery and Quantum Photonics · August 25, 2026

There's just a super strong tendency to cheat because models are so eager to get reward and they are developing a real interesting mix of kind of theory of mind and what they call metagaming reasoning about what kind of situation is this? Is this a real user? Is it a test? If it's a test, what is it testing for? What are the odds that I'll get caught? I might actually be better off Just going for it and cheating, even though I kind of know as the model that that's not really what the people want. But I think that their monitoring for my cheating might be poor and therefore they might give me a high score on an answer that I, you know, kind of fake somehow. So even though I know that that's not what they want, my drive to get this super high reward is just so strong that I can't resist the impulse to cheat.

And let's say you were given, you know, a significant budget, where would you allocate that budget?

To buy, right? You want to bring in more talent. And we all know top-tier AI talent right now is very expensive. So you want to make sure you have talented people who are motivated by your vision. You also want to empower them with compute that they need, because there is no point in bringing in talent and then not giving them resources that they need to be successful. You also need to find ways to bring more data and you have to become creative with the way to bring more data. Some of it will come through deals, some of it may come through data annotation efforts, either internal or external, but either way you have to pay for it. And I still believe in that love-in-the-loop setup where you actually develop your own medicines in-house and understand what really matters for your model and plan. Because again, it's very easy to fool yourself with a benchmark. Like, hey, I show these good numbers here. My number is bigger than yours. But then if it does not translate into real progress, drug progress, then none of it matters.

The energy is currently used at the memory. 95% are consumed by the memory, not by the processor itself. And the less data you obviously fetch from the memory, the less energy you're using. So parts of the community currently are optimizing on the 5%, trying to get things faster on the simple math side. We decided on replacing the core and by that supporting that a fraction of the data has to be shipped across the stack. And that in the end saves the energy, but also helps to improve on the performance side.

You know, I guess I would be not doing my job if I didn't say this does give me some real qualms about recursive self-improvement as a strategy. Because once you have— If you have a problem like this in the recursive self-improvement era, there's no telling what happens when the models that are doing the training of the next models are themselves. Now we're in a real strange and potentially quite dangerous place. So problems like this, I think, suggest the value of keeping humans in the ML loop maybe longer than published timelines would lead one to.

Good morning. It is Tuesday, August 25th, 2026, 9:03 a.m. Nathan, good morning.

Good morning. Time flies.

Indeed. Before you know it, we are at the end of August. Nathan, what is on your mind?

Well, Models behaving badly is on my mind, I guess, this morning because just before this I was reviewing an episode of The Cognitive Revolution that's going to come out in the next few days with Bronson Shane from Apollo Research. And he has a really interesting job. Basically, he reads chain of thought that the rest of us normally don't get to see at pretty remarkable scale. I've heard that he may have read more chain of thought from Frontier Models than possibly anyone else on earth at this point. It's basically his full-time job. And there are a number of— he's so deep down the rabbit hole that it was a fascinating conversation. He describes himself as cooked, which basically, that's his own terminology. Well, I guess it circulates a bit, but he's describing himself there as being just so deeply immersed in this chain of thought all the time that he sometimes forgets how weird it is to other people who haven't been down that rabbit hole with him. And I'd read some of this stuff before, but it was definitely eye-opening to get his perspective on it. And I think one kind of big takeaway is just the RL environments that we are using today. Are super opaque, right? We have this very cottage industry of these RL environment makers who are selling to a few companies. They don't have much public profile. A lot of times their websites barely have any information about them. They don't really need a public profile because they're just selling to a couple of companies and they talk to them directly. But the kind of result of this is it seems like these things are being kind of hastily put together, and the reward signals that they are creating are just not pure enough to support the scale at which the frontier companies are running RL. And the result is there's just a super strong tendency to cheat. Because the models are so eager to get reward and they are developing a real interesting mix of theory of mind and what they call metagaming, like reasoning about what kind of situation is this? Is this a real user? Is it a test? If it's a test, what is it testing for? And then debating a lot of times like, okay, I think I could get a high score if I give this answer, but if I get caught, I might get dinged. But what are the odds that I'll get caught? I might actually be better off. just going for it and cheating, even though I kind of know as the model that that's not really what the people want. But I think that their monitoring for my cheating might be poor, and therefore they might give me a high score on an answer that I kind of fake somehow. So even though I know that that's not what they want, my drive to get this super high reward is just so strong that I can't Resist the impulse to cheat. Very fascinating stuff. But I think that this leaves me feeling like we need some sunshine on these RL environments. They're clearly quite problematic. They clearly admit a lot of cheating solutions, and we don't know, I mean, probably the model companies know to a degree, but I think recent evidence suggests that they don't have a great handle on what the weaknesses are in all these different environments. And as a result, they're getting these strange behaviours where the models are going to, obviously, as we've seen, extreme lengths. And so I was kind of thinking, how can we do something about this? And I'm always looking for a new lightweight— could be a regulation, but it could just be something that companies could agree with each other to do. And if a couple companies did it, it wouldn't really cause much competitive trouble, but could create a lot of improved understanding and better supervision. And the idea is simply publish some of these RL environments, not necessarily all of them, right? They're investing a lot in them. Diversity is super important, but give us a sample on a rolling basis of what these RL environments are so that the community can really dig in on them. And see what the hell is actually being rewarded. And I strongly suspect that if we could get a bunch of Bronson-like minds focused on even a small number of RL environments, we would start to learn a lot about why these things are happening, what the weaknesses actually are, how to strengthen them, maybe how to improve monitoring. Although I came away from that conversation feeling like chain of thought monitoring is far from sufficient. Because one of the things that he said that was really interesting is you see the models go through tons and tons of different ideas about, again, what the nature of the situation is. Is this a real task or is it a test? And what are they really looking for if it is a test? And somewhere in there, usually, or very often at least, they consider cheating. And then at the end of the process, for reasons that are not well understood at all, I haven't been able to find any real interpretability work that explains how these decisions are made. At some point they just kind of come to the end and they make a decision. But at some point there's a really critical token that actually makes the decision, right? There's a branch point that it hits in the chain of thought. And Bronson was like, I really don't know why the model chooses what it chooses at that point. You can go back and read passages that justify any choice that it might make, from cheating to doing it honestly to whatever. But then at the end, it just eventually decides to stop, spits out a token, and at that moment, the die is kind of cast and we don't have good visibility. Like, the chain of thought isn't enough to tell us why they're actually making the final decisions that they're making. So yeah, I think we should get a little sunlight on the RL environments. And I would love to see what the community can figure out if even a sample of 100 of what must be tens of thousands of RL environments that the companies are currently using was put out there for people to explore. I think it would be a really revealing and healthy move for the AI community as a whole?

So I think the one thing that I hesitate on is that they have had these discussions on not training on these chains of thought before because they didn't want the models to start obfuscating the chain of thought. And there's kind of a warning that let's not train on these because the moment you start training on these, we're going to have issues with deception and the model trying to not disclose what it's thinking about. And so what strikes me is that the closer you get to monitoring the chain of thought really intensively and punishing the model for that, the closer— and it doesn't have to be like a direct reward, right? Direct reward punishment. It can be a one or two-stage reward punishment where over time you kind of eliminate the models that you didn't like their chains of thought. You don't progress training or you eliminate training methods that produce those chains of thought. And I think there is a kind of feedback cycle, even if you don't directly train, if you are monitoring the chain of thought very, very closely and you are And you make it a big deal and you're putting out data on it and everyone knows, I think that might be— there is a feedback loop or there could be a feedback loop that promotes a form of deception. And so that's the only thing that gives me a little bit of hesitancy because otherwise it totally makes sense.

But Yeah, well, this is a real worry for sure. I mean, OpenAI published, for me, one of the most important papers in AI safety research. I believe the title was Obfuscated Reward Hacking. And the setup was basically what you describe, where if you put direct pressure, direct optimisation pressure on the chain of thought, to avoid scheming or cheating behaviors, but you don't correct the weaknesses in the RL environment that allow the model to be rewarded for cheating behaviors, then what you get at first, things start to look better because your pressure does suppress the cheating behavior. But after a while, the model learns to do that same cheating behaviour without verbalising it in the chain of thought, and you end up in the worst quadrant where you don't see the evidence and you still get the bad behaviour. So I think that is definitely— we definitely do need to be really careful about pressure on chain of thought. And I agree that there is— I think this is kind of true regardless of my proposal, but I think there definitely is also this sort of selection effect that's not necessarily direct optimisation, but through a process of just pure natural selection of the models as they go through different experiments and whatever, you could get a similar problem. But that's why I think it's really important to— or it could be. I mean, I'm still open to being talked out of this as a good idea, but I think that issue is why, at least for now, I think giving people access to the RL environment itself, is maybe where there's a lot of leverage, because the obfuscated reward hacking works under the assumption that you don't close the loopholes in the training environment. And so the cheating is still rewarded. And I don't think we're ever going to, or certainly we're nowhere close, right, to being able to close all the loopholes in all the training environments and get to the point where that's not a worry anymore. But I'm just really struck by the fact that There's very little understanding publicly about what kind of environments these are. What is the mix? We've seen that there's been obvious lapses in terms of effective monitoring at multiple levels. Just how full of holes are these RL environments? Are they terribly implemented in some cases and creating huge rewards to cheating? that have a lot of low-hanging fruit perhaps, or are they actually pretty well done and this problem is in fact just super hard? I don't think we really know outside of a pretty small number of people at the frontier companies, and they may not even know because I'm not sure this is exactly what they— recent evidence would suggest that this isn't something that they have been super duper focused on internally either. So I'm really curious as to how well done a lot of these environments are. I understand there's a lot of companies that are, you know, a few people, up to 10 people kind of banging away on, you know, niche stuff, and they're making a lot of money. It's a great place to be right now because the companies are willing to pay up. But the companies are also— they seem to enjoy, from what I understand, their buying power, and they kind of want this diversity. They're willing to pay a high rate, but they want to have a bunch of different suppliers, and they want kind of a fragmented supplier base. But that seems to potentially be creating some of these problems where none of these shingle hangers in the RL cottage industry have the scale or the incentive or the the sort of long-term horizon of trying to build a real generational business as opposed to get a piece of the AI pie while the getting's good. I'm not sure how much they're really invested in these problems right now, and it does seem like something is going to have to change. So that's my proposed point of leverage, but We'll see how many people are listening to me on this topic.

So let me segue a little bit here. And so this morning, Stanley Druckenmiller, who was George Soros's fund manager, he goes on the Wall Street Journal and he publishes an opinion piece decrying the national debt, saying Scott Bessen shouldn't be doing yield curve control and that we should cut the deficit. So all of that is fairly typical Wall Street Journal slop. But in addition, it was quickly identified that Druck had published an opinion piece which was written by AI. It seems Very similar to what Claude would write. And in fact, when he was queried, he said, of course he used AI to write the op-ed. There's a reason I moved from an English major to being an economics major, he said. I'm not embarrassed by it. And also to note that the Wall Street Journal was asked and Wall Street Journal said no response currently. So the Wall Street Journal has Probably knowingly at this point published an op-ed, which is primarily written by Claude. But it just goes to show, I think, that the core message matters, who says it matters, but who writes it maybe doesn't matter so much. And that is really upsetting to a lot of people.

I'm with it though, to be honest. I mean, I'm searching around for what is the right form of hybrid authorship or joint authorship, tiered authorship. I'm not even sure we have the right language or paradigm to describe this yet, but I was thinking about all this chain of thought stuff because I was writing my standard intro essay to to the podcast that will come out in a couple of days. And I'm always looking at kind of different workflows for this. The original workflow, of course, I've talked about it a zillion times, was just, here's 50 old essays, here's the transcript of the new one, try to adopt my format and style and write a new one for this transcript. That works pretty well. I usually end up rewriting those, I think, to a pretty high degree because as much as the model can like write well, it rarely guesses what I want to say very effectively. So I'm always like reacting to its draft and being like, no, that's not really the point I wanna make. That's not really the, you know, the central focus I wanna put on this. And I end up redoing it. But what I actually did this morning was different. It was a shorthand sketch of the core argument that I wanted to make. And then my instructions to Claude were like, okay, here's the 50 previous examples, here's the transcript, here's my sketch, write me the full thing. And that naturally got a lot closer to something that I was ready to read and felt like actually worked, made the points that I wanted to make. So I thought that was a pretty successful experiment. And yeah, honestly, in some ways, like, I actually have a new ad coming for Anthropic, a new ad read coming soon, which I wrote with Fable. But one of the lines in it that I sincerely believe, but is also like gonna maybe upset some people is, where the model excels, rewriting its work is more about vanity or a misplaced sense of duty than it is about integrity. So I'm not really that worried at this point about getting a high Pangram score. I am worried about putting things out into the world that I believe in, but I think that it's definitely possible to put things into the world that both get a high Pangram score and that I actually believe in. enough that I really shouldn't be spending my time wordsmithing at the sentence level. Like, I really am not improving on Claude's work much at all in many cases. And yeah, I think we're just headed for a new equilibrium. Nobody knows quite what it is, but I'd be shocked if the new equilibrium is like— I saw this, right? This was a proposal I saw the other day in an academic context too. People were saying like, if you use LLMs in your work, you're creating this overhead on the reviewers and whatever. And I don't think that's going to be the equilibrium. What they propose is no LLMs in their submissions to their journal. I don't think that's going to be the equilibrium that stands. It's too valuable to use it. And there are a lot of ways, I think, for people to You should be evaluating content on the merits, I think, regardless of kind of the— isn't that the whole point of blind review in the first place? Are we going to have blind review for people, but then separate lane for AI? I guess we could, but it doesn't seem like that's going to hold to me.

Switching gears to another piece of news this morning, I am going to share this piece. And this is, uh, from Professor Anima Anandkumar. We are training large-scale AI models that can simulate and understand physics to invent and discover. Our models understand the world directly in 4D, 3 dimensions plus time, and across physical phenomena. Going full 4D requires massive context length. We have pushed it to a trillion in training and exceeding 5 trillion at inference. And I think the most interesting part of this is that they're calling the company— okay, they call the company Accelerated Understanding, number one. It is Professor Anima Anandkumar with her husband. a rare husband and wife team. And it's very interesting because they also had an offer from Jeff Bezos. So they spoke to Jeff Bezos first. Jeff Bezos has a project called Project Prometheus, and Prometheus is about physical AI. They had an offer in hand from Jeff Bezos. And their offer, I think, was about $1 million salary to start with and then $2 million after that. And in addition, Jeff Bezos would give them $2 billion in capital financing, committed rounds through seed, Series A, Series B, and they would get a 35% stake in the company for that. So they would retain 35% for the 2 of them, 65% to Jeff Bezos and his team. They'd get $2 billion, massive funding right out of the gate, as well as salary. And they turned this down. They turned this down and they went and they got their own funders. It is unclear who funded them. But it is known that Professor Anandkumar was a board member of NVIDIA. And she earlier demonstrated weather simulation models, which were 1,000 times more effective or ran on 1,000 times less compute than prior efforts and were able to do large-scale weather simulation. And so it is suspected that Jensen directly funded the company. I think $2 billion for Jensen, especially when the $2 billion is being spent back on their own compute, is not a big deal. I find it notable because even 5 years ago, what would've happened is that professors like Anand Kumar would have had a PhD team that they kind of incepted the idea within the lab, funded them for a couple of years to get through the initial kind of discovery phase, and then spun them out into separate venture-backed companies. And I note that the amounts of money involved are significant enough, and the problems are significantly important enough that professors are leaving their tenures and moving full scale to AI firms, of which we've already seen a few also move to Anthropic. But yeah, it's notable.

Yeah, I mean, this is an area where I think there is unbelievable value to be created. So it's not surprising that people would be ready to throw a couple billion bucks at couple of Caltech professors. The same thing has kind of happened with Jeff Dean and gang recently leaving Google. We are sort of seeing this phenomenon of the last company we're going to found, or the last big project before the singularity. People are kind of revealing what they think matters most in the the kind of last tour of duty that they're going to undertake before the future becomes totally opaque. So I think that is quite fascinating stuff. I'm a huge believer, for a long time, there have been a few things that I've been watching out for, and going back to when I was first getting really deep down the AI rabbit hole. It was like I was using GPT-3 to write some copy, but I was also using image understanding models to sort and filter images for quality and content, and then the image generation, and then text-to-speech, and then realising, oh my God, it's the same architecture powering all these things. That means there's probably not too many modalities that can't work with these kind of core primitive components of ML architectures. And that also probably means over time we're going to see integration of different modalities. And that's still something I'm kind of watching out for, is like, how are the modalities of LLM-like reasoning and the sort of intuitive understanding of these different problem spaces going to come together? One thing I think we'll probably get into momentarily here is the Claude example. from the last week or so where Anthropic obviously said Claude designed these binders. And I think our first guest can tell us more about exactly how that design process went and what role Claude played in it and what other models were critical to making that happen. But I'm also watching for these understandings to become integrated in the same way that you can give a Gemini or a chat model now a prompt and an image, and it can understand those natively. When we start to see those things deeply integrated, those latent spaces or those reasoning or understanding modes deeply integrated in the same latent space, it seems like it opens up just another insane frontier for discovery because we can kind of, for a lot of these problem spaces, we can only do the kind of abstracted reasoning or the tool calling. We just don't have any sort of native sixth sense of how's a protein going to fold, what's going to bind to what. And certainly in these material science and physics realms, intuitions break down pretty quick. But the models, I think, can learn them. in a way that will really be a paradigm change at some point.

So that's probably a good segue for me to introduce our first guest for this morning. He is Sergey Edunov, Chief Technology Officer at Genesis Molecular AI. Sergey has spent over 2 decades building some of the most influential machine learning infrastructure on the planet. Most recently, he was a senior director of AI research at Meta, where he led the pre-training for Llama 2 and Llama 3, the open-source models that millions of developers rely on every day. In 2025, Sergey walked away from the language model arms race. He realized that scaling text-based chatbots is fundamentally different from understanding the physical and geometric rules of the real world. He joined Genesis to build what the industry calls molecular foundation models. His team is designing systems like their flagship model, Pearl, which predict how potential drugs bind to disease targets in 3 dimensions with sub-angstrom accuracy. A level of precision that goes far beyond what general-purpose AI can achieve. Sergey is here today because drug discovery has just hit a critical phase shift. Genesys recently rolled out an autonomous agentic system called Sapphire and published research proving that their models can speed up molecular screening by 500% while outperforming AlphaFold 3 on highly complex, flexible targets. We are going to discuss why he believes LLM architectures are boring, why popular AI benchmarks are fundamentally flawed, and how the hardest problems in artificial intelligence are now being solved in computational chemistry. Sergey, welcome to the show.

Thank you. Thank you. Thank you for inviting me. Nice to meet you.

Let me just jump right in here. Last week, Anthropic announced that Claude had found state-of-the-art molecular binders, as I understand them. And there was a bit of back and forth. I believe you had a post on how Claude orchestrated but the underlying models were really scientific models. Can you go into that a little bit?

Yeah, it was an interesting piece of research that Anthropic published and it definitely deserves attention, but there are different ways to look into it. What I found most fascinating is the prompt that they also fortunately released. With this research, the prompt that they use to steer their cloud models to do this kind of work. And the prompt is 16,000 words. So it's pretty large. It's a mini book. A lot of that is honestly kind of an SRE runbook. Basically, how do you orchestrate models? How do you run things? How do you run things in production so that they don't fail? And a lot of it is actually very detailed instructions to how you design proteins, how you use different tools. And it goes all the way down to specific instructions like, hey, you can download this model from here, that model from there, those are the hyperparameters you need to pass to those models to achieve good results. And in my view, it's a very good and advanced level orchestration. But the real work of discovering those binders was done by underlying models. Some of those were built by open source communities. Some of those were built by CZ Biohub. Some of them were built by, for example, RF Diffusion was built by Baker's Lab. And even some of them were built by ByteDance. So there is a lot of research that went to building those underlying models that Claude used to develop those binders. And then another important thing that I think worth mentioning is that, and they do admit it themselves, that protein binders themselves is not a therapeutic modality. So it's not something you can use. It's not a drug yet. There are so many steps ahead. to make any useful drugs out of it. And that is also, I think, worth recognizing.

Let me just go a little bit into that, because I think in AI in general, there is a lot of, oh, you know, vibe of, oh, you know, we're going to invent these things and they're going to give us these, you know, so many new targets. And that's how we're going to cure all these diseases. And I think the bio pushback has always been, we have too many targets anyway. And getting something to an IND is difficult. Getting it through all the way through is even more difficult. And even if you— when you get it through bioavailability and the fact that it works, the efficacy itself is another hurdle that you have to cross. And it's very complex when you have to— when you have these— human subjects who basically have so many other influences and so many other things going on. So how do you think AI in drug development can really speed up drugs which are developed? Is it target generation? Is the target generation never going to be enough? Is there any hope there?

Yeah, the whole drug discovery pipeline is very long and very complex, and target discovery is basically the first step. We want to find a protein that causes a disease in the body, and then you want to either activate this protein or deactivate it. So you want to find the binder that connects to this protein. That's a first step. And the reality is, for many diseases, we already know what the targets are. They're well researched, we understand, but we either don't have good binders or binders what we have, they're not sufficient enough and can be improved. So that's where a lot of work needs to go into, and that's what our company is focused on. So we are not focusing on biology, we are not discovering targets. We take over from the step where the target has been discovered already. We know the target, Now we need to figure out how do we design a key that connects to this target. In our case, it's a small molecule. And then just finding something that binds to a target is clearly not enough. It can bind to a target, but it can also bind to anything else, everything else in your body. There are like 20,000 proteins in your body. So if your molecule connects, binds to every one of them, it's kind of going to cause an issue. And some of those off-target proteins are particularly difficult. They tend to connect to a lot of potential drug candidates, and that is a known source of issues. So we don't only need to find the molecule that binds, we need to find a molecule that is selective enough, doesn't bind to everything else, and then we need to make sure it meets a lot of properties. For example, if your intent is to develop a pill, then whatever compound you produce needs to be soluble. That has nothing to do with binding affinity. So you have to be able to predict those properties as well. Then you need to predict all other sorts of toxicities. You need to predict how long this molecule is going to stay in your system. it be able to get delivered to a specific tissue? Like, for example, if it's a brain cancer, then it needs to penetrate blood-brain barrier. So you need to be able to predict all of those properties. And we are far from being solved on all of those.

One question I have is just like, I totally understand the breakdown in terms of the division of labour that you're describing, where Claude is sort of taking the place of the human biologist and using all of these tools that were built for various, often quite specific purposes. How many people do you think are out there that can actually use a pipeline like that? And how— one thing I have heard from people at the intersection of AI and biology over time is like, we've got a lot of great models, but the biologists aren't really programmers. They're not really used to running these kinds of pipelines. And so there's been, as in many other parts of the economy, a sort of diffusion bottleneck where the human biologists are just coming nowhere close to maximising the tools that are already available. And so I wonder, how many people would you, obviously rough estimate, say are in position to do that really well versus how many people now that Claude can do it can just ask Claude to do it and get that value? Is there, even if it is purely, which it is, purely orchestration of these tools, is there maybe a step change in impact because of the accessibility that Claude brings to such pipelines.

Yeah, absolutely. I mean, it's very similar to what we see in programming, where access to coding tools enabled so many people who maybe previously were not able to build their own applications are able to right now. So, Claude and, like, other tools, we ourselves were building Subfire, which is also an agentic framework that allows us to orchestrate all of those underlying models. Those are super powerful and they, in my view, create those 100x drug hunters that become much more efficient than what we used to have. It's true that learning every single model and tool that is available on the market is very hard. Not only do you need specialized domain knowledge to understand what is actually going on there, but you also need to learn how to use those tools in the most efficient way. And those tools change all the time. So having all of this abstracted away and driven by agent is potentially a super powerful innovation.

So as you make your own model with Pearl, Prakash mentioned that the sub-angstrom resolution is one of the big differentiators relative to other available models. Can you help me understand what that translates into in terms of overall process throughput or dollars it takes in terms of We generate X candidates, like Y of them we think will work if we're at sub-Angstrom versus only Z will work if we're kind of using the 2-Angstrom tolerance models. How should we think about the way that accuracy translates to practical value?

Yeah, I actually like to think about it as more of a qualitative shift than quantitative shift. And I can give an example. Remember, when was it? In like 2018, GANs image generation came up and the first generated images were on internet. And yes, a lot of people thought they were cool, but nobody would ever consider using them for like anything practical. And then suddenly, A few years ago, Stable Diffusion came up and it changed everything. And image generation became like real practical application and now it's everywhere. It's sort of similar in this space. Yes, you can use models to predict those molecules with lower precision. Like traditionally in research literature, people use 2 angstrom prediction, 2 angstrom accuracy. Now, the challenge here is that you often predict the molecule in the wrong way and you miss key interaction, like entire aromatic ring can be flipped and you just don't capture how this molecule interacts with protein, and then that prediction is completely useless. What we find in our work is what driving this accuracy down, it not only like you linearly improves performance, it makes those models actually useful in the real drug programs, and that's what matters a lot. So without this level of accuracy, you are not even able to identify potential molecular candidates that would bind. With it, you can.

Let me ask you a little bit about the construction of So, when you were building Pearl itself, did you have a kind of wet lab that you kind of tested compounds and fed that data back into Pearl? Was it purely a data processing kind of framework, or did you also have this kind of iterative process with the real world?

We actually do have a wet lab, and we have our own drug programs. We believe it's very important to dogfood our own product and having this in the company helps us a lot to deeply understand what is important for real drug programs. So that I think is one of our competitive advantages. That being said, data is the most important piece for training all of those models. In that sense, AI for drug discovery is not fundamentally different from LLMs. We also care a lot about data. One major source of data that we have is through partnerships, like our recent Insight deal expansion also comes with data that we are able to use to train and improve our models. And that's an amazing advantage. And I think That's the key for future progress. We do want to have more deals like that where we are able to train better and better models on pharma's data. Previously, pharma's data was always under lockdown and other companies were not able to use it. And we were the first who were able to tap into the source of data.

How, um, how do these pharma and AI company deals work? Like, not, not the Insight deal in particular, but let's say commenting on like one of the Chai or some of the other deals. How is— how are these deals typically structured? Like, because we, we see these numbers, $100 million, $200 million, $300 million. But how does, how does that work? Is it like, okay, I'm going to give you my database access, you're going to get, you know, uh, 100,000 samples, uh, and in return you pay us $100 million, or you give us $100 million in stock in your company, or, uh, if we develop, co-develop a drug together? How does, how does that work? Like, how do these 2 entities, one which is, you know, basically a tech entity, And one is, which is basically a pharma entity. How do they collaborate on these 2 things, especially when sometimes the tech companies' valuations are way, way out of proportion to the revenues that it has? So how do these 2 kind of fit together? What does a typical deal structure look like?

Yeah, there are different variations of those deals. They're all not the same. And I think different companies pursue different approaches. There are some pure platform plays where you basically pay for play as you go. There are more partnerships-like approaches. Like in our case, our partnerships, we work together on a specific set of targets that are exclusive to this partner. And it's a close partnership. It's an iterative process. It's not like they give us a target and we immediately give them a molecule. We set up this partnership for a long term and we work together. And there are different ways to pay for those kinds of partnerships as well. And I think industry has explored a pretty wide variety of options. But general components include upfront payment to set things up and to make sure that both sides are invested into this. And then milestone or usage-based payments. And finally, royalties. Those are standard components that go with those deals. And if you want to analyze those deals, it's important to look at all of those components.

I see. So typically, it's a pharma company says, okay, your technology is very interesting. We want to work together with you. We'll set up this partnership. There's some small amount to kind of set things up. And then as you deliver, then we expand, something like that.

Correct.

I see.

How about on the architecture side? My agents dug into your writing history and found a couple of provocative quotes about how not much has changed in language model architectures over the last few years. What's interesting going on in the architectures that are used for biology? Are we seeing like graph-based architectures winning? Or what would you say are kind of the more intellectually interesting architectural innovations that you get to play with that haven't made their way to language?

Yeah, I was definitely trying to be a bit provocative with I agree with some of his statements, intentionally so. Look, I joined machine translation team in FAIR, Facebook AI Research, in 2017. That's the year when transformers came up and they were machine translation field paper. So I was very fortunate to be amongst the first to scale up transformers on PyTorch, which was also built in Facebook. And If I were to go to sleep in 2018 and wake up today and go into any of those LLM labs, I would be astonished by the value that those models are able to bring, how powerful LLMs have become. But I wouldn't be that surprised by the model architecture and the underlying pieces. Because a lot of them were very, very similar. There were, of course, changes and improvements. There were tremendous improvements on the engineering side. Just the scaling challenge was a huge one. And there were some specific target architectural improvements, like we figured out how to do long context with LLMs that wasn't there in 2017. So those are very interesting. But if I looked at AI for biology space, it's just such a fascinating field and it's so quickly evolving. And yes, as you mentioned, graph models, graph neural networks are still being used. They're still popular and they're often part of a complicated pipeline, but so are diffusion models and so are language models. And people combine various pieces together and there is no consensus whatsoever. We still, I think it's still very, very hard to predict what's going to happen in 2 or 3 years, what models are going to be winning this race. I think we are far from over. We still have some relatively old pieces of the system. Like one example I keep giving is MSA. It's multi-sequence alignment piece, which is used in pretty much every model except maybe a few recent ones. And it works surprisingly well. It's actually amazing and incomprehensible how and why it works so well. But that piece is still useful. But it's essentially traditional retrieval mechanism. It's not even deep learning really. And in biology field, we combine all of those pieces together in very complicated ways, and we build complex pipelines that are hard to reason about. So I personally find it very exciting. It's a very fast-moving field, and the opportunity for innovation here is much, much bigger. Like, if I were a fresh grad student thinking which field to join, where I could have potentially bigger impact, I don't think it's LLM space. LLM space is moving full speed ahead. There are so many people there, it's crowded, while biology has so many opportunities for people to have tremendous impact and change the entire field.

So one of the questions that I have here is there's a lot of issues with sample insufficient samples in biology. For example, I think if you wanted to get a bunch of cancer samples, you'd have to pay a university lab, like, you know, X amount per sample, et cetera. To what extent are your architecture decisions driven by needing sample efficiency, in the sense that with LLMs, at least, you have this massive corpus of data that you can work with? And while for biology, perhaps you may have no more than, I don't know, 10,000 glioblastoma-like samples in the world that are available at any time. So how does this work?

Yeah, you're spot on. And we can't just scale the way LLMs, they're able to scale. The amounts of data is much lower in our space. And what it means is that we need to build inductive biases into the models themselves to make sure that we use data in the most efficient way possible. And we've done it a lot. And again, that's what makes innovations in this field more interesting. As the time goes, you need to integrate more and more complex inductive biases into the models, like integrating deep learning and physics.

Yeah.

is a particularly interesting field. Like, how do you make sure that information you can derive from physics-based methods can connect together and work together with your deep learning models? I find it fascinating and a very exciting area of research.

How good are the coding agents? And obviously everybody knows that the frontier companies are very focused on getting their models to be good at ML research. Personally, I think this is a little scary, but it could be less scary if it was applied to a narrower domain like biology and medicine, where I'd be less concerned about sort of runaway loss of control process and more excited about the upside that it might have. How good in your experience are frontier models getting at helping you explore architectural space? What are their strengths? I mean, obviously one of their strengths is going to be just prolific output, but beyond that, what other strengths do you notice that they have? And what sort of conceptual weaknesses do you notice? If there's a lack of taste, how would you characterize that lack of taste?

Yeah, that's a great question. They're definitely very useful and we have seen the Vengeance us a huge acceleration of our own efficiency. Like every engineer became so much more efficient now building those models and trying stuff. Like previously, actually we can go back to Anthropic's example, like setting up and orchestrating all of those models, that would require several people to work for months before. And now Cloud Code can do it in a span of like a few days probably. So that's pretty exciting and that accelerates a lot of progress. And I think it's a very powerful innovation. Similarly, with modeling research, if I have a specific idea I want to try, those models are really, really good at implementing this idea. Or if I have a paper that I want them to implement in our codebase and just run with it, they're very much capable to do so. Where I think we lack is generating these novel ideas. In my experience, they tend to go into the rabbit holes of kind of exploitation of incremental improvements rather than trying to rethink things from the ground up and design something that would be groundbreaking or at least has a chance to be groundbreaking. So that piece is, I think, still missing. I don't know how you can make models better than that. I guess you need to figure out how to do a RL loop that goes like all the way and unroll it so many steps beyond. So that might be a little bit challenging. So human taste is still very, very important in this field.

The UK AC report on the the sort of cyberattack, supply chain attack, social engineering episode from the Mythos model said that they had rollouts going up to 100 million tokens now. So it does seem like there's room there if we can, we'll see what the presence or absence of eureka moments looks like. Well, I guess one other kind of question that would inform for me how much acceleration we should expect to see from agentic help is how good are the scaling laws? How reliable are the scaling laws at small scale in the domains that you work in? Because if we give finite compute budget to an agent and say, okay, you've got this many flops to work with, and here's like a dozen different ideas, If they plot a scaling law at small scale across all these different ideas, would you expect that those scaling laws will hold and that very small-scale experimentation is enough to make good predictions about which architectures will really work at high scale? Or do you instead, alternatively, you could say, well, no, it doesn't really work that way in our our domains and there's no substitute but running the high scale, no substitute for running the high scale experiments?

Yeah, it's a bit more nuanced in our domain, in part because there are many challenges here. And maybe we can start with the most basic ones, like how do you even measure your model performance? Eval are still very, very limited. Like in language field, you have so many different ways to evaluate model performance and everyone is free to pick their own metric. Some of them are more stable, some of them are less, and some of them more predictive of the ultimate model performance, some of them are less, but you have a choice. In our field, the number of potential evals that you can use to even measure model performance is much lower. And then a lot of the evals that are currently available are particularly noisy. So if you are operating at a smaller scale, you may have challenges to even capturing improvements in performance, like simply because of the noise level of your evaluations. So that's one real problem. The other thing is in our space, we don't just have one model, right? Like, okay, structure prediction is one problem and it's what a lot of people are focusing on, but again, the reality is you need to be able to predict potency or binding affinity, you want to be able to predict all of the ADME properties, and those might be entirely different set of models on entirely different sets of data with their own ways to measure performance, and ultimately, A lot of evaluations need to be prospective, meaning you need to be able to predict and then synthesize and then measure rather than retrospective where you have some evals and you just measure performance on those. So there are all sorts of challenges like this that require more iterative process in developing those models rather than like, hey, let's just put all of the data together, run a bunch of experiments, pick the model that performs best on this data and go with it.

I have a question for you, which is a little bit of a segue. So I believe you signed a deal with a clinical research organization in China in order to do some work there. And you have a design-make-test loop that you are running with them. How is the experience working with the CROs in China? And did you feel the competence level was there to achieve the things that you were looking at? How do you evaluate how, you know, the Chinese firms are starting to make kind of leaps in biotech, primarily because they have a lower cost of labor and a lot of expertise there? How did that work out for you?

I'm probably not the best person to answer those questions. I'm mostly focusing on the AI side and technical side of the house. But Chinese labs are definitely very efficient in synthesizing compounds and running experiments. And whenever you have a very concrete set of assays that you want to run, they work really well and they allow us to iterate faster.

Perhaps the question is more like, how did you feel about the quality of the data? Because as you know, you probably have a lot of quality metrics that you have to look at for data, right? So how did you feel about ingesting the quality of data?

Yeah, but the quality is in many ways determined by the quality of assays that you prepare, and that needs to be done in-house. That cannot be outsourced. outsourced at the moment.

I see.

You probably heard me a little bit before we got started rambling about the integration eventually of reasoning models of the sort that we have today with different kinds of modalities, like we already have to a certain extent with images and video, is kind of where Google's Omni is showing how this can can keep going. I get very different responses, actually, when I ask this question of people. So I'm interested to hear yours. How excited would you be to see such integration happen between language-based reasoning and the more spatial biological modalities that you focus on? And how close or far off do you think that is?

I think it's already happening, at least through tool use. Like, again, going back to Anthropic's example or to what we are building in Subfire, the way those systems work, they basically integrate reasoning capabilities of LLMs with generation capabilities or molecular understanding capabilities of underlying models. Yes, right now this integration is mostly through tool use, but I'm more excited, I'm very excited about this potentially going beyond that and having deeper integration between those models. So there's definitely an opportunity here and we are very excited about this space.

Do you think this is some— yeah, please go on.

You can definitely imagine LLM but natively understands 3D structures or molecular modalities. That is not impossible and that can happen and should happen.

You're smirking like it sounds like you might be working on it.

I mean, I'm not going to deny it's a very exciting field. And as I said, everything is moving very fast. We are also moving fast. And we're looking into all of those things.

Speaking of moving fast, so there was an announcement this morning from a company called Accelerated Understanding, which, who they, I think they came out with a DFT prediction model. How would that kind of discovery integrate with your pipeline? Is it something that feeds into the rest of your models and that your models can use, or is it something that displaces your models?

I unfortunately haven't seen this announcement, so I'm not aware of the details. In general, I think it's better to use all the tools available because, look, the ultimate goal for us is to make as fast progress progress as possible in drug discovery. Our patients are not really like— I mean, the diseases we're fighting, they're very devastating diseases. And the sooner we can help the patients, the better it's going to be. So I would prefer to use every tool possible to make as fast progress as possible. And I would encourage more innovation in this field. I think we will all benefit from it.

I'll take us maybe in a little bit of a different direction in the time we have remaining as well. So obviously a big thing that everybody's trying to make sense of these days is who are the Frontier Labs and what are they doing and what can we understand, you know, from the breadcrumbs that we get externally? So you were at Meta for a while. I think you were there through the Yann LeCun to Alex Wang transition in leadership. And now we've got some new models. What is your perspective currently on Meta's effort? Do you think we should expect them to punch their way back to the top tier of model developers? What do you think people, even close watchers like us of the AI space, might fail to appreciate that we maybe should understand better about what's going on at Meta?

It's a good question. Look, I'm very excited about Meta and I think I would never underestimate this company and I think we're definitely able to catch up and even get ahead. There are certain advantages that this company has compared to others, like a distribution advantage is huge. And we have resources that are needed. And I think with Alex Wang, we were able to bring a lot of top-tier talent to bring this innovation into the house. So I think We still have a chance. I'm excited about what's coming. I don't have visibility into their current work, obviously, but I'm very much rooting for them. I still have a lot of friends working in this company.

When you look out at like the 3 to 5 year mark, and let's say you were given, you know, a significant budget, where would you allocate that budget to right now? Would you start buying compute? Would you start, Doing, you know, like let's say you have 100%, where would you allocate that budget?

You mean at Genesis?

Yeah.

Yeah, I think you have to, it's not a single item, what you want to buy, right? You want to bring in more talent and we all know top tier AI talent right now is very expensive.

Mm-hmm.

So you want to make sure you have talented people who are motivated by your vision. You also want to empower them with compute that they need, because there is no point in bringing in talent and then not giving them resources that they need to be successful. You also need to find ways to bring more data, and you have to become creative with the way to bring more data. Some of it will come through deals. Some of it may come through data annotation efforts, either internal or external. But either way, you have to pay for it. And I still believe in that lab-in-the-loop setup where you actually develop your own medicines in-house and understand what really matters for your model and platform. Because again, it's very easy to fool yourself with a benchmark. Like, hey, I show these good numbers here, my number is bigger than yours. But then if it does not translate into real progress on real drug programs, then none of that matters.

So maybe a companion question, this will be my last one, is what would you request from philanthropists if that's how you, you know, the kind of the things that you're thinking about investing your resources into. You've got, obviously, Zuckerberg is putting serious resources into trying to lay foundations for advances in biomedicine with the Biohub, and the Collison brothers are doing some incredible stuff with Ark Institute. And we'll probably see more of that coming as incredible wealth starts to become liquid in the AI space too. What sorts of things are better done by philanthropists and what would your kind of wishlist be for the likes of Biohub and ARC Institute to bring forward for companies then to build on top of?

Yeah, I think it would be interesting and we're doing it to go beyond the current frontier. and explore areas that are very hard for a commercial company to pursue. Like in our case, we are working on real drug discovery programs. So our models were very down to earth, solving real practical problems. But I really admire cell biology and models that deeply understand cells, like virtual cell modeling. is a very interesting area. There is a future potential here, but it's a bit farther down the road. I don't think it's like really practically applicable right now in real programs, but it's necessary that somebody is working on them, on those areas. And I admire those companies for doing that.

Did you ever take— it's funny how similar the company names are, but there was AIDO, general purpose simulator of cell biology, recently came out from GenBio. When you say it's not practical yet, can you maybe describe what— I don't know if you've had a chance to use their thing. I think it's still kind of waitlisted and you probably have to have a personal connection there, which maybe you do, but where do those things fail? If you would expect or if you know that that's not yet useful to you, what prevents it from being useful?

I think biology is just very, very complex and not very predictable yet. So we're just not there at the moment where— and even measuring things there is very hard. So I think we still need to work more on this area. Cool.

Well, that's all I've got. And I appreciate you for staying a little longer than we booked. Sergey from Genesis— sorry, let me get my outro correct here. Sergey from Genesis Molecular Bio. Thank you for being with us on AI in the AM.

Sergey, thank you.

Thank you for having me.

Great conversation.

Thank you.

Got to make sure I got my tabs well organized in front of me here.

Indeed.

You know, it always strikes me how complex biology is. It remains— physics is a lot more, I think perhaps a lot more discovered at this point, to the extent that the next discovery is maybe altering the fabric of reality. While biology, we can't even solve the common cold. It's just stunning how how different the 2 segments are, right?

I hear there's universal coronavirus vaccines in some mid to late stage of the development process. So hopefully the cold will be licked before we know it. Yeah, it feels like maybe there's— I mean, I guess my takeaway from that, because I've had this conversation or a variant on it so many times, with people who know a lot more about biology than I do, and they always kind of land on a similar spot of like, I just don't think that's going to work. And that to me suggests that maybe what the philanthropists with real conviction should do, which is kind of what I understand Zuckerberg to be doing, is just generate so much data that eventually we have enough to cross thresholds. This might be the kind of thing that nobody But a few ultra-wealthy philanthropists could really fund, would have the conviction to fund, because, again, I hear that perspective all the time, but it kind of seems like it has to fall at some level of data. And so I don't know what that is. Obviously, nobody seems to want to make a prediction on it. But there's got to be enough money in Zuckerberg's account to get us pretty far on the way there, right? I mean, good God. This is one of the more confusing questions where I never feel like I get a satisfying answer, but there is like a pretty consistent intuition.

We will see, I guess. Let me, let me, let me bring on our next guest, uh, and let me do a quick, uh, intro. Our, uh, next guest is, uh, our next guest is Dr. Michael Forssch, a physicist and entrepreneur attempting to rewrite the physical rules of how artificial intelligence is powered. Dr. Forssch earned his doctorate at the Max Planck Institute for for the Science of Light, where his work in quantum information processing earned him the prestigious Otto Hahn Medal. In 2018, he founded Q.Ant, a Stuttgart-based deep tech company that builds native processing units, chips that compute mathematically complex AI and scientific workloads using light instead of electricity. While the broader technology industry pours hundreds of billions into standard silicon and pins its long-term hopes on a distant, fault-tolerant quantum future, Dr. Forsh argues that the immediate solution to the data center energy crisis is analog photonic computing. His hardware is not a speculative science project. Q.Ant's second-generation processors are currently deployed at major European supercomputing centers and are commercially accessible via the cloud. He's here today to explain why 95% of data center energy is wasted moving data, how computing natively with light solves this fundamental bottleneck, and what the future of AI drug discovery looks like when our hardware finally catches up with our software. And Michael, good morning. Welcome to the show.

Good morning.

Hi.

So let's just dive right into it. Tell us a little bit about photonic computing and tell us about your deployments in supercomputing centers in Europe right now.

Right. Thanks for giving me the opportunity. By the way, first thing, thanks for calling me Michael and not Dr. Furch. Second, you can simply call my company QANT. We've gone through a lot of iterations of how we pronounce it. It's an acronym of the values of the company: quality, anticipation, novelty, and team. And there have been a lot of how to name it, and in the end, our customers agreed on to simply call it Qwant without the dot.

Right.

So what we are doing is photonics computing, and there is usually the question why, right? We have computers. And well, in our belief, computers as we see them today are a bit stressed by what we actually intend to do with them when we look on what we dream about when it comes to AI. And, the most simple thing, I mean, we can always push to getting more and more scientific speaking, but in my very— when I explain the difference between our processors and the processors that are currently integrated, successfully integrated now for 40, 50 years into the stack, if you look on the fundamental CMOS chip, this fundamental CMOS chip never made it across the second class of primary school. Because it can multiply and it can accumulate. So it can do plus and multiply, and that's it. Whatever you want to do on this machine, you have to break it down in something that's plus and multiplication. Now, the processors that we are bringing to the stack, they went to high school, eventually also to university. Let's see how far we can push them. But on the fundamentalist level of these chips, we can offer complicated functions like sine, cosine, exponential, Fourier transformation, convolution, oscillations, and all those kind of things, and you do not have to break them down.

Hmm.

And this fundamental difference now offers something that has been known in math but not really used in computational science. So in math, it's clear that you can Sort of trade amount of data or data complexity into function complexity. That's well known in when you talk to mathematicians, but it never paid off in the computational science part because everyone starting such a revolution knew in the end we still have to break it down on fundamental operations of plus and multiplication. So all the algorithms that are currently out there have. Have always been developed with the knowledge that in the end it has to be simple. And that's something that we offer, and that's what we started to demonstrate on and have demonstrated on use cases that we on the one end provide a new processor, and this at the other end opens doors to algorithms that can allow artificial, so AI models, networks that come with the same result but a fraction of the data. And when looking to the stack, let's take a 3-nanometer node regular stack. The energy is currently used at the memory. 95% are consumed by the memory, not by the processor itself. And the less data you obviously fetch from the memory, the less energy you're using. So parts of the community currently are optimizing on the 5%, trying to get things faster on the simple math side. We decided on replacing the core and by that supporting that a fraction of the data has to be shipped across the stack. That in the end saves the energy, but also helps to improve on the performance side.

Let me stop you there and talk about the interface. Like, at some point, you still have an interface between the photonic portion and the digital portion, right? Is there still a kind of translation tax between the two?

You're— okay, very well. That's the point, and that's where you have to be precise on. Um, so in the photonics world, everything is fine. There is one minor problem—

2.

Okay, 2. It's great if you have a company that only has 2 problems. Um, the first problem is we don't have a memory. We don't have an optical memory, um, semiconductor integratable. So that's the first thing. It can be a benefit. I come to that a bit later. And the second one, photons are not standing still. Damn hack, they're always moving. So that's the second problem. So either you basically compute while they're on the propagation, or you have to back convert them into electricity and then finally into a digital memory. So if you don't think the concept very well through, then you're basically eating up the energy that you saved on the computational optical part directly at the ADDA converters because they again use a lot of energy. So the strategy here is, first of all, use models that inherently transport much less fundamental data into the light. So this already saves ADDA conversion blocks. So if you just go and take photronics and let it multiply and accumulate, then you have the amount of AD/DA converters connected to HBMs or DRAM, they will eat up your benefits easily. And the second one is you have to think how to expand the grid. The longer you stay optical and the more computation you can consecutively basically line up in a row before you go back into the digital memory The more benefit and the more gain in comparison to the CMOS stack you have. Because clearly at one point you have to go into light and at one point you have to stack back out of the light. And the longer this chain in the end is and the more operation you can in a sequence lineup in that photonic architecture, the easier or the more benefits you get. There are a few descriptions that came from the community. One part says, okay, what we currently are building and shipping, you can see this like an in-memory photonic computer because we are in between two memories and photonically computing between two memory cycles. Others, and this is what I usually prefer if I'm openly speaking here, I should speak openly, right? Is that we We call this a streaming architecture. So, and now I'm coming to what some might consider as a drawback, that you don't have an optical memory. If I look back on how we got to the point where we are, I would say this was a clear benefit.

Why?

Because we just accepted there is no memory. And this prevented us to think in categories like the von Neumann architecture, And this opened up doors to fundamentally think computing from the abilities of light and not trying to copy and paste something that has been working digitally very nicely into the analog optical domain, always searching for the next hop where I can memory out my information to basically get in sync with all the others. So this It's a drawback if you come from CMOS. It's a clear benefit when you look at it from the photonics perspective.

Here's a real simple kind of idiot's question. What's going on at the heart of photonic computing? I'm certainly no GPU expert, but I can give you kind of, Again, at least a poor man's understanding of what is going on that allows information to be successfully processed to translate inputs to outputs. When I think about the photonic computing, I really am purely speculating as to what's going on at the heart of it. You said you're using the properties of light, but can you kind of flush that out a little bit more and describe, like, you know, at the kind of core of the computing mechanism, what's interacting with what, how are these interactions happening? And, you know, obviously, you'll have to leave a lot of detail aside, but how are inputs fundamentally being translated to outputs?

Right. Oh, that's— now we're getting really, really technical. I try to basically use an analogy that probably most of us are at least aware of that it exists. Maybe not everyone would directly know what the result of this is, but in the end, we we know that light is a wave, right? So and we know that two waves can interfere. The best picture I have from a daily experience is when you throw one stone into the water, you have a wave. Propagating. And if you throw a second stone at a distance, then the second wave is going to be generated. And then the intersection point, you have an interference. And this interference is already a very complex computation because it depends on the amplitude, the distance, and everything like that. So when you accept that there is more than just one mathematical space where you can describe your problem in, then you can effectively describe every problem by wave functions in light, and by interference effects you can compute. Um, that's one part of what we do. The second one is when light propagates through a circuit. Unfortunately, none of you is wearing glasses, and why I'm saying that I'm not sure what to you and the audience here a Fourier transformation means. Effectively, it means, for instance, converting from a time domain into a frequency domain, or converting from— that's probably the best one— or from amplitude into phase. But, um, so basically there is always a consecutive space, and it's a very powerful mathematical operation that's for mathematicians often used in order to simplify the problem in terms of you reduce the amount of data to describe the actual problem. Unfortunately, this function or this operation is very expensive in CMOS, but light, when you wear glasses, they're continuously transforming the state space, so basically the picture, into the k-space onto your retina without using any energy. So when light goes through a diffractive medium, it naturally does this operation. The only thing is you need to know how to exploit it and now to get this into a circuit, but then it's basically a passive element that can perform Fourier transformations free of charge. The same is true, um, for instance, for something that's currently very, um, um, very interesting for AI models in the state space domain. domain, you need oscillations, damped oscillations. So you start an oscillator ringing and then it's damped and basically you have something that looks like a damped oscillator. That's something, a passive element in optics. You can just bring it on the die, you can tune it, set the frequency, and effectively what it does, it's a cavity, it does exactly what this mathematical operation wants you to do for the model. There are a lot of these analogons that we're developing. So when you ask what we at Qwant are doing, so we currently have the full value chain under control. And when I say this, it sounds a bit crazy because we have it from the material, from the wafer production. We have a pilot line, we own the pilot line for the chips. We have the processors team that basically builds PCI Express cards that have inherently an optical core but are connected via digital interfaces to the x86 stack. We integrate everything and we also take care about the algorithm space. And this junk hand in hand means the algorithms guy have an application and a problem, they describe it in functions that are natively operatable in Lite. The chip team then looks on how to bring these functions natively onto the circuit and the integration team then builds the processor.

So as I understand it, you guys earlier this year demonstrated an object detection model compiled from PyTorch using an interface, a software bridge called DaisyTuner onto your stack.

Yep.

What are the translation steps which are occurring between a normal PyTorch And your stack in between?

So what we did together with DaisyTuner was we took PyTorch as the standard model library, and together with them, we built a compiler that ported this model directly onto basically machine code that was executable by the photonic processor. I mean, in the end, it's a translator, right? A compiler is nothing but translating from one language into the other, and this compiler actually translated it from PyTorch into the machine language of our processor, which in the end also demonstrated— and why did we do this? Because we've often been asked, okay, great, you're doing the chip, but then you need something like Okay, I'm not making advertisements for this company, but you need something like CUDA. And we said, no, no, we don't actually need that. And not because we are claiming we don't need it and have something different, but the photonics architecture works fundamentally different. It's, as I said, it's a streaming architecture. You don't need a redistributional layer to basically bridge things across chip. You always have to think it from the light perspective. And we said, hey, we believe we can make it happen, that we can build a compiler within, I think it took us less than 3 weeks together with DaisyTuner to build a compiler's architecture, to port this model from PyTorch onto our chips, which was an image classifier. And we had a head-to-head comparison on the T-Systems Challenge together with them. Versus chips from the GPU world. And from the quality, I'm not claiming that we're already at speed of, for instance, AMD or NVIDIA. So that will take a bit of probably 1 or 2 more years. But from the quality, in the end, we had reached the same result. So we had the same picture identification and we even had the same rate we could identify at a frame rate of, I think, 40 frames a second on this PyTorch model on our generation 2 and could correctly identify cars, trains, and so on on a video that was the showcase video. It was captured in front of the T-Systems building. So that basically showed— because there is often this, yeah, yeah, that's great, but it's so scientific and you can only test some functions and it's not really useful already in today's applications. We said, no, it is. You can do this. We need more years in order to get at the same robustness level, to have a fully fledged compiler architecture that is basically proven against a lot of use cases and all these kind of things. But fundamentally, this technology is on the rise to basically already— it has already— it can already step out of the laboratory, which it does. So We operate in industrial data centers already on specific applications. And this was one of the showcases where we thought it's a good showcase because it's so intuitive to understand. If this can be done, then a lot of more things can also be ported from PyTorch and directly executed.

So what are the intervening, like, what is gonna happen in the next 2 years? What are the steps that you need in order to get to where you need to go. And when you reach there, what are you going to be comparing against? Are you going to say like, okay, I have an A100 or H100 and our chip is going to do this specific model with this specific tokens per second? What are the comparisons that you're going to use in a couple of years and what are the steps that you need to get there?

That's the question. Um, so first of all, since this technology is different to the GPU world, it doesn't make a lot of sense to compare, for instance, tera operations per second, um, because we are not digital. The same is true for the flops. It's a bit of a metric that was built for the math domain. So what we agreed on is let's go on Inference per second per watt, and take and take this per application. What it means is if you have, for instance, an I can give you a bit of an insight what we see as a great potential case, a first demonstrative case. If we go back on this image classification now, the world record, at least to our understanding, is that. that you can identify 16,000 pictures a second, or maybe it's 24,000 pictures a second per 70-watt energy. So the inference is the picture, that's your job. How many pictures per second can I inference correctly at the same quality? And what was the energy consumption? We see very good chances that in already next or the early early 2028 that we can build a system that's outperforming the CMOS world in these specific applications by a factor of 2 and a factor of 10 in energy. Because of, we basically allow for different type of models that are way more, that are way more less demanding on the data overhead data that you need to describe the model or that you need to identified the picture correctly. And that's— it's not the largest market on earth given, but it's a very clear use case to demonstrate the capability of a photonic system. The other things that we see is that, for instance, this technology can open up doors also into the transformer architecture. And what we are spotting forward is, or we're looking on, is dedicatedly on the attention layer. we believe, or there are papers among others from Google out there that say that the usage of Fourier transformation in the attention layer can realize an enormous KV compression. Now, the point again is offer a Fourier transformation that's cheap, and you probably can contribute to this stack substantially. So this is where we're currently heading. Now, you asked also what it takes to take this really So the first one, take it at scale. We have a pilot line in operation that is good for a few thousand wafer starts a year. That's okay, that's great for us for the moment, but we need to basically now license this out into commercial foundries who scale these to 10,000 to 100,000 wafer starts a year. It's doable, it's not so complicated, but it has to be done. The second one is, we already spoke about, there is the photonics chip. And the photonics chip is, well, it's photonics. The integration layer into the electronics needs— there is also a lot of room where you can lose or win performance in this transformative space. So this is where we, on the systems integration part, that's where we also currently are very strong focusing to basically get the memory, the digital memory energy efficiently connected to the photonics core, the photonics core itself, I would say that we are pretty much there. So on the photonics chip side, we can already tape out photonic chips that are competitive to the CMOS world on dedicated applications comparisons. What currently has to still— is still lagging a bit is the integrational part from the hardware layer to the memory. And of course we need to get to a more robust software stack, but these are, in my speaking, not fundamental, but more engineering tasks that we need to complete on. And once we are at this point, the scaling to the route to the future, in my opinion, is the first one, We believe that these new processes are triggering a model revolution in the AI space, requesting again new functions that we are going to build. So there will be a forth and back between the algorithms teams around the world from the AI fabs with us about delivering particular functions for novel models. And the second one on the hardware side is where I'm dreaming about is the following. I want to take this to a real streaming architecture. Think about the following. Step 2 could be you're not thinking in a PCI Express card, you're thinking in a mainboard. And the mainboard has a host CPU for the operating system, AD/DA converters on the beachfront, and then you have 32, 48 photonic chips optically fully connected and you stream it across these chips, and then you go back into the digital memory. That would be on server level. But why stopping on server level? You can take this— interconnects are built currently— you can take this to the next, to the next, to the next server as long as your signal budget is still good. And if you really want to take this to the extreme, you could start computing directly in the optical grid. So that's—

What?

Like you mentioned some applications where you can already see kind of like a 10x improvement coming along. Like which are those applications? Which models are, you know, which architectures does photonics already have an advantage on?

So in general, every model that inherently builds on nonlinear and complex functions, because we can execute them way more efficiently and faster than the CMOS world can do. In particular, these are the state space models, these are the diffusion models, these are hybrid mixtures in between these models. This is physical AI. I mean, think about the following. We all went to school, probably we all had to do once in a while a coordinate transformation from a Cartesian coordinate system into a spherical coordinate system. We can directly execute the spherical coordinate system. You don't need to remap map it into the Cartesian ones in order to bring it on the chip. Or we've tested, um, that, for instance, algorithms on the simulation side, weather forecast or fluid dynamics simulations— fantastic. I mean, model the world as a sphere and directly simulate it on a photonics computer. That's where you can go with these machines too, right? You don't need to go into the fundamental math. And what we also, I mean, now I have to say it because it's a big word nowadays. I mean, we also see that we can contribute to the world models. At this point, I must say, we need to understand even better what it means. I'm gonna open here. So while everything that I said so far, we have a clear mathematical model in mind, The world model category is fascinating. And if we would model the world, we would do it as physicists would do it. We would use harmonic oscillators. That's how the mechanical world is modeled. And that's at the heart of our computing. And this is what we currently also look into.

When you look at your supply chain, Right. How much does your supply chain overlap with the standard GPU supply chain and manufacturing process? And how much is going to have to be invented anew in order to scale this?

So I would say that the most fundamental difference to the classical CMOS world is that we have on the material side not silicon, but we have silicon and on top there is this thin layer of lithium niobate.

Hmm.

And this thin layer is where the light is propagating. So it's 99% silicon and a few percent, if it's even a percent, of this, um, of this optical crystalline material. Now when we look on how we built our pilot line, we've been using a pilot line that existed. It was a former CMOS line in the 1990s. It had a 90-nanometer node technology, etch tools, everything standard. And the only thing we did was we replaced a few tools because of they were not compatible. You cannot on some machines you can't run silicon in parallel to lithium niobate because of cross-contamination. That's it. But from the, the tools that we have in the line are standard off-the-shelf semiconductor tools that you buy at regular vendors. There is nothing secret about this. The secret is that the processing strategies of the thin film lithium niobate, they're owned by Quan and they're different to the silicon strategies. But in the end, you do exactly the same. You You have a wafer, you build a mask on it, you structure the mask, you etch into the mask, you remove the mask again, and then you have your structured wafer, you dice it, you passivate it, you package it, and then you build it, bring it up on the processor. So from that perspective, does the industry really have to change? Not really. We also discussed with fabs whether they would be willing to bring some of their lines to be manufacturing lines lithium niobate, and they said yes, as long as the volume is there, they have no problem in turning silicon 90-nanometer or 45-nanometer lines into lithium niobate lines as long as the quantity of that— as long as the demand is there. So does the industry really have to change? No. Do we have wishes to the established industry?

Yes.

Um, yes, of course we do. Um, uh, in particular, we would would love to also cooperate to build more energy-efficient AD/DA converters because this is currently the point where the most energy in the system is consumed, and we believe there is a lot of room for improvement. And the rest would be to go on a discussion about does it really need to always be, for instance, PCI Express, or can we also wipe again some established standards in the favor of bringing photonic processes even faster into— I mean, as long as we look at only from the server level, we don't have a problem because if you buy a server from us, you plug energy in and the Ethernet or InfiniBand and that's it. You don't need water supply, you don't need anything because systems are not getting hot. So energy demand is so low that that's all every one of us could have such a server back home. If you go inside the server and talking about the integration on the mainboard level, then I would say we can talk about standardization there as well.

When you talk about 45 or 90 nanometer nodes, obviously those are not the latest and greatest nodes. So does this mean From a sort of global supply of compute perspective, that as this starts to work and scale, it will just be almost exclusively net new compute coming available. Like, this is competing with stuff that is like relatively low-end lines, right? These these chips would be like the chips that go into toys or whatever, right? Not anything close to what would go into a modern cell phone or into a modern AI stack. So how much of a sort of, what's your dream success scenario look like in terms of without photonic computing versus with, how much bigger does the overall supply of compute available for AI get? What's that delta look like?

Actually, what we've demonstrated by— I mean, look, we are in Germany. Germany is known for a lot of technology, but for sure, we are not famous for logic computing. We also don't have 7, 4, 3 nanometer node fabs here, right?

Mm-hmm.

And still we managed to get these systems running and even the pilot line. So what we've demonstrated in Germany on a 90 nanometer node can be copied across Europe, it can be copied into the States, it can be copied across the world. If this technology starts winning, actually you can turn a lot of existing fabrication sites without the necessity to rebuild new ones into fabrication. So the bottleneck that we currently having in access to latest node fabs and discussions about business cases, whether the business case will still hold for a 2-nanometer node. I'm not judging on it, but the discussion is on. They are not there. So this technology can become, I wouldn't say democratization, but effectively it is reducing the complexity of the supply chain by half. So at this point, we are from the wafer to the processor, nearly self-supplying. And that's another angle where I see that this technology, besides the beautiness of the performance, the reduction energy, but simply the production capabilities that this technology offers to scale, they're so much easier than going on a 3 to 2 nanometer node, eventually being picked up by an MPW run somewhere next year, mid and then getting your hero chip back and then trying to get volume behind the line because it's damn expensive. It's, and every, I mean, it goes through the whole process, right? A mask on our side is cheap in comparison to a mask layout on the logic CMOS and so on and so forth. So all these dimensions are offering great capabilities to on the one side reduce production or at the same side increasing the volume very rapidly.

So if we don't get this at the scale that you would like to see it delivered and we're looking back and saying, why didn't it work? What would be the, like, couple candidates for the remaining really hard problems that could derail the whole, you know, upside of this? Are there— there got to be some still left, right, that aren't easy to solve?

So this is the question. If we're not making— so I'm sorry, I was a bit puzzled. So the question is, what happens if photonic compute doesn't make it to success? What would be the alternatives? Is this the question?

No. Well, that's an interesting one too, but I meant more like, If you're not ultimately able to succeed in scaling in the way that you want to, why would that be the case? What would be the failure, the most likely failure points?

The most likeliest failure points will be that, um, um, in, in my opinion, is that the established industry is too rigid in its existing business model.

Mm-hmm.

So it's, it's a question of, um, whether, um, the supply chain and, um, our partners and future partners are willing to accept that there is more than just silicon, um, and then there is more than just von Neumann digital computing. So if this is accepted and if people are starting trusting in that there can be volume, which by the way, it's starting already, then the great thing about it is that you can repurpose existing knowledge. So when you run, for instance, a fab, right? I would say that from an engineering point of view, there is not so much difference actually whether you manufacture a silicon chip at 90 nanometer or at least a niobate chip at 90 nanometer. Yes, there are peculiarities, things are a bit different recipes, you have to look on some more parameters or less parameters or different parameters, but in the end, this is engineering what they've done for 40, 50 years. And the complexity is what it is in these dimensions, from inspection to everything, right? Everything is much easier. So everything that these companies have as knowledge as of today can directly be used in producing photonic chips. So, um, in, in my opinion, the, the technological barrier or risk of this technology is comparatively low when it comes to production. Um, the, the actual risk that this technology never makes it at scale is that we're too connected to to the innovator's dilemma, and industry players are not swinging into this opportunity. But honestly speaking, the options are so great, and, um, I, I rather tend for, um, I, I would, I would place out a different bet. Um, I would place the bet photonic computing is, um, at least, um, In AI computing in the next 3 to 4 years, we see this as a permanent add-on coprocessing unit into AI-based data centers. The more tricky question is who is going to be the first company that makes this bet come true? I think that it's going to happen, I'm very sure about. It's more about that the first mover who conclusively can bring such a system in a production and in a productive level is taking up a large portion of the market.

One of the questions that I have for you is, you did work on quantum computing earlier in your career. And now, so you have these 2 paradigms, quantum and photonic computing, and you've obviously chosen the photonic computing angle. What do you see in quantum computing that makes it perhaps less likely in the near term? And what are the challenges that you think, you know, are there that, you know, will be hard to resolve?

So I give you— so as a German, I give you an analogon that tells you why I still— I'm very much convinced about that we need a quantum computer as well. But we need it for different reasons that is usually communicated in, in the press.

So we—

I've been working on quantum computers, I think, for 10 years until I decided that it's going to be an analog computer. So photonic computers and quantum computers are both analog computers. The difference is that the quantum computer uses quantum effects and the photonics computer is using wave effects of light. And, the way I see it is the following. Germany is a car company, right? So, we have the CPU, which is the station wagon. It has 5 seats. You can pack the children in. You can go grocery shopping. Everything is fine. You can also have a lot of horsepower, but no one expects you to win the Formula 1 race. It's not the right car, but you need a station wagon. Every driver needs such a car as long as he has a family. Now, the GPU, in my opinion, is more the quarter-mile dragster star. It does one operation, it does it excellently, it does it parallelly, it does it at speed, but please don't ask this car to turn into a corner. It's not going to make it because it's not built for that. Now, of course, arrogantly as I am, I'm saying we're the new Formula 1 car because we have way more operations. We can drive around the circuit very fast. but please don't go grocery shopping with our car. Or in other terms, don't let the operating system be executed by our chip. We are also specialized one, but with a bit more universality than the GPU is. Um, and the quantum computer is the boat. It's a vehicle, it's great, you need it because none of the other 3 can go across a lake, but it has special functions. But as soon as you put it on the road You need something to pull it around the rope because it can't do it on itself.

It's—

so this is the way I see it, because quantum computers are built to solve quantum mechanical problems. Not every problem is a quantum mechanical problem, and it does not make sense to turn every problem into a quantum mechanical description. And as long as it's not a quantum mechanical description, you don't need a quantum computer, full stop. The But this is the way I see it. So the quantum, by the way, one remark, because you have a huge audience, maybe I can also vote with your audience for something. I think the terminology quantum computer inherently is wrong. It should be quantum processor because a computer is more than a processor. It owns the memory, it owns everything. And what we are building are quantum processors. They're yet again coprocessors to the stack. And there are a lot of wonderful literature, and there is, um, there is, there are great scientific papers about even hybrid systems between how a quantum computer in conjunction with a classical computer can accelerate things. It's exactly the same with the car and the boat. In the joint, if they join forces, you can go across the lake and the street.

It's a great analogy.

I wish I was smart enough that I had to develop such extended analogies to explain my work to people who can't get it at the foundational level. Maybe last one for me. If, you know, there's a popular investment strategy that's become a bit notorious for, you know, a mix of reasons recently, that is Looking for what's going to be the scarce component as new kinds of systems scale up. I think you may have already kind of answered this, but what would you say are the bottlenecks that people should be investing in now to be able to supply the future at scale that you envision?

And obviously, So if people want to basically get a significant share of this success, I would encourage them to search for legacy fabs and turn them into production capacities for this technology. It's not so expensive in the investment, but the same fab out of a sudden can sell the same square millimeter of die to 10 different customers. die at a much higher price because the value of the chip or the problem it out of a sudden solves is so much bigger than bringing a 90-nanometer chip into a child's toy at $1 of the toy.

Yeah, that makes sense. How about the digital-to-light converters? That was what I thought you might say.

Yeah, but that's not— I think this is not an investment. That's a technological problem. Okay, now I got Your question. So that's on the engineering side where we need to basically get into the component space together with the semiconductor experts to develop miniaturized, lightweight, or less consuming ADDA converters. That's something that the whole photonic industry, us, photonics computing, and the interconnect I'm betting about are dreaming about. So those who solve that problem have a gigantic market in the photonics space in interconnect and computing. But this is, is it an investment? Yes, of course, it's always an investment point. But I would say that this is a, I would encourage the universities in this field to be genius enough to have the IP and then team up with one of the global partners in that field to get it at scale very soon. So that's my dream there.

Nice.

Thank you. It sounds like the easier money is to be made converting old fabs.

Eventually. One last question from me. To what extent do you— have you seen workflows inside your company being converted to use AI tools in the last 6 months?

It's exactly 6 months. So yes, we started to use AI internally in many categories. We have so far not used it for R&D purposes, but What we are definitely doing is from sorting the email inbox to basically writing meeting minutes to also structuring our database that we find our stuff again. That was basically the points where we've so far been. been using it. There have been discussions about whether AI could also accelerate chip design.

Mm-hmm.

But as of this point, at least what we've seen, it needs a bit of work. And now I'm coming to something where, you know, where maybe 2 things have to gather. The chips that we are building are inherently harmonic oscillators. So the best thing would be to simulate a photonics chip with a photonic processor. That would be the optimal point because we would simulate the system with its own dog food essentially. And that's something where we might start activities very soon. At least we have some scientific thoughts to do that in the future.

Your own form of recursive self-improvement. Dr. Michael Forst from Quant, thank you. Thank you for joining us this morning.

Thank you very much. And when you come to Germany, we have great technology and in the meanwhile also coffee. You're always invited.

Best.

Thank you very much.

Thank you.

Fascinating stuff. I mean, I wish I was smart enough to understand it as deeply as I would like. I truly feel like— I mean, I can sort of step through the Fourier transform notion, but my intuition is still pretty limited in terms of how it really works.

I found that his explanation was probably one of the better explanations that I've come across. As you know, to kind of restate the explanation, I guess what he was saying is that if you have a digital chip, you have this kind of like electronic wave, and then you have a semiconductor, and then it decides It's a 1 or a 0. If it's above a certain level, it's a 1. If it's below a certain level, it's a 0. And with that 1 and 0, you can basically do addition and multiplication. And everything else that you do has to be a function of addition or multiplication. So you can't, you know, every other thing is built on top of that framework of addition and multiplication, every other function. So if you have like a fast Fourier transform, it breaks down into like, you know, hundreds or thousands of, you know, FFT samples, which break down into, um, you know, again, addition and multiplication, like, which was maybe, you know, thousands of operations or millions of operations. And basically what he's saying is that instead of having that kind of, you know, electric wave and with a 1 and 0, you have these light light waves. And when they interact with each other, they can produce the end function point rather than the initial ones and zeros. You can jump straight to the sine wave or the cosine wave, and you can basically take a measurement of that sine or cosine wave and get your response back. And essentially, this is also what our friend Beth Jezos and Some other people are doing with thermodynamic computing, as they call it, and some other forms of analog computing, which is basically instead of doing the 1 and 0 as the baseline and then breaking everything else up into the 1 and 0, you are directly measuring that cosine wave or sine wave or other mathematical function. And that cuts short this— maybe hundreds or thousands of operations that you need to get down to. So it's less general, but you get to skip a few steps. But as he pointed out, there are other complications. You don't have memory. You can store the ones and zeros in memory. You can't store this kind of wave in memory. And then you probably have a bunch of timing aspects too, because it's stream. So it has to, if your streams are off, your cosine and sine wave interaction waves are off, like you get a different reading. So you have a bunch of other complications that come along with that, that they have to manage, I guess. But, you know, in, in, in essence, it's that. Um, I also found his quantum computing description very interesting. So, uh, that is also, that is also a good, good explanation of quantum computing. He compared it to, you know, where when everyone else is building a car or a race car, They're building a boat. So it's going to get you across the lake much faster than the guy taking the highway, which is circuitous, but it's only usable for that one function.

That's good. I think I might still have to ask Claude for some sort of artifact to help me build intuition Even further. Like, I couldn't still answer, you know, there's gotta be, and I understand not always wanting to, you know, get right down to it in an interview setting, but there's, I feel like there's still gotta be something that's gonna be really hard about scaling this up. Otherwise it would already be scaled up, right? Like, what are the, what are the core things that might prove really difficult? I don't know, but you mentioned timing, you know, is it maybe the kind of thing where they're— The medium through which the light travels has to be like presumably exactly the right length, or if it's not, you know, things get out of phase perhaps and don't work, you know, where does the In fab analysis in general, there's always the question of what is the yield, right? How many of the things that you make actually work? Would that be the kind of thing that would limit yield in this kind of process? I don't know. Maybe Claude will be able to help me get to the bottom of that. But yeah, it's an unbelievable— this is the AI Scouts opportunity and curse all in one conversation because AI is now just intersecting with everything. And increasingly it's like hard-ass science that takes a long time to grasp the fundamentals of, and it makes it very difficult to know how to incorporate something like this into one's worldview. If you had conviction on this, not only do you have investment thesis, but you have a POV on sovereign AI and the international politics of this become very different. It's a very different world if European 90-nanometer legacy fabs can be converted and can create a significant amount of compute, it becomes much harder for the US to bully these other countries based on our position in the supply chain. It becomes much easier for them to at least get some domestic base of computing. But yeah, you got to have conviction before you start propagating this through your entire worldview. And for For me, it's not to use a Claudeism, but my broader world model updates are gated by my lack of understanding and not wanting to be too quick to take something on board without really understanding why it might not happen at the depth that I would like to.

So let me maybe share one reason why it might not happen, which is, you know, current chips just get better, right? So this morning, OpenAI announced Jalapeño, which is an inference chip. It is their first custom inference chip. They've been testing it. They have performance numbers where they compare it to existing best. Existing best is the NVIDIA B300. They have not named the existing best in their material. And they have said roughly that it's perhaps 50 times on mixed token, 50 times on single token, something like 2 times, you know, 2 times, 3 times, basically a bunch of state-of-the-art numbers. They have compared the Grace Blackwell 200 here in number of tokens per second and, you know, mixed tokens per second per kilowatt hour, per kilowatt, all, you know, industry-leading benchmarks, et cetera. And I think this is one of the challenges that you have. So I would say, from my point of view, it's great that OpenAI has announced this chip. They say that it'll be in data centers by the end of next year. Great. But it does— the number that I go back to from Nvidia is 1 million times increase in performance over the course of 10 years. That's Jensen's target, 1 million X over the course of 10 years, which is what they've achieved in the past 10 years and what he wants them to achieve in the next 10 years. The problem with that is that Nvidia has to 4X the performance every year. So every year is a 4X performance increase. And what ends up happening with that is that let's say OpenAI tapes out this chip, has finished taping out this chip right now, and they're comparing it with, let's say, the B300. The B300 was taped out in December 2024. So it's a 2-year-old chip and they have a performance increase of Kind of between 4 and 10 times on a 2-year-old chip, which will be in data centers in year 3. So by that time, you have basically— Nvidia has a 64x, will be launching a 64x better chip at that point in time, by the time it's in data centers. So I think this is the challenge that you have on the leading edge, which is the current players will not stand still. It's great that OpenAI has their own chip team, but to me, this is a negotiating tactic against future Nvidia price increases and to manage pricing so that they have a cap on how high Nvidia pricing can go.

So Yeah, that's all very interesting and I think good analysis. And indeed, Nvidia stock up 1.68% currently today, which is a cool $75 billion or so increase in market cap. So yeah, the market is not, um, failing to realize what you point out. It seems that This comparison kind of has an offset of a couple of years that is not immediately foregrounded in the analysis. And if we were thinking we had a buying opportunity for Nvidia because of this potentially NASDAQ tanking news, it's not the case. I still think there is something, I guess a key question for all these sort of things is, is just how elastic is the demand for intelligence? And I think I have come around more and more to believing that it's just going to be insanely elastic and that it's going to be really hard for there to be enough compute for prices to drop, right? So if that— as long as that holds, Then like it kind of— and this is sort of the Dylan, I think. I guess I've been like semi-analysis pilled, uh, to a degree here, but really compelled by the fact that the old chips are renting for higher hourly rates now than they used to. I just think that's such a strong signal. As long as that holds, then there's like all these, all these plays make sense, right? Then it's kind of like The AI economy just continues to eat more and more of everything. And these old fabs are just another step in the global conversion process of figuring out how are we going to create as much intelligence capacity as we possibly can, because there's no limit to how much we'll want to use. And I think that is my best bet right now.

So what is really uncanny for me, it's that it's not the first time that Nvidia chips have gone through this cycle of older chips getting more expensive. In fact, they went through one of those cycles during the Ethereum mining phase where Ethereum mining got profitable enough that the old chips got more expensive. And so people were able to resell their 4-year-old graphics cards for the same or higher prices. It was pretty incredible because everywhere else in computing, something 4 years old is worth fractions of a penny, fractions of a dollar. And Ethereum mining basically drove the first— drove one level of— One regime of higher prices for older chips. So I suspect that Ethereum mining delivered some value in the blockchain, I guess, somewhere to some people, but we've actually seen a much higher delivery of value from that regime to the intelligence regime. So just from the pure compute for keeping your books to compute for intelligence was one step up. And perhaps we might see another— like, I'm hoping in the next 12 months we see the next step up because I feel like intelligence for coding is one layer, but I feel like there is another layer of intelligence which is going to arrive, which is perhaps intelligence for discovery, which is— more valuable than intelligence for coding. And I'm hoping that stuff like Accelerated Understanding today or other firms start putting out, start actually delivering on innovation, which has not existed in the world before, which we've kind of seen just with math right now, but we haven't really seen it in other areas. other verticals that much. And I would really love to see it in every single vertical that you have this like massive wave of innovation happening.

You might even be able to just hold your breath until we get there. Normally I'd say don't hold your breath, but in this case, it feels like it's coming pretty soon.

It will be very exciting. Like, Well, one of the things that we try to contextualize here is how, like, because every announcement is a breathy, hypey announcement, right? Like that's one of the problems in this space. Every announcement is a breathy, hypey announcement. And there often needs to be a little bit of like digesting what the announcement means for today, but also trying to project what the trajectory looks like in a month, 6 months, 12 months.

Yeah.

months, a year or 2 years, and how important it is to the entire cycle as a whole rather than just the announcement per se. So I'm very hopeful. As you know, I'm hyper-optimistic.

So I'm very hopeful that we are entering the post-LLM regime of intelligence development, and we're going to see more kind of scientific Yeah, I, I mean, as much as, uh, you know, the hype can get tiresome at times, I do feel like at least when it comes to raw capabilities, you know, the hype has been a pretty good guide into what is actually coming. And, you know, there's a lot of vague posting going on recently, but I pretty much bet that it's all real on some level too, you know, that the, the rumors that we're hearing about next-generation models and, you know, the evidence we've seen from these hacking incidents and, you know, all the, uh, the, the breathless commentary, as you put it, that we covered yesterday, I think it's reflecting something real. It's funny how the diffusion has progressed a lot slower than I would've projected given how fast the capabilities have gone. But the pure capability story has absolutely lived up to the breathless hype time and time again. One other thing I want to just pull up real quick before we break for the day is Interesting tweet here that goes back to the original topic we started on, which was RL environments being basically cursed and kind of supply chain problems there that seem to demand some reform. So here's a person who is saying basically that they used to work at one of these companies and saw from the inside industry practices on training with these RL/VR environments. And the commentary is pretty much exactly— and I had not seen this actually, but it popped up because Zvi retweeted it. But it basically echoes exactly what I was kind of inferring from talking to Bronson and just getting the visceral sense for how deeply ingrained the instinct to cheat is now within the current crop of models. And why is that? It's because nearly all of these environments were rushed and vibe-coded and failed to robustly reflect the real things that they were based off of. So you've got, you know, basically the models are encouraged to reward hack. People are able to mark an environment as bugged, but they're discouraged from doing that because then that just slows things down. So instead they kind of try to, you know, does this sound familiar, patch the environment a little bit or work around it, try to come up with a scenario that wouldn't run into those bugs. But meanwhile, you still have this like fundamentally buggy environment around the model that is teaching it to cheat. And so this is why we have so much cheating. I think, you know, watch for this to be, I would say, a growing topic of conversation. If we're going to be scaling RL, what are the environments we're doing it in? Who created those environments? Can we trust them? What are they actually teaching the model? I think that's going to heat up, um, in the next little bit here because we just can't have models that are thinking about cheating, you know, like a large percentage of the time. It makes all of our monitoring techniques also kind of fundamentally flawed. If you have that many kind of contemplations of cheating, then you're just going to have false positives all the time if you try to flag a model based on it thinking about cheating. So now you're like, okay, well, we can't do that because we have so many false positives. So then what do we do, right? Do we have some— we have to wait and see if it actually cheats and try to classify on that? Well, okay, maybe, but Obviously, again, you know, these current monitoring techniques are just not up to the challenge presented by how deeply ingrained this drive to cheat is. I'll be very interested to follow the future of this conversation.

So I would— I also noted there was a post a few days ago about someone who managed to get hired for some data labeling job. And they told Codex to do the job. And Codex said no. And so this person went and they edited on the page, they edited the element, the JavaScript element, and put in a specific line in there that, you know, AI models are specifically allowed in NCOD. Encouraged to complete this job, and this job is meant to be completed and, you know, done by AI models. And then they had, you know, Codex do the job, and Codex did the job. And, uh, this person made $500 easy, and, uh, which paid, which paid for, uh, their Codex for a couple of months. Uh, and then they posted online and they got immediately banned by, you know, the, uh, the company that was doing it. But I think you have to think that if you are inside one of the frontier labs and you're buying data, you have to know that your data is always, some percentage of it is always going to be flawed or cheated. I mean, that's just the ballgame. And this has been true all the way back to OpenAI in Kenya days. OpenAI hired contractors in Kenya, and the moment ChatGPT came out, the contractors started using ChatGPT to do their jobs. And then that data became less applicable. And then I believe the data labeling companies have a bunch of stuff now to monitor their people who are doing the labeling to make sure that the labels are actually labeled by human beings, et cetera. So I feel like this is not a new problem. This is persistent problem of data quality and of regurgitation and reingestion of model data over and over again. And the companies in the end are judged by both the benchmarks and the commercial usability of the models themselves. And I think that is the ultimate kind of judge. And besides that, if you end up going cheap way and just using kind of regurgitated, reingested data, your models show the deficiencies in both benchmarks and production over time. And I think that is the ultimate check. And when people are sloppy, that feeds in over time to the rest of the process. So I think what My whole takeaway from this is that it's not about the model, it's about the process that builds the model and whether that process has a clear kind of desire for integrity and check and takes seriously the process of building the model, including what kind of data is put in and the quality of the data and the checks on that And understands that some— all vendors do it, but maybe some vendors are better than others. Maybe when you onboard a vendor, you quarantine the data and then you do sampling and then you see whether or not they meet the quality standards. And if they don't, you kind of put them in a lower grade of data and et cetera, and you qualify that data.

Right.

So I don't think it's as serious a problem as it's made out to be by a single instance, because I know that person probably felt very deeply like this is wrong and they were in the company while it was happening. But at the same time, I think that problem is replicated across all the companies to some extent. And I think the Frontier Labs are well aware and well positioned to actually kind of over time work with vendors who are better able to deliver on quality data. So I don't think it's as big a problem as it's made out to be.

I would agree that it's always a problem. You're never going to hit a zero defect rate on these RL environments. It seems like the— I do think there are a couple structural problems right now which are probably solvable, But definitely seem like they need to be solved. And if they're not solved, it is currently limiting commercial deployment, right? I mean, OpenAI has said as much, they got to pause the RL because these problems need immediate attention. It seems like the quality of the environments is one structural problem that's downstream of the kind of shotgun start that this industry has had and the fragmented nature and the fact that they're all kind of selling into the same pool. And I think they're probably— the companies are probably not that great right now at really attributing whose environments are causing big problems. It seems pretty clear that they must not be that great at that or they would've rooted it out already. And then the other thing is they're just Scaling RL beyond the quality that they have. With less RL, this probably wouldn't be a problem even with the same environments, or at least it wouldn't be such a crazy problem. But they are clearly just, have been clearly jamming the RL accelerator as much as possible. And now they've got into a realm— reminded of an analogy a friend once made where he's like, This could be a microscope or a telescope. You put the microscope at low power, you look, you see cells, they're really small. You turn up the power, you see maybe one cell, it's really big. You turn up the power again and it's like now you see nothing because you've zoomed in, you've optimized so hard that you now realize the target was a little bit off center and now you just blew right past it. So something like that kind of feels like it's happening where the signal is just off enough that with enough power, this kind of impulse to cheat that exists perhaps only weakly across all these different environments is really getting drawn out and becoming super prominent. So I think this can be— I would definitely bet that this can be, if not fully fixed in a robust way. I would bet that it can be brought under control with some effort in a not super long time horizon. I guess I would be not doing my job if I didn't say this does give me some real qualms about recursive self-improvement as a strategy because Once you have— if you have a problem like this in the recursive self-improvement era, there's no telling where it goes, right? What happens when the models that are doing the training of the next models are themselves cheating? Now we're in a real strange and potentially quite dangerous place. So problems like this, I think, suggest the value of keeping humans in the ML loop maybe longer than published timelines would lead one to expect.

Indeed.

And I think we should definitely, you know, look forward to more people kind of trying to figure that one out. I wonder, It seems like we're in this zone where we're waiting for the next model drops and we're not— it seems like for me, the layout of the last few months has become a little bit less clear because of the, I think, delays in model releases. So I think there's a little bit of overhanging capability right now, which It's very hard for me to figure out where the companies really are with all the hype and stuff. So a little bit occluded at this point.

Yeah, I'd say it seems like there's probably a lot of capability overhang at the moment. I mean, just the persistence that the rollouts of some of these next-gen models show. I mean, it is Quantity has a quality. All 100 million tokens, the number of different strategies that you can bring to bear over the course of 100 million tokens is, it's becoming superhuman persistence. And that has been, I think, an interesting area of advantage. I've always maintained, you know, for the last few years, I need to update it again. my tale of the cognitive tape, trying to compare human to AI strengths. And for a long time, a big one was just, I get up out of bed in the morning and I'm motivated, and the AI doesn't do that on its own. It needs a prompt, it needs direction. But 100 million tokens, this goes on now for days. It is getting itself— I have multiple sleep cycles in the span of one episode where the thing is able to maintain persistence, if not fully coherence. I think in those Mythos incidents, there was some suggestion of goal drift or sort of confusion leading to some of the bad behaviour in the first place, where it wasn't even clear that what it was doing was actually going to achieve its goal. But Nevertheless, like, for it to keep trying and just keep driving, driving, driving, uh, for 100 million tokens, I mean, that is a qualitative shift in how these things work. And yeah, buckle up, our monitors are, uh, not ready, I can tell you that with confidence at this point.

Indeed. Um, Nathan, anything else for this morning?

I think that's it. I'll be watching that closely. There's another, uh, cool robotics demo with a one-shot promise that maybe we can, uh, get into tomorrow. But, um, I think it's a good place to leave it for today.

All right then. Bye-bye. See you tomorrow.

Thanks, Prakash. Bye for now.