What AI Developers Are Building Next, with Swyx and Alessio of Latent Space

Today on the AI Breakdown, Part two of my conversation with Alessio and Swix from Latent Space. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our Discord, our YouTube and our newsletter. Hello friends, back again with part two. If you haven't heard part one of this conversation, I suggest you go check it out. But to be honest, they are kind of actually separable. In this conversation we get into a topic that I think Alessio and Swix are very well positioned to discuss, which is what developers care about right now, what people are trying to build around. Hello friends, quick note before we get to the rest of the episode. You have probably heard me talk about the AI Education Beta. Over the past few months we've had a ton of you participate, which has been amazing. And now we're almost ready to announce something big and something new. If you want to be one of the first to hear about our new approach to learning AI that is hyper practical, hands on, immediately relevant, continuously upgrading and anchored by community, go to BeSuper AI and sign up to be notified when the project goes live. We're getting there in just a few weeks and I want all of you along for the journey. Once again, that's Besuper AI. I honestly think that one of the best ways to see the future in an industry like AI is to try to dig deep on what developers and entrepreneurs are attracted to build, even if it hasn't made it to the news pages yet. So, so consider this your preview of six months from now and let's dive in, let's bring it to the GPT5 conversation. I mean, so, so I think that that's a great sort of assessment of just how the stakes have been raised. You know, what is your. I mean, so I guess maybe, maybe I'll, I'll frame this list as a question, just sort of something that, that I, that I've been watching. Right now the only thing that makes sense to me with how fundamentally unbothered and unstressed OpenAI seems about everything, is that they're sitting on something that does meet all that criteria.

Right?

Because I mean even in the Lex Friedman interview that, that Altman recently did, you know, he's talking about other things coming out first. He's talking about, yeah, he's just like, he, listen, he, he's good and he could play nonchalant, you know, if he wanted to. So I don't want to read too much into it, but you know, they've had so long to work on this. Like, unless that we are like really meaningfully running up against some constraint, it just feels like there's going to be some massive increase. But I don't know. What do you guys think?

Hard to speculate at this point. They're pretty good at PR and they're not going to tell you anything that they don't want to. And they can tell you one thing and change their minds the next day. So I've always said that model version numbers are just marketing exercises. They have something and it's always improving and at some point you just cut, cut it and decide to call it GPT5. And it's more just about defining an arbitrary level at which they're ready and it's up to them on what ready means. We definitely did see some leaks on GPT 4.5, as I think a lot of people reported, and I'm not sure if you've covered it. So it seems like there might be an intermediate release. But I did feel coming out of the Lex Freeman interview that GPT5 was nowhere near. And it was kind of a sharp contrast to Sam talking at Davos in February, saying that it was his top priority. So I find it hard to square. And honestly, there's also no point reading too much tea leaves into what any one person says about something that hasn't happened yet or a decision that hasn't been taken yet. So, yeah, that's my two cents about it. Calm down, let's just build.

Yeah, The February rumor was that they were going to work on AI agents. So I don't know, maybe they're.

Yeah, they had two agent, I think two agent projects, right? One desktop agent and one sort of more general sort of GPT, like agent. And then Andre left, so he was supposed to be the guy on that.

What did Andre see?

What did he see? I don't know. What did he see?

I don't know. But again, it just like the rumors are always floating around, you know, but I think, like, this is, you know, we're not going to get to the end of the year without GPT 4.5 or 5. That's definitely happening. I think the biggest question is, are Anthropic and Google increasing the pace? Is the Cloud 4 coming out in 12 months? Like nine months? What's the deal? Same with Gemini. They went from 1 to 1.5 in five days or something. So when's Gemini 2 coming out? Is that going to be soon? I don't know. There are A lot of, lot of speculations. But the good thing is that now you can see a world in which OpenAI doesn't rule everything. So that's the best news that everybody got, I would say.

Yeah. And Mistral Large also dropped in the last month and not quite GPT4 class, but very good from a new startup. So yeah, we have now slowly changed in landscape. In my January recap, I was complaining that nothing's changed the landscape for a long time, but now we do exist in a world, sort of a multipolar world, where Claude and Gemini are legitimate challengers to GPT4 and hopefully more will emerge as well. Hopefully from Meta.

Yeah. So let's actually talk about sort of the open source side of this for a minute. So Mistral Large notable because it's not available open source in the same way that other things are. Although I think my perception is that the community has largely given them like the community largely recognizes that they want them to keep building open source stuff and they have to find some way to fund themselves that they're going to do that. And so they kind of understand that there's like they got to figure out how to eat, but we've got. So, you know, there's Mistral, there's, I guess Grok now, which is, you know, Grok 1 is from, from October is open sourced.

Yeah, yeah, sorry, and I thought you thought you meant Grok, the chip company. No, no, no, yeah, you mean Twitter Grok.

Although Grok the chip company I think is even more interesting in some ways. But, and then there's the, you know, obviously llama 3 is the one that sort of everyone's wondering about too. And you know, my sense of that, the little bit that Zuckerberg was talking about Llama 3 earlier this year suggested that at least from an ambition standpoint, he was not thinking about how do I make sure that, you know, Meta, you know, keeps. Keeps the open source throne, you know, vis a vis Mistral. He was thinking about how you go after, you know, how he, you know, releases a thing that's, you know, every bit as good as whatever OpenAI is on at that point.

Yeah, from what I heard in the hallway said a GDC Llama 3, the biggest model will be 260 to 300 billion parameters. So that's quite large. That's not an open source model. You cannot give people a 300 billion parameters model and ask them to run it. It's very compute intensive.

So I think it can be open source. It's just, it's going to be difficult to run. But that's a separate question of whether it's open source.

It's more like as you think about what they're doing it for, it's not like empowering the person running Llama on their laptop. It's like, oh, you can actually now use this to go after OpenAI, to go after Anthropic, to go after some of these companies at like the middle complexity level, so to speak. Yeah. So obviously, you know, we also meet Gentala on the podcast. They're doing a lot here. They're making Pytorch better. You know, they want to. That's kind of like maybe a little bit of a shot at Nvidia in a way, trying to get some of the Cuda dominance out of it. Yeah, no, it's great. I love the duck destroying a Lotter monopolies arc, you know, it's been very entertaining.

Let's bridge into the sort of big tech side of this because this is obviously like. So I think actually when I did my episode, this was one of the. I added this as one of. As an additional war. That's something that I'm paying attention to. So we've got Microsoft's moves with inflection which I think potentially are being read.

As.

A shift vis a vis their relationship with OpenAI which also the sort of mistral large relationship seems to reinforce as well. We have Apple potentially entering the race finally, you know, giving up Project Titan and, and kind of trying to spend more effort on this. Although Counterpoint we also have them talking about it or there being reports of a deal with Google which you know, is interesting to sort of see what their strategy there is. And then you know, Meta has been largely quiet. We kind of just talked about the main piece but you know there's. And then there's spoilers like Elon. I mean, you know what, what of those things has sort of been most interesting to you guys as you think about what's going to shake out for the rest of this year?

Let's take a crack. So the reason we don't have a fifth war for the big tech wars is that's one of those things where I just feel like we don't cover differently from other media channels. I guess in our anti interest this we actually say we try not to cover the big tech Game of Thrones or it's proxied through all the other four wars anyway. So there's just a lot of overlap. Yeah, I think absolutely. Personally the most interesting one is Apple Entering the race, they actually released, they announced their first large language model that they trained themselves. It's like a 30 billion multimodal model. People weren't that impressed, but it was like the first time that Apple has kind of showcased that, yeah, we're training large models in house as well. Of course they might be doing this deal with Google. I don't know, it sounds very sort of rumoury to me and it's probably if it's on device, it's going to be a smaller model, so something like a Gemma, it's going to be smarter, autocomplete. I don't know what to say. I'm still here dealing with Siri, which probably hasn't been updated since God knows when it was introduced. It's horrible. It makes me so angry. So one, as an Apple customer and user, I'm just hoping for better AI on Apple itself. But two, they are the gold standard when it comes to local devices, personal compute and trust. You trust them with your data and I think that's what a lot of people are looking for in AI, that they love the benefits of AI, they don't love the downsides, which is that you have to send all your data to some cloud somewhere and some of this data that we're going to feed AI is the most personal data there is. So Apple being one of the most trusted personal data companies, I think it's very important that they enter the AI race and I hope to see more out of them.

To me, the biggest question with the Google deal is who is paying who? Because for the browsers, Google pays Apple like 18, 20 billion every year to be the default browser. Is Google going to pay you to have Gemini or is Apple paying Google to have Gemini? I think that's like what I'm most interested to figure out because with the browsers it's like it's the entry point to the thing. So it's really valuable to be the default. That's what Google pays. But I wonder if the perception in AI is going to be like, hey, you just have to have a good local model on my phone to be worth me purchasing your device. And that's kind of drive Apple to be the one buying the model. But then like Sean said, they're doing the MM one themselves. So are they saying we do models but they're not as good as the Google ones? I don't know. The whole thing is, it's really confusing but it makes for great meme material on. On Twitter.

Yeah, I mean I think like they are possibly more than OpenAI and Microsoft and Amazon. They are the most full stack company there is in computing. And so they own the chips, man. They manufacture everything. So if there was a company that could seriously challenge the other AI players, it would be Apple. And I don't think it's as hard as self driving. So maybe they've just been investing in the wrong thing this whole time. We'll see.

Wall street certainly thinks so. Wall street loved that move, man. There's a big sigh of relief. Well, let's move away from sort of the big stuff. I think to both of your points, it's going to.

Can I drop one factoid about this Wall street thing? I went and look at when Meta went from being a VR company to an AI company and I think the stock, I'm trying to look up the details now. The stock has gone up 187% since Llama 1, which is $830 billion in market value created in the past year. Yeah, yeah, it's, it's a.

Remember guys haven't. Yeah. If you haven't seen that chart, it's actually like remarkable. If you draw a little arrow on.

It, it's like, no, we're an AI company now. Forget the VR thing.

It's, it is an interesting. No, it's, I think Alessia, you called it sort of like Zuck's disrupture arc or whatever. He really does. He is in the midst of a, of a total. You know, I don't know if it's a redemption arc or it's just, it's something different where, you know, he, he's sort of the spoiler. Like people loved him. Just freestyle talking about why he thought they had a better headset than Apple. Like even if they didn't agree, they just loved it. He was going direct to camera and talking about it for, you know, five minutes or whatever. So that, that's a fascinating shift that I don't think anyone had on their bingo card, you know, whatever, two years ago.

Yeah, it's the whole CM fight Elon though. So. Yeah, that's what I'm really looking.

Yeah. I mean, hey, don't, don't, don't write it off. You know, maybe just these things take a while to happen. But we need to see him fight in the Coliseum. No, I think in terms of self management, life leadership, I think there's a lot of lessons to learn from him. You might kind of quibble with the social impact of Facebook, but just himself in terms of personal growth and perseverance. Through a lot of change and everyone throwing stuff his way. I think there's a lot to say about, to learn from, from Zuck, which is crazy because he's my age.

Yeah, right.

Awesome. Well, so. So one of the big things that I think you guys have, you know, distinct and unique insight into being where you are and what you work on is, you know, what developers are getting really excited about right now. And by that I mean on the one hand, certainly, you know, like startups who are actually kind of formalized and formed to startups, but also, you know, just in terms of like, what people are spending their nights and weekends on, what they're, you know, coming to hackathons to do. And, and, you know, I think it's a, it's a, it's such a fascinating indicator for, for where things are headed. Like, if you zoom back a year, right now was right when everyone was getting so, so excited about AI agent stuff, right? Auto GPT and baby AGI and these things were like, if you dropped anything on YouTube about those, like instantly tens of thousands of views. I know because I had like a 50,000 view video like the second day that I was doing the show on YouTube, you know, because I was talking about auto GPT. And so anyways, you know, obviously that's sort of not totally come to fruition yet, but what are some of the trends and what you guys are seeing in terms of people's, people's interest and what people are building?

I can start maybe with the agents part and then I know Sean is doing a diffusion meetup tonight. There's a lot of, a lot of different things. The, the agent wave has been the most interesting kind of like dream to reality arc. So out of GPT, I think they went from 0 to 125,000 GitHub stars in six weeks. And then one year later, they have 150,000 stars. So there's kind of been a big plateau. I mean, you might say there are just not that many people that can start it. Everybody already started. But the promise of, hey, I'll just give you a goal and you do it. I think it's amazing to get people's imagination going. You know, they're like, oh, wow, this is, this is awesome. Everybody, everybody can try this to do anything. But then as technologists, you're like, well, that's, that's just like not possible. You know, we would have like, solved everything. And I think it takes a little bit to go from the promise and the hope that people show you to then trying it yourself and going back to say, okay, this is not really working for me. And David Won from Adept, you know, they, in our episode he specifically said, we don't want to do a bottoms up product. You know, we don't want something that everybody can just use and try because it's really hard to get it to be reliable. So we're seeing a lot of companies doing vertical agents that are narrow for a specific domain and they're very good at something, you know. Mike Conover, who was at Databricks before, is also a friend of latent Space. He's doing this new company called Brightwave, doing AI agents for financial research. And that's it, you know, and they're doing very well. There are other companies doing it in security, doing it in compliance, doing it in legal. All of these things that like people, nobody just wakes up and say, oh, I cannot wait to go on auto GPD and ask it to do a compliance review of my thing. Just not what inspires people. So I think the gap on the developer side has been the more bottom sub hacker mentality is trying to build this very generic agents that can do a lot of open ended tasks. And then the more business side of things is like, hey, if I want to raise my next round, I cannot just sit around and mess around with super generic stuff. I need to find a use case that really works. And I think that has worked for a lot of folks. In parallel, you have a lot of companies doing evals. There are dozens of them that just want to help you measure how good your models are doing. Again, if you build evals, you need to also have a restrained surface area to actually figure out whether or not it's good. Right, because you cannot eval anything under the sun. So that's another category where I've seen from the startup pitches that I've seen there's a lot of interest in the enterprise. It's just really fragmented because the production use cases are just coming live now. You know, there are not a lot of long established ones to test against. And so does that. That's kind of on the virtual agents. And then the robotic side is probably been the thing that surprised me the most at Nvidia gtc. The amount of robots that were there, there were just like robots everywhere. Like both in the keynote and then on the show floor, you would have Boston Dynamics dogs running around. There was like this like Fox robot that had like a virtual face that like talked to you and like moved in real time. There were industrial robots. Nvidia did A big push on their own omniverse thing, which is like this digital twin of whatever environments you're in that you can use to train the robots. Agents. So that kind of takes people back to the reinforcement learning days. But yeah, agents, people want them, you know, people want them. I give a talk about the rise of the full stack employees and kind of this feature the same way for stack engineers kind of work across the stack. In the future, every employee is going to interact with every part of the organization through agents and AI enabled tooling. And this is happening. It just needs to be a lot more narrow than maybe the first approach that we took, which is just put a string in autogpt and pray. But yeah, there's a lot of super interesting stuff going on.

Yeah, well, LESU covered a lot of stuff there. I'll separate the robotics piece because I feel like that's so different from the software worlds. But yeah, we do talk to a lot of engineers and that this is our sort of bread and butter. And I do agree that vertical agents have worked out a lot better than the horizontal ones. I think the point I'll make here is just the reason AutoGPT and maybe AGI, it's in the name we're promising AGI, but I think people are discovering that you cannot engineer your way to AGI. It has to be done at the model level. And all these engineer and prompt engineering hacks on top of it weren't really going to get us there in a meaningful way without much further improvements in the models. I'll go so far as to say even Devin, which is, I think the most advanced agent that we've ever seen, still requires a lot of engineering and still probably falls apart a lot in terms of practical usage. Or it's just way too slow and expensive for what its problem is compared to the video. So yeah, that's what happened with agents from last year. But I do see vertical agents being very popular. I think the word agent might even be overused sometimes People don't really care whether or not you call it an AI agent. Does it replace boring menial tasks that I might hire a human to do or that the human who is hired to do it actually doesn't really want to do. And I think there's absolutely ways in a vertical context that you can actually go after very routine tasks that can be scaled out to a lot of AI assistants. I would basically plus one what? Lest you'll sit there. I think it's very, very promising and I think more people should work on it not less. There's not enough people. This should be the main thrust of the AI engineer is to look for use cases and go to production with them instead of just always working on some AGI promising thing that never arrives.

I can only add that. So I've been fiercely making tutorials behind the scenes around basically everything you can imagine with AI. We've done about 300 tutorials over the last couple months. And the verticalized anything, right, like this is a solution for your particular job or role, even if it's way less interesting or kind of sexy, it's like so radically more useful to people in terms of intersecting with how like those are the ways that people are actually adopting AI in a lot of cases. It's just a thing that I do over and over again, by the way. I think that's the same way that even the generalized models are getting adopted. You know, it's like I use Mid Journey for lots of stuff, but the main thing I use it for is YouTube thumbnails. Every day, like day in, day out, I will always do a YouTube thumbnail, you know, or two with. With mid Journey. Right. And it's like you can, you can start to extrapolate that across a lot of things. And all of a sudden, you know, AI doesn't. It looks revolutionary because of a million small changes rather than one sort of big dramatic change. And I think that the verticalization of agents is sort of a great example of how that's going to play out too.

Yeah.

So I'll have one caveat here, which is I think that because multimodal models are now commonplace, like Claude, Gemini, OpenAI, all very, very easily multimodal, Apple's easily multimodal, all this stuff. There is a switch for agents for sort of general desktop browsing that I think people need to keep an eye on. It's not mature yet, but it is absolutely coming on the way. And so just as we're starting to talk about this verticalization piece, because that is mature, that is ready for people to work on, a lot of people are making really good money doing that. The thing that's on the rise is this sort of drive by vision version of the agent where they're not specifically taking in text or anything, they're just watching your screen just like someone else would and, and piloting it by vision. And in the episode with David that will have dropped by the time that this airs. I think that is the promise of adept. That is the promise of what a lot of these sort of desktop agents are. And that is the more general purpose system that could be as big as the browser, the operating system. People really want to build that foundational piece of software in AI. And I would see the potential there for desktop agents being that, that you can have sort of self driving computers. Don't write the horizontal piece out. I just think we took a while to get there.

What else are you guys seeing that's interesting to you? I'm looking at your notes and seeing a ton of categories.

Yeah.

So I'll take the next two as one category which is basically alternative architectures. Right. The two main things that everyone following AI kind of knows now is one, the diffusion architecture and two, let's just say the decoder only transformer architecture that is popularized by GPT. You can look on YouTube for thousands and thousands of tutorials on each of those things. What we are talking about here is what's next, what people are researching and what could be on the horizon that takes the place of those other two things. So first of all we'll talk about transformer architectures and then diffusion, so transformers. The two leading candidates are effectively RWKV and the State Space models, the most recent one of which is Mamba. But there's others like the Striped hyena and the S4H3 stuff coming out of hazy research at Stanford. And all of those are non quadratic language models that scale, they promise to scale a lot better than the traditional transformer. This might be too theoretical for most people right now, but it's going to be, it's going to come out in weird ways where imagine if right now the talk of the town is that Claude and Gemini have a million tokens of context and like, whoa, you can put in two hours of video now. Okay, but what if you could throw in 200,000 hours of video? How does that change your usage of AI? What if you could throw in the entire genetic sequence of a human and synthesize new drugs? How does that change things? We don't know because we haven't had access to this capability being so cheap before. And that's the ultimate promise of these two models. They're not there yet, but we're seeing very, very good progress. RWKV and MAMBA are probably the two leading examples, both of which are open source, that you can try them today and have a lot of progress there. The main thing I'll highlight for RWKV is that at the 7B level they seem to have beat Llama 2 in all benchmarks that matter at the same size for the Same amount of training as an open source model. So that's exciting. But they're at 7B now, they're not at 7TB. We don't know if it'll scale. And then the other thing is diffusion. Diffusions and transformers are kind of on a collision course. The original stable diffusion already used transformers in parts of its architecture. It seems that transformers are eating more and more of those layers, particularly the VAE layer. So the diffusion transformer is what SORA is built on. The guy who wrote the diffusion transformer paper, Will Peebles, is the lead tech guy on sora. So you'll just see a lot more diffusion transformer stuff going on. But there's more sort of experimentation with diffusion. I'm holding a meetup actually here in San Francisco that's going to be like the state of diffusion, which I'm pretty excited about. Stability is doing a lot of good work. And if you look at the architecture of how they're creating stable diffusion, three hourglass diffusion and link consistency models, or SDXL Turbo, all of these are very, very interesting innovations on the original idea of what stable diffusion was. So if you think that it is expensive to create or slow to create stable diffusion or an AI generated art, you are not up to date with the latest models. If you think it is hard to create text and images, you are not up to date with the latest models. And people still are kind of far behind, the last piece of which is the wildcard I always kind of hold out, which is text diffusion. So instead of using autogenerative or autoregressive transformers, can you use text to diffuse? So you can use diffusion models to diffuse and create entire chunks of text all at once instead of token by token. And that is something that Midjourney confirmed today because it was only rumored the past few months, but they confirmed today that they were looking into. So all those things are very exciting new model architectures that are maybe something that you'll see in production two to three years from now.

So the couple other trends that I want to just get your takes on, because they're sort of something that seems like they're coming up are one sort of these wearable kind of passive AI experiences where they're absorbing a lot of what's going on around you and then kind of bringing things back. And then the other one that I wanted to see if you guys had thoughts on were sort of this next generation of chip companies. Obviously there's a huge amount of emphasis on hardware and silicon and different ways of doing things. But love your take on either or both of those.

Wearables. I'm very excited about it. I want wearables on me at all times. I have two right here to quantify my health and I'm all for them, but society is not ready for wearables. No one's comfortable with device on recording every single conversation we have. Even all three of us here as podcasters, we don't record everything that we say. And I think there's a social shift that needs to happen. I am an investor in Tab. They are renaming to a broader vision, but they are one of the three or four leading wearables in this space in the AI pendants or AI OS or AI personal companion space. I have seen two humanes in the wild in San Francisco. I'm very, very excited to report that there are people walking around with those things on their chest and it is as goofy as it sounds, it absolutely is going to fail. But God bless them for trying. And I've also bought a rabbit, so I'm very excited for all those things to arrive. But yeah, people are very keen on hardware. I think the idea that you can have physical objects that embody an AI that do specific things for you is as old as the sort of golem in sort of medieval times in terms of how much we want our objects to be smart and do things for us. And I think it's absolutely a great play. The funny thing is people are much more willing to pay you upfront for for a hardware device than they are willing to pay like an $8 a month subscription recurring for software. Right. And so the interesting economics of these wearable companies is they have negative float in the sense that people pay deposits upfront. I paid like, I don't know, 200 bucks for the rabbit upfront and I don't get it for another six months. I paid 600 bucks for the tab and I don't get it for another six months. And then, then they can take that money and invest it in the next events or their next properties or ventures. And I think that's a very interesting reversal of economics from other types of AI companies that I see. And I think just the tactile feel of an AI, I think is very promising. Alessio, I don't know if you have other thoughts on the wearable stuff.

The Open Interpreter just announced their product four hours ago, which is not really a wearable, but it's still like a physical device.

It's a push to talk mic to a device on your laptop. Right? It's a $99.

Yeah, but again, going back to your point, it's like people are interested in spending money for things that they can hold. I don't know what that means overall for where things are going, but making more of this AI be a physical part of your life, I think people are interested in that. But I agree with Sean. I mean, I've been. I talked to Avi about this. But Avi's point is, like, most consumers, like care about utility more than they care about privacy, you know, like you've seen with social media. But I also think there's a big societal reaction to AI that is like much more rooted than the social media one. But we'll see. But again, a lot of work, a lot of developers, a lot of money going into it. So there's bound to be experiments being run on the chips.

Sorry, I'll just ship it one more thing and then we transition to the chips. The thing I'll caution people on is don't overly focus on the form factor. The form factor is a delivery mode. There will be many form factors. It doesn't matter so much as where in the data war, does it situation. It actually is context acquisition and maybe a little bit of multimodality. Context is king. If you have access to data that no one else has, then you will be able to create AI that no one else can create. And so what is the most personal context? It is your everyday conversation. It is as close to mapping your mental train of thought as possible without physically you writing down notes. So that is the promise, the ultimate goal here, which is like personal context. It's always available on you, low latency, all that stuff. But that's the frame I want to give people that the form factors will change and there'll be multiple form factors, but it's the software behind that and the personal context that you cannot get anywhere else that'll win.

Yeah, so that was wearables on the chip side. Yeah. Grok was probably the biggest release. Jonathan Ross. Well, it's not even a new release because the company I think was started in 2016, so it's actually quite old, but now recently captured the people's imagination with their mixrel 500 tokens. A second demo. Yeah, I think so far the battle on the GPU side has been either you go kind of like massive chip, like the Cerebras of the world, where one chip front cerebras is about $2 million. You know, that's compared. Obviously you cannot Compare one chip versus one chip but H100 is like 40,000, something like that. The problem with those architectures has been they want to be very general, but they wanted to put a lot of the ram, the SRAM on the chip. It's much more convenient when you're using large language models. But the models outpace the size of the chips and chips have a much longer turnaround. Cycle Groq today, it's great for the current architecture. It's a lot more expensive also as far as dollar per flop. But their idea is like, hey, when you have very high concurrency, we actually were much cheaper. You shouldn't just be looking at the compute power. For most people this doesn't really matter. I think that's the most, the most interesting thing to me is we now gone back with AI to a world where developers care about what hardware is running, which was not the case in traditional Software for maybe 20 years. Since as the cloud is getting really big, my thinking is that in the next two, three years we're going to go back to that. People are not going to be sweating. What GPU do you have in your cloud? What do you have? It's like, yeah, you want to run this model, we can run it at the same speed as everybody else and then everybody will make different choices whether they want to have higher front end capital investment and then better utilization. Some people would rather do lower investment before and then upgrade later. There are a lot of parameters. And then there's the dark horses, right? That is some of the smaller companies like Lemurian Labs MedX that are working on maybe not a chip alone, but also like some of the actual math infrastructure and the instructions on it that make them run. There's a lot going on. But yeah, I think the episode with Dylan will be interesting for people, but I think we also came out of it saying, hey, everybody has pros and cons. It's different than the models where you're like, oh, this one is definitely better for me and I'm going to use it. I think for most people it's like fun Twitter memeing. But it's like 99% of people that tweet about this stuff are never going to buy any of these chips anyway. So it's really more for entertainment.

Wow. I mean this is serious business here. You're talking about the potential new Nvidia. If Anyone can take 1% of Nvidia's business, they are a serious startup that you should look at.

Right.

So that's why I'm more talking about.

How should people Think about it. You know, it's like, I think like the. The end user is not impacted as much.

I disagree. Yeah, I love disagreements. Because, you know, who likes a podcast where all three people always agree with each other? You will see the impact of this in the tokens per second over time. This year, I have very, very credible sources all telling me that the average tokens per second right now we have somewhere between 50 to 100 as, like, the norm for people. Average tokens per second will go to 500 to 2000 this year from a number of chip suppliers that I cannot name. So that will cause a step change in the use cases. Every time you have an order of magnitude improvement in the speed of something, you unlock new use cases that become fun instead of a chore. And so that's what I would caution this audience to think about, which is, what can you do in much higher AI speed? It's not just things streaming out faster. It is things working in the background a lot more seamlessly and therefore being a lot more useful than previously imagined. So that would be my 2 cents on that.

Yeah, I mean, the new Nvidia chips are also much faster. To me, that's true. When it comes about startups, it's like, are the startups pushing the performance on the incumbents or are the incumbents still leading? And then the startups are like, riding the same wave. You know, I don't have yet a good sense of that. It's like, you know, it's next year's Nvidia release just going to be better than everything that gets released this year. You know, if that's the case, it's like, okay, damn, Jensen. You know, it's like the meme. It's like, I'm gonna fight. I'm gonna fight Nvidia. It's like, damn, Jensen got hands. Like, he. He really does so well.

Awesome conversation, guys. I guess just. Just by way of wrapping up, uh, call it over the next three months, between now and sort of the beginning of summer, uh, what's one. One prediction that each of you has? Can be about anything. Could be big company, it can be startup. Could be something you have privileged information that you know and you just won't tell us that you actually know.

What does it have to be something that we think it's gonna be true or, like, something that we think? Because for me, it's like, is Sundar gonna be the CEO of Google? Maybe not in three months. Maybe like six months, nine months? You know, people were like, oh, maybe Demis is going to be the new CEO. That was kind of like, I was busy, like, phishing some DeepMind people and Google people for like a good guest for the pot. And I was like, oh, what about Jeff Dean? And they're like, well, Demis is really like the person that runs everything anyway in the stuff. And it's like, interesting.

And so I don't know, Sergey could come back. I don't know. Like, he's making more appearances these days.

Yeah, we can just put as like, yeah, my thing is like CEO change potential, but again, three months is too short to make a prediction. I think that's, that's fine.

The timescale might be off.

Yeah, I mean, for me, I think the progression in vertical agent companies will keep going. We just had the other day Klarna, talking about how they replaced like 700 of their customer support agents with AI agents. That's just the beginning, guys. Imagine this rolling out across most of the Fortune 500. And I'm not saying this is like a utopian scenario. There will be very, very embarrassing and bad outcomes of this where humans would never make this mistake, but AIs did, and we'll all laugh at it, or we'll be very offended by whatever, you know, bad outcome it did. So we have to be responsible and careful in the rollout. But yeah, this is, it's rolling out. You know, Alessio likes to say that this year is the year of AI production. Let's see it. Let's. Let's see all these sort of vertical, full stack employees come out into the workforce.

Love it.

All right, guys, well, thank you so much for sharing your thoughts and insights here, and I can't wait to do it again.

Thanks for having us. Sa.