Decart’s Dean Leitersdorf on AI-Generated Video Games and Worlds

So we launched Oasis a few weeks ago, and really when we launched it, the incredible thing from a tech perspective was, oh, this is the first video model that actually runs real time and you can interact with it. It responds to user actions. You can move around the world, you can break blocks, you can place blocks. And so we got this nice game without a game engine. Okay, but that's not interesting. Why is this actually interesting? And so to answer that, Forget about Oasis, Oasis 1, think about, say, Oasis 3, okay? And imagine this. So imagine for a sec, just go. Just put tech aside for a sec. Imagine you're looking at a mirror, okay? And you have this magical mirror that you can talk to it, okay? You can tell it to do cool things. You can say, hey, I'm here and here's my hand and I want to hold a sword, okay, can you give me a sword? And then you look at yourself in the mirror and boom, there's a sword in the mirror where your hand is, okay? And. And you move your hand around and the sword moves and you can be like, no, no, no, make the sword bigger or make it blue and it changes. And you can be like, okay, now turn me into Game of Thrones and everything around you become Game of Thrones. And then you get a crown and everything and you can be like, I don't like my crown. Change it a bit. And then you start jumping and you move around and the mirror responds to that, okay? And that's interesting. Now the reason that's interesting is because it's a completely different experience than anything we've had before on Earth. And it allows us to kind of channel our imagination through screens that we can see. It connects two things. It connects what we see in our minds and what we can see with our eyes. And so that's where we're going with this. How can we. In a sentence, how can Genai really allow us to connect our imagination to what we see on our screens? And with that, we can take it into. Into really worlds that we didn't explore before. It can change everything from applications we can't. We can't do today, all the way to how we even interact with. With computers or with hardware.

Hey, everyone, I'm Sean McGuire. I'm a partner at Sequoia Capital. Today, my partner, Sonia Huang and I are going to interview Dean Ladersdorf. Dean is a brilliant young mind. He grew up back and forth between Israel and the United States. He was the youngest person to ever get a PhD from the Technion at Israel at 23 years old, at least until his younger brother beat him and got his PhD when he was 21. Descartes is trying to deliver delightful AI experiences, really trying to let people interact with their imagination and other people's imaginations in a way that's never been possible before. To do this, they are fully vertically integrated, optimizing everything from as low level as Cuda kernels up to designing their own models, training the models, and then at the end of the day, delivering experiences. Over the next few months, we're going to see some pretty impressive launches from these guys.

Dean, thank you for joining us today. I was just playing Oasis this morning. I had so much fun. So let me start by asking Oasis a fully playable AI game engine. Like, what is it? Why did you launch it?

About Oasis. So we launched Oasis a few weeks ago, and really, you know, when we launched it, the incredible thing from a tech perspective was, oh, this is the first video model that actually runs real time and you can interact with it. It responds to user actions. You can move around the world, you can break blocks, you can place blocks. And so we got this nice game without a game engine. Okay, but that's not interesting. Why is this actually interesting? And so to answer that, Forget about Oasis, Oasis 1, think about, say, Oasis 3, okay? And imagine this. So imagine for a sec, just go. Just put tech aside for a sec. Imagine you're looking at a mirror, okay? And you have this magical mirror. You can talk to it, okay? You can tell it to do cool things. You can say, hey, I'm here and here's my hand and I want to hold a sword, okay? Can you give me a sword? And then you look at yourself in the mirror and boom, there's a sword in the mirror where your hand is, okay? And you move your hand around and the sword moves and you can be like, no, no, no. Make the sword bigger or make it blue. And it changes. And you can be like, okay, now turn me into Game of Thrones. And everything around you become Game of Thrones. And then you get a crown and everything. And you can be like, I don't like my crown. Change it a bit. And then you start jumping and you move around and the mirror responds to that, okay? And that's interesting. Now the reason that's interesting is because it's a completely different experience than anything we've had before on Earth. And it allows us to kind of channel our imagination through screens that we can see. It connects two things. It connects what we see in our minds and what we can see with Our eyes. And so that's where we're going with this. How can we, in a sentence, how can Genai really allow us to connect our imagination to what we see on our screens? And with that, we can take it into really worlds that we didn't explore before. It can change everything from applications we can't do today, all the way to how we even interact with computers or with hardware.

I love the mirror. Let's take it further. Where are you going with that? Is this a social media thing? Is this. Are you building a game? Are you building a world? You know, a world model, an interactive world model. Like, how should I think about, you know, what is Descartes? What is Oasis?

So let me ask you this. What problem does ChatGPT solve?

Homework.

Homework, great. And what else does it solve?

It makes it easier to talk to computers.

Nice. Sean knows the answer because.

Because I spent a lot of time with you, Sean.

Classic Sean.

I spent a lot of time with you.

But exactly that. The TLDR's ChatGPT doesn't solve any given. It helps you do your homework better, it helps you write emails, it helps you summarize. Exactly. Now, it doesn't solve a problem. It overcomes some fundamental limitation, which is exactly what Sean was saying, that it overcomes this communication barrier between humans and computers. Computers speak in structured languages, humans in unstructured languages, or in languages with complex structure that LLMs just bridge that gap and let computers and machines interact with each other in a language that we can both understand that itself. The second you have that, you get 100 different things that are solved on top of that. So we get with the mirror, or what you get with generative interactive video, is you get that communication barrier now overcome not just with text, but also with what we can see. Now computers will be able to see the world the way we see it, and they'll be able to show us the world in ways that we can understand. And you solve that. You build a platform that allows you to build everything on top of that, from next gen Snapchat or TikTok to simulators for fighter pilots. Okay? And that's the cool thing here, and that's if, you know, now we're in 2024. I think one of the most fun things we had at the cart is that we're founding a company. When you have an opportunity to build something that doesn't solve a problem but overcomes a limitation, 99% of companies solve problems. When you look at companies that come to pitch Sequoia or Pitch Any other VC they start with, here's the problem, here's how big the problem is. That's our TAM and everything. And here's how we're going to solve the problem. And usually the first two stay the same. Otherwise you call it a pivot, right? You say, okay, this is the problem I'm solving. If you change the problem you're solving, you call that a pivot, and 500 times you change the way you're going to solve it. That's 99% of companies, and that's what you can do in any regular year. There are moments in history, recently, it's been like once every decade, maybe 15 years, that you actually have the chance to build something that doesn't solve a problem, but just overcomes a limitation. Let me ask you this in a different way. Is the Mac a consumer product or an enterprise product?

And is it a hardware company or a software company?

Is it a hardware company or a software company? And what problems does it solve? Okay. And if you try to give me a list of problems that the personal computer solves, you'd have everything from gaming to Excel. And that's the nice thing about this, that you're building an insane piece of tech that you'll be able to productize in so many different ways.

Yeah, I love that. One of the things that was so cool about what you've built is that there's no game engine inside, as. As far as I can tell. Like, what do you think that means? Like, do you think. Do you think that game engines are an artifact of the past? Or like, what does that mean?

Game engines were supposed to make it so that we can. So that one player, one. One person can create a world and a different person can interact with that world. Right? That, that's. That's the purpose of game engines. You have the game developer and you have the. The user that uses that. And it might go for. Also for movies people use game engines for. Right. Unreal has been used for movies a lot recently as well. Now that is a very valuable product and it has lots of advantages to it. The world is very consistent. You can really make things very accurate. The problem is that it does take a lot of time to interact with it. People like taking the basic game and like turning into a bunch of different things. And, you know, as we got into this and we actually saw what people do with it. Do you know there's an actual mod to put Pokemon inside Minecraft? Okay. You can walk around the forest and there's Pokemon running around. That's an actual mod someone built. Okay. And so people inherently have this. Oh, we have that. We got this platform and we want to change it. And so that's the nice thing about mods. What you get here is that because what's running your game or your environment is an AI, you can interact with it. In the ways we're used to interacting with AI, you'd be able to say, hey, can you turn this into, like, Elsa themed? And then, boom, everything becomes Elsa themed. And can you add a flying elephant? And there's a flying elephant in the game. And it's not just there as a picture. You can actually interact with it. You can punch the elephant and it'll punches you back or whatever you can do with an elephant. And so I think that if this trend were to replace game engines, it would have to be at the state that you can program for it so that it's some machine that one person can build worlds on and the other can interact with. And that is definitely coming. And not only that, it's going to be much easier to program for this. You can just use words. You don't have to write code. And even if you do know how to write code, you can iterate so much faster on it. So basically, to summarize this, I think what this will allow us to do is we'll get modding much, much, much, much, much faster and we'll get interactive.

Modding to get a little more technical for a second. You're the first video model I've ever seen that has real time inference. What are some of the things that go into having real time inference? How hard is it? And just give us some of the flavors of what goes into that.

If we go back like three, four months, like back to the summer. I didn't remember where this was published, but there were a few headlines about, oh, when black bowl chips come out. When Nvidia's black bowl chips come out, we'll get real time video. Okay, Hoppers just can't do it. The H1 hundreds can't do it. We have to wait for Nvidia's next generation. And I think I heard this from quite a few different sources. There were like two weeks during the summer where everyone was saying that for some reason, okay? And no H1 hundreds can actually do it. Okay. And to pull that off, you have to do two things at once. You have to change a lot of things around the model itself. Not every video model can be run real time. You have to train the model differently. The architecture needs to look different now. It's not major architectural changes, but you do have to make them. On the other hand, you also have to do lots of the systems level stuff. You actually have to write your own Cuda kernels. You have to write, we threw out like Pytorch's garbage collector and wrote like half of it from scratch. Okay. And you really have to write everything on the systems level as well to actually pull this off. Because if you do only one of the two, you'll be waiting for someone else to do the other half for you. If you're only doing the systems level part, then you won't be able to pull this off because you won't have a model that's ready to be interacted with this way. If you do just the modeling stuff, you won't have the systems level support to be able to make it run real time.

Can you say a word on how the model works? Is it transformer based? Is it similar to the soras of the world? What have you built on the model side?

Yeah, tldr. It's exactly like the soras of the world. The prompt is user actions instead of text. That's the easiest way to think about it. You have text to video models. You have sora that you put in a sentence and you get a video. Same thing here, just you put in. Your prompt is like your keyboard actions and your past frames and it generates the next frame.

Okay, so how do you get the data between actions and video?

So yeah, you do have to do some pre processing steps here that you don't do with regular video models. For example, you do have to take the raw recordings of hey, this is the gameplay and to label it at each step with the action that's being taken. And so, you know, we trained a small model that does that. It actually doesn't need too much data. Like you can solve that with a small model that doesn't need too many examples. And so you can just have our team just played for a bit, recorded that you get a small model and then you use that to label all your data.

Super interesting. And are you building a world model or is this just purely pixel representation?

Nice. So the beautiful thing here is that it's purely pixel representation. Now let's compare that to exactly what you were saying with world models or 3D stuff and the other things in AI, for more than a decade, there's been the general question of do you solve stuff end to end or do you take an existing workflow and make something more efficient? Okay, there could have been two ways to solve this problem. You could say, hey, game engines exist. Unity is amazing, Unreal is amazing. Let's just plug into that workflow. Okay, let's build text to 3D. So I'll describe an elephant and I'll get the 3D mesh of an elephant and that'll be embedded into Unity and Unreal, or whatever game engine you're using. Okay, so compare that to the end to end solution of at the end of the day, what I have is a screen. The screen needs to show something and that needs to work. Okay? And at the end of the day, what people do is they see their computer screen and they touch their keyboard and they move their mouse and that's your interface and you solve this end to end from keystroke to frame. Okay? So obviously these two are competing directions. Now, over time, I think that there will be some merging between them because from a technical perspective, they each have their own advantages. The first is much more consistent over time. It's much easier to say, oh, here's this object, here's how it looks, and when it'll come back in two hours, it'll look exactly the same. And the other one, the end to end pixel, the fusion version that does pixels in the pixel space. That one is much more easy to work around. It's much more flexible. You can really say, oh no, no, change the elephant's tail, it's too big. Or you can actually edit it live in a way that's just more dynamic. So I do think that long term though, these two things will converge and just if we roughly map this out. So today we really just have prompt two pixels, keystrokes to pixels. You could in theory say that the right way to solve this and say like the next two or three years is to have two models, to have a model that everything's Transformers, right? Transformers.

What?

You have one model that's in charge of holding some state, state of the game and that's unrelated to pixels. It's like literally just like a LLM wise transformer. Okay? It just gets the current state, it gets the new user's action and just outputs changes to that state. And you have one model that's doing that and then the second model takes that state and renders it to pixels. So it makes sense that that's roughly where we'll converge because that will really take into account both the advantage of world models and the advantages of diffusion models.

Do you want to build both of those models?

Of course. Yeah, definitely.

But yeah, one of the Things for.

Me, I will say that we are a bit off. It will take some time to reach that stage. Yeah.

One of the things for me that really caught my attention about Dean and Descartes is they have this ambition to be completely vertically integrated. Like these guys understand literally down to electrons and how, I'm serious, they understand how electrons move in logic gates and even alternate logic gates and how you can represent them in levels even below assembly, how you can change then in like in assembly. Cuda kernels, they literally go all the way from electrons to pixels that your eye sees. And they're optimizing every single level in there. And I think you, by doing that, I think they'll always have like a kind of 10x plus advantage over anyone that's just on the application layer actually.

So talk about this because Sean loves to talk about this. You know the, I think the counter argument would be specialization, right? You know, there's 10, 10,000 very smart people at Nvidia at, you know, choose your favorite company working on this. You should focus on building the best possible user experience and the viral loops and things like that. So talk about, talk about your decisions to be vertically integrated.

Let me actually say something because Dean can't brag about himself the way we can brag about him. But I, I've been studying business models my whole life, has been a passion of mine from a young age. And for myself, Google to me is one of the most amazing companies of all time, one of the most amazing business models. I worked at Google for a few years. I really feel like people have the wrong understanding of what was Google's moat. For it's worth. I also think people have the wrong understanding of what is Nvidia's moat today. But for me, with Google, obviously Sergey and Larry had invented PageRank. PageRank was a very beautiful algorithm, but it was like a deep insight, but it's very simple to implement. It's like a very basic graph theoretical idea and it was a published paper. So once PageRank came out, everyone replicated it very quickly. For me, the real advantage of Google was that these guys were some of the best in the world at distributed systems and at low level systems optimization. And they had this very profound insight from early on that basically all the other search engines were buying Sun Microsystems like server racks. The way they would get fault tolerance was by buying expensive hardware. Whereas for Google they realized that they can buy just cheap consumer commodity hardware that fails all the time. You buy Intel Pentium processors that are in your gaming computer or like SanDisk memory. And you need five times as many total flops or five times as many bits to get the same performance because of all the failure rates. But the cost per flop is like 1 50th. So you can have like a 10x cost optimization 10x cost advantage by really leaning into distributed systems and getting the most out of the hardware. And what that led to with Google is for me, when I look back on when I first started using it, it was this very, very simple front end. It was literally just a white webpage with like a search box. It was, I think, a worse front end than Yahoo at the time. You know, Yahoo also had chat rooms and other like these kind of flashier, exciting things. But Google had this magical backend. Like all the magic to me of Google was on the back end. And I think that backend, the performance came from this cost advantage and it came from the fact that they had optimized all the way down to the bare metal. And with Dean and Descartes, the story really rhymes with me. And look, we need to stay humble. This company hasn't done jack shit yet. It's a very long way before they deserve a comparison to Google and Fortis where it's Sequoia, you know, led the Series A. Co. Led the Series A in Google. Very proud of that. Also led the Seed in Nvidia. So, you know, we have good history, good track record, Good track record. Also, you know, Series A and Apple. But commercial break is over. Commercial break is over. But anyways, I think to really deliver these like delightful, like say a delightful mirror experience, which is a very similar, very simple front end, I think you need this absolutely insane backend that is optimized to the bare metal. And I think it's kind of all or nothing if you can't deliver real time. I don't think it's very good. And I don't think you can deliver real time in the next year without going all the way to the bottom. And so I just, I don't know. For me, I think you kind of have to do that. And these guys are the only ones I've seen doing that.

Wow. I love what Sean just said because two things really caught my attention. One is about the vertical integration. We'll touch about that in a sec. And it goes back to your original question. The second is really about. So I won't name names, but I was speaking to someone who's very, very, very executive at Google recently. Okay. And just, you know, reminiscing about the past and Trying to hear. Because I was three months old when Google was founded, okay? So I was around back. I was around back then, but not really paying attention.

And knowing you, Dean, you might have been paying attention.

So I was trying to understand exactly what happened there, why that was interesting. It came from an unrelated conversation and the way that person brought it up. We're talking about how GPU clusters are just unreliable. Okay? Just, you know, in general, today, if you try to train a model like the one we trained on, any cluster, okay, whether it's hyperscalers or GPU clouds, that thing's gonna crash every few hours, okay? And you're gonna have, like, the weirdest things, okay? You'll have one node crashing, and it'll be because two other nodes have dust on the cable between them, okay? And there won't be any error to really tell you that that's what's happening, okay? So your training room will just crash, and you're like, okay, why did it crash? And you'll try rebooting it and it won't work. And then you'll try removing random nodes until you understand what happens. And that's the state of the entire industry, pretty much. Like, the only ones training that don't see this are probably Google and OpenAI, because they really built everything down to, like, Google built everything down to the hardware as well. OpenAI had a lot of time to really focus a lot on these reliability stuff. But anyone else who's training, from the big companies to the small startups, they're all experiencing this. And so I was talking to this person who's very, very high up at Google, and they said, hey, we're today with, like, training. Today is, like, back where CPUs were in the 90s. Like, forget, like, Kubernetes. There was no VMware, okay? Nothing was reliable. And your servers would just crash all the time. And you had the exact same thing that most companies didn't want to deal with that. And so they just either paid for the premium service that was somehow better, A, so they both paid more money, but B, they also paid with time. Like, the broken hardware exists before the stable hardware exists. Sure, we'll get to stable training runs in a year, in two years, whenever that will happen, okay? Nvidia will make their chips more stable, they'll make their code more stable. The GPU clouds will figure out stuff around this that'll happen. It's not the state today. If you want to train a model today, you're going to face all of that. And so One of the things that. It's really a challenge you have to deal with. And at Descartes, we just deal with it. Okay. The reason we can. So the model that you saw, Oasis, okay, Oasis 1, Oasis 1 converges from start to finish in 20 hours. Wow. And compare, you can compare that. Like, we know what the. We have lots of joint work or communication with other AI labs. They were all shocked by this. Now I'm talking really about the best labs, training diffusion models. For this model, their convergence would usually take around two weeks. And it's not, it's both because they're not using optimized systems layer stuff, but also because they crash every few hours or every few days or whatever. We can actually hold it. We can. Okay, we can, we can, we can actually hold the training run end to end without crashing. We can also hold a training run for a week or for two weeks without crashing. And that reliability part really, really resonates with, with what happened back then. Now the thing about it is that it's really not simple to pull off. It's. You see, like, we have this internal doc, I think it's around 200 pages now, of everything that can go wrong when you're training a model. And it's everything from. If you see this error on this node, then, yeah, tell, tell your hardware operators that these two nodes have a problem between them, these other nodes have a problem between them and all the way to. And here's, here's a fun one. At a certain point as we're training Oasis, we were doing the training run and we needed, we needed some synthetic data to generate as well. And so we said, okay, well, we have this cluster, it has a shit ton of GPUs, CPUs as well. Like, great, it has lots of GPUs, but there's lots of CPUs and they're being like they're utilized by 3% or something. Okay, we can just use this and just generate lots of synthetic data on the same cluster as the training is happening. Okay. By the way, this like blew, blew the minds of our GPU cloud. They were like, you guys are using the cluster to like 200%. You're using the CPUs, you're using the GPUs, and you're using, we even use like the Infiniband to send data around during training. So, like, we're getting a lot more out of the cluster than is like, should be expected. Okay, now that all makes sense. So on one hand, you know, you have this like the GPUs are utilized, the CPUs are not utilized. So you run like synthetic data in parallel. It's not supposed to utilize, it uses just the CPUs. And so it's not supposed to hurt anything. And then your training run doesn't work. Okay? And you get a random error that literally says, the team will know how to say this better. But the error that you get is something like missing lock file and the data loader. Okay? And it's like, how are these two related? Do you want to know how they're related? They're related like this. The synthetic data gen was using up more ram, which is, which is fine. But it caused. Sorry, no. It was as okay to move the data around between the different nodes as the synthetic data was being generated. It was using more network bandwidth than before. And that caused Python's data loader to take one of its log files that's usually network mapped and move it to be swap it out to disk. Okay? And that caused the state that different nodes had different lock files and that caused the data loader to crash. Okay, now I'm probably saying this wrong and the team's probably like listening to this like, no, Dean, you're getting it all wrong. But that's the TLDR of what happened. Okay? You did something that was supposed to make sense and you got a random error. And that's the day to day. And we have a 200 page doc of all of these things. And so that's my part.

And this is a simple example that Dean is happy to share. Like there's, you know, there's.

It's one of the simpler ones.

There's 100x harder, more important things that they've had to figure out. One that I think is also relatively simple, but it's just kind of shows the current state of AI and Dean, feel free. If you don't want to talk about this, don't talk about it. But they got access to a new cluster and somehow the cluster had not installed memory yet. But the GPUs have some very small amount of onboard memory. And so most people would just not even be able to use the GPUs. Can you share anything about this story? Yeah.

So this is actually a nice story. So, you know, we call this the best place on earth to train a video model. Training a video model isn't just the cluster, it's everything surrounding the cluster. Okay? You need to have the storage there, you need to have the networking there. There's so much that needs to go into building the best place on earth to train a video model. And we're actually very far away from them. Okay. Like, I'm assuming that roughly over the next half year, lots of this will stabilize and lots of the GPU clouds are working on this. But yeah, with one of the clusters that we got to, there wasn't any storage. And by the way, it wasn't even with one. It happened with a few clusters and different clouds. Okay. That, you know, the clouds, they bring the GPUs and they try to.

They're so focused on getting the H1 hundreds that, you know, they forgot the.

Memory or the storage and it's fine and it's okay. And they were going to install it, they would get there. But, you know, they try to release everything as fast as possible, which is great, which makes sense. And so, okay, there was no stable storage, storage optimized nodes that you can use, or an S3 bucket or something that you can use. And so we said, okay, well, every node has a few SSDs connected to it. What if we just build our own mini fake distributed file system on top of that? Okay. And that's what we did. And it worked. And it was. There were so many things to overcome to make that happen, but it works at the end of the day. And that's. I think. And it goes back to your question about vertical integration. Vertical integration. So Shaw knows business much better than I do and has been around all these fields way longer than I have. Okay. I did PhDs. And like, I think he just called you old.

I was using Google when, when it first came out and I bought Nvidia shares in the IPO, which also right around nice. Born.

So, yeah, Nvidia, I think IPO'd before I was born. No, 96.

99, I think.

99. Okay. Okay. But yeah, as far as I see it, and correct me if I'm wrong, vertical integration usually gives you two things. It gives you a cost reduction, like higher margins or whatever, and it gives you the ability to move faster, maybe gives you a third thing, because usually things give you three things, but who knows? So I think here in AI, the more important part, sure, they're both important, but I think the second one is even more important than the first because at the end of the day, if you look at all the problems we're facing, great, they will be solved, but it'll take time for them to be solved. And if you, you know, I think that there was a great article, I think at the information about how it Was like a few months ago that people who leave Google to start startups suddenly realize that nothing works because everything works inside Google and then you go outside like, oh, there's no storage or oh, my cloud provider doesn't provide me with this. I actually need to take care of this. And so, okay, fine, over time these things will stabilize and your clouds will provide you what the cloud needs to provide you. And you'll have great companies that provide you with like middle layer for the system stuff or even for the model training stuff will make lots of easier, lots of easier for you. But if you really do everything end to end, you can, you can get to market a year before everyone else, you can get to market two years before everyone else. And that's, and that's, I think what's key here because even if we go to the Google Story or open the eyes story, tech modes don't last. Right? Sure. Google, Google is a great search engine. Bing is probably not that bad. Okay, sure, maybe Google has more data so they're able to do that. But Microsoft's a huge company. They've been working on Bing for so long and it's a good search engine. They have the tech. It still doesn't mean that now Bing and Google are balanced. Right? So at the end of the day, the entire game here is get your tech mode quickly and then convert fast two years before everyone else, like Google and like OpenAI did, and work as fast as possible to convert that to a different mode. And that's, and that, and that's the game here. That's what you have to play. Because we can all say, okay, you know what? Sequoia invested. All good. Let's put the money in the bank for a sec. Okay, let's get some interest on that. We'll go be on the beach for like two years, wait for everything to stabilize, we'll come back in two years and then we'll build the same company and that'll be great. But someone else would have done it before. And that's, I think, why we chose to be vertically integrated.

I love it. What's your remote going to be?

Long term or short term? Both. Both. Perfect. Short term tech. Okay. Short term tech. And that's, and that's great. And we have the best systems layer stuff and we're also doing the model layer stuff as well. So we're fully integrating. And that, and that's your, that's your moat at the end of the day. Short term, long term, long term. I think that's a great Question and let me, let me share something that I found really interesting. Okay, so there is a new weaker version of network effects that exists today that didn't exist before. And that network effect is called what people say on TikTok. Now why is that interesting? Okay. One of the companies that we learned a lot from and that I think is an actually really, really good company they did end up selling to is character AI. They didn't end up selling to Google and wanting to go back to training big mo models, but character, there's a lot to learn from character. And one of the things that the second they took off they had lots of competition instantly. Like fine techmote lasted for like half a year until Meta released open source models. And then other people started running this. They were still vertically integrated and so they were able to be 10x cheaper than everyone else, which was great. But one of the things that really stood out to me was their TikTok mode. If you go on TikTok and you look for any character AI competitor, fine. You'll find a video of that competitor and then you'll scroll and you'll see 100 videos of character. And if you even, you know, even if you go on the videos which are not character, all the comments are full of character. And if you talk to a random character AI user, they don't even know the competition. And so we have somehow, literally because of TikTok, there is a new moat of what people say about you on TikTok and do you have a mini network effect there? I'm not sure if it's a network effect or brand effect, but why is.

This different from just brand?

So it's very similar, it's similar to brand, but it's in your face. Brand 20 years ago was okay. Did you hear your friends talking about this or your parents talking about this here you're always on like the younger generation especially, they're always on TikTok and so they just see this instantly. And so there's even a big question of whether a moat like that could survive for the two, three years until you need, until you get your long term moat of like insane brand like Google's or a distribution brand or something like that. Or a distribution moat or something like that. So I think we're really in this new market here. Yeah. That we're not necessarily going to have the same moats we had 10 years ago.

Super interesting.

Hardware is always the best moat though. And for Google, I think they elevated what was initially like a software moat and a distributed systems moat to becoming a hardware moat. I personally think that Google has not leveraged that moat enough on the application layer. They haven't had that many really fantastic breakout consumer products since the early days. But they have an absolutely gigantic cost advantage really on the hardware layer. When I was at Google, there was this project that just absolutely blew my mind and gave me a prepared mind for a few investments, which is basically Google built optical interconnects to move data in data centers. One of the papers, if you Google Jupiter Rising, Google Data center, you'll find the papers. And basically these optical switches, by turning them on, basically about doubled the performance of the data centers like these. One switch is mainly rack to rack in data centers, moving from electrons to photons. And these switches were insanely hard to build. And basically everyone outside of Google, if you asked them at the time, is it possible to build? They'd say, no way, 100 terabyte per second switch or whatever. They'd say, absolutely no way. But they did it. People didn't even know for years that Google had this and it reduced power consumption of the Data center by 30% or something. Just like those things are real fundamental moats. I think it's always hard to know what the moats will be for a company in the future, but I strongly believe hardware is the ultimate moat, in part because there's always going to be an extreme delay to move atoms, to spin up fabs, to get power, to build a power plant. Even in a world with AGI, you know, the timescale of hardware, even in a world with a billion optimus robots, like the timescale to make new hardware will be much slower or the time scale will be longer. So anyways, I hope Descartes has a hard remote sense.

I think I agree with you on that. Like long term. Okay. You know, this actually goes back to when we were founding Descartes. So we said, okay, we're in this. We called it the golden ticket. We got this ticket that you get once in your life of starting a company and a time where going back to what we were discussing before starting company, at a time, we can solve some fundamental limitation and not like there's some huge tech shift going on. And we said, okay, there are three huge companies you can build here. That was our analysis of the field. A, you can build an Nvidia competitor, and if you like, the next gen chip that's actually built for AI is. And it'll be very tough to do. But Nvidia, you know, Nvidia is not just A chip giant, but they're a supply chain giant. Okay, insanely hard to do, but you know, if you hustle your way around, everyone in the industry wants to help you. And so it's doable if you really, if you really excel on the business side. Two was to build the next aws. Like there is. There is an opportunity because the, the workloads themselves are changing. There is an opportunity to be able to build a new cloud. Very, very, very tough. Because in that market there's a default winner. If you all lose, the big three will still win. The big three plus Oracle or the other clouds as well. And the third was create new experiences. That new experiences will happen and these experiences will be drastic enough so that the next trillion dollar company can come out of these in five years and not in 30 years. And so we had to choose one to start with. We chose the experiences one. But a definitely strong second was let's build an Nvidia competitor one. And so we have that lingering thought of one day we'll get back to this.

I see why you two are friends. I want to close it out with one last question. If everything goes right, what is Descartes in 10, 15, 20 years? And what experiences have you crafted and like, what is the future of consumer entertainment? I don't know if that's the right market.

And I'll say this, and I'll give credit to James from Sequoia here because he's the one who coined this term generate experiences gx. Okay. And we call this UX is dead. Long live gx. Okay. Basically we're going to have new experiences that are generated in ways that match how humans want to interact with computers. And that encapsulates everything from character AI as a generated experience to real time video models or generated experiences. And that's what we're going to see with the card. At the end of the day is a generated experiences company. We're implementing this with being fully vertically integrated, with having the systems layer. At the end of the day, you're a generated experiences company. You're creating the new wave of experiences that's going to touch every single person on the planet. And that's where the cart is now. The only question is whether it does take 10 or 15 years. In today's age, it might take less. It took a long time for the previous Titans to rule the world. I don't know if it'll take that long this time. It'll definitely take at least five years.

You operate on a different time scale than a Lot of the best AI researchers that are in our orbit and I really respect that about you. Should we close it out with a rapid fire round?

Sure. Start it off.

Favorite AI app other than Oasis has.

To be between ChatGPT and character. Has to be between ChatGPT and Character.

What do you use character for?

Not using character, but on the basic notion of that, we'll have these apps that are entities that hold some kind of relationship, whether it's friendship or whether it's utilitarian. With hundreds of millions of people. I think that's an insane platform that's going to be the basis for so many things going forward.

Yeah, I love that. Favorite AI company. Could be the same as the last.

Same as the last answer.

Same as last answer. Okay, let's see.

When did you first program a computer?

First program a computer when I was 13. Bots for Runescape. Okay, great game, Runescape. I botted the hell out of it for years until six years in, I used a bot that I downloaded from the Internet 24 hours later got banned.

Are we gonna have AI generated video games first or AI generated novels? And I mean at the, you know, at the level where I would actually pay for it.

You're going to have. The first thing you're going to have is a platform that lets other people use their creativity to create this content because AI is still far away from creating creative content.

Ah, super interesting.

Okay, who's your favorite scientist ever?

Favorite scientist. That one, I like. That one I like. It's. You know, there's a reason we chose the name, the cartoon. We chose the name Descartes because so specific. Okay, well, first of all, I'll answer the question. Favorite scientist is Da Vinci because I think he's both an insane scientist and engineer and somehow was able to get people to fund his projects. Okay. He was like, if you go back to Da Vinci, like he literally was a great scientist, engineer and somehow knew how to raise money from VCs back then, which were kings. Okay. So yeah, definitely DaVinci and the card and Tesla are close seconds. The reason we chose the name Descartes was we looked at Tesla. We're like, okay, we love both that company and the name and we needed someone who resembles the same thing that Nikola Tesla resembled to the company Tesla. And for that, that was the cart. Because, you know, I think therefore I am resembles almost a lot of what AI is to do.

Perfect. Note to end on Dean, congratulations on what you've done. Thank you for joining us today. We love this conversation.

I'm not gonna congratulate you. Haven't done jack shit yet. Let's build something insane. But I love the sentiment.

We can't celebrate until we really win. Okay? There's no celebrating small wins, Sam.