George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387
George Hotz discusses AI safety risks like wireheading and addictive generative media, the limits of current LLM training methods, and his work on Tiny Grad, a minimal framework that removes the matmul primitive to simplify accelerator design. He's building Tiny Box, a wall-pluggable device with six GPUs to run 65B models locally, and using simulation and reinforcement learning at Comma.ai to teach cars to drive like humans.
Chapters
What possible ideas do you have for the how human species ends?
Sure. So I think the most obvious way to me is wireheading. We end up amusing ourselves to death. We end up all staring at that infinite TikTok and forgetting to eat. Maybe, maybe it's even more benign than this. Maybe we all just stop reproducing now, to be fair, it's probably hard to get all of humanity.
Yeah. The interesting thing about humanity is the diversity in it.
Oh, yeah.
Organisms in general. There's a lot of weirdos out there. Well, two of them are sitting here.
I mean, diversity in humanity is we do respect. I wish I was more weird.
The following is a conversation with George Hotz, his third time on this podcast. He's the founder of Comma AI that seeks to solve autonomous driving and is the founder of a new company called Tiny Corp that created Tiny Grad, a neural network framework that is extremely simple with the goal of making it run on any device by any human, easily and efficiently. As you know, George also did a large number of fun and amazing things, from hacking the iPhone to recently joining Twitter for a bit as an intern in quotes, making the case for refactoring the Twitter code base in general. He's a fascinating engineer and human being, and one of my favorite people to talk to. This is the Lex Friedman podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's George Hotz. You mentioned something on a stream about the philosophical nature of time. So let's start with a wild question. Do you think time is an illusion?
You know, I sell phone calls to comma for a thousand dollars, and some guy called me and, like, you know, it's a thousand dollars. You can talk to me for. For half an hour. And he's like, yeah, okay. So like, time doesn't exist. And I really wanted to share this with you. I'm like, well, what do you mean time doesn't exist? Right. Like, I think time is a useful model whether it exists or not. Right. Like, does quantum physics exist? Well, it doesn't matter. It's about whether it's a useful model to describe reality. Is time maybe compressive?
Do you think there is an objective reality? Or is everything just useful models? Like, underneath it all, is there an actual thing that we're constructing models for?
I don't know.
I was hoping you would know.
I don't think it matters.
I mean, this kind of connects to the models of constructive reality with machine learning, Right?
Sure.
Like, is it just nice to have useful approximations of the world such that we can do something with it.
So there are things that are real. Columnograph Complexity is real.
Yeah.
Yeah. The compressive thing. Math is real. Yeah.
Should be a T shirt.
And I think hard things are actually hard. I don't think P equals np.
Ooh, strong words.
Well, I think that's the majority. I do think factoring is in P.
But I don't think you're the person that falls the majority in all walks of life. So it's good for that one.
I do.
Yeah. In theoretical computer science, you.
You.
You're one of the sheep.
All right.
But to you, time is a useful model.
Sure.
What were you talking about on the stream with time? Are you made of time?
If I remembered half the things I said on stream, someday someone's going to make a model of all of it and it's going to come back to.
Haunt me someday soon.
Yeah, probably.
Would that be exciting to you or sad that there's a George Hotz model?
I mean, the question is, when the George Hotz model is better than George Hotz, like I am declining and the.
Model is growing, what is the metric by which you measure better or worse in that if you're competing with yourself.
Maybe you can just play a game where you have the George Hots answer and the George Hotz model answer and ask which people prefer.
People close to you or strangers?
Either one. It will hurt more when it's people close to me, but both will be overtaken by the George Hotz model.
It'd be quite painful, right? Loved ones, family members would rather have the model over for Thanksgiving than you.
Yeah.
Or like significant others would rather sext with the. With the large language model version of.
You, especially when it's fine tuned to their preferences.
Yeah, well, that's what we're doing in a relationship, Right? We're just fine tuning ourselves. But we're inefficient with it because we're selfish and greedy and so on. Our language models can fine tune more efficiently, more selflessly.
There's a Star Trek Voyager episode where Catherine Janeway, lost in the Delta Quadrant, makes herself a lover on the holodeck and the lover falls asleep on her arm and he snores a little bit and, you know, Janeway edits the program to remove that. And then of course, the realization is, wait, this person's terrible. It is actually all their nuances and quirks and slight annoyances that make this relationship worthwhile. But I don't think we're going to realize that until it's too late.
Well, I think a large language model could incorporate the. The flaws and the quirks and all that kind of stuff.
Just the perfect amount of quirks and floor flaws to make you charming without crossing the line.
Yeah, yeah. And that's probably a good, like, approximation of the, like, the percent of time. The language model should be cranky or an asshole or jealous or all this kind of stuff.
And of course it can and it will, but all that difficulty at that point is artificial. There's no more real difficulty.
Okay. What's the difference in real and artificial?
Artificial difficulty is difficulty that's, like, constructed or could be turned off with a knob. Real difficulty is like, you're in the woods and you gotta survive.
So if something cannot be turned off with a knob. Is real?
Yeah, I think so. Or, I mean, you can't get out of this by smashing the knob with a hammer. I mean, maybe you kind of can, you know, I into the wild when, you know, Alexander Supertramp, he wants to explore something that's never been explored before. But it's the nineties. Everything's been explored. So he's like, well, I'm just not gonna bring a map. Yeah, I mean, no, you're not exploring. You should have brought a map. Dude, you died. There was a bridge a mile from where you were camping.
How does that connect to the metaphor of the knob?
By not bringing the map. You didn't become an explorer, you just smashed the thing. Yeah, yeah. The art, the difficulty is still artificial.
You failed before you started. What if we just don't have access to the knob?
Well, that maybe is even scarier, right? Like, we already exist in a world of nature, and nature has been fine tuned over billions of years. To have humans build something and then throw the knob away in some grand romantic gesture is horrifying.
Do you think of us humans as individuals that are, like, born and die? Or is it. Are we just all part of one living organism? That is Earth, that is nature?
I don't think there's a clear line there. I think it's all kind of just fuzzy. I don't know. I mean, I don't think I'm conscious. I don't think I'm anything. I think I'm just a computer program.
So it's all computation? Everything running in your head is just computation?
Everything running in the universe is computation. I think I believe the extended Church Turing thesis.
Yeah. But there seems to be an embodiment to your particular computation. Like, there's a consistency.
Well, yeah, but I Mean, models have consistency too. Yeah, models that have been RLHF'd will continually say, you know, like, well, how do I murder ethnic minorities? Oh, well, I can't let you do that, Hal. There's a consistency to that behavior.
It's all RLHF. Like, we are RLHF each other. We provide human feedback and thereby fine tune these little pockets of computation. But it's still unclear why that pocket of computation stays with you for years. It just kind of fall, like you have this consistent set of physics biology. What, like whatever you call the. The neurons firing, like the electrical signals and mechanical signals, all of that, that seems to stay there and it contains information. It stores information and that information permeates through time and stays with you. There's like memory. There's like sticky.
Okay, to be fair, like, a lot of the models we're building today are very even. RLHF is nowhere near as complex as the human function.
Reinforcement learning with human feedback.
You know, when I Talked about Will GPT12 be AGI? My answer is no, of course not. I mean, cross entropy loss is never going to get you there. You need probably RL in fancy environments in order to get something that would be considered like AGI. Like, so to ask like the question about, like, why? I don't know, like, it's just some quirk of evolution. Right? I don't think there's anything particularly special about where I ended up, where humans ended up.
It's okay. We have human level intelligence. Would you call that AGI? Whatever we have gi.
Look, I actually, I don't really even like the word AGI, but general intelligence is defined to be whatever humans have.
Okay, so why can GPT12 not get us to AGI? Can we just like linger on that?
If your loss function is categorical, cross entropy. If your loss function is, just try to maximize compression. I have a soundcloud, I rap and I tried to get chat GPT to help me write raps. And the raps that it wrote sounded like YouTube comment raps. You know, you can go on any rap beat online and you can see what people put in the comments and it's the most like mid quality rap you can find.
Is mid good or bad?
Mid is bad. It's like mid.
It's like every time I talk to you, I learn new words. Mid.
Mid. Yeah.
I was like, is it, Is it like basic? Is that what mid means?
Kind of. It's like, it's like middle of the curve. Right? So there's like there's like a. Like that intelligence curve and you have like the dumb guy, the smart guy, and then the mid guy. Actually being the mid guy is the worst. The smart guy is like, I put all my money in Bitcoin. The mid guy is like, you can't put money in bitcoin. It's not real money.
And all of it is a genius meme. That's another interesting one. Memes, the humor, the idea, the absurdity encapsulated in a single image. And it just kind of propagates virally between all of our brains. I didn't get much sleep last night, so I'm very. I sound like I'm high. I swear I'm not. Do you think we have ideas or. Ideas have us.
I think that we're going to get super scary memes once the AIs actually are superhuman.
Ooh, do you think AI will generate memes?
Of course.
You think it'll make humans laugh?
I think it's worse than that. So Infinite Jest, it's introduced in the first 50 pages, is about a tape that you, once you watch it once, you only ever want to watch that tape. In fact, you want to watch the tape so much that someone says, okay, here's a hacksaw, cut off your pinky, and then I'll let you watch the tape again and you'll do it. So we're actually going to build that, I think, but it's not going to be one static tape. I think the human brain is too complex to be stuck in one static tape like that. If you look at like ant brains, maybe they can be stuck on a static tape. But we're going to build that using generative models. We're going to build the TikTok that you actually can't look away from.
So TikTok is already pretty close there. But the generation is done by humans. Algorithm is just doing their recommendation. But if it's do. If the algorithm is also able to do the generation.
Well, it's a question about how much intelligence is behind it. Right. So the content is being generated by, let's say, one humanity worth of intelligence. And you can quantify a humanity, Right. That's a, you know, it's. It's exaflops, yotta flops. But you can quantify it. Once that generation is being done by a hundred humanities, you're done.
So it's actually scale that's the problem, but also speed. Yeah. And what if it's sort of manipulating the very limited human dopamine engine for porn? Imagine just TikTok, but for porn.
Yeah.
It's like a brave new world.
I don't even know what it'll look like. Right. Like again, you can't imagine the behaviors of something smarter than you. But a super intelligent and an agent that just dominates your intelligence so much will be able to completely manipulate you.
Is it possible that it won't really manipulate, it'll just move past us? It'll just kind of exist the way water exists or the air exists.
You see? And that's the whole AI safety thing. It's not the machine that's going to do that. It's other humans using the machine that are going to do that to you.
Yeah, because the machine is not interested in hurting humans.
It's just the machine is a machine. Yeah, but the human gets the machine. And there's a lot of humans out there very interested in manipulating you.
Well, let me bring up Eliezer Yudkowski, who recently sat where you're sitting. He thinks that AI will almost surely kill everyone. Do you agree with him or not?
Yes, but maybe for a different reason.
Okay. And then I'll try to get you to find hope where we could find a no to that answer. But why? Yes.
Okay. Why didn't nuclear weapons kill everyone?
It's a good question.
I think there's an answer. I think it's actually very hard to deploy nuclear weapons tactically. It's very hard to accomplish tactical objectives. Great. Can nuke their country. I have an irradiated pile of rubble. I don't want that.
Why not?
Why don't I want an irradiated pile of rubble? For all the reasons. No one wants an irradiated pile of rubble.
Because you can't use that land for. For resources. You can't populate the land.
Yeah. What you want a total victory in a war is not usually the irradiation and eradication of the people there. It's the subjugation and domination of the people.
Okay. So you can't use this strategically, tactically, in a war to help you, to help gain a military advantage. It's all complete destruction. All right, but there's egos involved. It's still surprising, still surprising that nobody pressed the big red button.
It's somewhat surprising. But you see, it's the little red button that's going to be pressed with AI that's going to, you know, and that's why we die. It's. It's not because the AI if there's anything in the nature of AI it's just the nature of humanity.
What's the algorithm behind the little red button? Well, like what, what, what possible ideas do you have for the. How human species ends?
Sure. So I think the most obvious way to me is wireheading. We end up amusing ourselves to death. We end up all staring at that infinite tick tock and forgetting to eat. Maybe, maybe it's even more benign than this. Maybe we all just stop reproducing. Now to be fair, it's probably hard to get all of humanity.
Yeah, yeah. It probably doesn't always go like the, the interesting thing about humanity is the diversity in it.
Oh yeah.
Organisms in general. There's a lot of weirdos out there. Well, two of them are sitting here.
I mean, diversity in humanity is.
We do respect.
I wish I was more weird. No, like I'm kind of. Look, I'm drinking smart water, man. That's like a Coca Cola product. Right?
Do you want corporate George Haas corporate?
No. The amount of diversity in humanity I think is decreasing just like all the other biodiversity on the planet.
Oh boy. Yeah, right. And social media is not helping.
Go eat McDonald's in China. Yeah, yeah. No, it's the interconnectedness. That's, that's, that's, that's doing it.
Oh, that's interesting. So everybody starts relying on the connectivity of the Internet and over time that reduces the diversity, the intellectual diversity. And then that gets you everybody into a funnel. There's still going to be a guy in Texas.
There is. And yeah, to be fair, do I think AI kills us all? I think AI kills everything we call like society today. I do not think it actually kills the human species. I think that's actually incredibly hard to do.
Yeah, but society, like if we start over, that's tricky. Most of us don't know how to do most things.
Yeah, but some of us do and they'll be okay and they'll rebuild. After the great AI, what's rebuilding look.
Like, how far, like, how much do we lose? Like, what has human civilization done? That's interesting. Combustion engine, electricity. So power and energy. That's interesting. Like how to harness energy.
Well, they're going to be religiously against that.
Are they going to get back to like fire?
Sure. I mean there'll be a, there'll be a little, be like, you know, some kind of Amish looking kind of thing. I think, I think they're going to have very strong taboos against technology.
Like technology. It's almost like a new religion. Technology is the devil and nature is God.
Sure.
So closer to nature. But can you really get away from AI. If it destroyed 99% of the human species, isn't it somehow have a hold like a stronghold?
What's interesting about everything we build, I think we are going to build super intelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a. Like a bird. Right. A bird is an incredibly robust organism. We've built nothing like this. We haven't built a machine that's capable of reproducing.
Yes, but there's, you know, I work with LEGO robots a lot now. I have a bunch of them. They're mobile.
Mm.
They can't reproduce, but all they need is. I guess you're saying they can't repair themselves, but if you have a large number, if you have like a hundred.
Million of them, let's just focus on them reproducing. Right. They have microchips in them. Okay, then do they include a fab?
No.
Then how are they going to reproduce?
Well, they're. It doesn't have to be all on board.
Right.
They can go to a factory, to a repair shop.
Yeah, but then you're really moving away from robustness.
Yes.
All of life is capable of reproducing without needing to go to a repair shop. Life will continue to reproduce in the complete absence of civilization. Robots will not. So when the, if, if the AI apocalypse happens, I mean, the AIs are going to probably die out because I think we're going to get again super intelligence long before we get robustness.
What about if you just improve the fab to where you just have a 3D printer? That can always help you?
Well, that'd be very interesting. I'm interested in building that.
Of course you are. You think, how difficult is that problem to have a robot that basically can build itself?
Very, very hard.
I think you've mentioned this, like to me or somewhere where people think it's.
Easy conceptually, and then they remember that you're going to have to have a fab.
Yeah. On board, of course. So 3D printer. That's prints. A 3D printer.
Yeah, yeah.
On legs. Why is that hard?
Well, because, I mean, a 3D printer is a very simple machine. Right. Okay. You're going to print chips, you're going to have an atomic printer. How are you going to dope the silicon? Yeah, right. How are you going to etch the silicon?
You're going to have to have a. A very interesting kind of fab. If you want to have a lot of Computation on board. But you can do like structural type of robots that are dumb.
Yeah, but structural type of robots aren't going to have the intelligence required to survive in any complex environment.
What about like ants type of systems? We have like trillions of them.
I don't think this works. I mean again, like ants at their very core are made up of cells that are capable of individually reproducing.
They're doing quite a lot, a lot of computation that we're taking for granted.
It's not even just the computation, it's that reproduction is so inherent. Okay, so like there's two stacks of life in the world. There's the biological stack and the silicon stack. The biological stack starts with reproduction. Reproduction is at the absolute core. The first proto RNA organisms were capable of reproducing. The silicon stack, despite as far as it's come, is nowhere near being able to reproduce.
Yeah, so the fab movement, digital fabrication, fabrication in the full range of what that means is still in the early stages.
Yeah.
You're interested in this world even if.
You did put a fab on the machine. Right. Let's say, okay, we can build fabs. We know how to do that as humanity. We can probably put all the precursors that build all the machines and the fabs also in the machine. So first off, this machine is going to be absolutely massive. I mean we almost have a, like think of the size of the thing required to reproduce a machine today. Right? Like is our civilization capable of reproduction? Can we reproduce our civilization on Mars.
If we were to construct a machine that is made up of humans, like a company that can reproduce itself.
Yeah.
I don't know, it feels like like, like a hundred fifteen people.
It's so much harder than that.
120.
I believe that Twitter can be run by 50 people. I think that this is going to take most of like, it's just most of society. Right? Like we live in one globalized world.
No, but you're not interested in running Twitter. You're interested in seeding like you want to seed a civilization and then because humans can like.
Oh, okay, you're talking about, yeah, okay, so you're talking about the humans reproducing and like basically like what's the smallest self sustaining colony of humans? Yeah, yeah, okay, fine. But they're not going to be making 5 nanometer chips over time.
They will. I think you're being like we have to expand our conception of time here going back to the original timescale. I mean over, across maybe 100 generations. We're back to making chips. No, if you seed the colony correctly, maybe.
Or maybe they'll watch our colony die out over here and be like, we're not making chips. Don't make chips.
No, but you have to seed that colony correctly.
Whatever you do, don't make chips. Chips are what led to their downfall.
Well, that is the thing that humans do. They. They come up, they construct a devil. A good thing and a bad thing, and they really stick by that. And then they murder each other over that. There's always one asshole in the room who murders everybody, and he usually makes tattoos and nice branding.
Do you need that asshole? That's the question. Right. Humanity works really hard today to get rid of that, but I think they might be important.
Yeah, this whole freedom of speech thing, it's. It's the freedom of being an. Seems kind of important.
That's right.
Man, this thing, this fab, this human fab that we constructed, this human civilization is pretty interesting. And now it's building artificial copies of itself or artificial copies of various aspects of itself that seem interesting, like intelligence. And I wonder where that goes.
I like to think it's just like another stack for life. Like we have, like, the bio stack life. Like we're a bio stack life and then the silicon stack life.
But it seems like the ceiling, or there might not be a ceiling, or at least the ceiling is much higher for the. For the silicon stack.
Oh, no, I don't. We don't know what the ceiling is for the biostack either. The biostack. The biostack just seemed to move slower. You have Moore's law, which is not dead, despite many proclamations in the biostack or the silicon in the silicon stack. And you don't have anything like this in the biostack. So I have a meme that I posted. I tried to make a meme. It didn't work too well. But I posted a picture of Ronald Reagan and Joe Biden. And you look, this is 1980, and this is 2020. And these two humans are basically like the same. Right? There's no. There's no. Like. Like there. There's been no change in humans in the last 40 years.
Yeah.
And then I posted a computer from 1980 and a computer from 2020. Wow.
Yeah. With their early. Early stages. Right, which is why you said, when you said the fab. The size of the fab required to make another fab is like very large right now.
Oh, yeah.
But computers were very large 80 years ago, and they got pretty tiny. And people Are starting to want to wear them on their face in order to escape reality. That's the thing. In order to be live inside the computer.
Yeah.
Put a screen right here. I don't have to see the rest of you.
I've been ready for a long time.
You like virtual reality?
I love it.
Do you want to live there?
Yeah.
Yeah. Part of me does too. How far away are we, do you think?
Judging from what you can buy today, Far, Very far.
I got to tell you that I had the experience of Meta's Kodak avatar where it's a ultra high resolution scan. It looked real.
I mean, the headsets just are not quite at like eye resolution yet. I haven't put on any headset where I'm like, oh, this could be the real world. Whereas when I put good headphones on, audio is there, like we. We can reproduce audio that I'm like, I'm actually in a jungle right now. I. If I close my eyes, I can't tell. I'm not.
Yeah, but then there's also smell and all that kind of stuff.
Sure.
I don't know. I. The. The power of imagination or the power of the. The mechanism in the human mind that fills the gaps, that kind of reaches and wants to make the thing you see in the virtual world real to you. I believe in that power.
Or humans want to believe.
Yeah. Like, what if you're lonely? What if you're sad? What if you're really struggling in life and here's a world where you don't have to struggle anymore?
Humans want to believe so much that people think the large language models are conscious. That's how much humans want to believe.
Strong words. He's throwing left and right hooks. Why do you think large language models are not conscious?
I don't think I'm conscious.
Oh, so what is consciousness then, George Hodtz?
It's like what it seems to mean to people. It's just like a word that atheists use for souls. Sure.
But that doesn't mean soul is not an interesting word.
If consciousness is a spectrum, I'm definitely way more conscious than the large language models are. I think the large language models are less conscious than a chicken.
When is the last time you've seen.
A chicken in Miami? Like a couple months ago. How?
No, like a living chicken.
Just living chickens walking around Miami. It's crazy.
Like on the street. Yeah, like a chicken.
A chicken.
All right.
All right.
I was trying to call you out like, like a good journalist, and I. I got shut down. Okay, but you don't think much about this kind of subjective feeling that it, it feels like something to exist. And then as an observer, you can have a sense that an entity is not only intelligent, but has a kind of subjective experience of its reality. Like a self awareness that is capable of, like suffering, of hurting, of being excited by the environment in a way that's not merely kind of an artificial response, but a deeply felt one.
Humans want to believe so much that if I took a rock and a Sharpie and drew a sad face on the rock, they'd think the rock is sad. Yeah.
And you're saying when we look in the mirror, we, we apply the same smiley face with rock?
Pretty much, yeah. Doesn't.
It's not weird though that you're not conscious that.
No.
But you do believe in consciousness. It's just, it's unclear. Okay, so to you it's like a little like a symptom of the bigger thing. That's not that important.
Yeah, I mean, it's interesting that like human systems seem to claim that they're conscious and I guess it kind of like says something in a straight up, like, okay, what do people mean when. Even if you don't believe in consciousness, what do people mean when they say consciousness? And there's definitely like meanings to it.
What's your favorite thing to eat?
Pizza.
Cheese pizza. What are the toppings?
I like cheese pizza.
Don't say pineapple.
No, I don't like pineapple.
Okay.
Pepperoni pizza, as they put any ham on it? Oh, that's real bad.
What's the best, what's the best pizza? What are we talking about here? Like, you like cheap crappy pizza?
Chicago deep dish cheese. Oh, that's, that's my favorite.
There you go. You bite into a deep dish Chicago deep dish pizza and it feels like you were starving. You haven't eaten for 24 hours. Just bite in and you're hanging out with somebody that matters a lot to you and you're there with the pizza.
Sounds real nice.
Yeah. All right. It feels like something. I'm. I'm George motherfucking Hot. Eating a fucking Chicago deep dish pizza. There's just the full peak light living experience of being human. The top of the human condition.
Sure.
It feels like something to experience that. Why does it feel like something? That's consciousness, isn't it?
If that's the word you want to use to describe it? Sure. I'm not going to deny that that feeling exists. I'm not going to deny that I experience that feeling when I guess what I kind of take issue to is that there's some like. Like, how does it feel to be a web server? Do 404s hurt?
Not yet.
How would you know what suffering looked like? Sure. You can recognize a suffering dog because we're the same stack as the dog. All the bio stack stuff. Kind of. Especially mammals. You know, it's really easy. You can.
Game recognizes game.
Yeah. Versus the silicon stack stuff. It's like you have no idea. You have you. It. Oh, wow. The little thing has learned to mimic, you know. But then I realized that that's all we are too.
Oh, look.
The little thing has learned to mimic.
Yeah, I guess. Yeah. 404 could be. Could be suffering. But it's so far from our kind of living organism, our kind of stack. But it feels like AI can start maybe mimicking the biological stack. Better, better. Better. Because it's trained.
We trained it. Yeah.
And so in that maybe that's the definition of consciousness is the biostat consciousness.
The definition of consciousness is how close something looks to human. Sure. I'll give you that one.
No, how close something is to the human experience.
Sure. It's a very anthropocentric definition.
But wait, that's all we got.
Sure. No, and I don't mean to like. I think there's a lot of value in it. Look, I just started my second company. My third company will be AI girlfriends. Oh.
Like, I mean, I want to find out what your fourth company is after.
Oh, wow.
Because I think once you have AI girlfriends, it's. Oh, boy, does it get interesting. Well, maybe. Let's go there. I mean, the relationships with AI that's creating human like organisms. Right. And part of being human is being conscious is being having the capacity to suffer, having the capacity to experience this life richly in such a way that you can empathize. The AI system can empathize with you, and you can empathize with it, or you can project your anthropomorphic sense of what the other entity is experiencing. And an AI model would need to create that experience inside your mind. And it doesn't seem that difficult.
Yeah. But okay, so here's where it actually gets totally different. Right. When you interact with another human, you can make some assumptions.
Yeah.
When you interact with these models, you can't. You can make some assumptions that. That other human experiences suffering and pleasure in a pretty similar way to you do. The golden rule applies with an AI model. This isn't really true. Right. These. These large language models are good at fooling people because they were trained on a whole bunch of human data and told to mimic it.
Yep. But if the AI system says, hi, my name is Samantha, it has a backstory. I went to college here and there. Maybe you'll integrate this in the AI system.
I made some chatbots. I give them backstories. It was lots of fun. I was so happy when Llama came out.
Yeah, we'll talk about Llama. We'll talk about all that. But the rock with the smiley face, it seems pretty natural for you to anthropomorphize that thing and then start dating it. And before you know it, you're married and have kids.
With a rock.
With a rock. There's pictures on Instagram with you and a rock and smiley face.
To be fair, like, you know, something that people generally look for when they're looking for someone to date is intelligence in some form. And the rock doesn't really have intelligence. Only a pretty desperate person would date a rock.
I think we're all desperate deep down.
Oh, not rock level desperate. All right.
Not rock level desperate, but AI level desperate. I don't know. I think all of us have a deep loneliness. It just feels like the language models are there.
Oh, I agree. And you know what? I won't even say this so cynically. I will actually say this in a way that, like, I want AI friends. I do.
Yeah.
Like, I would love to. You know, again, I. The language models now are still a little, like, people are impressed with these GPT things, and I look at like, Or. Or, like. Or the. The copilot, the coding one, and I'm like, okay, this is, like, junior engineer level, and these people are, like, Fiverr level artists and copywriters. Like, okay, great. We got, like, Fiverr and, like, junior engineers. Okay, cool. Like, and this is just the start. And it will get better. Right? Like, I can't wait to have AI friends who are more intelligent than I am.
So Fiverr is just a temporary. It's not the ceiling.
No, definitely not.
Is it. Is it count as cheating when you're talking to an AI model? Emotional cheating.
That's. That's up to you and your human partner to define.
Oh, you have to. All right, you can.
Yeah, you have to have that. Have to have that conversation, I guess.
All right. I mean, integrate that with. With porn and all this.
No, I mean, it's similar kind of to porn.
Yeah.
Yeah. Right? I think people in relationships have different views on that.
Yeah, but most people don't have, like, serious, open conversations about all the different aspects of what's cool and what's not. And it feels like AI is a really weird conversation to have.
The porn one is a good branching off point, like these things. You know, One of my scenarios that I put in my chat bot is I, you know a nice girl named Lexi, she's 20, she just moved out to LA. She wanted to be an actress, but she started doing only fans instead. And you're on a date with her. Enjoy.
Oh man. Yeah, and so is that if you're actually dating somebody in real life, is that cheating? I feel like it gets a little weird. Sure, it gets real weird. Like what are you allowed to say to an AI bot? Imagine having that conversation with a significant other.
I mean, these are all things for people to define in their relationships. What it means to be human is just going to start to get weird, especially online.
Like how do you know? Like, there'll be moments when you'll have what you think is a real human you interacted with on Twitter for years and you realize it's not.
I spread. I love this meme. Heaven banning. You know what? Shadow banning. Yeah, shadow banning. Okay. You post, no one can see it. Heaven banning. You post, no one can see it. But a whole lot of AIs are spun up to interact with you.
Well, maybe that's what the way human civilization ends is. All of us haven't banned.
There's a great. It's called My Little Pony. Friendship is Optimal. It's a sci fi story that explores this idea.
Friendship is optimal.
Friendship is optimal.
Yeah, I'd like to have some, at least on the intellectual realm, some AI friends.
Yeah.
That argue with me. But the, the, the romantic realm is weird. Definitely weird, but not out of the realm of the, the kind of weirdness that human civilization is capable of.
I think I want it. Look, I want it. If no one else wants it, I want it.
Yeah, I think a lot of people probably want it.
There's a deep loneliness and I'll fill their loneliness and you know, it just will only advertise to you some of the time.
Yeah, maybe the conceptions of monogamy change too. Like I grew up in a time like I value monogamy, but maybe that's a silly notion when you have arbitrary number of AI systems.
This, this, this interesting path from rationality to polyamory. Yeah, that doesn't make sense for me.
For you? But you're just a biological organism who was born before like the Internet really took off.
The crazy thing is like culture is whatever we define it as. These Things are not. You have the is ought problem in moral philosophy. Right. There's no okay. What is might be that computers are capable of mimicking girlfriends perfectly. They passed the girlfriend Turing test. But that doesn't say anything about ought. That doesn't say anything about how we ought to respond to them as a civilization. That doesn't say we ought to get rid of monogamy. Right. That's a completely separate question. Really? A religious one.
Girlfriend Turing test. I wonder what that looks like.
Girlfriend Turing test.
Are you writing that? Will you be the. The. The Alan turing of the 21st century that writes the. The girlfriend Turing Test? No.
I mean, of course, my. My AI girlfriends, their goal is to pass the girlfriend Turing test.
No, but you. There should be, like, a paper that kind of defines the test. Or. I mean, the question is if it's deeply personalized or there's a common thing that everybody.
Yeah. I mean, you know, look, we're a company. We don't have to get everybody. We just have to get a large enough clientele to stay.
I like how you're already. Already thinking company. All right, let's. Before we go to company number three and company number four, let's go to company number two.
All right.
Tiny Corp, possibly one of the greatest names of all time for a company. You've launched a new company called Tiny Corp that leads the development of Tiny graduates. What's the origin story of Tiny Corp and Tiny Grad?
I started Tiny Grad as a toy project just to teach myself. Okay, what is a convolution? What are all these options you can pass to them? What is the derivative of convolution? Very similar to. Karpathy wrote micrograd. Very similar. And then I started thinking about AI chips. I started thinking about chips that run AI, and I was like, well, okay, this is going to be a really big problem if Nvidia becomes a monopoly here. How long before Nvidia is nationalized?
So you. One of the reasons to start Tiny Corp is to challenge Nvidia.
It's not so much to challenge Nvidia. Actually, I like Nvidia, and it's to make sure power stays decentralized.
Yeah. And here it's computational power. And to you, Nvidia is kind of locking down the computational power of the world.
If Nvidia becomes just like 10x better than everything else, you're giving a big advantage to somebody who can secure Nvidia as a resource.
Yeah.
In fact, if Jensen watches this podcast, he may want to consider this. He may want to consider Making sure his company is not nationalized.
You think that's an actual threat?
Oh, yes. No.
But there's so much, you know, there's amd.
So we have Nvidia and amd. Great, all right.
But you don't think there's like a push towards selling, like Google selling TPUs or something like this? You don't think there's a push for that?
Have you seen it? Google loves to rent utpus.
It doesn't. You can't buy it at Best Buy.
So I started work on a chip. I was like, okay, what's it going to take to make a chip? And my first notions were all completely wrong about why? About, like, how you could improve on GPUs. And I will take this. This is from Jim Keller on your podcast. And this is one of my absolute favorite descriptions of computation. So there's three kinds of computation paradigms that are common in the world today. There's CPUs. And CPUs can do everything. CPUs can do add and multiply. They can do load and store, and they can do compare and branch. And when I say they can do these things, they can do them all fast, right? So compare and branch are unique to CPUs. And what I mean by they can do them fast is they can do things like branch prediction and speculative execution. And they spend tons of transistors on these like super deep reorder buffers in order to make these things fast. Then you have a simpler computation model, GPUs. GPUs can't really do compare and branch. I mean, they can, but it's horrendously slow. But GPUs can do arbitrary load and store, right? GPUs can do things like X, dereference, Y. So they can fetch from arbitrary pieces of memory. They can fetch from memory that is defined by the contents of the data. The third model of computation, DSPs. And DSPs are just ADD and multiply, right? Like they can do load and stores, but only static load and stores, only loads and stores that are known before the program runs. And you look at neural Networks today, and 95% of neural networks are all the DSP paradigm. They are just statically scheduled adds and multiplies. So tinyguard really took this idea and I'm still working on it, to extend this as far as possible. Every stage of the stack has Turing completeness, Python has Turing completeness. And then we take Python, we go into C, which is turn complete, and maybe C calls into some CUDA kernels which are Turn complete. The CUDA kernels go through llvm, which is turn complete into ptx, which is Turing complete. Sas, which is Turing complete on a Turing complete processor. I want to get Turing completeness out of the stack entirely because once you get rid of Turing completeness, you can reason about things. Rice's theorem and the halting problem do not apply to add model machines.
Okay, what's the power and the value of getting Turing completeness out of. Out of. Are we talking about the hardware or the software?
Every layer of the stack.
Every layer?
Every layer of the stack. Removing Turing completeness allows you to reason about things, right? So the reason you need to do branch prediction in a CPU and the reason it's prediction and the branch predictors are I think they're like 99% on CPUs. Why do they get 1% of them wrong? Well, they get 1% wrong because you can't know, right? That's the halting problem. It's equivalent to the halting problem to say whether a branch is going to be taken or not. I can show that. But the ADMOL machine, the neural network, runs the identical compute every time. The only thing that changes is the data. So when you realize this, you think about, okay, how can we build a computer? And how can we build a stack that takes maximal advantage of this idea? So what makes Tiny Grad different from other neural network libraries is it does not have a primitive operator even for matrix multiplication. And this is every single one. They even have primitive operators for things like convolutions.
So no matmul.
No matmul. Well, here's what a MATMUL is. So I'll use my hands to talk here. So if you think about a cube and I put my two matrices that I'm multiplying on two faces of the cube, right? You can think about the matrix multiply as, okay, the N cubed. I'm going to multiply for each one in the cubed, and then I'm going to do a sum which is a reduce up to here to the third face of the cube. And that's your multiplied matrix. So what a matrix multiply is, is a bunch of shape operations, right? A bunch of permuth three shapes and expands on the two matrices. A multiply N cubed, a reduce N cubed, which gives you an N squared matrix.
Okay, so what is the minimum number of operations that can accomplish that if you don't have Matmul as a primitive?
So tiny grad has about 20. And you can compare tiny grad's opset or IR to things like XLA or Primtorch. So XLA and Primtorch are ideas where like, okay, torch has like 2,000 different kernels. Pytorch 2.0 introduced Primtorch which has only 250. Tiny grad has order of magnitude 25. It's 10x less than XLI or Primtorch. And you can think about it as kind of like RISC versus cisc. Right. These other things are cisk like systems. Tiny GRAT is risk and risk. One risk architecture is going to change everything. 1995 hackers.
Wait, really? That's an actual thing?
Angelina Jolie delivers the line. Risk architecture is going to change everything in 1995. And here we are with ARM and the phones and ARM everywhere.
Wow. I love it when movies actually have real things in them. Right, okay, interesting. And so this is like. So you're thinking of this as the risk architecture of MLStack25. What? What? Can you. Can you go through the. The four OP types?
Sure. Okay, so you have unary ops, which take in a tensor and return a tensor of the same size and do some unary OP to it. X log reciprocal sine. Right. They take in one and they're point wise.
Relu.
Yeah, relu. Almost all activation functions are unaryops. Some combinations of unary ops together is still a unaryop. Then you have binary ops. Binary ops are like point wise. Addition, multiplication, division, compare. It takes in two tensors of equal size and outputs one tensor. Then you have reduce ops. Reduce ops will take a three dimensional tensor and turn it into a two dimensional tensorflow or a three dimensional tensor turn into zero dimensional tensor. Things like a sum or a max are really the common ones there. And then the fourth type is movement ops. And movement ops are different from the other types because they don't actually require computation. They require different ways to look at memory. So that includes reshapes, permutes, expands, flips. Those are the main ones probably.
And so with that you have enough to make a matmul and convolutions, convolutions.
And every convolution you can imagine dilated convolutions, dried convolutions, transposed convolutions.
You're right on GitHub about laziness showing a matmul matrix multiplication. See how despite the style, it is fused into one kernel with the power of laziness. Can you elaborate on this power of laziness?
Sure. So if you type in pytorch A times B plus C. What this is going to do is it's going to first multiply A and B and store that result into memory and then it is going to add C by reading that result from memory, reading C from memory and writing that out to memory. There is way more loads and stores to memory than you need there. If you don't actually do A, B as soon as you see it. If you wait until the user actually realizes that Tensor, until the laziness actually resolves, you can fuse that plus C. This is like. It's the same way Haskell works.
So what's the process of porting a model into Tiny Grad?
So Tiny Grad's front end looks very similar to Pytorch. I probably could make a perfect or pretty close to perfect interop layer if I really wanted to. I think that there's some things that are nicer about Tiny Grad syntax than Pytorch. But the front end looks very torch like. You can also load in ONNX models. We have more ONNX tests passing than CoreML Cornwall. Okay, so we'll pass Onyx runtime soon.
What about like the developer experience with Tiny Grad? What it feels like, what A versus Pytorch?
By the way, I really like Pytorch. I think that it's actually a very good piece of software. I think that they've made a few different trade offs and these different trade offs are where, you know, Tiny Grad takes a different path. One of the biggest differences is it's really easy to see the kernels that are actually being sent to the GPU, right? If you run PyTorch on the GPU, you like, do some operation and you don't know what kernels ran, you don't know how many kernels ran, you don't know how many flops were used, you don't know how much memory accesses were used. Tiny grad type debug equals 2 and it will show you in this beautiful style every kernel that's run how many flops and how many bytes.
So can you just linger on what problem Tiny Grad solves?
Tiny Grad solves the problem of porting new ML accelerators quickly. One of the reasons tons of these companies now, I think Sequoia marked graphcore to zero, right? Cerebus tens, Torrent, Groq. All of these ML accelerator companies, they built chips. The chips were good, the software was terrible. And part of the reason is because I think the same problem is happening with Dojo. It's really, really hard to write a Pytorch port because you have to write 250 kernels and you have to tune them all for performance.
What does Jim Jim Culler think about Tiny Grad? You guys hung out quite a bit. So he was involved. He's involved with Ben Stor. What's his praise and what's his criticism of what you're doing with your life?
Look, my prediction for ten's torrent is that they're going to pivot to making risc V chips. CPUs.
CPUs?
Yeah.
Why?
Because AI accelerators are a software problem, not really a hardware problem.
Oh, interesting. So you don't think, you think the diversity of AI accelerators in the hardware space is not going to be a thing thing that exists long term?
I think what's going to happen is if I can finish. Okay. If you're trying to make an AI accelerator, you better have the capability of writing a torch level performance stack on Nvidia GPUs. If you can't write a torch stack on Nvidia GPUs, and I mean all the way, I mean down to the driver, there's no way you're going to be able to write it on your chip. Because your chip's worse than an Nvidia gpu. The first version of the chip you tape out, it's definitely worse.
Oh, you're saying writing that stack is really tough?
Yes. And not only that, actually the chip that you tape out almost always because you're trying to get advantage over Nvidia, you're specializing the hardware more. It's always harder to write software for more specialized hardware. Like a GPU is pretty generic and if you can't write an Nvidia stack, there's no way you can write a stack for your chip. So my approach with Tiny grad is first write a performant Nvidia stack. We're targeting amd.
So you did say a few to Nvidia? A little bit. With love.
With love?
Yeah, with love.
So, like the Yankees, you know, I'm a Mets fan.
Oh, you're, you're, you're a Mets fan? A risk. A risk fan and a Mets fan. What's the hope that AMD has? You did a build with AMD recently that I saw. How does the, the, the 7900 xtx compare to the RTX 4090 or 4080?
Well, let's start with the fact that the 7900 xtx kernel drivers don't work. And if you Run demo apps in loops, it panics the kernel.
Okay. So this is a software issue.
Lisa sue responded to my email.
Oh.
I reached out. I was like, this is, you know, really like. I Understand if your 7 by 7 transposed Winograd conv is slower than Nvidia's, but literally when I run demo apps in a loop, the kernel panics.
So just adding that loop.
Yeah. I just literally took their demo apps and wrote like while true do the app semicolon done in a bunch of screens. This is the most primitive fuzz testing.
Why do you think that is? They're just not seeing a market in machine learning.
They're changing. They're trying to change. They're trying to change. And I had a pretty positive interaction with them this week. Last week I went on YouTube. I was just like, that's it. I give up on amd. This is their driver. Doesn't even like. I'm not gonna, I'm not gonna. You know, I'll go with Intel GPUs. Right. Intel GPUs have better drivers.
So you're kind of spearheading the diversification of GPUs.
Yeah, and I'd like to extend that diversification to everything. I'd like to diversify the. Right. The more my central thesis about the world is there's things that centralize power and they're bad. And there's things that decentralize power and they're good. Everything I can do to help decentralize power, I'd like to do.
So you're really worried about the centralization of Nvidia. That's interesting. And you don't have a fundamental hope for the proliferation of ASICs except in the cloud.
I'd like to help them with software. No, actually there's only. The only ASIC that is remotely successful is Google's tpu. And the only reason that's successful is because Google wrote a machine learning framework. I think that you have to write a competitive machine learning framework in order to be able to build an asic.
You think Meta with Pytorch builds a competitor?
I hope so. They have one. They have an internal one.
Internal. I mean, public facing with a nice cloud interface and so on.
I don't want a cloud.
You don't like cloud?
I don't like cloud.
What do you think is the fundamental limitation of cloud?
Fundamental limitation of cloud is who owns the off switch.
So it's power to the people.
Yeah.
And you don't like the man to have all the power.
Exactly.
All right. And Right now the only way to do that is with Nvidia GPUs. If you want performance and stability. Interesting. It's a costly investment emotionally to go with AMD's. Well, let me sort of on a tangent ask you what you've built Quite a few PCs. What's your advice on how to build a good custom PC for let's say for the different applications that you use for gaming, for machine learning?
Well, you shouldn't build one. You should buy a box from the Tiny Corp.
I heard rumors, whispers about this box in the Tiny Corp. What's, what's this thing look like? What it, what is it, what is it called?
It's called the Tiny Box.
Tiny box.
Um, it's $15,000.
Yep.
And it's almost a paid a flop of compute. It's over 100 gigabytes of GPU RAM. It's over 5 terabytes per second of GPU memory bandwidth. I'm going to put like 4 NVMEs in. In. In RAID. You're going to get like 20, 30 gigabytes per second of drive read bandwidth. I'm going to, I'm going to build like the best deep learning box that I can that plugs into one wall outlet.
Okay, can you go through those specs again a little bit from your, from memory?
Yeah. So it's almost a pay to flop of compute. So AMD intel today I'm leaning toward amd, but we're pretty agnostic to the type of compute. The, the, the main limiting spec is a 120 volt 15amp circuit. Okay, well I mean it, because in order to like, like there's a plug over there, right? You have to be able to plug it in. We're also going to sell the tiny rack which like what's the most power you can get into your house without arousing suspicion? And one of the, one of the answers is an electric car charger.
Wait, where does the rack go?
Your garage.
Interesting. The car charger.
A wall outlet is about 1500 watts. A car charger is about 10,000 watts.
What is the most amount of power you can get your hands on without arousing suspicion? That's right, George. Hots. Okay, so the, the tiny box and you said nvmes and raid. I forget what you said about memory, all that kind of stuff. Okay, so what about what GPUs again?
Probably, probably seven 900 UXXs, but maybe 3090s, maybe a 770s. Those are intels.
You're flexible or still exploring.
I'm still exploring. I want to deliver a really good experience to people. And yeah, what GPUs I end up going with again, I'm leaning toward AMD. We'll see. You know in my email what I said to AMD is like just dumping the code on GitHub is not open source. Open source is a culture. Open source means that your issues are not all one year old stale issues. Open source means developing in public and if you guys can commit to that, I see a real future for AMD as a competitor to Nvidia.
Well, I'd love to get a tiny box to mit, so whenever it's ready, let's do it.
We're taking pre orders. I took this from elon. I'm like $100 fully refundable pre orders.
Is it going to be like the cybertruck is going to take a few years or.
No, I'll try to do it faster than that. It's a lot simpler. It's a lot simpler than a truck.
Well, there's complexities not to just the putting the thing together but like shipping and all this kind of stuff.
The thing that I want to deliver to people out of the box is being able to run 65 billion parameter llama in FP16 in real time in like a good like 10 tokens per second or 5 tokens per second or something.
Just. It works. Llama's running or something like Llama experience.
Yeah. Or I think Falcon is the new one. Experience a chat with the largest language model that you can have in your house. Yeah.
From. From a wall plug.
From a wall plug. Yeah. Actually for inference it's not like even more power would help you get more.
Even more power and get you more.
Well, no, there's just the biggest param. The biggest model released is. Is 65 billion parameter llama as far as I know.
So it sounds like tinybox will naturally pivot towards company number three because you could just get the girlfriend and I mean. Or boyfriend. That one's harder actually the boyfriend is harder.
Boyfriend's harder.
Yeah, I think that's a very biased statement. I know a lot of people just think what's what. Why is it harder to replace a boyfriend than the girlfriend with the artificial LLM?
Because women are attracted to status and power and men are attracted to youth and beauty. No, I mean that's what I mean.
Both are. Could be unmimicable easy through the language model.
No, no, machines do not have any status or real power.
I don't know. I think you both. Well first of all, you're using language mostly to. To communicate youth and beauty and power and status.
But status fundamentally is a zero sum game. Right, Whereas youth and beauty are not.
No, I think status is a narrative you can construct. I don't think status is real.
I don't know. I just think that that's why it's harder, you know? Yeah, maybe it is. My biases.
I think status is way easier to fake.
I also think that, you know, men are probably more desperate and more likely to buy my product, so maybe they're a better target market.
Desperation is interesting. Easier to fool.
Yeah, that's.
I could. I could see that.
Yeah. Look, I mean, look, I know you can look at porn viewership numbers, right? A lot more men watch porn than women.
Yeah.
You can ask why that is.
Wow. There's a lot of questions than answers you can get there anyway with the, with the tiny box. How many GPUs in tiny box?
Six.
Oh, man.
And I'll tell you why. It's six.
Yeah.
So AMD Epic processors have 128 lanes of PCIe. I want to leave enough lanes for some drives and I want to leave enough lanes for some networking.
How do you do cooling for something like this?
Ah, that's one of the big challenges. Not only do I want the cooling to be good, I want it to be quiet. I want the tiny box to be able to sit comfortably in your room. Right.
This is really going towards the girlfriend thing because you want to run the LLM.
I'll give. I'll give a more. I mean, I can talk about how it relates to company number one.
Come. AI. Yeah, well. But yes, quiet. Oh, quiet because you maybe potentially want to run in a car.
No, no, quiet because you want to put this thing in your house and you want it to coexist with you. If it's screaming at 60 decibels, you don't want that in your house. You'll kick it out.
60 decibels? Yeah, yeah.
I want like 40, 45.
So how do you make the cooling quiet? That's an interesting problem in itself.
A key trick is to actually make it big. Ironically, it's called the tiny box. Yeah, but if I can make it big, a lot of that noise is generated because of high pressure air. If you look at like a 1U server, a 1U server has these super high pressure fans. They're like super deep and they, they're like gen engines versus if you have something that's big. Well, I can use a big, you know. You know they call them big ass fans. Those ones that are like huge on the ceiling and they're completely silent.
So Tiny Box will be big.
It is the. I do not want it to be large according to ups. I want it to be shippable as a normal package. But that's my constraint there.
Interesting. Well, the, the fan stuff, can't it be assembled on location or. No, no, no, it has to be. Well, you're, you're.
Look, I want to give you a great out of the box experience. I want you to lift this thing out. I want it to be like, like the Mac, you know, Tiny Box, the Apple experience.
Yeah, I love it. Okay. And so Tiny Box would run Tiny Grad. Like what, what, what do you envision this whole thing to look like? We're talking about like Linux with a full software engineering environment and just not Pytorch, but Tiny Grad.
Yeah, we did a poll. If people want Ubuntu or Arch, we're going to stick with Ubuntu.
Ooh, interesting. What's your favorite flavor of Linux? Ubuntu. I like Ubuntu Mate, however you pronounce that mate. So how do you. You've gotten LLAMA into Tiny Grad, you've gotten Stable Diffusion into Tiny Grad. What was that like? Can you comment on like, what are, what are these models? What's interesting about porting them, what are the challenges, what's naturally, what's easy, all that kind of stuff.
There's a really simple way to get these models into Tiny Grad and you can just export them as onnx and then Tiny Grad can run onnx. So the ports that I did of llama, Stable Diffusion and now Whisper are more academic to teach me about the models. But they are cleaner than the Pytorch versions. You can read the code. I think the code is easier to read. It's less lines. There's just a few things about the way Tiny Grid writes things. Here's a complaint I have About PyTorch NN Relu is a class, right? So when you create an NN module, you'll put your nnrelus, as in init. And this makes no sense. Relu is completely stateless. Why should that be a class?
But that's more like a software engineering thing. Or do you think it has a cost on performance?
Oh no, it doesn't have a cost on performance, but yeah, no, I think that it, it's. That's what I mean about like Tiny Grad's front end being Cleaner.
I see. What do you think about Mojo? I don't know if you've been paying attention to the programming language that does some interesting ideas that kind of intersect Tiny Grad.
I think that there's a spectrum and like on one side you have Mojo and on the other side you have like ggml. Ggml is this like we're going to run Llama fast on Mac? Okay. We're going to expand out to a little bit, but we're going to basically like depth first. Right. Mojo is like we're going to go breath first. We're going to go so wide that we're going to make all of Python fast and Tiny grads in the middle. Tiny grass. We are going to make neural networks fast.
Yeah. But they, they try to really get it to be fast, compiled down to specific hardware and make that compilation step as flexible and resilient as possible.
Yeah, but they have Turing completeness and.
That limits you turn. That's what you're seeing. It's somewhere in the middle. So you're actually going to be targeting some accelerators. Some like, some. Some number, not one.
My goal is step one, build an equally performant stack to Pytorch on Nvidia and amd, but with way less lines. And then step two is, okay, how do we make an accelerator? Right, but you need step one. You have to first build the framework before you can build the accelerator.
Can you explain MLPerf, what's your approach in general to benchmarking Tiny Grad performance?
So I'm much more of a, like, build it the right way and worry about performance later. There's a bunch of things where I haven't even like really dove into performance. The only place where Tiny Grad is competitive performance wise right now is on Qualcomm GPUs. So tiny grad's actually used in OpenPilot to run the model. So the driving model is Tiny Grad.
When did that happen, that transition?
About eight months ago now. And it's 2x faster than Qualcomm's library.
What's the hardware that OpenPilot runs on the comma?
It's a Snapdragon 845. Okay. So this is using the GPU. So the GPU is an Adreno GPU. There's like different things. There's a really good Microsoft paper that talks about like mobile GPUs and why they're different from desktop GPUs. One of the big things is in a desktop GPU you can use Buffers on a mobile gpu image textures are a lot faster.
On a mobile gpu image textures. Okay. And so you want to be able to leverage that.
I want to be able to leverage it in a way that it's completely generic. Right. So there's a lot of. Xiaomi has a pretty good open source library for mobile GPUs called Mace, where they can generate, where they have these kernels, but they're all hand coded. Right. So that's great if you're doing 3x3 comps, that's great if you're doing dense matmuls, but the minute you go off the beaten path a tiny bit, well, your performance is nothing.
Since you mentioned OpenPilot, I'd love to get an update in the company number one common AI world. How are things going there in the development of semi autonomous driving?
You know, almost no one talks about FSD anymore and even less people talk about Open Pilot. We've solved the problem like we solved it years ago.
What's the problem exactly? Well, what does solving it mean?
Solving means how do you build a model that outputs a human policy for driving? How do you build a model that, given a reasonable set of sensors, outputs a human policy for driving? So you have companies like Waymo and Cruise, which are hand coding these things that are quasi human policies. Then you have Tesla and maybe even to more of an extent, comma asking, okay, how do we just learn the human policy from data? The big thing that we're doing now, and we just put it out on Twitter. At the beginning of comma, we published a paper called Learning a Driving Simulator. And the way this thing worked was it's a, it was an autoencoder and then an RNN in the middle, right? You take an autoencoder, you compress the picture, you use an rnn, predict the next state. And these things were, you know, it was a laughably bad simulator. Right? This is 2015, our machine learning technology. Today we have VQ, VAE and transformers. We're building drive GPT, basically.
Drive GPT. Okay, so. And it's trained on what? Is it trained in a self supervised way?
Yeah, it's trained on all the driving data to predict the next frame.
So really trying to learn a human policy. What would a human do?
Well, actually our simulator is conditioned on the pose. So it's, it's actually a simulator. You can put in like a state action pair and out the next state.
Okay.
And then once you have a simulator, you can do RL in the simulator and RL will get us that human policy.
So it transfers. Yeah.
RL with a reward function. Not asking is this close to the human policy, but asking, would a human disengage if you did this behavior?
Okay, let me think about the distinction there. Would a human disengage? Would a human disengage? That correlates, I guess with human policy, but it could be different. So it doesn't just say what would a human do? It says what would a good human driver do? Such that the experience is comfortable but also not annoying in that the thing is very cautious. So it's finding a nice balance. That's interesting.
It's asking exactly the right question. What will make our customers happy.
Right.
A system that you never want to.
Disengage because usually disengagement is almost always a sign of. I'm not happy with what the system is doing.
Usually there's some that are just. I felt like driving. And those are always fine too, but they're just going to look like noise in the data.
But even I felt like driving maybe. Yeah, that's even. That's a signal. Like why do you feel like driving here? You need to re. Calibrate your relationship with the car. Okay, so what that, that's really interesting. How close are we to solving self driving?
It's hard to say. We haven't completely closed the loop yet. So we don't have anything built that truly looks like that architecture yet. We have prototypes and there's bugs. So we are a couple bug fixes away. Might take a year, might take 10.
What's the nature of the bugs? Are these, these major philosophical bugs, logical bugs? What kind of, what kind of bugs are we talking about?
They're just like. They're just like stupid bugs. And like also we might just need more scale. We just massively expanded our compute cluster. At comma, we now have about two people worth of compute. 40 betaflops.
Well, people. People are different.
20 fade flops. That's a person. I mean it's just. It's just a unit. Right. Horses are different too, but we still call it a horsepower.
Yeah, but there's something different about mobility than there is about perception and action in a very complicated world. But yes.
Well, yeah, of course not all flops are created equal. If you have randomly initialized weights, it's not gonna.
Not all flops are created equal.
Doing way more useful things than others.
Yeah. Yep. Tell me about it. Okay, so more data scale means more scale and compute or scale in scale of data?
Both. Diversity of data Diversity is very important in data. Yeah, I mean we have. So we have about, I think we have like 5,000 daily actives.
How would you evaluate how FSD is doing?
Pretty well. Driving pretty well.
How's that race going between come AI and fsd?
Tesla is always one to two years ahead of us. They've always been one to two years ahead of us and they probably always will be because they're not doing anything wrong.
What have you seen that's since the last time we talked that are interesting? Architectural decisions, training decisions. Like the way, the way they deploy stuff, the architectures they're using in terms of the software, how the teams are run, all that kind of stuff. Data collection. Anything interesting?
I mean, I know they're moving toward more of an end to end approach.
So creeping towards end to end as much as possible across the whole thing. The, the training, the data collection, everything.
They also have a very fancy simulator. They're probably saying all the same things we are. They're probably saying we just need to optimize, you know, what is the reward? Well, you get negative reward for disengagement. Right. Like everyone kind of knows this. It's just a question who can actually build and deploy the system.
Yeah, I mean this good. It requires good software engineering, I think.
Yeah.
And the right kind of hardware.
Yeah. The hardware to run it.
You still don't believe in cloud in that regard?
I have a compute cluster in my office. 800amps.
Tiny grad.
It's 40 kilowatts at idle. Our data center dives me crazy with 40 kilowatts just burning just when the computers are idle.
Just when I.
Sorry, sorry. Compute cluster.
Compute cluster. I got it.
It's not a data center.
Yeah.
Now data centers are clouds. We don't have clouds. Data centers have air conditioners, we have fans. That makes it a compute cluster.
I'm guessing this is a kind of legal distinction.
Sure, yeah. We have a compute cluster.
You said that you don't think LLMs have consciousness, or at least not more than a chicken. Do you think they can reason? Is there something interesting to you about the word reason? About some of the capabilities that we think is kind of human to be able to integrate complicated information and through a chain of thought arrive at a conclusion that feels novel, a novel integration of the. Of disparate facts?
Yeah, I, I don't think that there's. I think that they can reason better than a lot of people.
Hey, isn't that amazing to you though? Isn't that like an incredible thing that A transformer can achieve.
I mean, I think that calculators can add better than a lot of people.
But language feels like reasoning through the process of language, which looks a lot like thought.
Making brilliancies in chess, which feels a lot like thought. Whatever new thing that AI can do, everybody thinks is brilliant. And then like 20 years go by and they're like, well, yeah, but chess, that's like mechanical, like adding, that's like mechanical.
So you think language is not that special. It's like chess.
It's like chess and it's, I don't.
Know, because it's very human. We, we take it, we listen. There is something different between chess and, and language. Chess is a game that a subset of population plays. Language is something we use non stop for all of our human interaction. And human interaction is fundamental to society. So it's like, holy shit, this, this language thing is not so difficult to like create in the machine.
The problem is if you go back to 1960 and you tell them that you have a machine that can play amazing chess, of course someone in 1960 will tell you that machine is intelligent. Someone in 2010 won't. What's changed? Right, Today we think that these machines that have language are intelligent, but I think in 20 years we're going to be like, yeah, but can it reproduce?
So reproduction, yeah. We might redefine what it means to, to be, what is it? A high performance living organism on Earth.
Humans are always going to define a niche for themselves. Like, well, you know, we're better than the machines because we can, you know, when, like they tried creative for a bit, but no one believes that one anymore.
But niche is, is that, is that delusional or is there some accuracy to that? Because maybe like with chess you start to realize like that, that we have ill conceived notions of what, what makes human special. Like the apex organism on Earth.
Yeah. And I think maybe we're going to go through that same thing with language and that same thing with creativity.
But language carries these notions of truth and so on. And so we might be like, wait, maybe truth is not carried by language. Maybe there's like a deeper thing, the.
Niche is getting smaller.
Oh boy.
But no, no, no, you don't understand. Humans are created by God and machines are created by humans. Therefore. Right. Like that'll be the last niche we have.
So what do you think about just the rapid development of LLMs? If we could just like stick on that. It's still incredibly impressive. Like with ChatGPT, just even ChatGPT. What are your thoughts about reinforcement learning with human feedback on these large language models.
I'd like to go back to when calculators first came out and. Or computers and, like, I wasn't around. Look, I'm 33 years old and to, like, see how that affected, like, society.
Maybe you're right. So I want to put on the. The. The big picture hat here.
Oh, my God, A refrigerator.
Wow. Refrigerator, electricity, all that kind of stuff, but.
