Elon Musks Shocking New AI Prediction

Elon Musk just had a recent Twitter space where he discussed with another investor called Cathie Wood the future developments coming in 2024. Things like Grok, AI, open source AI and just a lot of stuff that you do need to know if you are in the AI space. I do think that it is a really interesting talk because he touches up on some of the things that people have been discussing on Twitter regarding open source AI. He also does talk about the timelines for open source AI and how in around six months things could change because GPT4 might not be the market leader we want to know it to be. So with that being said, hopefully you enjoy this Twitter space audio. I'll leave a link to the full clip where he does actually talk about Tesla, but this is a supercut version to only hyper specific focus on the bits where he's talking about AI which is relevant for all of you. So with that being said, it basically.

Said, it seems that this is Grok talking. Seems, seems that people are eager to hear about the latest developments in AIs. I mean in Tesla's AI technologies. We definitely are. Such as FSD 12 and the next generation platform for Robo Taxi. They're also interested in learning more about the company's plans for the Optimus project. Additionally, there is a lot of curiosity surrounding Elon Musk's thought on the AI space as well as his take on the future of Bitcoin. I guess that was a little repetitive AI there, but these, these topics are sure to spark, spark some engaging discussions. That's, that's Grok for you.

Okay, well that's pretty good. Yeah. So AI and, and I do, I do have more time than you know. If you would like to go longer, longer than 15 minutes, we can go longer than 15 minutes.

Right.

I mean, I try to distill my thoughts here on AI because I could really talk four hours on AI.

You know, both Charlie and Brett are steeped in it, so is Frank and of course Tasha as she's done our work on autonomous. But one of the things we've been very curious about is open source versus closed AI. And I, I don't know if you, I don't know if you've seen the chart that Frank and Joseph did on our team, but they, they drew a line. They basically calculated performance improvements in both the closed, closed or private AI companies and models, large language models and then the open ones and open is behind closed. But the steep, the performance is improving at a steeper rate than closed. Do you have any Thoughts about that and about open versus closed generally when it comes to AI.

Sure. Well, I mean, I think a lot of people know that, you know, I was instrumental in the creation of OpenAI and the, in fact I came up with the Name the opening. OpenAI means open source.

Yes.

And it started as an open source nonprofit. And the reason I wanted to create it was to create, establish a counterweight to Google DeepMind because it was a unipolar world at that point. It was, you know, Google DeepMind had probably 2/3 or more of the top AI talent and of course, or fast computing and financial resources. And at the time I was close friends with Larry Page and it did not seem to me that he was taking AI safety seriously. So I felt there needed to be some, like I said, some counterweight, some competitor to sort of the Google DeepMind. Practically instrumental in creating the company, was recruiting Ilya Susgar, which was a back and forth thing where actually Ilya changed his mind a few times and ultimately decided to come to OpenAI. And so I won that recruiting battle and that is actually what caused Larry Page to stop being friends with me, frankly. But now, ironically, because Fate loves irony, OpenAI is super closed source and for maximum profit, it should be renamed Close for maximum profit. AI, it would be more accurate because it's literally the polar opposite of how it was started. I guess I generally have a bias in favor of open source and you can see that certainly with the X platform where we've open sourced the algorithm and for example, for community notes, we open source not just the algorithm, but all the data as well. So you can see exactly how a node was created, can be audited by third parties. There's no mystery, nothing hidden at all.

That's one of, that seems to be one of the confusing things about open source as pertains to large language models. It's not as simple, and I think this has been lost in some of the debate as just open sourcing the code. It's open source is the shades of gray here, of course, with parameter weights, data, there's all of that spectrum. And I think that's been maybe lost from some of the debate. So it's great to hear.

Yeah, there's actually very little code, very little actual traditional lines of software. Certainly at the inference level, there's very little. It's a remarkably tiny number of lines of code. So it's just these giant, you know, weights files and you know, what your hyperparameter, you know, numbers are, and it's basically a giant comma Separated value file. So I always find it amusing to contemplate that perhaps our digital God will be a CSV file. You know, like, and, and then you make a new CSV file that has maybe more weights and the weights are better, and then you pretty much delete the old one. So just the AI is evolving and deleting its prior self constantly.

So.

So we're going to worship a giant Excel file, essentially.

That's perfect, kind of. I mean, you need a very big. Yeah, it's a lot of cells, but it's really just a bunch of numbers. And then those numbers are, they're getting smaller in magnitude too, you know, going from FP16 to FB FP32 to FP16 to INT16 to INT8. And now it looks like things are trending towards, are mostly into four.

So.

They'Re not particularly large numbers.

Yeah, and back to the.

Yeah. I'm not sure if I'm answering, I'm probably not answering the question, but I mean, it's true that closed source is not that far from a time standpoint, not that far ahead of, of open source. But let's say, and if you assume open source is. You have everything six months because of the immense rate of improvement of AI, that six months is a. Is actually a massive amount of time. I mean, or at least the delta in a. An exponentially improving situation, six months will I think, feel like an eternity. So I think closed will outperform open by a meaningful amount at any given point in time.

And given the. I was just going to say given the questions we were talking about before about truth and not truth and do you think in 10 years time AI will have been a net truth improver or detractor from where we are today?

I think it will be, well speaking for, at least for Grok, I think it will be a significant truth improver. That is literally our goal is to be maximum truth seeking maximally curious, minimizing the error between. Between perceived reality and described reality and actual reality and always acknowledging the error, not being too confident about it. So I, and I think there will be a competition for truth and you will go, people will tend towards the one that they think is most accurate. So. And if there's at least one AI that is aiming for maximum accuracy, I think it pushes all of the AIs to aim for maximum accuracy. Just as with the X platform. Fka, Twitter, as soon as X pushes for maximum truth and accuracy, it forces the hand of others. The others now also have to do that, right?

The Truth, the truth arms race among the CSVs. And one of our theses, I think with ARC is that Tesla is one of the most undervalued AI companies there is because of the insane amount of data you've got on all, you know, to feed into autonomous. I don't know if there's any commentary on that because I think, you know, obviously with the rise of these LLM companies which are by and large working off essentially available data that is not proprietary, the benefits coming to the companies like Tesla that understand how to use the AI benefits, but do have these enormous proprietary data pools, seems like that's going to be the next frontier. Yeah.

I think that's accurate, an accurate description. Tesla is one of the leading AI companies in the world and with respect to real world AI, it is obviously by far and away the leader. So you know, the word large language model with phrasal large language model LLM is massively overused. But what I do see happening is somewhat of a convergence towards intelligence. You know, Tesla like really to make full self driving work, you kind of need baby AGI in the car because you need to understand reality and reality is messy and complicated and just as a side effect, the car AI has to be able to read, for example, it has to be able to read arbitrary science as a little side effect of understanding reality in every language. So you know, everything is coming down to, you know, different layers of transformers and diffusion and how you put together the transformers and the diffusion. That's right, I sort of made that somewhat niche AI joke on the X platform. Who do you think will be president in 2032? Transformers or diffusion?

Do you think that switching to full stack AI for Tesla's full self driving reduces the kind of forecast error for you in terms of when the problem will be solved? As in you famously said it's a year away for a few years now. Do you think that like actually line of sight, wherever it ends up, you'll have a better like line of sight on when you're going to cross a particular threshold of performance where it can go without human intervention.

Yeah, I mean the car is already incredibly good at driving itself. So now especially if you say like okay, drive in California, which is, you know, both generally easy driving because you don't have heavy rain and snow and that kind of thing in most parts of California. And Tesla engineering is primarily in California. So you're going to get, it's going to overfit for solving California. But if you're just driving say around Palo Alto, the probability that you'll need an intervention at this point is incredibly low. In fact, even if you're driving through San Francisco, the probability of an intervention at this point is very low. So we're really just going through a march of nines. Like how many nines of reliability do you need before somebody does not need to monitor the system?

You know, it's so interesting, GM basically shutting down crews. It wouldn't have been possible. Right, but we didn't know that. Is that how you're viewing it now?

Well, it's certainly true that transformers are transformational.

Yes.

So, you know, pretty much everyone's using transformers and it's really just how many transformers? What kind of transformers are you using? Auto aggressive transformers, Gutierrez. Very memory bandwidth intensive. You know, there's. But I think you really. I'm not sure you can really do AI without transformers in a meaningful way. Diffusion is also very important. So I mean, it just really looks like AGI is some combination of transformers and diffusion.

That's interesting. Does diffusion help Elon, in the things that transformers are not necessarily that good at yet, like planning and remembering rather than sort of a current state?

I think transformers are important for all aspects of AI. Yeah, I mean, computers are very good at remembering things. Your phone remembers the video that you took down to the last pixel, whereas most humans cannot remember who they met last week. So memory is the easiest. I mean, we've already outsourced memory to computers in that, like you say, how much of human knowledge is in digital form versus in and contained in human neurons. It's overwhelmingly in silicon rather than biological neurons. Right. So I mean, and there, there's. I mean, I think it's always, you know, it's good to think of.

Like.

Try to consider what are the most fundamental ratios. This is like in physics, you always think about looking at fundamental ratios and some of the, you know, one of the fundamental ratios is the ratio of digital to biological compute. Just, you know, just in RAW compute and how much. And the ratio of digital memory to biological memory. So, you know, at what point it like, I think at this point, I think we're probably over 99% of all memory is digital as opposed to biological. So you've got over, I think over 100 digital to biological ratio in memory at this point, trending towards a thousand, you know, in many orders of magnitude. Beyond that, then. Then you've got the digital to biological compute ratio. And since the number of humans is more or less constant, I'm worried about humans declining quite significantly in the years to come. Because the birth rate is so low. So human computers, more or less, it looks like a flat line versus time, whereas the digital compute is exponential.

Right.

And you know, at some point it's that. That will also be above 100, meaning more than 99% of all compute will be digital instead of biological.

Right.

And if we're not there already, we will be within a year or two.

Although it is amazing how much data biology can store, I reckon. I believe it's estimated that a gram of dried DNA can store 4 and 500 exabytes. So maybe there's still things to learn from biology. But yes. Incredible, the pace of change.

Yeah. I mean, if you encode memories in DNA like a ticker tape, then the DNA memory potential is immense. But I'm not sure that's actually what's happening or if it is, it's. I don't know. Not sure why my memory is so terrible.

But actually, I mean, your memory in terms of the raw details, like you have an amazing search function. Right. For humans, it's kind of like. Yeah, well, for any. I mean, I would argue kind of in some ways, like the transformer architecture enables effective search of a very, very broad amount of data. Or it's like data compression. It seems like the. Yes. We have. There's many more, like data bytes encoded or gigabytes encoded in disk. But actually being able to effectively search that space is pretty still primitive with compute at least at large scale relative to humans ability to access information.

No.

I mean, there's still some things that humans do better than computers. I mean, if you say like computers have not yet discovered any fundamental physics, but humans have discovered a lot of fundamental physics. The computers have not yet invented a useful technology. Humans have invented many useful technologies. So, you know.

Do you ascribe to the notion that essentially training AI systems on language is not enough and that you have to have some kind of embodied source of data, like through an optimus robot to actually get kind of the raw learning in that AI system would need to understand fundamental physics.

Yeah. So that's what I think is coming. I think. I think what's coming is AI understanding fundamental physics and AI inventing new technologies.

It definitely feels like rather, even though some things now are apparently pass the Turing test, most importantly, passing the Moneypenny test, as in, can an AI actually pay bills, do expense reports to the extent James Bond did those, and actually pay parking fines and things like that.

That would actually be a very good.

Use of AI that I think would have a lot of users immediately and maybe the Billy Connolly test of can an AI actually be consistently funny? Those feel like they're a long way out.

Grok is pretty funny.

That's fair.

I think one of our goals for Grok is to be the funniest AI. So, I mean, if you ask Grok to provide a vulgar roast, it's really good.

And speaking of Grok to not let down our future AI overlords. The final question on the GROK summary that Kathy touched on was the future of Bitcoin. Not sure if we want to touch on that or not, or whether that's out of scope.