Sam Altman says GPT-4 releases will be paced by safety, not hype. PaLM scores 58% on benchmarks versus a 12-year-old's 60%, leaving room for improvement but not the leap to AGI some expect.
Questions this video answers
January 20, 2023.
540 billion parameters.
PaLM ~58% vs average 12-year-old ~60% on the cited task set.
"It'll come out at some point when we are, like, confident that we can do it safely and responsibly."
"We are going to release technology much more slowly than people would like. We're going to sit on it for much longer than people would like."
Chapters
You may have heard rumors that ChatGPT4 is going to be released imminently, say in January, and that it's going to dwarf ChatGPT3's ability, just like you can see in this graph. Well, I'm going to let Sam Altman correct the record here on both fronts. And then at the end of the video, I'm going to discuss what I think ChatGPT4 will be capable of. Here's Sam Altman on the timing of the release of ChatGPT4.
Can you comment on whether GPT4 is coming out in the first quarter, first half of the year?
It'll come out at some point when we are, like, confident that we can do it safely and responsibly. I think in general, we are going to release technology much more slowly than people would like. We're going to sit on it for much longer than people would like.
Then referring to ChatGPT4, he talked about whether it will be an exponential increase in terms of its abilities or more of an incremental upgrade.
Given the magnitude, magnitude of the economic impact we expect here, more gradual is better. And so putting out a very weak and imperfect system like ChatGPT and then making it a little better this year, a little better later this year, a little better next year, that seems much better than the alternative.
Then Sam directly addressed the hype train that had been generated by graphics such as this one.
I saw a visual, and I don't know if it was accurate, but it showed GPT 3.5 versus, I guess, what GPT 4 is expected.
I saw that thing on Twitter.
Did you. Was that completely.
No.
Okay. Because that was a little bit scary.
The. The GPT4 rumor mill is like, a ridiculous thing. I don't know where it all comes from. I don't know why people don't have, like, better things to speculate on. I get a little bit of it. Like, it's sort of fun, but that it's been going for like, six months at this volume. People are begging to be disappointed. And they will be like, it's, you know, people are gonna, like, the hype is just like, we don't have an actual AGI. And I think that's sort of what is expected of us. And, you know, yeah, we're going to.
Disappoint those people in a moment. You're going to see Sam respond to a question about ChatGPT putting Google out of business. And he made a fantastic point, which I'm going to go into more detail on after he gives his remarks.
Yeah, I think whenever someone like talks about a technology being the end of some other giant company, it's usually wrong. Like, I think people forget they get to make a counter move here and they're like, pretty smart, pretty competent.
The countermove that Sam is referring to may well be Palm, which is a 540 billion parameter transformer model. And you can see from the graphic the improvements that have been made by increasing the number of parameters. But I've done some research on Palm. See this graph. Notice the line crossing, the performance of the average human. It admits when you go into detail about the tasks that Palm can achieve, that we're still talking about solving 60% of problems that 9 to 12 year olds can solve. And that is incredible. So the average 12 year old can solve 60% of them and Palm can solve 58% of them. That isn't exactly AGI. Not yet at least. So if ChatGPT4 is anywhere in that kind of ballpark, we're going to notice the difference. We're going to see the improvement. Just as that graph increased quite dramatically by increasing the number of parameters, maybe it will be able to explain jokes, for example, understand books a bit better. But we're definitely not talking about AGI, not yet. I am certain that the initial buzz around ChatGPT4 is going to be amazing. I'm going to let Sam address the difference between the initial buzz, the impressive achievements it can do, and, and the robustness of the entire model.
One of the sort of strange things about these technologies is they are impressive but not robust. And so you use them in a first demo, you kind of have this like, very impressive, like, wow, this is like incredible and ready to go. You use them 100 times, you see the weaknesses. And so I think people can get a much sort of a false impression of how good they are. However, that's all going to get better. The critics who point these problems out and say, well, this is why it's like, you know, all like a, like, you know, fake news or whatever are equally wrong.
Thank you for watching and do subscribe for more such content.
