Dylan Patel breaks down why AI models are releasing faster than ever. He points to three drivers: scaling up both pre-training and reinforcement learning, the investment boom after ChatGPT, and the lag time between ordering chips and getting them into production.
Chapters
I think it's a product of a couple of things. One is that there's now this two-paradigm regime where previously you did pre-training scaling and reinforcement learning scaling, and now we're in a mix of the two, basically. And so I think that gives you more opportunities to update models because it means that you can make advancements along multiple frontiers. And that means that you end up shipping more frequently. I think it's also a reflection of the fact that this is now 2-ish years after ChatGPT, 2.5 years after ChatGPT. And so the post-ChatGPT investment cycle is finally hitting where compute availability is increasing and all of this. And so it means that you should expect actually the pace of progress to be— because there's lead times in commissioning chips, basically.