Enrico Schaefer argues that the internet's data may soon run out for training AI models. Researchers are exploring whether AI can generate its own high-quality synthetic data to keep scaling. If AI can create training data better than human intelligence, growth could become exponential.
Chapters
We tend to think that the amount of information available on the internet is large, that it's a huge dataset. And clearly it's the largest public dataset that's ever been created. However, it is not enough data in order to continue to scale these AI models. So one fascinating thing that's being worked on right now is to have AI create the next set of public data. And there's a lot of issues with that. Can AI create Information that would be analogous to human intelligence or might even be better than human intelligence? That'd be more valuable data than human-created data. If so, if AI is able to create data, synthetic data for it to train on, you can see the exponential nature of the ability of AI to scale, then the capabilities of AI to scale to become much more intelligent than us.