New STUNNING Research Reveals AI In 2030...

Epoch AI is a research initiative focused on investigating trends in machine learning and forecasting the development of artificial intelligence. Now they've recently released a report on the future of AI, and some of their predictions are probably the most accurate. And it's rather surprising considering what most people are saying. So essentially in this video I'll dive into their findings and show you how why the AI hype is truly far from over. And I'll show you the actual conservative estimates that show we're in for a pretty wild ride over the next six years up until at least 2030. So one of the craziest things that I saw from the report, and I've just, you know, picked up a few things because the entire report was, I think around 60 or so pages, I'm not exactly sure how many pages, but it was rather extensive incentives. So I decided to just show you guys a few snippets from that report. Now, one of the things that was there was that they talk about how the potential for sufficiently large economic returns that could emerge from scaling beyond GPT4 to a GPT6 equivalent model, coupled with substantial algorithmic improvements and post training improvements. It says, okay, and this is the bit that I've highlighted that this evidence might manifest as newer models like GPT5 generating over $20 billion in revenue within their first year of release. Now that is absolutely incredible, but I think later on in the article they talk about how the entire economic output is around $60 trillion per year. And they're basically stating that, look, if an AI model is able to automate small portion of that, it being able to get $20 billion of economic value is not that hard when you actually think about the amount of value the economy produces. So what you can see here is that they're talking about significant advancement in AI functionality, allowing for models to seamlessly integrate into existing workflows, manipulate browser windows or virtual machines, and operate independently in the background. So basically what they're talking about here is, is that allowing models to seamlessly integrate into existing workflows, manipulate browser windows or virtual machines, and operate independently in the background, what they're referring to here is agentic capability. So operating independently is where we have these systems that don't longer require humans as much. Now, currently, if we want AI systems to perform well at nearly any task, what we have to do is, is we have to prompt that AI model. So we open up a chat, we say, hey, can you do this? Can you do that? And then of course we have to, you know, refine the prompt and get the AI system to do a lot of different things. Now in the future, these things are going to be operating independently in the background, which means that there's going to be quite a lot more scale. I didn't mean to draw a box there, but this is going to be one of the biggest things. Now the thing about this is that if you saw another video that I spoke about, you know, the trends in machine learning and how we're going to evolve for Future models, a GPT4 6 to GPT6 level equivalent model, coupled with of course, as they say, substantial algorithmic improvements and post training, that is going to be absolutely incredible because when I looked at another part they basically talked about GPT4 to GPT6 could be a 10,000 times x scale up, or future models by 2030 could be entirely a 20,000 times scale up. So it's going to be super intriguing to see how models scale up from GPT4 to GPT6, because there's going to be likely two giant training runs, there's going to be substantial algorithmic improvements. And considering the fact that GPT5 is likely to be released later this year or early next year, it's going to be interesting to see exactly what those improvements are with every iterative cycle. So this being $20 billion of economic revenue or economic value is going to be absolutely incredible. But the point is that it should show you what is going to come in the Future. And if GPT5 could generate $20 billion in revenue within its first year of release, I'm wondering what future models are going to be able to do at that time. Now you can see right here, like I said before, this is where we talk about the $60 trillion economy. And it says here that the potential payoff for AI that can automate a substantial portion of economic tasks is enormous. It's plausible that an economy would invest trillions of dollars, basically stating that of course, you know, it's plausible that the economy would invest trillions of dollars building up that stock of compute related capital, including data centers, semiconductor fabrication plants and lithography machines. And it says, of course, here the part I highlighted. To understand the scale of this potential investment, consider that global labor compensation is approximately $60 trillion per year. Basically stating that this is how much we pay people to do tasks that move the economy. And even without factoring accelerated economic growth from AI automation, if it becomes feasible to develop AI capable of effectively substituting for human labor, investing trillions of dollars to capture even a fraction of this $60 trillion flow, would be economically justified, basically stating that, look, like I said before, $60 trillion, okay, is a lot of money. And if we get even a slice of that, like even if you get $1 trillion, like think about these companies and what they're trying to do. This is why a lot of people can't understand why these companies are spending millions and millions of dollars on AI. Like there was an article recently where it's talking about, okay, you know, AI, they're spending millions and millions of dollars on these training runs on these researchers. But Wall street just can't understand the long term picture because Wall street, they're thinking about cash flow, thinking about all these metrics, stock valuations. But I'm going to show you guys this article right now. You can see here, it says, has the AI bubble burst? Wall street wonders if artificial intelligence will ever make money. And you can see that there has been one question in the minds of Wall street tech's earnings season. When will anyone start actually making money from artificial intelligence? And in the 18 months that's kicked off the arms race, they've promised that this is poised to revolutionize every single industry. But like I said before, of course they're spending billions of dollars on data centers and semiconductors needed to run the AI models. But like I said, these guys on Wall street, they're not thinking about completely 2030 when things start to get a little bit more crazy. I like to think of it like this. Where AI right now, yes, it's having its chatgpt moment, but once a lot more capabilities are on the line, these AI companies are going to become so much more valuable. The, the money that they're going to make is just going to go up and up and up and up like that. I think it's really going to be like that. Of course it's probably going to be a level off, but we're definitely still on that sigmoid curve where there's going to be huge gains towards the end. And I think that many of these companies just can't seem to fathom that. Look, in the future they're predicting that even this company, this research organization, they are predicting that I think 100% of tasks get automated by something like 2043. And I mean, even you have to think about it, okay, if the global economy is going to be outputting $60 trillion per year, I'm not sure how much, okay, you know, global label compensation is going to grow or decrease by. But you have to think about it. These top companies, they're going to be getting A lot of that value. Now none of these companies make trillions of dollars per year. But you could argue in the future that with AI and automation that this is going to be something for the first time it's probably going to happen. So I do think that those companies, their valuations are going to be know, astronomical in the future. This isn't like a stock, you know, point video. But here the researchers are saying that, look, investing trillions of dollars to capture even a fraction of the flow is economically justified, which is what a lot of people can't seem to think. Which means that like when you actually think about, you know, the future, 2030, 2040, what the years are going to look like, it truly is, you know, something that's going to blow my mind now. So now this is where we talk about 10x output. So it says here that standard economic models predict that if AI automation reaches a point where it replaces almost or all human labor, economic growth could accelerate by tenfold or more over just a few decades. This accelerated growth could increase economic output by several orders of magnitude. And given this potential, achieving complete or near complete automation earlier could be worth a substantial portion of global output. And recognizing this immense value, investors may redirect significant portions of their capital from traditional sectors into AI development and essential infrastructure, such as the energy production, the distribution and the semiconductor fabrication plants and data centers. And it says that this potential for unprecedented economic growth could drive trillions of dollars in investment in AI development. Now if you remember previously earlier this year where a certain someone, a certain Sam Altman, was talking about how much money he is going to be spending on AI and some of the future valuations that he was talking about, you can see here it says Sam Altman has a mind boggling price tag, according to the Wall street journal, somewhere between 5 and 7 trillion. And you can see here that pretty much everyone is clowning him. It says such numbers are preposterous. The fact that they're being talked about with anything about approaching a straight face is indicative of a degree to which the broader AI discourse has become unmoored from reality. However, we're seeing that these guys that do research and they try to truly understand with conservative outputs where the AI growth is actually going to be. And remember, this isn't some lab that's doing like a clickbait article. They are literally just publishing their research for anyone to view and they're just tweeting it out. It's not like this hypey hypey thing. But what we're seeing here Is that they're also stating that, you know, trillions of dollars being invested in here is not that crazy. But you can see here, but you can see here that because Sam Altman has been seeking trillions of dollars to reshape the business and chips of AI, many people are stating that this is insane. This is incredible. Look at the guys. I mean, look at the research guys. Like, this is something that they're saying that, look, okay, when you start to see, okay, how much AI, you know is going to be automating the economy and how much economic value AI is going to eat up, like putting trillions of dollars into that doesn't seem that crazy when you factorize that. It says, recognizing this immense value, investors may redirect significant portions of their capital into traditional sectors of AI development. So when Sam Altman was talking about trillions of dollars, he wasn't just completely going off the rails in terms of AI hype. This is something that certain research organizations are, are already starting to talk about. Now this is where we talk about some of the compute for larger models. You can see here that it says frontier training runs by 2030 are projected to be 5,000 times larger than llama 3.1. And it says, however, we don't expect power demand to scale as much. And this is for several different reasons, but a 5,000 times larger training run than llama 3.1 in the next six years is. It seems crazy, but I mean, you can just imagine, okay, and this is actually a conservative estimate because they do have values that are on the high end. But in this writing, they've actually put the conservative estimate because like I said before, it's not like this hypey journalistic article. It's actually just people that are doing research based on what they see, based on the data that they're looking at. So, I mean, when you actually take that into account, it seems that the future is going to be absolutely incredible. Now you can see right here, it says that they also expect training runs to be longer. Okay? And it says since 2010, the lengths of training runs has increased by 20% per year among notable models. Since we expect power constraints to become more pressing, training run durations could lengthen to spread out energy needs over time. Of course they're talking about many different things, but basically they're stating that training runs could take around a year or around a few hundred days. So they do state that, look, it's going to be unlikely that training runs will exceed a year, as labs will wish to adopt better algorithms and Training techniques on the order of timescale, which these will provide substantial performance gain. So basically saying that, look, no point training it for an entire year, because by the time you finish training it, there's going to be algorithmic improvements that you're going to need to go ahead and retrain the model completely once again. So it's going to be completely intriguing to see what these future models are and how they're going to be trained. But you can see right here that llama 3.1, which trained over 72 days, just over three months. But GPT4 was trained over 100 days, which is actually three months. No, this one is two months and this one is actually three months. But the point is, is that it's going to be interesting to see how these training techniques differ. Now, one thing that we are seeing is that companies are starting to absolutely buy into this. We can see that Meta bought the rights to a power output of 350 megawatt solar farm in Missouri and a 300 megawatt solar farm in Arizona. And Amazon owns a data center campus in Pennsylvania with a contract for 960megawatts for the adjoining 2.5 gigawatt nuclear plant. So you can see that Amazon is really, really pushing the envelope when it comes to the amount of power that they're going to need, because they are really going all in on this stuff. And you can see here that it says that the primary motivation behind these deals is to save on grid connection costs and guarantee a reliable energy supply. In the coming years, data centers might allow for unprecedentedly large training runs to take place. And a 960 megawatt data center would be over 35 times more power than the 27 megawatts required for today's training runs. But we can see that this stuff is already happening behind, behind the scenes. These companies are ramping up for 35 times more power needed than current AI models. And you can see here that it says that some companies are investigating options for gigawatt scale data centers. And basically they're stating that we're going to have gigawatt scale data centers that actually seem feasible by 2030. And it says that this assessment is supported by industry leaders and corroborated by recent media reports. This is. The CEO of NextEra, the largest utility company in the United States, recently stated that while finding a site for a 5 gigawatt AI data center would be challenging, locations capable of supporting a 1 GW facility do exist within the country. So they're basically stating that, look, whilst 5 gigawatt AI data centers are pretty insane. A 1 gigawatt data center, the facilities currently do exist within the country and of course, if you do remember that, you know, OpenAI and Microsoft have the 2028 Stargate that will require several gigawatts of power with an expansion up to 5 gigawatts by 2030. Now that's a huge feat and that's going to be really difficult to accomplish. But I mean, this is, you know, a race. There's going to be lots and lots of money invested in this. And you have to understand that they're talking about capturing $60 trillion of economic value. So I think a few billion dollars into some data centers and is something that they're not going to scoff at. So now I can see here that this is where they talk about the future training runs. They say that training runs, we will presume that they will not likely exceed six months. And we will assume that training runs will last around two to nine months on the higher end if progress in hardware and software stores and on the lower end if progress accelerates relative today. So it could be two months or it could be nine months. So this is pretty crazy because it seems that it's still going to get pretty longer. And then of course, this is where we get into some incredible statistics. It says since the chinchilla scaling laws suggest that one ought to scale up data set size and model size proportionately, scaling up training data by a factor of 30 times by using the entirety of the Index Web would enable labs to train models with 30 times more data and 30 times more parameters, resulting in 900 times as much compute. Okay, if models are trained to be chinchilla optimal, which is absolutely insane. Okay. And people have been saying that we've exhausted all our data, but we haven't actually done that yet. So can you imagine a model being trained with 30 times more data, 30 times more parameters and 900 times more compute? I mean, it's going to be truly incredible with as to how these systems are going to be working. Now, like I said before, many people have spoken about this data war, which is a thing where people are thinking that, okay, we're were going to run out of data. But you can see right here that they say that if the recent trend of four times a year scaling continues, we would run into this data war for text data in about five years. So basically where we completely run out of data. But it also does state here that however, data from other modalities and synthetic data generation might help Mitigate this constraint, we will argue that the multimodal data will result in effective data stocks and of about 450 trillion to 23 quadrillion tokens, allowing for impressive training runs. And of course, synthetic data might enable scaling much beyond this if AI labs spend a significant fraction of their compute budgets on data generation. Now, the synthetic data conversation is one that's rather interesting because there was this recent report and basically there was this paper that actually addresses an issue with synthetic data. Now, basically with synthetic data, there was this issue called model collapse. And I need to show you guys what this is. It's not really a real issue, but this is something that people always bring up and I'm going to show you guys. I know this isn't the best image that you're going to see, not from the best article either, but essentially what they're stating is that you have real images, then those real images produce fake images. Those fake images are used to train another model that produces even more fake images. And by the fourth iteration you have a system that collapses, essentially. Basically they're saying that this lack of human data is going to limit AI progress. However, what these studies show is that models that are just completely just trained on their own data again and again and again, they weren't really filtering with humans and stuff like that. This is why I'm talking about this paper, because this paper came out recently and basically they've had a new method, and this method is called reinforcement, to improve the quality of AI generated data. And this involves having a system which could be a human or an AI, which, which checks the generated data and then only selects the best examples for training future models. And basically they provide mathematical proof that under certain conditions, using reinforced data can prevent model collapse and even lead to perfect performance in some cases. So without reinforcement, training on AI generated data would indeed lead to worse performance, which is model collapse. But with reinforcement and selecting the best AI generated data, they could prevent model collapse and sometimes even improve model performance beyond the original model. And the quality of both the data and the generator and the reinforcement system are important for good results. So whilst many people are thinking that synthetic data is simply this hole that is never going to be filled, there is a lot of research that is out there that suggests that this isn't the truth. Now what we also do have is this graph that shows us the largest feasible training runs in given the different constraints. Many people talk about AI hype and they talk about how AI is just complete, overly hyped in terms of the future progress. But like I said before, these are people who've researched the stuff and they said that this is what the largest feasible training runs are given the actual different constraints. So we have different constraints here. We've got the power constraints, which are the energy supplies, of course, we've got the chip production capacity, which is inventory Nvidia being able to even produce enough chips. Recently we had news that there were delays on I think the B2 hundreds and of course we've got the data scarcity and of course the latency war. Now you can see here that they state that the most binding constraints are power and chip availability. And you can see that essentially these are the ones here that are pretty crazy. But you can see that it says that data stands out as the most uncertain bottleneck with its uncertainty spanning a range of 4, 4 orders of magnitude. You can see on the graph here that data is all the way down here and it's all the way up here. So they're not sure, but you can see that by 2030 this is where they expect things to be. And I'm going to show you guys another image that basically explains everything. But basically the worst case scenario, okay, like the literal worst case scenario is that we have systems that are 10,000 times greater in terms of the scale. So this is pretty insane when you actually think about it. You can see that there are other areas where we could get to 50,000 times greater chip capacity, 80,000 times greater, a million times greater in terms of the latency. But yeah, it just shows us that, you know, by 2030 things are going to get rather incredible. And I mean, this is taking, you know, the average, you know, of all of these. And then of course you can see it's brought it down here. So it's not like the highest, the complete highest. We can see that the 2030 compute projection shows we're going to have 10,000 times more compute to train these models by 2030. Which means I'm not like there's going to be just like an explosion in terms of how these models are going to be in terms of their effect. Now the takeaway from this that you should think about is basically they're stating that by the end of the decade, so by 2030, we're going to be able to train a model that is 10,000 times larger because GPT2 to GPT4, if you remember GPT2 to GPT4, that scale was 10,000 times larger. And they're basically saying that we're going to be able to do that by the end of the decade. So if you can imagine that with all the progress that we've had just in the past three years, which has been quite a lot, but now with all the investment, now with all the money, now with all the eyes on AI, with all the major players in robotics, with all of the companies, anthropic, Google, Amazon, with all of those companies competing, the fact that we're also going to have 10,000 times more compute available, and the fact that by that time in 2030, we're going to be able to train a model that is going to be 10,000 times larger in scale, what kinds of systems are we going to have in place? I mean, it's going to be pretty crazy. But I think this should let you understand that even, even in the conservative estimates of these people that have done the research, it shows us that we're going to have a incredible time in terms of AI. So hopefully this video educated you guys a little bit in terms of how the future is going to be in terms of compute. The full thing is actually really long. You can see here that I'm scrolling down for quite some time. It's called Can AI scaling continue through 2030? And it says we investigate the scalability of AI training runs, we identify all of the stuff. But you can see right here that guys, this is something that is really, really long. I read through this entire thing. It's super, super detailed, super, super long. They've got so many different re, you know, people that have done research on this and you can see that all of the sources are cited here. You can walk through on the right hand side, click through different things sometimes or images. But if you do want to do this link will be in the description. If you guys have any comments down below, let me know what you think about this and if the future is going to be crazy and I'll see you guys in the next one.