AI News: AGI Is Closer Than You Think! Opus BEATS GPT-4o World Spirals into Doomerism, GeminiLEAKS

So with an actually crazy week in AI, here are some of the stories that you maybe did miss. Now one of the stories I actually did want to talk about was this. So someone posted on Twitter the LLM ranking. And I think the LLM ranking is probably one of the most discussed things because it kind of shows us what company is at the frontier in terms of AI capabilities. And with these large language models or now, you know, large multimodal models, I guess we can see where they are. But this was really interesting because if you don't know, this is the ranking on the mmlu. But the problem is that the MMLU is essentially a part of how these large language models and AI systems are graded. And the thing is, is that the MMLU has issues. Now on the LLM ranking, the MMLU calculated by an actually independent third party Stanford calculates the helm MMLU that uses simple standardized prompt, resulting in fair reproducible comparison of models. And essentially these numbers are uniformly below what the various companies have reported in the past. And I think it's important to actually have third parties do this because it's like saying I'm going to investigate myself and then I'm going to give you guys the results on what that is. Of course I'm always going to say that the results that I've got are better than everyone else's because it simply makes sense to do that. And I think what's important to see here is that Claude Opus actually inches out GPT4 and even their most recent model, not by a large margin, it's literally just a percent of a percent. If you even look at it like this is 0.842, this is 0.846. So it's not even a huge, huge deal. But I think it is important because it shows us that what we're seeing here is that what companies are reporting might not match up to what independent investigations are seeing. Now of course some people are going to say that MMRU is just only one benchmark. You've got, you know, Math, you've got GSM8K, you've got coding, you've got all these other crazy things that you can benchmark an AI on. And of course people are going to say that why not just look at the arena? Because that is where people actually get to interact with an AI system. And stuff like this doesn't matter. Of course the arena is an area where you can actually get data on whatever you think is relevant, depending on whichever company is going to be the best at the time for the actual user on a day to day basis in these blind tests. But I think it's important that other companies do this a lot more because the MMOU already there's like 3% of the questions that just don't make sense. So that's a big issue because we know that 3% can be the difference between a model being classed as state of the art and a model not being classed as state of the art. So I think this shows us that Claude is better than we think and that GPT4O might not be as good as we think. But at the same time I do think the arena is good. But I do know that there could be the potential for some. I wouldn't say things going wrong there, but I did see a few tweets about a few shady things going on there. I didn't follow up on that. But I think third party evaluation is always important. And I will leave a link to where you can actually view this, the Helm MMOU. And you can see right here Claude, GPT4O and then all of these other different models here. And I mean, maybe this is going to be good, you know, so that you can like look around where the models are really good at in an unbiased way. Because these, you know, independent third party researchers don't have any bias towards any of these companies unless they were like paid a lot to do that. But yeah, I mean it's pretty, pretty interesting on the kind of results that we're seeing here. Now this is something that I really did want to talk about and this is the form of doomerism. Now I know most people won't say that this is AI news, but I think this is important to cover because it covers a certain aspect of AI news that is something that the general population might not realize, but a small subset of the population who are engaged with AI might start to take this take. So this person said, for the record, I'm currently at 70%, they were all dead in 10 to 15 years from AI. I've stopped saving for retirement and I've increased my spending and the amount of long term health risks I'm taking. Now some people would say that this is just, you know, being hyper, just, you know, being just sarcasm. She's being like extreme, just trying to get reaction out of people. And it certainly did. But I do know that like the sentiment that I've seen change is one that has changed over the last year and a half. And as I've seen comments on my channel, as I've seen people in the, you know, comments on Reddit, I've seen people like stop caring about a lot of stuff. Like I've seen people, you know, like the mentality has changed because a lot of people are starting to realize that, look, okay, if AGI or artificial superintelligence is going to be here, I'm meaningless. I'm just a cork in the wheel. It's going to be smarter than me, it's going to be faster than me. There's going to be companies that are just going to be completely run by AGIs. You know, I've got no way to work, you know, what on earth am I going to do? And a lot of people are saying that, you know, I stopped saving for retirement because at the end of it we're all going to be dead anyways. Now I. That is what you call a doomerist defeatist mindset. And I would urge any of you who are watching the channel to not at all take into any consideration what this tweet is saying. Because if this is something that is, you know, perpetuated, you know, like consistently along social media and there's some kind of movement that's going on, I truly do believe that this is the wrong take. Because on the other side, you do have this post abundance world. And whilst yes, you can fluctuate between dystopia and, and abundance, I think it's important to put yourself in the best position right now. If you're in the AI community and if you're someone that's actively paying attention, you have to understand that 90% of the world isn't really. And when I say really, I mean really paying attention to the day by day updates. Look at the videos on this channel. It gets between sometimes 15,000 to 20,000, 30,000 views on a video. What that means is that if that's the amount of people, and I'm not saying that I'm the barometer of how many people are paying attention, but if you look at how many people are actively using AI, trying to understand AI and how it's going to impact their industry AI, it's not as much people as you think. Which means that if you do take action now to sort of, you know, protect your future and actually be someone who benefits from AI, the future isn't going to be bleak. I mean, of course you've got this AI safety debate and I've said to people time and time and again, there's nothing we can do unless you are actively an AI, you know, safety researcher. The most you can really do is sign a few petitions, retweet a few things. But there's nothing you can really do to hope these companies are doing their best they can do in AI safety. It's up to the people who are actively in these companies and the billionaires who are investing in these companies to ensure that some of the money that they're investing, some of the compute they're using is going towards AI safety. There's literally nothing you can do, so why worry about it? Because worst thing that could happen is that post abundance does come here. And of course in a few years you don't have money, you haven't saved for retirement, you've got some chronic health issue that just makes your life worse. So even now that the economy shifted, you can't adapt. Just trust me, guys, just don't take this through thing right here because whilst, yes, I understand why people are doing this, it is just so defeatist. And I think that right now is probably the biggest opportunity for anyone in the space. So as long as you're in this space, you can, you know, join communities. I have my own private community. You know, it's a link in the description. It's just a patreon where we like literally focus on how to better ourselves for the post AGI world. And honestly, it just makes me a lot more excited for the future because I'm seeing all of the capabilities and all of the different nooks and crannies where there's just vast opportunities for the taking because entire industries are being built right now. So I know that this was just a quick rant, but it's important to just not take this into account because it's just way too extremely negative.

My naive conception when we started is that we would like get to a moment where we didn't have AGI and then we did and it would be a real discontinuity. And I still think there's some chance of a real discontinuity, but on the whole I think it's going to look much more like a continuous exponential curve where what matters is the pace of progress year over year over year. And you and I will probably not agree on the month or even the year that we're like, okay, now that's AGI. We can come up with other tests that we will agree with, but even that is harder than it sounds. And you know, GPT4 is definitely not over a threshold that I think almost anyone would call an AGI. And I don't expect our next big model to be either. But I can imagine that we're like only maybe one or two or some small number of ideas away and a little bit more scale from something. We're like, this is now kind of different.

So this was an interview clip from Sam Altman's recent interview where he basically talks about how he thinks with regards to AGI. There's only around two or three major breakthroughs left. And I mean, like he's previously stated before, I think it's going to be interesting to see how that scale changes year over year, because every year we're going to get increasing, basically increasingly more capable systems and increasingly more capable frontier models from all of the competing companies. And I think it's going to be hard to determine at that point where we do have AGI. And I think recently with GPT4O, someone recently said this is essentially AGI because, you know, you can talk to it, you know, it can laugh, it can speak, it can giggle, it can change its intonations, which I just, I honestly found just completely crazy. And the worst thing is, is that literally two weeks before that where I spoke about how, you know, these, these, these models, like the way they speak is just so monotone and even the way they speak is just so monotone that you could always, always know that it's a, you know, always know that it's basically an AI system. I basically said that, you know, like the way how I sometimes I speed up, speed up, speed up and then sometimes I slow down to keep people interested. I said that AI systems wouldn't be able to do that. And then literally two weeks later we got the four way demo. And that was something that just upended one of my predictions, which goes to show that this thing is moving pretty, pretty quickly. But the point is that every single year there's going to be increasing capabilities. And I think it's important to remember that when Sam Altman talks about AGI, he doesn't talk about AGI in a traditional sense of an AI being able to embody robot and clean the dishes. He talks about an AI that's literally able to do autonomous research in an AI field that's rather effective. That was his personal definition. So when he's talking about AGI, that will be his personal definition. So I think maybe he's talking in terms of that time frame. There might be two or three more breakthroughs before they get to that area, but I'm excited to see what they have with the next frontier of models. So this is where something actually missed me and I was like, okay, this is pretty crazy. So it says GPT4O was a whole team of it. Huge credit to especially Fuller because he's the head of Omni. But what he said that was actually rather interesting was that he said, you know, this took course over the past 18 months, which is pretty, pretty incredible, guys, because what that means is that they were working on this for the, for 18 months, which means they've been working on this since 2022, which is pretty, pretty crazy. Okay, like that is pretty, pretty insane. And I think that shows us just how big of a lead OpenAI does have. Because if they're able to demo something that's this crazy, I mean, you can just imagine the kind of projects and things that they're working on now. Whilst, yes, there will be some failures, there will be some successes, I think it's important to realize just how far ahead they are because I mean, 18 months, guys, like literally, if we look at the pace of AI innovation over 18 months, like, I mean, I'm guessing that this was obviously very, very hard to do because, you know, if you look at some of the other capabilities that were, you know, shown in the paper that I actually did a video on, where I spoke about, you know, GPT4O's secret capabilities, I think it's important to realize that like this here is just absolutely insane, that this has been going on for the past 18 months. And, and the fact that they're only iteratively releasing some of this stuff goes to show that like I said before when I spoke about LLMs and stuff and AI just slowing down, essentially what I meant was things are slowing down externally, but not internally, meaning that these teams are having their internal research projects that just keep going. Because the problem is that, like I said before, OpenAI, they no longer publish their research, they no longer come up with demos that are, I guess you could say, no longer profitable in terms of business standpoint, because now they're a private company. Anytime they have research that could advance the company in a proprietary way, which they need to do because they're competing with Google and of course Anthropic for customers and market share, they're not going to publish that. They're just going to keep that to themselves and they're going to extend the lead, which they already have, which is, I think it's important to note that OpenAI just so far ahead and 18 months just shows you where we could be in the next 18 months. So I think it's important to just remember this, because I didn't see this And I thought that that was pretty crazy. Also there was a GPT4 generated image. And this is photorealism if I see it. I remember previously we looked at Mid Journey and how Snapchat raw photos were really, really photorealistic. But this is a GPT4O generated image. So much to explore with GPT4O's image capabilities alone. Team is working hard to bring those to the world. So yeah, like I spoke about ages ago, I said that, you know that the new model, whichever one it was going to be, we're going to get an even more updated version. Because if you haven't seen already, even if you use image generation now, compared to two months ago, there's a new version of Dall E. I don't know if it's Dall E3 fine tuned or whatever it is, but it is better than what we had before. It just seems more consistent in what it's able to do. But this is GPT4O's image generation model, which isn't in the model yet. But from this we can see something that looks remarkably photorealistic in terms of the drawings and the text, which I think is really, really important. I mean, maybe this image is cherry picked, but I think that this goes to show. Like I said before, GPT4O is probably one of the most important models because I think once different markets start to realize what they can use 404, then that's when OpenAI is going to be expanded to the mass market. Because I know that I have a few people, if I were to show them the 4.0 demo, they'd completely freak out. And if I were to give some people the 4o demo in real life, they would certainly be like, I'm going to use this every single day. It's pretty, pretty crazy. It's pretty, pretty crazy. I mean, I'm sure kids are gonna love it. I'm sure the elderly are gonna love it. I mean, you know, the amount of industries, private tutoring, just lessons. I mean there's, there's a whole range of different things that you could use this for. And I can't imagine what the API is going to be like as well. So there was also another piece of information. So as you know, I'm not gonna talk about people leaving OpenAI, but this person says, I'll be leaving OpenAI after three and a half years. I'll be joining my good friend Andy Barry, Boston Dynamics and Pete Florence and andy Zhang at DeepMind on a brand new initiative. And their initiative is that was what they think will be fully necessary to fully realize AGI in the world. And I'm excited to share more about it soon. So basically he's stating that what they want to do is they want to fully realize AGI in the real world. And the humanoid hub is a very good account for, you know, anything humanoid robotics. So I recommend giving this kind of follow and basically, you know, you can see that they have a very, very, very stacked resume in terms of what they've done. So something that I'm going to speak about in another video is how like, you know, all of these researchers, a lot of them are leaving like big companies to go start their own things. And I think it's kind of interesting because we get this like dynamic landscape where new companies are being formed, big companies are shrinking and changing, and the landscape is just, you know, so, so different. But I'm guessing they might be targeting embodied AGI. Maybe they figured out something, maybe they're going to combine their talents. But whatever it is, I'm excited for it. I'm guessing maybe they're going to be raising money, I have no idea. But I think it's important to, you know, keep these updates here because they're going to be sharing more soon. So if you want to give his account a follow, you know, I'll leave a link in the description or his, you know, Twitter account is just eom. I'm not sure what character that is, but. And of course with, you know, talking about someone leaving OpenAI to go build robots. If you didn't watch the previous video, OpenAI is, I wouldn't say it's in tatters, because it's not. But it's a bit strange because people are leaving and the people that are leaving, getting fired are all, all to do with AI safety. We can see that, Jan, like Ilya Sutskova, concerned safety researcher, leaves. And I don't think it's good because it's not like they were just standard safety research. These guys were working on Super Alignment. And I spoke about that in the previous video, which means that Super Alignment is either solved or Super Alignment just has a big issue that, you know, we can't really talk about. I mean, either way, like, I really don't know what's going on. I did make a 40 minute video so you guys can watch that. But the point is, is that this doesn't look too good in terms of AI safety because the bottom line of that video is basically like Super Alignment team is gone. Not entirely, but like half of the initial team is gone. And with that, half of the initial team is gone. They're like the key founding members. Like Ilya Setskova is regarded as one of the smartest people in terms of AI. Jan, like, was also the lead of the team and he's now gone. And then two other people on that team were fired. Okay, so they were fired for apparently, you know, leaking information. So the point is that OpenAI, since haven't, you know, on the Super Alignment block, they haven't posted any updates since then. And since he resigned and Ilya resigned, it brings us to the question, you know, what on earth is going on with Super Alignment? Do you guys figure it out? Did you guys figure out how to make an automated alignment researcher? Because that was essentially the goal of Super Alignment. So is this trend going to continue? But an update would be nice because for the AI safety folks who are always saying that AI safety isn't this big thing, they do have a real point now because if your safety team, like half of them, like five of them, five out of the 12, I don't know how people are there right now. And you guys haven't posted any updates. I mean, you know, they're going to be asking questions. So it's a bit. It's a bit weird. So this thread here that we can see is basically one that highlights some of Sam Altman's opinions and some of his thoughts and some of his ramblings on where he thinks generative AI and just AI in general is going. And this was actually an ask me anything he did on Reddit. So if you don't know Reddit is a social networking place, I guess you could say it's like a forum. I don't know. However you want to describe it. The point is, is that he did something where people can literally ask him any question. And these were some of the highlights from that. So he said, when is your prediction that fully generated AI film is going to outperform human efforts at the box office? And he said, I don't think this is the most important question. The most important thing that he's excited about are new kinds of entertainment that will be possible. So imagine a movie that is a little different each time that you can interact with. I believe that human creativity will remain super important and that humans will know what other humans want and care about and what other humans want to make. So what? That is an important point. Like I've said before, if you want to remain relevant, one of the key things that you have to understand is what humans want and what it means to be human. I know it sounds really, really basic, like we're human, of course we know what it means to be human. I don't think people do. I think people realize that and people forget that humans want human connection. They want the social aspect, the status aspect, all of the things what drive humans intrinsically. I think if you look into that psychology on what that is, then you can use that information to kind of like predict what industries are going to remain relevant regardless of how powerful AI gets. Because whilst yes, it is kind of hard to predict, but I think as long as you can understand what skills, like for example, you know, like if you're trying to sell something to someone, like talking to an actual person who's selling you, the product is going to be a lot more like, for example, on YouTube. I know that, you know, I always want to be in the videos and actually making the videos because the human to human connection is a lot more valuable than you just hitting hearing like an AI generated script and an AI generated voice and an editor piecing a few things together that doesn't even care about AI. That's why I'm saying that if you can pay attention to what humans will kind of value in the near short term to long term, you can position yourself in not a bad way. Now, I think one of the biggest predictions here, and this is what I talk about a lot on the post, AGI Private Community, basically, he basically said that AI will cause the price of work that can happen in front of a computer to decrease much faster than the price of work that happens in the physical world. This is the opposite for what most people, including me, have expected. And this also have strange effects. And he still believes in this prediction. Now why is this something I'm talking about in this video? Because I don't know what, you know, what the demographic is of, you know, what your career is or where you work. But I think that is an important distinction to make because in the future, when that leads to short term, before AGI is embodied, we are going to see whatever it is being done on a computer, and this is literally anything being done on a computer, the price of that task will drop. Like that is something that you can know and that is something that you're hearing right now. So with that information, okay, you at least need to be like, okay, if that information is real, what am I going to do in my scenario and how can I act and position myself so I'm not completely screwed when this, you know, strange effects start to occur. Because stuff like this, a lot of people are only going to be like, at the last minute, they're like, hey, wait, I used to be able to make this much amount of doing this, that computer. Now I don't make this much. Like for example, it's like if you were a writer and you heard about GPT 3 and you knew that GPT 3.5 was coming out in GPT 4, you might not have pursued a career in certain areas because of these tools coming. That's the point I'm trying to make is that like you can predict kind of where things are going to go. And I think it's going to be strange how the economy operates when you know, people that are doing in physical labor do have like, I guess you could say a longer career than some people who are in, you know, desk job, you know, white collar versus blue collar. It's really going to be just a strange, strange time. So it's important to understand that there. And I'm not saying, you know, drop your software engineering career and go become a plumber. I'm just saying that stuff is about to get weird and it's important to see how you can put yourself in a position where you're less automatable. Because I don't think anyone wants to be out of a job. And recently I was actually watching a few videos, this guy, where he lost his job to AI and he actually spoke about how he lost his job to AI due to graphic design. But if you watch the video, if he was someone that actually paid attention to the AI space and paid attention to certain things, I, I'm 100% sure that he would have been able to use certain AI tools, get his work done a lot faster. He would have applied to certain jobs, certain careers, maybe shifted. But he was someone that, it's not his fault. Okay? I'm not saying that this is this guy's fault at all, okay? But the point I'm trying to make is that you who are watching this video now, you're in a very privileged position because if you're passionate about AI and you like this stuff, it means that whatever career you're in, you're going to be able to see things before other people are. And it gives you a strategic advantage. I can guarantee you if this guy was keeping up to date with all the generative AI tools and all the updates on whatever channel it may be, he probably wouldn't have been in this position. And that's not like, you know, haha, you didn't know it's just something that I'm like, guys, if you're here, okay, good, okay, that's good for you. But other people are just not paying attention and they don't know what they don't know. So the point is that it's important to pay attention to the industry because it's really going to impact a lot of stuff. And as long as you know what the impacts are going to be, you can position yourself best. Like, for example, he could have maybe applied to more jobs, tried to switch roles, started taking certain courses, and it's just better because he just got a short notice saying that he was made redundant. And I'm sure that a lot of people in the coming years are going to be unfortunately going through that. So it's best to start to think about some of the short term solutions and some of the long term solutions to those kind of problems now. Now Google is actually also preparing the release of a memory feature. And what's crazy is that I did see this kind of leak before the oh I O update, but there wasn't anything after this. So it was a bit weird that they didn't release this memory update. And I'm guessing that maybe this is going to be an update that's coming in the next two weeks. So hopefully when this happens, like I'm hoping that this is out in the EU as well, because I know that the EU doesn't get a lot of features currently, so I'm just waiting to see if this one is going to actually happen. So that is something I'm patiently, patiently, patiently waiting on because I know that people that live in the UK live in the eu. There are cool features, but we don't get them yet. We just get them super, super late, which is kind of frustrating, especially if your goal is trying to create actually good content. We also had improvements to data analysis in ChatGPT. It says interact with tables, charts and ad files directly from Google Drive and Microsoft OneDrive. This is something that you can kind of do right now. I did a tutorial yesterday about how you can actually do that and it says today we're rolling out enhancements, data analysis, upload latest file versions directly from Drive and Microsoft OneDrive. Interactive tables and a new expandable view. You can see right here how it's changing. This looks really, really, really nice. Customize and download chance for presentations and documents and like, stuff like this is like, you know, I'm sure that this is going to affect a lot of jobs. It's going to make you a lot more efficient. So I would say, you know, try and learn this take on, take it all on board and if this is part of your career, part of your work, just be a lot more efficient. Like try to get twice the work done in as much less time as you can because you know when they're looking around for the next layoffs, you want to be like a super employee that like, you know, is one that doesn't get layoffs. In fact, you know, some of that moves up in the company. So with this stuff kind of here, I'm guaranteeing that most people are going to use it for work. But it's a small update. But, you know, it's deployment, yada, yada, yada. But yeah, if you guys enjoyed today's video, don't forget to leave a like, subscribe all the good stuff and I'll see you guys in the next one.