GPT-5s New STUNNING Capabilities, Autonomous Software Engineer, Shocking AI Research

So with another interesting day in artificial intelligence, let's take a look at some of the most pressing stories that you probably did miss, because, boy, oh boy, there was one that I'm pretty sure you missed, and it's pretty, pretty crazy. So let's take a look at some of these. So one of the first stories I was actually rather interesting was that Anthropic did new research where they measured model persuasiveness, and they developed a new way to test how persuasive language models are and analyzed how persuasive scales across different versions of clause. Why this is pretty fascinating is because this is a big deal. If you think about the future implications. Now, you might be thinking that this is just some, I wouldn't say terrible research at all, but terrible in terms of, like, it's not hypey news. Okay, some news isn't hypey. This news isn't hypey. But that doesn't mean that it's not important. And the reason why this is important is because we have looked at LLMs and AI systems for what they're able to do to humans in terms of being able to write certain pieces of text, in terms of being able to have a decent voice to be able to fool humans with some sounding like the president, sounding like your loved one so they can empty your bank account and drain your financial assets and scam you. There is just a variety of different concerns with AI, however. AI persuasiveness is far, far one of the greatest threats that really, really will exist. But it's something that isn't talked about because it's something that is not objective, but it is quite subjective. Meaning that it's pretty hard to quantify how persuasive someone is, because it kind of depends on the person. Like, some people are persuaded easily more than others, while some of us are just a little bit more stubborn on our ideas, which can be good and can be bad. But the point is, is that if these models get to a point where they are extremely persuasive, then that means that these tools are going to be extremely powerful. Which is quite interesting because if you have a model and then, you know, you essentially use this large language model to persuade someone of a different opinion. I mean, it's. It begs the question, how far do models get before they have superhuman persuasiveness to where you could literally have someone that has point A and they are, you know, they live their entire life about point A? And then we could use an LLM to construct text before, you know, getting information about them. And we can see here that these models, you know, as they scale up in, you know, capabilities, we can see that the frontier models, you know, higher, becomes more persuasive. And we can see that. That these models are becoming more and more persuasive as these capabilities are scaled up. Which is quite concerning because it breaks the question, are there going to be models out there that could really persuade you? And I think it might not even be persuasiveness that could be necessary. I think what people need to understand is that sales is the thing. If you've ever looked into sales, you understand that certain words just trigger you and certain phrases, the way how you say certain things can get people to act. It's a whole thing. Certain pages convert more. Certain words would make you buy. Certain words or not make you buy. You might think you realize this, but trust me, guys, when you land on a web page, There are like 10 to 20 different variations of that web page that other people saw. And they're just feeding you the one that, you know, has the right words that makes you buy. People have tested this. I've been in that industry for a little bit, and I know a decent amount about that. The point is, okay, is that words do have an impact on what we do. And if that is the case, if these LLM systems are really, really persuasive, then we need to actually pay attention to this because it could be one of those things that kind of slides under the radar because it's not something crazy like AI agents or, know, synthetic compounds that could kill us all. I think this is something to be wary of. And it's definitely going to be something that I'm paying attention to, because once AI gets to that level where it can write a paragraph that could move you, that could make you cry, it could make you happy, it could make you sad. That is going to be a level where we're going to be in some real trouble. Because, you know, sometimes, you know, you've been browsing Twitter, maybe even browsing an online thing, and you've seen something and it's probably upset you. What happens when we have a world where AI systems are running rampant and you are just seeing stuff that just annoys you, annoys you, annoys you. Or what if you see stuff like you see five pieces of text and subtly it changes your opinion without you even knowing. So this is definitely some important research. You can see they spoke about how they focused on polarized issues, such as views on new technologies, space Exploration education. We did this because we thought people's opinions on these topics might be more malleable than their opinions on polarizing issues and basically their research. And like I said, you know, it's a phenomenon shaped by many subjective factors and further complicated by the bounds of the ethical experimental design. Of course, you know, you got to be pretty ethical because you don't just want to convert someone's beliefs because sometimes when you're presented with new information, you might actually believe the new information, even if it isn't true, just because of how it's presented. And you can read more about the research here, but essentially it found that, you know, newer models tended to be more persuasive and it has more implications as large language models continues to scale. It's rather important. So yeah, Anthropic, the company behind Claude 3 doing some really important work, you likely did see this, but Elon Musk did a prediction and some people, I don't know why they hate Elon Musk predictions, but I guess now we're going to have a prediction which is going to be either right or wrong within the year. And I think most people's predictions around Elon Musk, the hate comes from the fact that, you know, he said Tesla would be doing this and then it got delayed. And then, you know that Tesla would be doing this and they got delayed. But understand that running a company is hard. Delivering products on time, it's very hard. Supply chain issues, a million different things. I'm not making excuses, I'm just saying as someone who's done a variety of entrepreneurship, I can understand why things get delayed. But this is a prediction about artificial intelligence that I say it's probably true. He said that the AI will be smarter than the smartest human next year, which is a profound statement. Which might mean we have some super level AGI, which we probably already do somewhere in the labs of opi. But there was an interview and he said if you divide AGI as smarter than the smartest human, I think it's probably next year. Within two years. Musk said about the timeline development AGI. So the thing is, okay, is that, you know, two years isn't that far away. Currently in 2024, by 2026, the mid-2026, within two years, are we going to have an AGI system? Probably with how fast things are moving. As long as there isn't some major catastrophe that wipes out everything, you know, if there is some supply chain issue or something like that. As long as there isn't some Insane regulation. I do think we get AGI, but remember, it's probably going to come from one of the top labs. Now, whether or not that's going to be from OpenAI X, AI, Google, DeepMind, Anthropic. I mean the competition is severely, severely. But I think it's going to be either Google, OpenAI or Meta, because that's what those three companies are working on. And it will be interesting because I personally don't believe that AGI is going to be just widely distributed as many people think and that, I'm not saying that's a fear among you, like OpenAI is some scary congulment that's going to take AGI. I just think that if AGI is real, I'm trying hard to understand how they control that system to the point where if it's jailbroken, it isn't a massive danger. Like imagine a system that is actually at that level that can pretty much do anything a human can do and better borderline super intelligence. Like, how is that not a major, major national security issue? I think it is. Personally, I think it might be in the best interest if AGI is an open source. And that might be a controversial opinion, but take this quote, okay, if AGI is going to be as powerful as nukes, as some people would say, then why would it be open source and free for everyone to use? Because we know what humans are like. So I mean, I do think also that the technology should be distributed in a safe manner for everyone to access. But you know, to the point where, you know, everyone has access to the raw files, I think, I think it's going to be pretty, pretty hard. I know that OpenAI is going to find a way to get this technology out there in a safe manner, but it will be interesting because like I said, times change. Maybe they're going to be like, AGI is so smart, this level is so smart that we can't release it. We have to, you know, only assign it to this kind of narrow AGI, I guess you could say. But yeah, it will be interesting to see how that thing actually does develop. Because, you know, like I've seen and what led me to believe that AGI might not be developed is because how they spoke about Voice Engine and they said, you know, we're not releasing Voice Engine, we're only releasing it for certain use cases. I think the AGI is probably going to be released in that same way where they're like, look, we can only use AGI perhaps in self driving cars. We can use it for Research papers in certain laboratories and we can use it in certain AI systems. But for the average person trying to do like a random task, probably not. But I think the benefits will be there. I just think it will be on the back end, just like Voice Engine is for certain use cases in certain industries. And I think, I think, you know, I think that's okay. I don't think that's a horrible thing. So then OpenAI actually rolled out a secret model, they said, majorly improved GPT4 Turbo model, available now in the API and rolling out in ChatGPT. Do apologize for talking really quickly, but I am extremely tired. So that is the point. The point is, okay, is that we have a situation on our hands because this is fascinating. So they rolled out a new model and you might be thinking, ah, it's just an updated version of GPT4. However, okay, OpenAI have been secretly updating their models. I wouldn't say secretly, but they've been continually updating the models again and again and again and again. And I think it's kind of frustrating for people because we don't get GPT 4.5, but I'm pretty sure that this, what we're Getting is basically GPT 4.5 because we've had so many GPT 4 Turbo 0125, this update, that update. And I'm going to be honest, okay, I think that this model is actually an actual improvement. But of course we know that GPT5 is coming. So a lot of people right now, when they see this, they're not really that hyped up. However, benchmarks say otherwise. If we take a look at this benchmark right here, we can see Nick Dobo's tweeted, GPT4 is back coding. If coding benchmarks are to be believed, it smashes Claude and previous GPT4 versions. We can see GPT4 Turbo 2024 of the April 9th. We can see that the pass is 83.8, 45.9 and the other pass. But we can see compared to GPT4 Turbo, Claude 3 Opus, GPT4 Mistral Large, Claude 2, Google's Gemini 1.5. So, yeah, we can see those. That's crazy that Mistral Large is actually beating Gemini 1.5 and Claude sort of add job. Like that's pretty crazy. But the point is, is that, yeah, we're getting major, major upgrades across the board. And I think it's going to be kind of interesting if we're actually going to get just like a 5 lines versus GPT 4. Now what I think actually happened was I think OpenAI just had this in a locker and I think they waited for people to release their systems and then they were like, ah, you know, we want to remain state of the art. We've seen too much backlash about people stating that, you know, we've been surpassed so many times. Look at this, guys. Alpe here tweeted. I ran Humaneval on the base and the plus on the new GPT4 Turbo 2024, April 9th and it ranks number one on both. We can see here that the pass is 86.8. The pass one here is 19.2. So a stark difference, which is pretty interesting because on this leaderboard. Okay, which is kind of fascinating, I'm going to get to another leaderboard that some people say is the only one that matters. But we can see that Claude 3 Opus is in fourth place, Deepsea Coder Wizard Coda, which is pretty crazy. So I'm guessing that what we have here is a situation on our hands where maybe GPT4 Turbo has retaken the lead. And that would be kind of fascinating because I think it would be annoying for people because people are like, ah, if GPT4 still in the lead, that means we're gonna have to wait more time for GPT5, which is not what people want to do. A lot of people do like AI news and AI hype, but trust me, guys, there's still a lot going on in the industry. So even if this isn't some major, major news, there are some changes that you might notice using it today. Today. One thing I actually did notice is that if you're using ChatGPT and you're trying to talk to it and you're trying to get certain characters, it just won't refuse as many responses. So now we'll try to make an image based on what you tried to do. So that is something that I did notice and I'm pretty sure that that was something that they did add with the update. Then of course we actually did get Mistral. If you don't know who Mistral. Mistral is a French open source AI company that just drop open source stuff just out of nowhere. They just literally just tweet it out. They don't say introducing Mistral new model. They just literally dropped the torrent link, let people go ahead and do with it and they let the work speak for itself. So this is Mistrals 8x22B. This is a mixture of experts model. It's pretty, pretty crazy. We can scroll down and see some of the benchmarks here, someone, Jan P. Harris ran the AGI eval results on Mistrals new model and you can see right here that it beats Quinn 72 base. If you didn't know, Quinn 72 base was actually something that was released the other day and it was pretty state of the art. I was actually pretty excited for this because I was like, these guys have actually produced some really, really cool models of the past. But now Mistral, okay, mixture of experts 8 times 22 billion parameters, boom. We can see that the AGI eval based zero shot is up to 52%. That is pretty, pretty crazy. Okay, ladies and gentlemen, this is another state of the art model. And I'm wondering how quickly a Mistral going to move because it seems like they just continually improve, improve. We can see Mistral 7B missile 8 times 7B. And I wonder, I wonder, I wonder because this is important. You see what we have here is Mistral's mixture of experts model. But okay, if you were paying attention to some of the news, llama 3 comes out I think next week or you know, very soon. There's been some rumors so we can't confirm nor deny that it's coming out next week. But the point is, is that, is it better than this? Is it better than this? Because Mixtral just released this out of nowhere. So I'm wondering if llama3 is going to be better than this. And I'm wondering where those benchmarks are going to be because as far as I know, they've been working on Llama 3 for quite some time and they've devoted a lot of money and a lot of researchers on that product. So it'll be interesting to see if open source beats out, you know, meta, even though it's open source again. But you know, I guess we're gonna have to see on the benchmarks here you can see that we also did have Command R. Command R is a rag model by Cohere, which is a really, really important architecture because essentially allows you to use your models with basically without any hallucination. Not hallucination as in like it's perfect, but it's basically perfect in terms of the citations and the things you can use it for. And basically a lot of people were excited about this because this is the Arena ELO leaderboard which some would regard as the leaderboard that is actually the real one. The reason some people regard this one as the real one because whilst benchmarks are important for certain tasks, some benchmarks are flawed. Some benchmarks are also very I guess you could say biased in certain aspects. And some people would ask that some tasks are also just fine tuned for the benchmarks, just so that people can say that our models inherently subjectively, I mean, objectively better than theirs, if you look at the benchmarks. But some people are like, look, forget the benchmarks, forget what the papers say, look at how people are actually using the model. Do people actually believe that this model is better than it? And if so, it will show on the arena Elo. And so far it seems that Claude 3 Opus seems to have that cake. And of course Command R was there and people were saying that, look, this also defeated GPT4, which was pretty surprising. So, yeah, now I have so many other stories that I do want to cover, but I really just want to get to the meat of the video, which is this. Okay, so it says OpenAI and Meta have new AI models capable of reasoning. Now, this is insane. Okay, sounds boring, but it's insane because this means that we're going to unlock some new capabilities because it means his model is going to be a lot smarter. But as Sam Altman said in his, you know, conference, he's had like a billion. But he said that models are going to be smarter. So basically it says. Executives at OpenAI and Meta both signaled this week that they were preparing to launch the next version of their LLMs, the system that power their generative AI applications. Meta said it would begin rolling out LLAMA 3 in the coming weeks, while Microsoft said it would be, you know, rolling out the model called GPT5 soon, which is extremely vague, which is just very, very teaser ish. And it says, we are working very hard and figure out how to get these models not just to talk, but to actually plan to have a memory. Vice president of AI research at Meta. So it'll be interesting to see if Meta actually, you know, does stick to open source. Or what if Meta just releases like their own chatbot, which is just like ChatGPT. Like, what if they actually just do that? But yeah, they said actually to reason, plan to have memory. And it says OpenAI's chief operating officer Brad Lightcap told the Financial Times, and the next generation of GPT would show progress on solving hard problems such as reasoning. And reasoning is pretty crazy because it allows you to do long things. And when I say long things, I mean like do steps and steps and steps that allow you to achieve a lot more complex tasks. And the problem is LLMs struggle with reasoning because inherently, by design, wherever you stand on the argument, okay, the architecture basically means that these things predict the next tokens. And some people say it's just predicting the next tokens, and some people say it's not predicting the next token. Wherever you stand, the point is, is that these things aren't really grounded in reality, so they struggle to truly reason. Okay. And of course, you know, like they say here, I think we're just starting to scratch the surface on the ability for these models to reason. We've seen some examples of models reasoning, like Mice's KPU and Devin being able to, you know, do multiple different steps, which is pretty interesting. But the reason I'm so excited for this is because people have been wrapping GPT4, the current versions of GPT4, and they've been achieving some incredible results. And with that, that does mean now that if we get other models that are even better and they're plug and play into these wrappers, I'm truly wondering how advanced these systems are going to be. And essentially, they say reasoning and planning are important steps to what researchers call AGI, human level cognition, because they allow chatbots and virtual assistants to complete a sequence of related tasks and predict the consequences of their actions. And this is very true. AGI is coming sooner than you think. This is pretty, pretty big news. And it says meta's chief AI scientist, Yann Lin Kun said current AI systems produce one word after the other without really thinking and planning, which is pretty true. And he said because they struggle to deal with complex questions or retrain or retain information for a long period, they still make stupid mistakes. Adding reasoning would mean that the AI model searches over possible answers, plans a sequence of actions, build a mental model of what its effects and actions are going to be. And this is the big missing piece that we are working to get to machines, to the next level of intelligence. And Lecant said it was working on agents that could, for instance, plan and book each step of a journey from someone's office to Paris in another in New York, including getting to the airport. And it says Meda plans to embed its new AI model into WhatsApp and its Ray Ban smart glasses. And I did talk about this previously because I actually did get a pair of these and they're actually pretty cool. This isn't a plug, but, like, if you've seen them, if you use them, you know that they're actually one AI wearable that I think people have sunglasses anyway. So if your sunglasses did just magically have a camera inside it, you probably wouldn't care. You probably wouldn't, wouldn't be obliged to using it as with other AI devices, it's like an additional device that you have to carry. Whereas glasses, most people carry glasses. So I mean, it's kind of fascinating. So since I think over time we'll see the models go towards longer, more kind of complex tasks that implicitly requires improvement in their ability to reason. So, yeah, it's pretty, pretty crazy. And he says we will be talking to these AI assistants all the time until digital diet will be mediated by AI systems. So this is clearly, you know, the next coming models here are going to be, you know, the next step up, which is of course reasoning, energetic behavior, which is planning and thinking and sequencing all these steps and thinking what's going on and mental models and that kind of stuff. Because whilst talking to your AI system and getting a response is good, it's very, very basic in terms of like, I talk to you, I get a response, I'm happy, okay. But, you know, being able to plan, being able to reason, be able to think, okay, I'm going to do this, I'm going to look over all these things. This is getting closer and closer, closer to AGI. So whatever models drop next, GPT5 and LLAMA3, I'm really, really intrigued because they kind of set the stage for what to expect, especially since GPT5 has high expectations that will be the benchmarks that other companies will be chasing. And it's going to be interesting to see where they are because if anything repeats itself, we know that GPT5 is going to be likely holding the cake in terms of the benchmark for another year and a half. So, yeah, that will be kind of interesting to see how that develops. And if you wanted to learn more, there was a paper here which basically says, based on the analysis, the paper concludes that despite its occasional flashes of analytical brilliance, GPT4 is utterly incapable of reasoning. Now, that might sound a little bit harsh, but you can read the paper if you want more information. And of course, Microsoft are going to be investing $2.9 billion in AI and cloud infrastructure in Japan while boosting the nation's skills in research and cybersecurity. So if you weren't bullish on Microsoft, maybe you should be now, because Microsoft are expanding Japan's cloud and AI infrastructure capacity. So that's a pretty significant investment. They're going to be training 3 million people over the next three years, giving them the skills they need to build and work with AI technologies, which is pretty interesting. They're actually opening up a Microsoft Research Asia. So that is going to be fascinating too. And yeah, if you're a software developer, this is news that you probably don't want to see, but you probably have already looked at this because you're a software developer. So this is autocode Revolver and this is something you don't want to see. So it says, presenting our autonomous software engineer from Singapore takes a GitHub issue and fixes it in minutes with minimal cost. 5 cents. Not good. You know, I'm not dogging on software engineers at all. I'm not saying you guys are going to be completely out of work at all. I think it's a really complex issue with how AI systems moving. The point is, is that it seems, you know, since the release of Devon, like every single week we've got an open source border that takes over the last one. And if the trend continues, by the end of the year, we're going to have fully autonomous software agents that are going to be able to do pretty much like 90% of the tasks. Whether we will get to 100, I don't know because I don't know if there's any AI system that can currently do like 100% of all tasks. I mean, I guess some, probably basic ones. But I think the point here is that this is actually pretty interesting and I looked at a certain piece of paper which was actually quite. This is the fascinating part of the paper. It says, interestingly, our approach resolved 67 GitHub issues in less than 10 minutes each, whereas developers spent more than 2.77 days on average. So it's pretty, pretty crazy in terms of what it's able to do. So I'm wondering, okay, if Devin's next update, they're going to have a benchmark which is like, oh, we're at 40% and then another benchmark a year later, oh, we're at 70%. And then like ChatGPT comes out with like GPT code or something and they're like, oh, we're at like 85% or something like that. So it's pretty crazy because this is actually an agent means it goes, it plans, it actually figures out what to do. So this isn't just like an LLM system where you ask it what to do. It like plans everything and actually solves everything. So I mean, yeah, the industry is kind of interesting and I want to see how the industry reacts to this because this is real, this is happening. This is going to be something that continues to go on. It's not something to ignore. The point I'm trying to make here but yeah, let me know what you think about this. Do you think new GPT4 Turbo model? If you want to know how to use that, you can check out the second channel. I uploaded a bunch of tutorials there including how to use the new Command R model and and how to sign up for Gemini 1.5 Pro. Oh yeah, by the way, what kind of channel would I be if I didn't tell you guys? Gemini 1.5 Pro is now available in 180 countries and for some reason if you're in the UK and I don't know if it's just me but anytime I click the link I don't really have access to it in the way that other people have access to it. So what you want to do is just use a VPN or just use it in the other Vertex AI studio which does work as well. So you can finally get access to that million context window which is pretty fascinating. So if you enjoy the video, let me know and I'll see you in the next one.