Worlds NEWEST AGI AGENT Just SURPISED EVERYONE! (Beats CLAUDE, GPT-4, Gemini) (Maisa AI)

So there has been a new AI startup that actually claims to have completely thrashed everything that we know in terms of these state of the art systems. And it's pretty insane if what they're claiming is true. So take a look at this. They've said introducing MESA or MESA kpu, the next leap in AI reasoning capabilities. The knowledge processing unit is a reasoning system for LLMs that leverages their reasoning power and overcomes their intrinsic limitations. But that of course isn't the craziest part. Okay, take a look at these benchmarks. Okay, this is absolutely insane if this is true. Okay, we can see that on the left hand side. Okay, this Knowledge processing usage. The KPU is achieves 96.92% which is pretty much 97% on the GSM 8K, 86.2% on the drop benchmarks. Like look at these benchmarks guys. 100% on multi step arithmetic, whereas GPT4 gets 4%. This is absolutely insane. Now it isn't like an entirely new LLM system, it's like GPT4 on top of a system. So of course you can kind of credit that with GPT4 Turbo. But someone did point out why don't we Compare KPU with GPT4 rather than KPU with GPT4 Turbo? Because where is GPT4 Turbo? The benchmarks? Because GPT4 Turbo is slightly better than GPT4. So it doesn't make sense to compare the KPU with GPT4 Turbo than with GPT4. It just doesn't make sense. But the point being is that even if this is like a GPT4 wrapper, it's still poses some very, very interesting capabilities. Because as I was making the video, they recently released a demo and it is pretty cool. So this is like kind of some agent because it has multi step reasoning and the reasoning allows it to achieve much better things. Now another thing here that's pretty crazy. What you can see is that these ones are zero shot. Okay, so zero shot just means that they gave it one question and it comes back with one answer. Whereas three shot, they kind of prompt it in a way before and then they ask it the question and then it's able to get that and you can see that Gemini Ultra, that's 32 shot, that one' five shot. This one is chain of thought prompting and this one is five shot. Okay, so this zero shot is just like one question. It uses all its reasoning steps and then it comes to the knots. So that is why this is pretty crazy. Now they actually talk about more of this in their blog post and we're going to dive do a deep dive onto this because they didn't release that much stuff, but they did release a demo which I'm going to dive into and then after I show you guys a demo, we're going to look at some of the other stuff. So they said okay, that today Mesa is thrilled to announce the kpu, the knowledge processing unit. And KPU is a proprietary rich framework that leverages the power of LLMs with the decoupling of reasoning and data processing and an open ended system capable of solving complex tasks. This white paper aims to show what the KPU is, which is its architectural overview and how it has outperformed the more advanced language models such as GPT4 or Claude3 opus in several reasoning tasks. So you know Claude3 opus that everybody knows and loves this system looks like it beats Claude 3 and GPT4 standalone. Now it does talk about the limitations of LLMs. It says these large language models based on their current arguments architecture have several innate inherent problems that persist no matter how much they advance their reasoning capacity on the number of tokens they can work with. Number one is the hallucinations. When a query is given to an LLM, the veracity of the response cannot be 100% guaranteed no matter how many billions of parameters the model has. This is due to the way that the model is and of course the context limit. And it says lately more models are appearing that are more capable of handling more tokens, but we must wonder at what cost. And essentially usually the cost is slower times with more compute. So this is where they give us the architectural overview. So essentially, essentially this is where we get into how this thing actually works. So they actually talk about the transformer architecture. So they just talk about the transform architecture as the famous lost in the middle, which means that sometimes the model is unable to retrieve key information if it's in the middle of the context window. And it says that LLM is not always up to date because it's not be able to up to the Internet and of course limited capabilities to interact with the digital world. They're fundamentally language based systems lacking the ability to connect with external services. And this can pose challenging things. Is there restricted abilities to interact with files, API systems or other external software basically saying that they're not inherently designed for that. So this is the architectural overview for this system and this could be game changing because achieving this kind of reasoning just on top of GPT4 is absolutely incredible because that means that more advanced systems Combined with this could be even better. So essentially they state what they have is they have this reasoning engine right here and this is the brain of the KPU which orchestrates a step by step plan to solve the user's tasks and design the plan. It relies on an LLM or a VLM and all available tools. And the LLM is plug and play and currently extensively tested with GPT4 Turbo. So the reasoning engine here, this is where you put the LLM or that's what they said, it's the brain of the kpu. And this relies on the LLM, which is here, I'm guessing that's inside the reasoning engine, I'm not sure. Then of course, execution engine, this is receiving the commands from the reasoning engine which it executes and its results are sent back to the reasoning engine as feedback re planning. So I'm guessing that this is some kind of feedback loop, which is what they have. So reasoning engine execution, then reasoning again. So probably reasoning thinking, what do I do? It executes and then once it executes, it comes back again and then reasons again. So here's where they talk about the virtual context window. And they state that it manages the input and output of data and information between the reasoning engine and the execution engine, ensuring that information arrives at the reasoning engine and the data stays in the execution engine. In this way, the LLM context window underlying the reasoning engine is maintained only with reasoning and not with data, maximizing the value of the tokens. And this input and output management of data and information is not only covered by the user's prompt and files, but also external services and systems such as the Internet, Wikipedia and other things. And it says this system has been inspired by the architecture of operating systems where they're responsible for managing and orchestrating the various hardware. And it says this decoupling between reasoning and command execution allows the LLM to focus exclusively on reasoning, relieving it of any runnable operation of hallucination, data processing or retrieval of current information. And it says the articulation of these three components, in general, the KPU architecture opens the door to future analysis of the quality and performance on tasks with large volumes of data and multimodal content, open problem solving interaction with digital system such as APIs and databases. So what they have here is a pretty, pretty insane thing. They also do say unlimited multimodal data. They do say they have a giant virtual context window. I'm not sure, I'm not sure if it was unlimited, but it was pretty insane from what I heard. And of course so yeah, they got a reasoning engine and it seems that the reasoning engine was where all of the work is done. I don't know how entirely it's done because the problem is they haven't released a technical paper just yet, they just released a blog post and blog posts are fairly limited and what they do state, but if we take a look at the benchmarks, you can see right here, it says that we are pleased to show the excellent results we have achieved by testing our system against the reasoning benchmarks which the state of the art LLMs are usually evaluated. And as you'll see when compared to GPT4, Mistral Large Claude Opus or Google Gemini against our reasoning LLM and of course we kind of do better than that. So they also said it sets a new paradigm of math reasoning. This is the GSM 8k, the grade school math, which is composed of 8.5 thousand high quality linguistically diverse mathematics problems for elementary schools. And essentially they actually talk about how numerous methodologies have been explored in the context of experimental setup, including chain of thought, few shot prompting, code based self verification. And then they said we deemed it pertinent to evaluate our system using a zero shot approach to closely mimic the standard operational conditions. Basically what they're saying is that if you're a normal person and you're going to be using these AI systems, you're not going to be using five shot, you're not going to be Using 32 shot, you're not going to be using few shot, you're going to ask it one question and when you ask it one question, you usually expect one response. So whilst chain of thought is good, they're saying that we're going to evaluate it using Xeroshot and we've refrained from employing any form of prompt engineering or iterative attempts. So it's a one, it's a one size fits all thing apparently. And you can see on the GSM 8K it performs at, I think it doesn't even say right here, but it's clearly like 97% above Claude 3 opus, which is chain of thought. This is zero shot. That's pretty insane. Above Gemini Ultra, above GPT4 and it says surpasses all state of the art models in the GSM 8K test. Now they've tested this with GPT 4 Turbo and I'm really wondering which other models, if they test it with, is this going to be better? Because if GPT4 is way down here, what about if they test it with the OPUS and then use OPUS to reason could they even go further? I don't know, maybe they already experimented with that. Then we've got the math benchmark, which is, they talk about the drop benchmark, which is a 96,000 question benchmark. They said contrast to GPT4 which utilizes three shot approach. KPU was benchmarked using zero shot approach without the application of any prompt engineering techniques. This approach resulted in the KPU establishing a new state of the art benchmark in performance, showcasing its strong capability of complex reasoning. And we can see right here that this achieves. I'm not even sure what percentage, I think 86% or something like that, but that's clearly above Claude 3, Opus, Gemini Ultra and GPT 4. And here's the thing again, this is zero shot. And also, also, also we are going to get into the actual demo of this because the demo of this is pretty crazy and I probably should have shown you guys the demo. In fact, let's actually skip to the demo now. But yeah, the demo is here. So the demo of this thing, I should have probably did this before, but the demo of this software, essentially what they did was they had this right here, okay. And it's going to zoom in. Okay? So essentially what they had here, and I don't know why they didn't, the way how they did this demo, it was good, but they just didn't say exactly what was going on. So they have this email from a user. The user is having a problem. So the user says, you know, urgent delay in delivery from foodanddrinks.com I hope this email finds you. Well, I'm Steven and essentially the problem is that this person has sent an email and he's wrote some wrong things in the email. This guy's writing an email to a company, right? Okay, long story short, he wrote an email. People get emails all the time. Then here's we can see the software. And then of course we have the orders in a CSV file. So we have all this data of, you know, CSV files. So we got all of this data, all this order data, I'm guessing this is from the company from the last couple of weeks or days or whatever. Then essentially all they did was they added that data into the prompt menu. You can see after scrolling through that, they added it into prompt. They see now answer the customer's email and then of course send it to contact Mesa AI. So you can see now it's reasoning and it's thinking, okay, what do I need to do here? So essentially what we do is and they did Actually Test us against GPT4. We'll show you that in a second. But you see that it is able to reason. It's reasoning, reasoning, reasoning. And then we get the first step. So it then says the process begins by focusing on understanding the content of a specific email. This is achieved by interpreting the text found within a file named email.txt. the goal here is to grasp the customer's request fully by examining the email's content. This step is crucial as it lays the foundation for any actions that will follow, such as responding to the customer's email or addressing their needs based on new information from extracted from the email. Then you can see here that this thing will continue to reason and reason, reason. And it says after understanding the need to comprehend yada yad access and read the email. By doing so, extract the exact words and request. You can see it's reading after reading the email file containing the orders need to be accessed and examined. Then it continues to reason, having guided the necessary data. This is done by searching for the unique identifier. Continuing to reason continues, continues, continues to reason. And you can see step by step how this model is starting to read. And here's the thing as well is that this order, this order that this person wrote. So this email that the person wrote their order number. So they said that their order number was. I'm not sure where I can find it, but the order number somewhere here, right here. So their order number fd, whatever, this order number doesn't exist. Okay, but what's crazy is that after reasoning and identifying it out, it's able to somehow reason and find the correct email, realize that the user entered a wrong number and then get it right. And then able to, it says reasoning completed in eight steps. Check the reasoning process and then you can see it's able to send that email immediately. Now you can see right here as well, they added a little prompt of where they compared this to GPT4 and you can see error analyzing. It seems there's no record of number FD240188 in the orders database provided this discrepancy might indicate an issue with the order processing or record keeping system. Given the situation, a customer focused response that acknowledges the issue outlines blah, blah. So basically GPT4 failed at this and find it kind of fascinating that their reasoning engine was able to do this so quickly and just in eight steps. And it might seem like something basic, but clearly their reasoning engine, whatever it is that they're using, is rather, rather advanced. And if we take a look at Some of the other benchmarks, we can see that the KPU is essentially nearing 100% on these kind of benchma marks on multi step arithmetic. I'm not sure why they added GPT 3.5 Turbo there. It doesn't really make sense. But yeah, so at first a lot of people were stating that, you know, it appears out of nowhere, claims state of the art performance with a wrapper, offers a chart with no paper to dig into, shows no demos, gets a date wrong on their site. Part of my skepticism. But do you have a product to show us or just extraordinary claims? Now of course with this I am pretty skeptical because look at the benchmarks again, guys. Like zero shot. Zero shot. Like this is literally what we would expect from GPT5. Like literally, if GPT5 was here and they said zero sort. Zero sort. Zero sort 97, 87 like this is what we would expect. Like this is a GPT5 level system, especially with reasoning as well. Like Sam Altman literally said in a recent interview that GPT5 will be able to reason very, very well. So this kind of reasoning here, this is what we would expect from GPT5. So the fact that they've come out of the blue and just said, look, boom, we've done it. We've done what GPT5 is supposed to be able to do. And they didn't drop a technical paper, they didn't drop a demo straight away. I mean, they have six hours after and now it's in beta. It is a pretty, pretty crazy thing either A this company is absolutely insane. They've managed to do something rapidly and incredibly and surprise everyone, take everyone by surprise, and they figured out how to reason with LLMs in a way that we didn't know before, or B this is just a marketing gimmick and things aren't as good as they seem. Now, of course, right now, ac currently it's in beta development, it's not ready for commercial use just yet. However, with its imminent release on the horizon, we invite you to sign up for our waiting list which grant you access to the beta version. So it will be interesting for other people to benchmark this as well. And I would say that we need to see other benchmarks as well compared to the other LLMs, because that will give us the very, very, very accurate description on how this thing performs. And I do think their architecture overview will be interesting, especially for those of you who are more technically inclined on that aspect, to see if this even kind of makes sense. Some people said that it's just GPT4 with the rag wrapper could be. But I guess until we get the technical report, we're never really going to know. And hopefully as long as this demo wasn't faked. Because I know a lot of people were actually saying the Devon demo was, you know, it was cherry picked because essentially with a lot of demos, people will cherry pick things. But I think the proof is going to be in the pudding. Either way, we're going to be able to see this, people are going to get access, and then we're going to find out the truth. Because what they're claiming is pretty insane. And if this is true, this just goes to show how crazy AI development is. So yeah, will be interesting. Let me know your thoughts down below and I'll be seeing you guys in the next one.