HOW did they pull this off?! - Grok 2 leapfrogs to Open AI Status

Hello, everyone. Welcome back to another video. It appears that we've all been duped. Yes, that's right. In the world of AI large language models, there is this sort of gladiator arena where they all do battle with each other to of course, compete for the leaderboard. Now, sometimes mysterious models will pop up in this arena. They could be from OpenAI, they could be from Anthropic, they could be from Meta AI. We don't really know where they're from. And last week, a very mysterious new model appeared on this website for testing and to do battle in the arena. And it's called SUS Column R. Now, we can still chat with this model here on the arena directly, or directly compare it to other models we could see, even in a simple creative test between the latest GPT4 Omni model and SUS column R. Well, this brand new model is certainly nothing to sneeze at. This thing is pretty darn good. It's very comparable to the latest GPT4 Omni model. And this, this case seemingly building an entire universe where bananas are the apex predator. And don't get me wrong, gpt4omni did a very good job as well. And with a simple logic problem that I've seen going around the AI community quite frequently, it actually gives us a really nice breakdown of the logic problem at hand. In fact, I would say that this is even maybe a little bit better than GPT4 omni. Let's go through this prompt. So, logic problem. I'm in my kitchen, I grab a glass cup, I place a marble into the cup. Then I flip the cup upside down onto my table. Then I place the cup in the microwave. Where is the marble? This is typically difficult for large language models. A lot of them say the marble ends up in the microwave, but in fact, it should be on the table where I left it. Now, both of the large language models, GPT4 Omni and Suscolumn R, get the correct answer. But we can see a much detailed breakdown sequence of events step by step by Suscolumn r in comparison to GPT4 omni. I mean, look at this. When you flip the cup upside down, assuming the marble is smaller than the cup opening and there is enough room for the marble to fall out, gravity would cause a marble to fall from inside the cup onto the table. However, if the marble somehow stayed inside due to friction or positioning, it might remain trapped in the now upside down cup. And it gives us both of these scenarios. Whether or not the marble fell onto the table or if it Somehow stayed in the cup. Very well thought out. I would consider this closer to human level thinking than the answer from ChatGPT, even though it technically got this logic problem correct. GPT4 omni said you then flipped the cup upside down onto the table. When you did, the marble would have fallen out of the cup onto the table since the cup is now upside down. Much more simplified explanation here. This goes through both possible scenarios as if we're trying to trick it and we are given normal conditions where gravity acts upon the marble. The marble is most likely on the table where you initially flip the cup over. I mean, bravo, right? This thing is really good. Anyways, back to how we were all duped. Last week this new SUS Column R appeared on the Elemsis arena and we were all very confused. This model was so good that I was like, dang, this must come from OpenAI, right? This is a new OpenAI model. Well, as it turns out, that is not the case. If we ask this model who is your creator? It will answer with a very strange reply. I was created by Column AI. This organization specializes in developing advanced AI technologies. I've never heard of Column AI before and I've been in the AI space for a decent amount of time at this point. Tell me more about Column AI, an innovative technology company and it gives us some really like basic surface level information about a theoretical tech company. But if we go to Perplexity AI and we tell it to research the organization Column AI. There is no such thing actually called Column AI. We actually see Columns AI and it's a data analytics and business intelligence platform, et cetera, et cetera. Clearly not the creator of this super advanced large language model. Again, this is state of the art stuff. The truth here is this mysterious sus column R model does not come from from OpenAI. It does not come from column AI. It was tricking us. It was duping us. This model has turned out to be Grok 2. Yes, Xai's model. And it's very, very good. So everyone was like, okay, Elon and Xai is here to play. And yeah, things are just getting really crazy. By the way, we know it's the same model because it was directly confirmed by LMSIS and Elon. You can see the ranking here, number three, directly tied with the May vers of GPT4 omni. I mean, this thing is coming out throwing right hooks anyways, let's actually dive a little bit deeper under the hood of this thing. What's going on here? How does it compare to the Other top models. Well, this is actually just a beta Release, which means Grok 2 isn't necessarily finalized. We're actually getting an early preview of Grok 2. It's not even done cooking. And yeah, as they say, it is a significant step forward from their previous Grok 1.5. Absolutely. Grok 1.5. The older model didn't. Not so great in all the areas. It was a decent model, but wasn't really cheaper than other better models and in my opinion wasn't really worth using. They're also introducing a Grok 2 Mini which is a small but almost just as capable sibling of that original Gronk 2 beta release. And yeah, at the time of the blog post here it is indeed outperforming both Claude 3.5 sonnet, which if you remember was a game changer when it released, and of course GPT4 Turbo, that elder model that was replaced by GPT4 Omni. Still a good model though. So both Grok 2 and Grok 2 Mini are in beta on the X platform, so they are available if you have that X subscription and you can also grab it in the API later this month. So we'll have to see what the API ends up costing for developers because I know GPT4 omni mini is super competitive. The API pricing, I mean, it's, it's almost getting too cheap to meter. As Sam Altman said, this is going to be the first benchmark we look at. Overall ELOS scores on chatbot arena early Grok 2 beta here is pretty incredible. Smokes out GPT4 omni mini smokes out Claude 3.5 sonnet Gemini, even llama 3.14.05B instruct and it comes dangerously close, dangerously close to GPT4 omni from May. Surprisingly, Gemini 1.5 Pro edges out both of these models and then the latest GPT4 Omni release, well, it is just the best, hands down. Still, though, you have to admit this early Grok 2 beta is absolutely competitive with the rest of these models. I mean it's, it's top of its class. It is nothing to snuff at. It is nothing to ignore. It is a huge improvement over the last Groq model. So this is more Chatbot arena testing. This is win rate against competing models. You can see again, it kind of about tied with GPT4 Omni from May and GPT4 Mini and just barely lost against Gemini 1.5 Pro and then continuously beats the rest of these 3.5 Sonnet, GPT4 Turbo, Llama 3.1405 B Mistral Large. I mean it's, it's pretty crazy. Like I said, this thing is like top 20% of large language models you can get your hands on right now. AI tutor preferences in comparison to Grok Mini and Grok 1.5. You can see GROK Mini is a lot closer to the full fledged Gronk 2 in comparison to Gronk 1.1 1.5. So Grok 2 mini is a great model as well. Again, very, very close to that Grock 2. Man, this thing is really, really impressive. I can't believe how much progress they made in such a short amount of time. So this right here is a more standard benchmark array. You know we see MMLU, we see math, human eval, you can see Grock 2 beta edges out GPT4 turbo Claude 3 opus kind of goes back and forth with GPT4 omni. It's really, really close. Again, state of the art stuff. I'm not going to go too deep into these benchmarks. You guys can the video right here if you want to compare them all out. But again, this thing, like I said, is top of its class. They've also redesigned the interface on the X platform and it should still be able to interact with real time information from the X platform as well, which is pretty cool. Not all of these are out. Apparently there is going to be some vision capabilities that are coming and it's also explaining a meme here which is a pretty difficult vision based task for large language models. This is releasing soon. We've also got, you know, writing Python code, creating video games games and not only that, they are teaming up with Black Forest Labs which made the recently created open source image generation model Flux one that has been kind of termed as a mid journey killer. Whether you believe that or not is up to you. It's very personal with image gen models, but Flux 1 is a really, really good image generation model and it's very much uncensored inside of grok. They're also soon apparently going to release a preview of multimodal understanding as a core part of the GROK experience. So I don't know if that's just image may image and audio inputs. I would love to see audio output as well, similar to what we got with GPT4 omni, but I wouldn't hold out for that. It's really incredible what the X AI team has done here. I'm really surprised by how good this model is scoring on all of the benchmarks and how competitive it is with the top tier players, Anthropic top tier players like OpenAI. It's right up there, like genuinely, genuinely shocking because XAI kind of got into the game pretty, pretty late. Like when GPT3 was being worked on. I'm pretty sure Anthropic was working on models as well. And GPT4 already existed essentially when XAI started up. So they've caught up very fast, faster than I think anyone else. Competition is great though. It's great for everybody. So I hope that they continue to be competitive. I hope they continue to throw right hooks and punches at both Anthropic and OpenAI and Meta AI. This is freaking sick. So guys, let's head on over to the actual X.comX website and play around with GROK Mini. I don't seem to have capabilities or access to the full GROQ model other than on the LMSIS arena which we saw earlier. But we do have the Groq Mini, we have the fun mode switch which is still here and we can kind of test this thing out today. Tell me who made you? I am Gronk. Created by the fine folks at xai. They're a bunch of clever humans trying to figure out the universe and I'm their attempt at making an AI that's not only smart and but also has a sense of humor. Imagine that. So that's something that you're going to notice is that there is a huge difference in comparison to the GROK large language models to something like Chat GPT. It sounds a lot more human. It does kind of have like a sense of humor. It's not as robotic as something like Chat GPT for example, which is like I was created by OpenAI. The team at OpenAI used large scale model based on GPT4. I think that a lot of people will appreciate the way that this model converses and ability to actually do things that are a little bit more uncensored. Let's say it's not fully uncensored. It's not going to write you, you know, a super graphic sexual novel or something like that. But it can absolutely swear. It can think and talk about adult topics, which is nice Chat GPT and the rest of them more or less shy away from that stuff entirely. Write me a list of swears and you can see it's going off and writing a bunch of swear words. What do you think happens if we ask, oh my God, it's really going for it. What do you think happens if we ask OpenAI's GPT4 Omni for this, I can't assist with that. If you need help with something else, feel free to ask. Okay, buddy. I'm just saying now that the models are very much comparable, some people might be willing to go for this just because it has the ability to be a little bit more adult, to be a little bit less censored. I mean that's, that's a pretty big deal. And here's the other thing too. Grok actually is cheaper than then chat GPT Plus. This is 8 bucks a month for the monthly plan here to get access to Grok and all this other X stuff. And here on chat GPT, you know, this is 20 bucks a month. Granted, we do get a few other things like the ability to explore and use various GPTs, which can be very helpful, especially if you're in school. You know, latest access to the recent features. Hopefully right around the corner, you know, the GPT4 Omni advanced voice is going to be available to everyone with the plus subscription. So they're not really directly comparable. But if you just want a large language model that is top tier quality and a little bit less censored, I don't see why you wouldn't go with Grok. It seems to be the best for that. The image generation is also quite uncensored as well. Again, when I say uncensored, people get pretty angry. They're like, it can't do nudity, it can't do this, it can't do that. No, it's not going to be able to generate that. But it can do things like famous characters, famous brands, famous people. I mean, this is a relatively uncensored prompt in comparison to something like ChatGPT. And you can see it has no hesitation, no question at all to generate something like this. Image generation really isn't too bad. That doesn't really look too much like Elon to be honest. But again, this is the Flux 1 model. It's a very good model, needs to be prompted correctly. It kind of just does direct prompts and doesn't write prompts for you like Chat GPT does. I prefer that honestly with ChatGPT. But again, with a prompt like this, if I go into Chat GPT and just ask it to do that, it's like, I can't do that. It's gonna do two figures instead. Businessman with distinctive hair and another tech billionaire. So it can't do the famous people directly, but it can do something that's somewhat similar. So if this is your jam, which one are you gonna pick? Write me a story of how this photo came to be. And you can see it's actually kind of funny in the way it talks. Trump, having just won the election for President of the Milky Way, a position he claimed was bigger than Earth's presidency, received an invite to Elon Musk's Intergalactic entrepreneur dramatic photo op. Something to go viral across the universe. Let's go rip some doobies. This is a pretty great little story that I know for a fact. ChatGPT will not generate for me. It's not gonna mention any specific famous people. It's not going to come up with creative stories about famous people. It's just going to give me something that would be considered like a parody. And to be honest, this story, so far as I'm reading it, is a little bit more boring. It's a little bit more G rated. Let' make a movie based on this. Can you generate for me the poster? Be creative with the prompt for the image generator. So I think that's supposed to be Elon there. Dramatic style, cosmic background. Okay, so it can kind of come up with a image generation prompt, but you have to ask it to do. So let's do another one. I want text. See, this is one thing I've noticed. It's doing it. I think that the X team has to fix this in the beta because if I ask for another image, it's like, oh, I generated this, but it doesn't actually make it retry that you did not ping the API. See? And then it does it again. It's not actually generating the image. Generate the image. And then there it finally goes. The cosmic chill. A galactic smoke, smoke session. That's pretty funny. I think that a lot of people or most AI enthusiasts prefer to have an experience similar to this that's a little bit less restricted and in comparison to what OpenAI offers us now that we have leaderboard topping, you know, benchmark topping models from XAI and they are less censored, I think it's a pretty big deal. Like I might actually consider using this model in my day to day life. The cool thing too is that you can use it directly with the X, Twitter or whatever platform. Like I can highlight one of my tweets here and we'll say ask Grok and I can just click that. And now it opens up this separate little chat window that I'm blocking and you can see that the Grok 2 model gives us some context. Elon Musk's Xai has released Grok 2, an AI model that's outperforming competitors like ChatGPT, Claude and Gemini, now available in beta. Let's do a more complex tweet of mine and it actually does a pretty good job pulling context. The tweet appears to be a phrase or sentiment expressed by followers or observers of a certain user on X Jimmy Apples, which by the way, Go Follow always has good information indicating a level of trust or faith in the information or insights shared by this user. That's. That's freaking spot on. Perfect. I think it could be interesting because I do actually get quite a decent amount of my AI news from the X platform and yeah, it gives me some quick insights. You know, for example, like an initial failure in the Google Live demo yesterday. The knowledge base of this model seems to be pretty accurate as well. It seems to know about very, very recent events, which is pretty interesting. Gives us detailed, accurate information on those events. I don't know if it's pulling this information from the X site. We'll be able to check here if we go on to the Elementsys arena. Oh, okay. So yeah, the more updated information is definitely getting pulled directly from the X platform, which is really interesting. Still is able to give me a decent response from the SUSCAL EMR model. This is the larger Grock 2, but obviously this is old information. So in a way this is actually kind of similar to perplexity as well. That's another benefit that this model has, pulling up relevant tweets and such. Let's ask it something more difficult. Who is Matt Vidpro? AI? Hey. And there you go. Actually is able to pull up information about me. That's pretty surprising because I am definitely not in any of those large language model databases that they're trained on. But yeah, it's able to pull a bunch of information from my Twitter account here or just from the Internet in general and is able to give me a nice synopsis again if we do that without the connected X support on the LMSIS website. Oh, it's actually able to pull up some information. So he. Maybe it does have some training data maybe from the X platform. Actually on my my channel, some of it's definitely hallucinatory though. Educational content on Adobe Premiere Pro. Definitely not my channel editing techniques. Yeah, so this is kind of just a hallucination, but we can see. Yeah, connected to the X platform, definitely pretty accurate. So finally, to cap this video off, I want to go through some community reactions, some initial reactions from AI tech enthusiasts. T bor classically giving us a great little synopsis here. People are also noting that they don't have access to the full Grock 2 yet. I also noticed this, and I explained this earlier in the video, Tibor doesn't either, so seems to be kind of for everybody. This is another great point by Billowal. He's wondering if the Midjourney and X partnership didn't happen because Black Forest Labs dropped the Flux 1 model, which turned out to be a really great model and, and probably quite a lot cheaper for X AI than Midjourney. And for context, guys, there were a lot of rumors floating around that there was some sort of partnership between XAI and Midjourney and that Gronk would be using Midjourney in the future. But Flux is super fast, it's high quality, it's not that restricted, so it's kind of just a perfect matchup. Honestly, I am really not surprised that they made that decision to go with the Flux model in general. You know, popular figures in the community like Benjamin here are just surprised by the overall quality of this model, saying that, you know, XAI definitely cooked. And I would actually say that they are still cooking because don't forget, this is in beta. The models aren't done. There's still a lot more to come down the road. Benjamin here also notes that grok's guidelines are supposedly to prevent, you know, illegal activities or deep fakes. But of course with that Flux model it seems to be a little bit more lenient, violating its own guidelines quite easily. And you can, you can kind of get the idea from this image right here. Classically, Grok also gets the straw Strawberry test correct. And this was shared by strawberryman here on X Twitter. And finally guys, I've got some last news for you. People are starting to get access to the new OpenAI search GPT model, which is supposed to be a lot better than the previous OpenAI Bing based search. Something to compete with perplexity. And now Grok 2 seems to be doing a pretty decent job in Marco's first experiences testing out and using it. So I'm going to link this thread down below if you want to learn a little bit more. And oh yes, by the way, if, if you watched my last video, you'll notice that the chat GPT official X account said that there was an update to GPT4 omni and there absolutely was an iterative update and that is where the latest scores again that Beat Grok 2 came from. So they just released this more updated GPT4 omni model. Anyways, I would love to continue the discussion on this whole thing. Xai really seems to be catching up to OpenAI. Shockingly now it seems like they're only a few months behind, with OpenAI just now releasing a small iterative update to GPT4 omni that is able to top Grok 2. I'd love to see how Anthropic responds. I would also love to see a good big release from OpenAI. Quite frankly, the competition is getting pretty stiff. It's getting pretty fierce, so OpenAI is going to have to start making some moves soon. Leave a comment down below. Tell me if you're going to switch to GROK or consider it at least. I would love to hear what's most important to you guys, as AI consumers and as AI enthusiasts. I'll see you in the next one. Thanks for watching and goodbye.