EP 163: Google Gemini - ChatGPT killer or a marketing stunt?

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

Google got so many things wrong when it came to the release and the marketing behind their new Gemini model. It's been almost a week. Yes, I took that long to decide how I'm going to have or how I'm going to deliver today's episode because I think that there's a problem going on right now with Google and I really want to talk about whether that this new Google Gemini model, its latest large language model, is a chat GPT killer or just a marketing stunt gone horribly wrong. All right, so we're going to talk about that today on Everyday AI. This is your daily live stream, podcast and free daily newsletter helping everyday people like you and me not just learn what's going on in the world of generative AI, because there's a lot, but how we can all actually leverage it. How we can use everything that's going on day to day. These new tools, tips, techniques and strategies, how we can use them, understand them to help grow our companies, grow our careers. My name is Jordan Wilson. I'm your host and I want to know, I want to know your thoughts. What do you think so far of this Google Gemini? Have you used it yet inside Google Bard? Do you think Google handled it well, I want to know from you, today's going to be one of those shows where I ask a lot of questions of our live audience. If you are joining us on the podcast, thank you. As always, we appreciate your support. Support. Please follow us, leave us a rating, but also check the show notes. Every single day we leave show notes in there so you can come back here, you can join the conversation going on on LinkedIn and other social media. So please, I want to hear from everyone out there because I think today's episode on Google Gemini is extremely important to talk about because it's actually about the bigger picture. It's not just about Google Gemini, it's about the company Google. It's about the state of large language models. So I do want to hear from you. So before we get started, as we always do, let's talk about what's going on in the world of AI news. So it was only a matter of time, but AI can officially read our minds. Scientists have developed a new groundbreaking technology that can read human thoughts and convert them into text through an EEG cap in an AI model. So this technology was developed by the Graphene X uts Human Centric Artificial Intelligence center at the University of Technology in Sydney in Australia. And it was recently featured at a prestigious AI conference. That was a mouthful. Right? But this system uses an EEG cap to record and decode brain activity and an AI model named De Wave to translate EEG signals into coherent words and sentences. Yes, this is not science fiction. This is real science. But yeah, apparently now this was just unveiled today. Yeah, I can read our minds and actually put it into coherent words and sentences. I'm actually happy for this, for those times when I'm thinking about something AI related and can just get it down on paper. Right. All right, next piece of news. How prominent are hallucinations in AI models? Well, it's so prominent, the word hallucinate is now officially the Dictionary.com Word of the Day or, sorry, word of the year. Yeah, word of the year. So hallucinate, if you don't know it, means when an AI model gives you false or inaccurate information. So every single year, Dictionary.com toward the end of the year names their word of the day or word of the year. And this year it's hallucinate. Right. It's something we talk about on the everyday AI show all the time. We've done multiple episodes specifically on how to get large language models to hallucinate less. So if you're like every other person out there and you're worried about hallucinations, make sure to go to your everyday AI dot com, search the word hallucination or hallucinate and you'll find so many different episodes that will help you. And so a little more about hallucinate. So this term emerged from technical jargon in the 1970s, which I didn't know, and it has seen a 46 increase in lookups this year. Oh, man. I used to side note, I used to just look up words all the time. The dictionary for fun. Weird, right? All right, last but not least in AI news, political robo calling is changing due to AI. So a Democratic congressional candidate in Pennsylvania's 10th district is using an AI powered phone campaign to reach voters and fundraise, making it the first of its kind officially known, at least in US politics. So the company is set Box. And that's the company behind the AI campaign, hopes to start a public conversation about the role of AI in politics. So I'm not sure how y' all feel about this, but especially here in the US Political robocalls and robotechs are terribly annoying, but they're extremely prominent, you know, especially when we get into that election season with an election now about less than 11 months away here in the U.S. so expect, if you haven't started getting calls from actual AIs that you can talk back and forth with, unfortunately I think you're going to have to expect that soon. All right, so there's more. There's always is more news, more what's happening in the world and how it affects you. Make sure you go to your everydayai.com, sign up for that free daily newsletter where every single day we break down the conversation that we have on the podcast, on the live stream. But also we cover just about everything else we have our AI news, which is, you know, some of the things that I previewed as well as others fresh finds which are just different, important things happening in the AI world kind of across the Internet and we have a whole lot more as well as previewing other shows that we have coming up on Everyday AI. All right, big wind up this morning. If you're joining us, let me know, I want to know from you, how hot should this hot take be? Should this hot take Tuesday be when I'm talking about Google Gemini? Let me know if you're joining this live. Should I take it easy on them? Should I tell how it really is or should I just burn all the bridges? Should I just go full hot take mode and, and make Google never want to talk to the everyday AI show? Let me know, I want to know from you. And thank you everyone for joining us as always. Mike joining us, good morning. Woozy and Tara joining us as well. Thank you all. Mike joining us. Liz, so many people, thank you. But let me know, what are your thoughts so far on Google Gemini? I'm going to get into it. I'm going to get into it, but I always want to hear from our audience. So what are your thoughts? What are your questions? I love this from, from Douglas saying quick de wave. What am I thinking now? Yes, that AI brain model. If I could get it live as well, that, that would be fun. All right, but let's, let's talk a little bit. And so far the votes are in. It's, it's a lot of flame emojis. People, people want full, full heat, full heat. All right, well, Cecilia says never burn your bridges, but everyone else is giving me like 50 flame emojis. So I'm gonna have to, I'm gonna have to go with the, the majority of people here. So. All right, we're going, we're going full hot take. But let's, let's talk high level overview. Talk High level overview on Google Gemini. What this is what it means. So yes, I've waited almost a full week. So they announced this. Almost. Let's see, it's been six days, today's day seven. So about a week ago Google announced Gemini. So here's the overview. Right, so Google Bard has been using the Palm 2 large language model. So Google is replacing that with a much more sophisticated and a much better much model in Gemini. All right, so Palm is out, Gemini is in. So there's essentially three flavors of Gemini. So you have Ultra Pro and Nano. All right, so Ultra is essentially the, the largest use cases. Pro is kind of like the, the day to day use cases and Nano is, is a much smaller version of the model that you can actually fit on a physical device. So Google is going to be putting it in their hardware. So let's start off with the good things. That's awesome, right? I love seeing smaller, more capable models that don't even necessarily need to be connected to the Internet and can run locally on a device. That is one of the big pieces of the future of large language models. So let's start with the good, right before I just start, you know, busting out the flamethrower. Let's start with the good also. It is the first true multimodal base model. Okay. So what that means is, you know, when you think of GPT, the model from OpenAI, it is multimodal. So being able to input not just text but images and audio and then on the output being able to receive, you know, kind of multiple medias. So you know, being able to input and output text, photo, audio. But GPT was built as a text model first and then kind of these multimodal functionalities were built on top of it, so to speak. So Gemini is at least according to all reporting and everything that I can see, the first true multimodal first model. So that's a big, that's a big step in the correct direction on generative AI and large language models being the future of how we work. Right. So another good thing is obviously built in integration with Google services. All right. And then like we talked about the ability to run on physical devices. So what most, well what most people do know is Ultra is not out yet. Right. So that's what everyone's talking about this, this Ultra mode, you know, inside Google Gemini. But if you go onto Bard right now it is actually the Pro model mode. So more on that later. But keep that in mind. Google launch with kind of giving people access inside of Google bar to this middle tier mode. That's extremely important. It's extremely important. All right, so now as I sip my good morning coffee, let me know, what do you. I mean, are y' all drinking coffee when we go over this? Are you walking, Walking the dog? Are you on the treadmill? Let me know. Getting caffeinated up, because I saw a lot of people wanting the hot ticks. All right, so let's talk about this. Google pretty much lied about Gemini's capabilities. I said pretty much. All right? I'm not going to say they straight up lied. We'll say they hallucinated. All right? We'll say they hallucinated. They did not tell the full truth. All right? So let's. Let's look at. Let's look at why and what they said. All right? So much of this was based around a video release, all right? And everyone everywhere, if you care about generative AI, if you're on Twitter, if you're on LinkedIn, if you follow AI news, you saw this video. Okay? So Google released a short video, and it was essentially showed a person interacting with Gemini in real time, right? And I have some screenshots of it that I'll show you here in a second. But essentially it was someone on the left side interacting with. And there was an overhead video with different objects asking questions and talking to the Gemini model. And Gemini was seeing what was going on on the screen and interacting in real time. All right? And so the only piece of. The only note that Google put up initially was on the video. They said this. They said, we've been testing the capabilities of Gemini, our new multimodal AI model. We've been capturing footage, okay, to test it on a wide range of challenges, showing it a series of images and asking it to reason what it sees. All right? So the thing that I want to point out is interactions, right? That's what I have underlined there. So here's one example. Here's one example of what was shown. Okay? So again, this is a video. So this is a screenshot from a video. But essentially in the video, the actor or the main participant who is talking with Gemini does paper, rock, scissors, right? And yes, I say paper, rock, scissors. I know that's weird. I know 99.9 of the world says rock, paper, scissors. I just can't. I grew up calling it paper, rock, scissors. So the person's playing paper, rock, scissors. And then Google, Gemini says, I know what you're doing. You're playing rock, paper, scissors. And the collective Internet lost its mind. Everyone was Going nuts. But here's the problem. None of that was actually in real time like Google said it wasn't. Right. The person was not actually talking to Google Gemini, Google Gemini was not actually listening. But that is the, the exact message and it was a very well done marketing video. Right? And that is what people really started running with immediately when Google announced this video. Just about everyone who's anyone in the world of AI, right? I was going to call people out by names. Yeah, I was going to call them out. All those, you know, quote unquote influencers on Twitter and LinkedIn, those 22 year old crypto bros that you're buy your g your chat GPT prompts from. They all said Chat GPT killer. Look at this model in real time. This is the future. This is the best thing ever. Google is taking over. But none of it was actually in real time. Yeah. And here now the former journalist in me has a lot of problems with this. All right? A lot of problems. Because yeah, there were falsehoods, untruths, white lies all over the place. The only other thing that Google said aside from that message on the screen was this. In the, in the YouTube video description it said for the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity. That is nonsense. That is hot garbage. Untruths. That is a hallucination. That is terribly inaccurate because let's, let's break it down, shall we? Let's break down Google's own words and let them know that no wrong BS detectors. Blaring, blaring. Latency has been reduced. Yeah, latency has been reduced by completely reconstruct, completely reconstructing how this was done. It was not done in real time. I'm going to show you. And then it says, and Gemini outputs have been shortened for brevity. No, they haven't. They've been completely reproduced in a different way. That was not represented in this demo video that was seen by millions of people that was shared by, by tens of thousands of the biggest names in, in tech and AI. All those people you're following but should stop following because everyone is just blindly reposting anyone's marketing messages without digging into it. Y'. All. I saw this. I was at the AI Summit in New York City speaking. When I saw this Wednesday, I saw the demo video. Two, two thoughts immediately came through my mind. Number one, I said, that's impressive. Number two, I said I gotta dig into that because I don't think it's possible. Right? I, I Wasn't gonna share it with you all. I wasn't gonna say, hey, this new Gemini model is the best thing ever. We talked about its release, but we didn't say anything about it's a chat GPT killer. It is correctly doing this in real time. It's not. Let's take a look at how they actually did it and break down those words. Yeah. So latency has been reduced. No, it didn't. The video was, was produced. Gemini was not in real time. And then when they said Gemini outputs have been shortened for brevity. No, those multimodal input outputs were faked. They were text and photos. They were not real time. So we're going to leave links in the newsletter. So make sure if you're listening on the podcast or on the live stream, if you are not already signed up for the newsletter, what the heck is going on? Go to your everydayai.com, sign up for the newsletter. We will leave a link to the paper because essentially on Google's developer blog, they, they eventually, I, I don't know the timing. I, I believe it was after, after people started questioning the validity of this video, but I'm not sure on that because I don't think it's timestamped. However, on the Google developer blog, to their credit, Google did release how this video was actually made and how it was put together. So this paper, rock, scissors. Yes, Paper, rock, scissors. Where in the video the person goes, paper, rock, scissors. And Google says, oh, I see what. Or Google Gemini says, oh, I see what you're doing. That's not how it worked. There was no real video. What they did is they recorded the video, then they took many screenshots, and then they uploaded those screenshots with text into Google Gemini. So, no, for this demo that set the Internet world on fire, there was no seeing Google Gemini was not seeing real time anything. It was not hearing anything real time. It was just the same as every other model. They took screenshots, many screen grabs. Go look at it on the developer's platform blog that we'll link to. And in this example, so we have the three photos, one of a paper, one of a rock in one of scissors. It's someone's hand doing these different techniques. So not only did they not just upload the video, right? Look at how many levels of, of hallucinations from Google's marketing, right? They didn't upload it. They didn't even. Number one, it wasn't real time. Number two, they didn't upload a video into Gemini. Number Three. They didn't just do screenshots of the paper, rock, scissors. They did photos. Three photos. One of a hand making the paper, one of a hand making the rock, one of a hand making the scissors. And guess what? Even at that point, they couldn't just upload it and Google Gemini could tell them they had to say, what do you think I'm doing? Hint. It's a game. Hmm. Hmm. What do y' all think? Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everyday AI.com partner to get in contact with our team. Or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen. Is Google's pants on fire? Did they lie? Did they hallucinate? Is this marketing? What I would like to say is it's not truthful. It is so far from the truth. Jerry says, don't hold back more flame emojis. All right, all right. Jerry. Yeah. Mike says the hot takes are scorching. All right, well, we'll keep it going then. We'll keep it going. But let's keep investigating. And let me tell, Let me tell everyone this. Yeah, Since. Since people are wanting the hot takes, stop listening to other people on the Internet. Right? I know that sounds weird, like, oh, only listen to me. I'm a former journalist. When a company tells me something, I never just talk about it on the everyday AI show. I never put it blindly in the newsletter where thousands of people are reading and trusting what we're putting out there is true. If your mother says she loves you, get it in writing, y'. All. I. I was an award winning journalist. When I see this kind of stuff, I know it is hot marketing garbage. And I'm Never going to go on and just trumpet what all these other crypto bros are doing to get you to sign up for their newsletter so they can sell you some crap AI products that are going to go out of business anyways. That's not me. That's not everyday AI. If you're listening, if you're watching, if you're reading our newsletter, we are giving you the truth. All right? And this was not the truth from Google. It was not the truth. So, yeah, in other words, the entire premise of the video was faked. Okay. It's not like Google was announcing, you know, new features on a camera or Google was announcing, you know, some, some new Gmail feature. No, the whole premise of this video and the whole secret sauce or the whole usp. The unique selling proposition of this new Gemini model was the multimodality, right? Was the fact that presumably Gemini could see and hear and talk in real time. And guess what? It was no better or no different than what we can already do with other models. And eventually, when people caught on days later, Google got scorched. Rightfully. So you see these articles says Google's Gemini marketing trick. An article from cnbc, Google faces controversy over edited Gemini AI demo video. Yes, but the initial reaction was Google's stock shot up through the roof. And now since that happened and since people realize, yeah, Google, you faked this, this wasn't truthful. Their stock has been going down and I think it should continue to go down. But also, I'm curious for our live stream audience, did you believe the video? 99 of people did. I'm a skeptic. I'm a little bit cynical at times. My old, the old journalists in me just didn't believe a single thing. But I think most people, even very smart people that I, that I trust were duped and they. Google has been getting dragged through the mug mud ever since. Now, you thought that takes were hot before. Here we go. I'm settling in my seat because the takes are going to get even hotter. Here's the thing, Google, why would you put out a video like this? Did you really think that no one was going to notice? Did you think that no one was going to dig in? Did you think that no one was going to explore how this seemingly revolutionary, that, that piece of technology, y', all, if it worked like that, live and in real time, that is not only bigger than chat GPT, that's one of the biggest technological innovations since the smartphone. But it wasn't true. Here's the thing. No one's talking about this, you can do the exact same thing in Chat GPT. I literally took the exact same prompt in the same pictures. And obviously I. I uploaded the same thing. I said, what do you think? The same photo inside of Chat GPT inside of their multimodal GPT4. I uploaded the photo. I said, what do you think I'm doing? Hint, It's a game. The exact same thing that Google did, it wasn't a video. They uploaded a screen. They uploaded photos and text. My gosh, this is embarrassing for them. But guess what? GPT4 did it just fine, obviously. So here's. Here's my other bone to pick, right? Because right now we talked about the three models, Ultra Pro and Nano. Right now Google Bard has the Gemini Pro model, which is not very good, right? So we are comparing, even in these screenshots, we are comparing this Ultra model that's not even out yet. It will reportedly be available sometime in 2024. So we are comparing an unknown model that no one out there can test or use to GPT4, which is already out there. So guess what? We could have recreated, in theory recreated this entire thing in GPT4. That's already out. And I'm showing you an example of it. Huge fail from Google Marketing. Huge fail. But that, not. That might not be the most concerning part, right? The fact that, hey, it actually wasn't live. Hey, we actually just took a bunch of screenshots and had to do a lot of prompting, a lot of prompting inside, inside Gemini to get this to work correctly. And then we took those responses and narrated them back. Oh, my gosh, so deceitful. But that might not be the most concerning part, because like I talked about, the public perception was already set by the time, three days later that people realized, wait, this. This isn't how it worked. It wasn't real time, right? Because every single person out there, every little influencer on Twitter trying to get you to buy something crappy from them, every single news outlet just blindly said, you know, Gemini more powerful than GPT4. And, you know, everyone on Twitter essentially put the same thing out there. Here is the chat GPT killer. No, Gemini was not a chat GPT killer. They were promoting a model that none of us can see, use or benchmark. All right, so here's the problem. Here's the problem. Yeah. Like Douglas said, I thought it was an impressive video, but we need to see it, touch it, feel it, before we can believe it. Absolutely. The truth is important. Like Brian is saying, Cecilia brings up a great point. She said Did Google lie or did we all assume and you know what that makes out of you and me? Yeah, I think it was both, Cecilia. I think the majority of people just assumed and took, you know, their spoon fed information from big Google and just reposted it to try to get people to sign up for whatever crappy product or service that they're offering ultimately in the end. But I think also, yes, Google faked it. Google represented that this was happening live and in real time. And not only did they misrepresent it, but even their language, right, when they said, hey, these are interactions, those weren't interactions. Those were not interactions. Right. And they would. And, and when they said it's been edited for brevity. No, it wasn't. No, it wasn't. It was just screenshots. It was completely reproduced and reconstructed from scratch. None of those live interactions actually happened, like I said. That's why I said probably lied in parentheses. But they faked it. This was full blown big tech hallucinations. Bad. It was bad. So the problems. So Gemini Ultra, which was quote unquote, better than chat gbt, isn't even out yet. Okay. Gemini Pro, which we're going to get to in a second from a benchmarks perspective. So what we can all go use right now. Yeah, you can go use Gemini inside Google Bard. Like I said, they swapped out the Palm 2 model for Gemini Pro. It is in line with GPT 3.5. Okay. The benchmarks were not favorable. So the combination of Those things, number one, the best model you can't use. Number two, the model you can use is testing like GPT 3.5. And number three, the benchmarks were not favorable. Let me ask Google, why did you release this now? This was a huge, huge failure, releasing this now, not just the way that you released it, but the timing of it all. Because actually, about four days before this announcement, there was all this reporting that was saying that the Gemini model was getting delayed until 2024. Right. So I'm guessing it might have been a, you know, shareholder response. Hey, we need to get something out there. You know, initially this was supposed to be a live event and instead it just got released through a press release and a YouTube video. So obviously there's a lot of things going on behind the scenes that you and I don't have access to. However, anyone can see now that this release was disastrous. Not only the deliberately, the, the delivery and what I will say is the deceit, but the timing. Why would Google release this when we can't even use or Test this model that they said is so fantastic. And when the one that we can use is comparatively running at the same speed as models and the same power as models that are almost two years old. Let's look at those benchmarks. Let's dig in. Dig in, shall we? So here we go. I gotta make my screen bigger for this. So the big thing is there's all these different benchmarks. Okay? And again, we're going to have the link and you can go read this paper. It's a 70, 60, 60 page PDF. I read. I read through most of it. Specifically, people are talking about the benchmarks in this. In this table here, okay. Where it's comparing Gemini Ultra, Gemini Pro, GPT4, GPT 3.5, and some other models across different benchmarks. And what Google said was Gemini Ultra was ahead of GPT 4 in 30 out of 32 benchmarks. Is that the truth? No, no, not the truth. Not the full truth. Let's take a look. All right, so now again, if you're listening on the podcast, I'm going to do my best to describe this, but it is some kind of technical benchmarking. Okay? But right now, keep in mind, Google is comparing a model in Ultra. No one has access to Ultra. You only have access to Pro. Okay. That no one can test. And they're also using different testing methodologies. So I have highlighted here Gemini Ultra and Gemini. Or, sorry, Gemini Ultra and GPT4. And you'll see that for the most part, in every single one of these, except hella swag, which is which GPT4 wiped the floor with Gemini Ultra in. In, hello, swag, which is actually a very unique test. But in every other test, Gemini Ultra outperformed GPT4. Well, number one, I would hope so. I would hope that an unreleased model would outperform the model that's already been in production for almost a year. Gosh, I would hope. How embarrassing would that be? Okay, but look in the details, y'. All, look in the details. I know it's kind of hard to see, so let's look at this test, which is GSM8K, which is grade school math. All right? The difference is the Gemini Ultra went through a 32 shot. And the same thing with the MMLU, which we're going to talk about here in a second. It went through a chain of thought, 32 shot. So without getting too technical and too into the weeds, chain of thought is a methodology of prompting that makes it much easier or much more increases the likelihood of a model getting it correctly. So a 32 attempt chain of thought. That's what they are comparing to the GPT where they're also looking at the five shot. Okay, so they're, they're kind of cherry picking, right? So when you look at the GSM 8K, which is grade school math, it's a test. All of these are different tests and benchmarks that you can see how well and you can see definitively how models perform against each other. So you can say, oh, this one's best at A, B and C. This one's best at D, E and F. Right? But when you even look at the GSM, AK, Gemini Ultra did a 32 shot and GPT4 did a five shot. So you're comparing apples and orange slices here, Google. And no one else can run these tests to confirm them. Let's keep looking because here's, here's, here's the real story that no one is talking about what we can all use right now, which is Gemini Pro. It gets straight up smoked in almost all of these benchmarks by GPT4, which again, Google, what's the logic behind releasing a model? You know, all this hype, all this marketing and say, ah, well, number one, we didn't actually do this. It wasn't actually real time. And that's our way better, better model. And you can't use it yet. And obviously everyone rushes out to go try Gemini Pro inside Google Bard. And it's straight garbage. Anyone that used it said, oh yeah, it's way worse than GPT4 and the benchmarks even confirm it. So Google, why on earth would you even. I would not have released Gemini Pro number one, I wouldn't have done this announcement at all. It was botched on every level. Number two, if you're gonna do this, don't, don't put Gemini Pro out there at all. Again, I know there's a lot of things that are above my pay grade and I don't understand, but if, if you put out and you release something and people go out and all they can use is Pro, all they can use is Pro number one. The average consumer is not going to know the difference. I'm not the average consumer. Most of the people listening to the show aren't, but the average person, they saw this on social media and they go into their Google Bard and they start using it and they see it's still hot garbage. It is outperformed by GPT on almost every single benchmark and it is not even close. It is not even close. Not close. All right, and here's Something that no one else is talking about. So MMLU testing that is a massive multitask language understanding. So this is a newer and what a lot of people who are smarter than me call the gold standard for evaluating large language models. So it's essentially think of it as like an satisfaction, right? It's, it's, it's not that simple. But this is kind of a standardized test to see how well different models can do in this massive multitask language understanding. Okay, so even when we look at Gemini Ultra, okay, the model that's not out yet, and if you look at the same testing methodology, a five shot, right, which is much more realistic, a 32 shot chain of prompt, that's not how people use large language models. You know, a five shot test is much more realistic, that is much more indicative or truthful and how most people might be using one of these models, right? So y', all, when you're comparing average use case across the quote unquote gold standard of large language model testing, Gemini Ultra scored in 83.7% on this MMLU, whereas GPT4 scored a much higher score at 86.4%. Again, Google, did you not think, did you not think this through? Because here's, here's the other reality. This is not even out yet. This is not out yet. Okay? So by the time it comes out, you know, whether it's 1Q20, 24, 2nd quarter, how long until OpenAI releases GPT5? Right? Yes, it looks like Ultra is going to outperform GPT4 on some metrics and through some testing methodologies. But for the most part, I think your best use case is the average person sees it as pretty similar to GPT4. Why would you do that? Why did Google release Gemini? I, I, I think y', all, it was a massive misstep. It was a master class in what not to do in marketing. And I actually think it set the gen AI industry back, right, because you had thousands of news organizations first reporting out, oh, hey, this is the new chat GBT killer, right? Because they're essentially just copying and pasting Google's press release. Stop doing that. Tech journalists investigate. Because it is not a chat GPT killer because we don't know we can't use it. All we can use right now is Gemini Pro and it is not good. It is not good at all. Google shouldn't have released Gemini. They shouldn't. And they dropped the ball with Bard again. Again. Yeah, Kevin, Kevin says GPT 4.5 will probably be released this month. Yeah, Kevin, if you know anything, let me know. I'm always following to see when, when things are going to be updated and released. But regardless, Google dropped the ball with Bard again. So I don't know if y' all remember this, but when Google first announced Google Bard, right, they did it in a big way, but also a huge failure. Because in the demo video, if y' all don't remember this, in the actual demo video that they showed that millions and millions of people ended up seeing there was inaccurate information that took all of five seconds to disprove, right? So there was some, you know, something flashed up on screen about, you know, being able to see something with a telescope and it got it wrong. And Google didn't even realize until after the fact, didn't even fact checked. They just took whatever Bard produced in this demo as truth, just like they wanted us all to do with Google Gemini Ultra. They botched the Bard release, they botched the Ultra release, and I think they're going to pay for it. And what I mean by that is their stock. All right, so if you look at what's called the magnificent seven tech stocks, which now make up a majority of the S P500 here in the US so we're talking Microsoft, Apple Meta, Nvidia, Amazon, Google, Tesla. So Tesla's kind of in it almost in a different category than the rest of them. Yes, Tesla's in AI with their auto drive, but they're having issues that are completely different than what these other companies are going through. But if we look at the other six, Microsoft, Apple, Meta, Nvidia, Amazon and Google, those six companies are heavily involved in generative AI. They're heavily involved in large language models, whether it's their own or investing billions of dollars in other companies that are creating them. And Google, at least out of those six, is the only one whose stock is down in the last three months. Microsoft crushing it up 12%. Apple crushing it up 10%. Meta eating people for lunch, up 8%. Nvidia up 4%. Amazon up 3%. Tesla's obviously getting crushed, down 10%. But they have some other issues with their auto driving scandal. But Google is down 1%. Here's what happened. When the announcement first came out, their shock shot up and it stayed up for, for a day or two. And then when the reporting came out that said, oh, actually this was faked, this video was faked, their stock has continued to go down, and I think it will continue to go down as more and more people talk about this, as more and more people report about this. And as you look at the big picture that Google had months, months to get this right, after completely fumbling the Google Bard release, they did it again with Google Gemini. There's no transparency there. There's no trust. And that is paramount, right? If you want big companies to start using your product, if you want enterprise companies to use and promote and to feel good about your product, you to establish trust, you have to be transparent. And what Google did is the exact opposite of that. All right? So they not only hurt themselves, they hurt their stock, they hurt their credibility. But I think this was a shot to the generative AI industry, right? Because they are now planting seeds of distrust that will impact all other companies, right? Because now when you have big Fortune 100, big Fortune 500 companies who were already on the fence about adapting gen AI, now they're just going to be like, oh no, we can't trust this. Look at the Google, look at the Google stuff, right? This gave large language models a black eye because Google fumbled it. Google fumbled it. Hey, thank you for the support, Michael. He says I'm a force to be reckoned with. You know what I think I was. My background sets me up to do this, right? I've been a tech geek my whole life. I have a digital strategy company. We've been using different generative AI tools with our clients for four years. We've been using the GPT technology since it was publicly released. You know, in. What Was that copy? AI was the first third party service to offer it back in 2020. So we've been using the GPT technology now for almost three years for ourselves, for our clients. So I'm not, I'm not new to this, right? I'm not an expert, but I'm not new. But y', all, you have to investigate when a company says, oh, look at this great new thing, don't just repost it. Don't just copy and paste and say, oh, they're a chat GPT killer. No, take your time and investigate. All right, I hope this was helpful, y'. All. If it was, please, as always, go to your everyday AI dot com. Why? Well, we have a newsletter that's actually based in facts, right? We're not like everyone else. We don't just, you know, take whatever big tech company says and give it to you. We investigate everything. We talk to the experts, we ask them the hard questions. Because if you are going to grow your company, grow your career with generative AI, you have to actually know the truth. You have to actually know how to use it. All right. So please, if you haven't already, go to your everydayai.com Sign up for that free daily newsletter. Hit us back right. I answer all messages. It does take me a while, especially the LinkedIn DMS. But drop me an email. I try to respond to every single message. Whatever your question is, I try to respond. We try to point you in the right direction. We are here to help you learn and leverage Generative AI to grow your company and to grow your career. If this was helpful, please share this episode with a friend. Leave us a rating, but also share this episode with a friend. So if you're on social media, hit that repost share whatever button is on social media. I don't know. But also if you're listening on the podcast, if this was helpful, please share it with a friend. And I hope to see you back tomorrow and every day for more Everyday AI. Thanks y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get get left behind. Go break some barriers and we'll see you next time.