Google’s New UPGRADED Gemini Pro 0801 Just Blew Past GPT-4 and Shook Up the Industry!

LARRUZ breaks down Google's new Gemini 1.5 Pro 0801, which scored 1,300 on the LMSYS benchmark, edging past GPT-4's 1,286. The model processes up to 1 million tokens and handles text at a scale competitors can't match yet.
Questions this video answers
1,300
GPT‑4: 1,286; Claude 3: 1,271
1,261
Up to 1,000,000 tokens
2 billion parameters
Chapters
So you know how Google's been kind of struggling in the AI world lately, right? I mean, they had some pretty rough moments, like that one time their Bard Chatbot gave wrong info about the James Webb Space Telescope during its first big demo. That little mess up cost Google's parent company Alphabet, a casual $100 billion in market value. Yeah, that hurt. And don't even get me started on their Gemini image generation tool. People were tearing it apart for historical inaccuracies and some bias issues, so Google had to pull the plug on that one really quick. With all that, it seemed like Google was falling way behind OpenAI and Microsoft, who've been absolutely killing it with their AI stuff, especially with GPT4O and all the AI integration Microsoft's been doing. It was almost like Google was this giant who was stumbling trying to keep up in a race it used to dominate. But here's the plot twist. Google just pulled off a major comeback and it's got everyone in the AI world buzzing. They've been steadily improving their AI game and now they've rolled out some impressive upgrades. Earlier this year they introduced Gemini 1.5 Pro, which really shook things up and now they're back at it with a new version. We're talking about the experimental version 0801 of Gemini 1.5 Pro, which is now available for early testing and feedback through Google AI Studio and the Gemini API. The LMSYS Chatbot arena is a popular benchmark that tests AI models on various tasks and gives them an overall competency score. In a recent evaluation, GPT4 got a score of 1,286, and Claude3 wasn't far behind with 1,271. Interestingly, an experimental version of Google's Gemini model called Gemini 1.5 Pro 0801 scored an impressive 1,300. This surpasses even the scores of its well known competitors. For context, an earlier version of Gemini 1.5 Pro scored 1,261. While this suggests Gemini might be more capable overall, it's important to remember that benchmarks don't tell the whole story about an AI model's strengths and weaknesses. The AI community is buzzing with excitement about this latest Gemini version. Social media is flooded with users praising it as insanely good, with some even claiming it blows GPT4 out of the water. So yeah, Google's definitely back in the game with this one. This release marks a turning point where users now have a variety of strong AI chatbot options. Ultimately, the best model for a user will depend on their specific needs and preferences. It remains to be seen if this experimental version of Gemini will become the standard. Although it's currently available to the public, its early release status means it could be modified or even taken down in the future for safety or alignment reasons. So it's safe to say that Gemini 1.5 Pro is a top contender in the world of AI models. One of its most impressive features is its ability to handle massive amounts of data, like processing up to 1 million tokens. Now, the other model they launched, Gemma 2.2B, is kind of a game changer too. It's got 2 billion parameters, which might sound small compared to some other models out there, but this thing punches way above its weight class. It's actually outperformed models that are way bigger, like OpenAI's GPT 3.5 and even Meta's Llama 2. Google's basically showing everyone that bigger isn't always better. In the AI world, it's all about how you optimize and tweak what you've got. Now, while Google is making strides with their AI models, not everything they touch turns to gold. Case in point, the deer Sydney Ad fiasco so Google had this ad running during the Olympics on NBC, which where a dad uses the Gemini AI to help his daughter write a fan letter to her hero, Olympic track star Sydney McLaughlin Levrone. Sounds harmless, right? Well, not exactly. In the ad, the dad asks Gemini to help his daughter write this heartfelt letter, and the AI goes ahead and does it. But here's where things get weird. The whole premise of using AI to write something as personal as a fan letter didn't sit well with a lot of people. Critics were quick to call it out as tone deaf and just plain bizarre. I mean, who wants an AI written fan letter? It's supposed to be personal, right? Something that comes from the heart, not a machine. The backlash was almost immediate. People were all over social media roasting the ad for being out of touch. Linda Holmes, a novelist and podcast host, pretty much summed up the sentiment by saying, who wants an AI written fan letter? And honestly, she's got a point. The ad came off as a bit of a misfire, like Google was trying too hard to show off their AI's capabilities without really thinking about whether it made sense in this company context. So how did Google respond? Well, they decided to pull the ad from their Olympics rotation. They said the ad tested well before it aired, but after hearing the feedback, they thought it was best to phase it out. However, they didn't completely abandon the idea they defended it by saying their goal was to show how AI can enhance creativity, not replace it. They wanted to create an authentic story celebrating Team usa. And in their minds, the Gemini AI was just supposed to be a tool to help get the creative juices flowing, not do all the work. But even with that explanation, the damage was kind of done. The whole situation just highlighted the fine line tech companies have to walk when it comes to integrating AI into everyday life. Sure, AI can be an amazing tool, but there are some areas, like personal expression, where people might not be so keen on letting a machine take over. Not everything is great for OpenAI either. Despite attracting significant investments exceeding $11.3 billion since 2015 from big players like Microsoft and Sequoia, OpenAI is facing a tightrope walk. A recent report suggests they're on track to spend a staggering $8.5 billion this year alone, raising concerns about their long term sustainability and even sparking whispers of potential bankruptcy. Now what's driving these massive costs? Running cutting edge AI models like ChatGPT doesn't come cheap. OpenAI is projected to shell out around $7 billion just to keep these models up and running. A significant chunk of this approximately $4 billion is is dedicated to renting server capacity from Microsoft, highlighting the immense computational power required for their AI operations. An additional $3 billion is earmarked for training their AI models, including licensing agreements with news organizations like News Corp to access and learn from their vast content libraries. And let's not Forget about their 1,500 employees. Their salaries are estimated to reach $1.5 billion this year, while OpenAI is generating revenue projected to fall between $3.5 billion and $4.5 billion this year, primarily from ChatGPT and other paid AI services. It's not enough to offset their massive expenditures. This puts them on track for a potential loss of at least $5 billion for the year. To put this into perspective, Google backed Anthropic, a competitor expects to burn through $2.7 billion this year. This situation has ignited concerns about OpenAI's future. Some experts are even questioning the viability of their business model, particularly with the emergence of free and open source AI models like Meta's Llama 3.1. This model, which allows developers free access to its code, could potentially undercut OpenAI's paid offerings, especially as some businesses are already hesitant about AI's cost and accuracy. This isn't the first time OpenAI's financial stability has been questioned. Last year, reports of ChatGPT's high running costs nearly $700,000 per day sparked similar concerns. However, OpenAI's strong backing from giants like Microsoft with a $13 billion investment, offers some reassurance. Microsoft's CEO Satya Nadella has emphasized that their investment secures them significant rights to OpenAI's technology, ensuring continuity even if OpenAI were to face difficulties despite these financial challenges, OpenAI remains committed to pushing the boundaries of AI. They continue to release new products, including the AI powered video generator Sora and the AI search engine SearchGPT, which even caused a dip in Google's stock price. OpenAI CEO Sam Altman remains steadfast in his ambition to achieve artificial general intelligence, even if it means spending billions. He envisions AGI as highly autonomous systems that surpass human capabilities in most economically valuable tasks. In essence, OpenAI is engaged in a high stakes gamble. They are pouring immense resources into developing groundbreaking AI, even if it means operating at a significant loss. Whether their bet pays off with the dawn of AGI or leads to financial difficulties remains to be seen. One thing is certain, the world is watching OpenAI's every move with bated breath. Alright, don't forget to hit that subscribe button for more updates. Thanks for tuning in and we'll catch you in the next one.










