GEMINI 1.0 Beats GPT-4 | The NEW MultiModal Model From GOOGLE

Google finally did it and Gemini is here. We've been waiting for this for quite a while now. It's multimodal in nature, so it has seamless reasoning abilities across text, images, videos, audios and code. Probably the closest thing to AGI today. The best part, it's able to outperform GPT4 on almost all the benchmarks. More on that later in the video. Google is releasing Gemini in three different sizes. Ultra is the biggest and most capable model, but it's going to be released in 2024. Second one is the Gemini Pro, which is going to be available in Google Bard starting today. And the last one is Gemini Nano, which is the one I'm actually really excited about because you will be able to run this on device and pixel phones already will be able to run this. Everybody's excited about Gemini because it's supposed to be a foundation model which you can use to build very capable applications on top. We all been waiting for Gemini because it's supposed to be Google's answer to GPT4. It seems like it's able to outperform GPT4 on almost every task that is given to it. More on that in a little bit. As I said in it's a multimodal model and they're calling it an anything to anything model. Now, the difference between this and other multimodal models is that it's built as a multimodal model from the ground up. So let me explain what it means. Multimodal models like GPT4 are actually language models with a visual encoder on top of it. So it does not natively have multimodal capabilities, but there is an extra layer on top of it. However, Gemini is trained to have the understanding of code, text, videos, images from the ground up. So it's part of the foundation model. Now, Gemini is different because all of these capabilities are built into the model from the ground up. Now, I really like some of these incredible demos that they are showing in here so you can provide a video as an input and it can write code to simulate the behavior of birds in the video, which is absolutely incredible. It's able to understand music notes from virtual inputs. So before looking at its comparison to GPT4, let's watch a quick demo video, which I think really summarizes its abilities. Hi. Here we go. Tell me what you see.

I see you placing a piece of paper on the table. I see a squiggly line.

What about now?

The contour lines are smooth and flowing with no sharp angles or jagged edges. It Looks like a bird to me.

What if I add this?

The bird is swimming in the water. It has a long neck and beak. It is a duck.

Yes. Now this video does seems to be speed up but it shows the abilities visual understanding of Gemini. Here's an example of it helping you solve a visual puzzle. So not only it's able to keep track of the objects, but it can identify which cup has that piece of paper underneath it. It's also able to do logical reasoning based on visual inputs. So I'm going to put a link to this video in the description. I would highly recommend everybody to watch it. It's around six minutes long, but it really shows all the different capabilities that Gemini have out of the box. I think the question that everybody is going to be thinking right now is how does it compare to GPT4? Well, it's beating GPT4 on almost all the benchmarks, but there's a small caveat which I'm going to explain. So according to Google, it's their most capable AI model to date and it shows when you look at some of the benchmarks results. So for example, here they are showing results on the massive multitask language understanding which is one of the most popular methods to test the knowledge and problem solving capabilities of AI models. Now based on the results, it's able to beat GPT4 by a large margin. In the initial technical report of GPT4 they reported an accuracy of 86% but Gemini is able to achieve 90% accuracy which is a huge margin. However, something to keep in mind, they are comparing the results with the previous version of GPT4. So we will need to look at the current results of GPT4 and MMLU for a fair comparison. As I said, it's multimodal from ground up and it actually shows up in the results as well. So here are the results for some of the text benchmarks. We already looked at the mmlu. Now, based on the reasoning, math and coding abilities, it's outperforming GPT4 on almost all the benchmarks. Now, some people have been highlighting that the differences that we see in here are not that much. However, you need to keep in mind that however, it seems like the performance of these LLMs or multimodal models are reaching a plateau. So now in order to make incremental performance improvements, they will need a large amount of effort. When you get to higher accuracies, then incremental updates will need a lot more effort and capabilities from the model. Results on multimodal data sets is very Similar Gemini is able to outperform GPT4 Vision, which is the latest model on almost all the benchmarks. Now the results that we are seeing in here are based on the Gemini Ultra which is not yet available. But if you have access to Bard, you can try the Gemini Pro starting today. So with this they also upgraded Bard and now Bard is going to be using Gemini Pro. So it's going to have some added capabilities. So if you have access to Bard, check it out and let me know what you think. I'm going to be creating detailed comparison video between GPT4 and Bard with Gemini Pro. Apart from the model, they also released a technical report which highlights some of the capabilities and technical details of Gemini family. And according to the technical report, Gemini Ultra model is able to achieve state of the art results on 30 out of the 32 data sets it was tested on. I will highly recommend everyone to to go through the technical report. It's an incredible read and I'll break it down in another video. So let's look at some of the next generation capabilities of Gemini 1.0. So the first one there they are highlighting is the sophisticated reasoning. So according to them, Gemini 1.0 sophisticated multimodal reasoning capabilities can help make sense of complex written and visual information. This makes it uniquely skilled at uncovering knowledge that can be difficult to discern amid vast amount of data. Now here is a video in which they use it for a test case of extracting information from scientific papers. So here they're showing an example use case in which scientists have to collect a whole bunch of data from thousands of papers and collect it in a table. And, and now Gemini is going to help them and expanding that. So here they write a simple query for Gemini to go through all the research papers and update the knowledge base, extract all the relevant information and there are even some plots that are being updated within the code base. It has the understanding of text, images, audios and more. The one thing I'm personally interesting is its advanced coding abilities. So according to Google, the first version of Gemini can understand and explain and generate high quality code in the world's most popular languages. So Python, Java, C and Go. And it has the ability to work across languages and reason about complex information. That makes it one of the leading foundation models for coding in the world. Now we'll have to see, now we'll have to see how it performs. But they have released a new system so they created a new code generation system which they're calling AlphaCode, which excel at solving competitive programming problems. That go beyond the coding to involve complex math and theoretical computer science. So they also released a technical report with AlphaCode 2 which I would highly recommend everybody to read through. So essentially it's a fine tuned version of Gemini Pro, so not even ultra specifically on programming tasks. Now what are the results? So they say Alpha Code two shows massive improvement, solving nearly twice as many problems, and we estimate that it performs better than 85% of competition participants, up from nearly 50% for alpha code. So if you put this in competitive programming competitions, it's able to beat 85% of the top human coders, which I think is incredible. Then when programmers collaborate with alpha code 2 by defining certain properties for the code samples to follow, it performs even better. So this is a perfect example of augmenting human intelligence rather than replacing it. And I think that's exactly the way the technology should be moving forward, that the technology is not going to replace humans, but is going to augment their abilities. We are excited for programmers to increasingly use highly capable AI models as collaborative tools that can help them reason about the problems, propose code designs and assist with implementation so they can release apps and design better services faster. If you're thinking of building on top of Gemini, you will be able to do that pretty soon because this is going to be released as an API on the Google AI Studio and Google Cloud Vertex AI starting December 13th. So I think we are going to see a lot more apps that is going to integrate Gemini Pro into their workflows and provide better user services. Anyways, it's really incredible to see that Gemini is being finally released and now we have a worthy competitor to GPT4. 2024 is going to be very exciting year when it comes to generative AI and I can't wait to see what the future holds for us. Hope you found this video useful. Thanks for watching and as always, see you in the next one.