Googles GEMINI Just SHOCKED The ENTIRE INDUSTRY! (GPT-4 Beaten) Full Breakdown + Technical Report

So I'm not going to waste your time. This video will be a summary of everything you need to know about Google, Gemini and what we're about to watch. First of all is of course, the trailer that Google just released. Later on in the video, there will be the of course, benchmarks, which are rather surprising, and absolutely everything you need to know about Gemini.

You know, one of the reasons we got interested in AI from the very beginning is that we always viewed our mission as a timeless mission. It's to organize the world's information and make it universally accessible and useful. But as information has grown in scale and complexity, the problem has gotten harder. So we always knew we needed to have a deeper breakthrough to make progress.

I've worked on AI my whole life because I've always felt would be the most beneficial and consequential technology for humanity. Human beings in our society would have five senses, and the world we've built and the media we consume is in those different modalities. So I'm super proud and excited to announce the launch of the Gemini era, a first step towards a truly universal AI model.

The Gemini approach to multimodality is all the kinds of things you want an artificial intelligence system to be able to do. And these are capabilities that haven't really existed in computers before.

Traditionally, multimodal models are created by stitching together text only, vision only, and audio.

Only models in a suboptimal way. At a secondary stage, Gemini is multimodal from the ground up, so it can.

Seamlessly have a conversation across modalities and.

Give you the best possible response.

Gemini is our largest and most capable model. It means that Gemini can understand the world around us in the way that we do and absorb any type of input and output. So not just text like most models, but also code, audio, image and video. What's amazing about Gemini is that it's so good at so many things. As we started getting to the end of the training, we started seeing that Gemini was better than any other model out there on these very, very important benchmarks. For example, each of the 50 different subject areas that we tested on, it's as good as the best expert humans in those areas.

It's very rare that you can work on a technology at a foundational level and it simultaneously can impact all our products.

We created a family of models that.

Can run on everything from mobile devices to data centers, each of which is.

Actually best in class. Gemini will be available in three sizes. Gemini Ultra, our most capable and largest model for highly complex tasks. Gemini Pro, our best performing model for a broad range of tasks. And Gemini nano, our most efficient model for on device tasks.

We want to provide the best foundational building blocks, and then we know developers and enterprise customers are going to figure out really creative ways to further refine our Gemini foundational models. And the potential is almost limitless.

So at Google, there's this healthy disregard for the impossible, and that has oriented us to be both bold and responsible together. As these systems become more capable, all of those capabilities also raise new questions.

We have to think about what it means to have an image be a part of, for example, the input, because an image might be innocuous on its own, or text might be innocuous on its own, but the combination could be offensive or hurtful. Safety and responsibility has to be built.

In from the beginning.

And at Google DeepMind, that's what we've done with Gemini.

We develop proactive policy and adapt those to the unique considerations of multimodal capabilities. We then do rigorous testing against those policies to prevent the harms that we've identified with approaches like classifiers and filters.

If I were to look at the foundational breakthroughs in AI over the past decade, Google has been at the forefront of many of those breakthroughs. And I think Gemini continues that rich tradition.

It's been an enormous sort of monumental engineering task, which has been very challenging, but also very exciting.

I have been at Google for quite a while and the reason I'm here is really I believe in the company's mission and I think Gemini is a great step in that overall mission.

We end up with a world that has more knowledge and that people have.

More access to information that otherwise would not be available to them.

That's what excites me. The chance to make AI helpful for everyone, everywhere in the world.

So now what you're about to see is Google's multimodal capabilities. One of the things that Google has talked about previously was exactly how Gemini would be focused mainly on multimodal stuff and that they would increase the capabilities of the multimodality. This example that you're about to watch clearly demonstrates how advanced Google Gemini's multimodal capabilities truly are.

Alright, testing Gemini. Here we go. Tell me what you see.

I see you placing a piece of paper on the table. I see a squiggly line.

What about now?

The contour lines are smooth and flowing with no sharp angles or jagged edges. It looks like a bird to me.

What if I add this?

The bird is swimming in the water. It has a long neck and beak. It Is a duck?

Yes.

A duck is a type of waterfowl in the family Anatidae. Ducks are closely related to geese and swans. The duck appears to be blue. This is not a common color for ducks. However, there are some breeds of blue ducks. These are typically not as common as other duck breeds. It is more common for ducks to be brown, black or white.

Are you sure?

What the quack? I was just talking about a blue duck. And now you're holding one. It looks like blue ducks are more common than I thought.

Would this float?

I'm not sure what material it's made of, but it looks like it might be rubber or plastic.

What if I tell you it's making a squeaking sound?

Oh, if it's squeaking, it's definitely going to float.

Nice.

It is a rubber duck. It is made of a material that is less dense than water.

Can you teach me how to say this in a different language and give me a few options? Oh, can you explain how to pronounce the Mandarin one? Is it Yaatsu?

Ya zi is pronounced with a first tone in Mandarin. There are four tones which are essential to distinguish the meaning of words. The first tone is a high level tone.

Oh, okay.

The duck is in the middle of the ocean. There is no land nearby for the duck to rest or find food. It is unlikely to find ducks here naturally.

Okay, based on what you see, come up with a game idea. Oh, and use emojis, please.

How about this? We can play a game called guess the country.

Sure. Let's do it.

Clue 1. This country is the home of the kangaroo, the koala, and the Great Barrier Reef.

Oh, that's easy.

Clue 2. This country loves football and has won the most Men's World Cups in football history. You are trying to get me to find the paper ball under the cup. I accept the challenge. The cup to the left.

Nice.

I know what you're doing. You're playing rock, paper, scissors.

What do you see? Now?

The fingers are spread out to look like the wings of a butterfly. What's this big ear and barking mouth? A dog.

Sweet.

The coin should be under the right hand.

I don't know.

The coin is in the left hand using a sleight of hand technique to make it appear as if the coin has disappeared. Both objects are round and flat.

There you go.

Both are food. The orange is a healthier choice than the cookie. Citrus can be calming and so can the spin of the fidget toy. They are both examples of popular toys that have been enjoyed by children and adults alike. For Many years.

Give me some ideas for what I could make with this.

I see pink and green yarn. How about a dragon fruit? Or how about a green cake with a pink heart?

Okay, how about these colors? And maybe show me some animals.

Okay, now I see blue and pink yarn. How about a pig with blue ears? Or an octopus? Or a bunny with a pink nose?

Ooh, I'll keep those in mind. Which way should the duck go?

Going left leads to a duck, which is a friend. Going right leads to a bear, which is a foe. It is better to make friends than foes. So the duck should go left.

Smart choice.

This is a picture of a crab.

That's right. Is this the right order?

No, the correct order is sun, Earth, Saturn.

Based on their design. Which of these would go faster?

The car on the right would be faster. It is more aerodynamic.

Which looks more fun?

The one on the right, because it has a loop.

What do you think this person might be saying?

How about a woohoo?

Oh, I love that.

I see you're drawing a guitar. You've added an amp. Now it's an electric guitar. We can make some seriously loud music. Now, given that you added drums, how about some 80s hair metal? All right, I see you've added a palm tree to the drawing. Change it up with some beachy vibes.

What movie are they acting out here?

I think they are acting out the famous bullet time scene from the Matrix.

Ooh, nice. What do you think might happen next?

The cat is going to jump to the wall and stick the landing. It's going to be a perfect 10. Oh, and it's a miss. It looks like it was just a little too far away. But don't worry, the cat is a natural athlete and it will be back up there in no time.

Okay, I've got one more for you. Describe the drawing I made.

It is a simple line drawing of the constellation Gemini. You did a good job of capturing the beauty of Gemini.

Nice. That's it. I think we're done.

Now it's time to take a look at one of the most important things regarding Gemini. And that is of course, the benchmarks. Now you can see here that these are the benchmarks compared to GPT4, as it is the next level in large language model. But currently GPT4 has been surpassed in nearly everything by Gemini Ultra. We can see that in the general capabilities, the MMLU that Gemini Ultra received a 90% and GPT4 received an 86.4%. If we scroll down to the reasoning tab, you can see that on the big bench hard on the drop, Gemini Ultra achieved better results, although maybe only marginal. It did achieve better results in this category. The only category it didn't was the Helicewag where it achieved a 87.8%, which is of course still very respectable. If we do look at the math benchmarks and the code benchmarks, we can see that on the GSM 8K, Gemini Ultra received a 94.5% compared to GPT4's 92%. On the math tasks, which include challenging problems including algebra, geometry, pre calculus and others, it achieved a 53.2% compared to GPT4's 52.9%. And on the coding with a human eval it achieved a 74.4% compared to GPT4's 67%. And on a natural two code it achieves a 74.9% compared to GPT4's 73.9%. So overall, Gemini Ultra does surpass GPT4 in seven out of these eight categories, which does make it right now the very best large language model that you can use in the AI space. So now we have the multimodal benchmarks, and these are the multimodal benchmarks in which Gemini and GPT4 were compared against. What's really incredible was that across every single benchmark on this page, Gemini Ultra surpassed GPT4's previous capabilities. So you can see that across all of the image benchmarks, Gemini achieved 59%, 77%, 82% and 90%. And in all of these categories it does surpass GPT4. And this is of course GPT4 with Vision, which is the latest model that can analyze images. In addition, since GPT4 doesn't have video, what they've done is they've compared Gemini Ultra to DeepMind's Flamingo and you can see that it achieved a 62.7% compared to DeepMind's score of 56, which is of course the same model that the same company did before. It just shows a decent improvement. In addition, what is also very interesting was that we did see that even in audio, Gemini Pro outclasses OpenAI's Whisper V2 and Whisper V3, which shows us that Google Gemini is currently a truly state of the art large language model multimodal AI system. Now let's take a look at Gemini's Ultra impressive capabilities in terms of reasoning and truly understanding what a user needs.

Here you will see a demo of Gemini's multimodal reasoning capabilities to understand and reason about users intent use tools and Generate bespoke user experiences that go beyond chat interfaces. Let's say I'm looking for inspirations for a birthday party theme for my daughter. Gemini says, I can help you with that. Could you tell me what she's interested in? So I say, sure. She loves animals and we're thinking about doing something outdoors at this point, instead of responding in text, Gemini goes and creates a bespoke interface to help me explore ideas. It's got lots of ideas, it's visually rich, it's interactable. Now, none of this was coded up. It was all generated by Gemini. Gemini uses a series of reasoning steps, going from broad decisions to increasingly high resolution of reasoning. Finally getting to code and data. First, Gemini considers does it even need an ui? Is a text prompt best? Okay. This is a complex request that needs lots of information to be presented in an organized way. Gemini then tries to understand if it knows enough to help. There is a lot of ambiguity. I didn't see what my daughter's interests are or what kind of a party I wanted, so it had asked a clarifying question. When I said, we're thinking about an outdoor party and my daughter loves animals, Gemini reasoned it had enough information to proceed. But it made a note that there was still ambiguity about what kind of animals and this is important and what kind of outdoor party. Next is a critical step. Gemini writes the product requirement document, or prd. It contains the plan for the kinds of functionality the experience will have. For instance, it should show different possible party themes, some activities and food options for them. Now, based on this prd, Gemini tries to design the best experience for the user's journey. It thinks that the user will like to explore a list of options, but will also want to delve into details. It uses this to design a list and detailed layout that we saw earlier. With this design, it writes the flutter code to compose the interface out of widgets and write any functionality needed. Finally, it generates and retrieves the data needed to render the experience. You can see it filling in content and images for the different sections. Ah, farm animals. She would like that. Clicking on the interface regenerates the data to be rendered by the code. It wrote. Oh, I know she likes cupcakes. I can now click on anything in the interface and ask it for more information. I could say step by step instructions on how to bake this. And it starts to generate a new ui. This time it designs a UI best suited for giving me step by step instructions. I want to find some suitable cake toppers for this. Show me some farm animal cake Toppers. At this point, Gemini again decides to create a visually rich experience. It generates a gallery of images. Notice the dropdowns at the top. It decided that maybe it should help me explore by showing different options. Sheep sounds interesting. I know she likes that. And now it helps me pick sheep cake toppers. These look great. This is going to be a fun birthday party. I hope you saw a glimpse of what Gemini is capable of. I'm really excited about what's possible here. This is such an interesting time in AI and I'm excited to be part of this.

Now let's take a look at Gemini's ultra multimodal capabilities in terms of helping someone with their homework.

As a parent, you may have to help your kid with their homework. I've certainly had to. Here's where Gemini can help. For this demo, we've created a simple interface. And with some clever prompting under the hood, we can really leverage Gemini's map reasoning and multimodal capabilities to learn a subject like physics. With gemiini, you could upload a photo of handwritten answers on a worksheet. Not only can gemiini solve these problems, but this is the amazing part. It can read the answers and understand what was right and what was wrong and explain the concepts that need more clarification. So Gemini identified some mistakes with problems one and three here. Let's take a look at three here. Gemini identifies that the formula was correct, but there was a mistake in calculating height. We can ask Gemini to explain in more details why the height is 50 meters instead of 6. I can ask Gemini to explain further.