Googles AI Bard Can Now SEE! (Major Bard MULTIMODAL Upgrade Shocks Everyone!)

Of the most underrated AI technologies recently got an update that will fundamentally change how users interact with this software. The update is actually one that adds a multimodal capability to Bard. And as many of you know, Bard has been likely overshadowed by ChatGPT's various highly skilled abilities. But many people haven't understood that Bard is just as good as ChatGPT and in terms of usability in certain use cases, and one of those being Internet research and of course now image capabilities. Now Bard continuously gets updates and you can see right here that this is essentially the changelog or what was added. So right now you can see that this was literally only four days ago and Bard has now added over 40 new languages. So you can upload images alongside text in your conversations with Bard Bard, allowing you to boost imagination and creativity in completely new ways. And of course, what's also cool is that Bard can read responses out loud. Now, I know that many people are going to just simply state that if we look at the human evaluation tests and if we look at what GPT4 is able to do on certain technical tests, that Bard isn't even in the realm of possibilities. But you have to understand that what we are now seeing is a diverse AI landscape. And essentially what that means is that every single AI tool that you now use is, is going to have fundamentally different strengths and weaknesses and different areas where they simply outclass the other AI. For example, if we go onto ChatGPT, we know that ChatGPT is of course one of the smartest AIs, but the problem is, is that it isn't connected to the Internet and of course they disabled browsing with Bing. Now browsing with Bing, even in and of itself, was actually quite slow. I remember using it and it wasn't that effective at searching through many different articles and. And it just wasn't as great. Whereas if you use Bard to search the Internet, it's actually much quicker with certain responses and has up to date information on every single article. When using ChatGPT, browsing with Bing, what it will, essentially what it would do sometimes is that sometimes it would click on a link and it would say reading failed, content failed. And this is just something that you don't get with Bard. So Bard actually does have certain strengths that I'm going to show you, which is hopefully going to change your opinion and show you that I don't think people should limit themselves to just to using one artificial intelligence tool. I think there are different tools for different applications. So the point I'm trying to make is that there are many different AIs for many different uses and you're going to be a much more effective individual if you know which is best for which scenario. Currently, the only multimodal AI that's released to the public is Bard. And once again, like we've stated before, it's quite underrated. So let's take a look at what some of the community is doing with Bard and why. I do think it is much better than ChatGPT. And if you are trying to do updated Internet search information, and that does include in comparison to Bing. So essentially, when you do come over to Bard, what you'll see is you'll see a new user interface. You'll see this plus button right here, which shows you that you can easily upload a file. Now, when you click the upload button, you see you can pretty much upload any image. Now, I was actually messing around with this and one thing that Bard finally added, which should have been there from the beginning, was of course, this recent. You can see this is where you can actually search through your chats and where you can view your chat history. Now, it's important to remember, although this is going to be really fun to play with, if you are going to be messing around with images, please try to not submit any personal information because they are currently reviewing the training data that is submitted and they are going to be making adjustments based on Bard's response. For example, if I were to enter a prompt to Bard and Bard was to give me a prompt that responded in a way which wasn't suited or Google didn't approve of it, they're going to ensure that in future updates, Bard's responses are much better. So that just essentially means that human reviewers are going to be seeing the data that you enter. So here you can see that there are two images that I submitted to Bard because I wanted to quickly see how good Bard is with image recognition. One of the first images you can see right here is just an image of a house, which is a 3D render. Now, I've got to be honest, it actually gets this very, very accurate. It states that the image you sent me shows a 3D model of a house with a garage. It is a transparent PNG image, which means that the background is transparent and you can see through the house. Although there is some minor hallucination, as they do say that the house is surrounded by trees and shrubs, which isn't true, which is quite unfortunate, because if this software is going to be used to help someone who might be visually impaired. If they do rely on this, then of course, that wouldn't be too good. Now, of course, we do know that Bard's hallucinations are still facing some early tweaks. But what this has shown us is that this AI race between Microsoft and Google is proving that we are going to get more and more AI features quicker than we anticipated. Even though Google did announce that at Google IO Bard was going to get images, we didn't really realize that it was going to get shipped out this quick because as you do know, OpenAI has been working on images for quite some time, and they haven't currently shipped that model out yet. They've only actually shipped out images to a select group of users on the Bing experience. Currently, I'm guessing what they're doing is alpha testing it to ensure that the AI remains safe for everyone to use and of course, to prevent any kind of abuse. As you know, many kind of jailbreaks and abuse can happen with AIs. Then, of course, I decided to send it a basic picture of Sonic. And it says, the image you sent me shows a blue cartoon character standing on a black background. Of course, once again, it's not a black background. It is completely transparent. So maybe that's where the image got this kind of wrong, because sometimes transparency can appear black. It does, however, identify that the character is Sonic the Hedgehog. I'm guessing, according to some sources who, what Google has actually done is essentially combined its reverse image search feature with whatever image you input it in. And then based on the data that you do get from Google's reverse image search. And essentially, if you don't know what Google's reverse image search is, you can see that right here you have a button that says search by image. So, for example, if I was to put that same picture of Sonic in here, I'm guessing that a bunch of different search results would actually pop up here. And then I'm guessing what Google's done in order to expedite this process is of course, have Bard summarize what some of the largest search results are. And I'm guessing that that is probably going to be very, very effective, since that it seems to be working at a decent amount. And what this also does prove is that Google is willing to do anything at its current moment to beat OpenAI, which shows us that Google have actually stepped up the notch in terms of their development speed and in terms of what they want out AI programs. Now it's time to take a look at what the community has done with Bard. Because Although my examples are quite basic, some of the examples floating around on Twitter showcase that Bard does have a vast range of capabilities that most people don't currently explore. So on Twitter, this person called Dr. Sintas actually did one of the very best experiments that you're likely to see. And the reason this is one of the very best experiments with Bard is, is because it is a direct comparison into what GPT4's image search is able to accomplish. So if you don't know why this is such a great example is because in the GPT4 trailer, Greg Brockman, the co founder of OpenAI, demonstrated the future multimodal capabilities of GPT4. Essentially what he did was he took a picture of a poorly written handnote and then immediately converted it into a website. I'll play the clip from the developer livestream because it's better if he explains.

It than me hand drawn mock up of a joke website definitely worthy of being put up on my refrigerator. So I'm just going to take out my phone, literally take a photo of this mock up. So the thing that's amazing in my mind is that what's going on here is we're talking to a neural network. And this neural network was trained to predict what comes next. It played this game of being shown a partial document and then predicted what comes next across an unimaginably large amount of content. And from there it learns all of these skills that you can apply in all these very flexible ways. And so we can actually take now this output. So literally we just said to output the HTML from that picture. And here we go, actual working JavaScript filled in the jokes for comparison. This was the original of our mock up. And so there you go, going from hand drawn beautiful art if I do say so myself, to working website.

So now that you've seen the clip, let's take a look at this example from someone that tried to do the same thing with Bard and see if they're actually successful. So essentially he decided to do his own sketch of the website and apparently it did work. So essentially what he did was of course he decided to add his sketch in which you can all see. Then of course you can see that essentially, then of course you can see that the prompt that he added was write a brief HTML JavaScript to turn this mock up of a colorful website where you replace the jokes with two real jokes. So essentially what he shows you here as well is that if you do get some drafts that aren't particularly accurate, like for example, sometimes with ChatGPT, it generates code that doesn't work. All you can do is click the regenerate draft button or look at the other drafts to see if that code does work as well. Because one thing that's really great about Bard is that rather than having to regenerate the entire response like ChatGPT, it automatically instantly generates three different variations, which is very good. And if you don't know why they do that, I think that one of the reasons they actually have three separate drafts is because, of course, this might be a secret form of reinforcement learning with human feedback. So essentially, when people do pick a specific draft over others, I think Google realizes that in certain scenarios and then looks at what made that response specifically, specifically much better. And then a Google looks at what made that response much better than the other two, which of course over time is going to improve Bard's efficiency. So you can see the final version here does actually work. And when he pushes to reveal the punchline, the punchline actually does completely work. So this goes to show that although Bard's coding capabilities aren't widely touted as something that is the best, they are still pretty decent, as you can still get usable code that does work with basic stuff. Now, of course, we aren't building websites like this anymore, so applications may be limited, but it does go to show that Bard is still being upgraded at a pretty quick speed. So another Twitter user by the name of Amar Reshi actually showcases something really cool by Bard. And I was actually about to say GPT4, but of course, this is surprisingly Google's Bard. So essentially what he wanted to do was, was recreate the iPhone timer app. So you can see right here he put recreate the app you see in this screenshot. Make sure it's fully functioning, provide all of the necessary code for it to work, write your work or your code in SwiftUI, do not make any mistakes. Then what we can see here is Bard get to work. Then what he does is of course he imports it into SwiftUI. Then of course he then. And of course you can see that the user goes ahead and pastes it. And of course there are certain things that you do need to fill fix. Understand that if you've ever coded anything before, you'll understand that coding is never a one trick point. Now then of course you can see that his build succeeds, which means that the app actually works. And just like that, you can see that he was able to get this simple app coded within seconds. Now, those of you who are going to say that Bard might be something that doesn't always work. I can't get the code to work. Understand that this is pretty impressive, considering Google didn't really have a large language model a year ago and and already they've managed to create a product under intense heat, intense competition, and managed to ship out multimodal coding features and capabilities before OpenAI managed to do it. Now, of course, everyone's going to argue that Bing's multimodal capabilities are going to be absolutely incredible, which they are. But it's just very interesting that Google, who was very, very behind on AI, managed to somehow catch up to ChatGPT.