Major AI BREAKTHROUGH!, Meta AI, VideoGPT, GPT Vision HUGE AI NEWS #17

This week in AI was absolutely incredible. There was a major breakthrough that many people haven't picked up on. And it is about to change many different industries. And from that to ChatGPT to Dall E3, there are many things that happen this week that will fundamentally change the future of not only AI, but some other technological developments that you will likely see within the coming years. So this is one of the things that has recently changed the game in terms of a major breakthrough in terms of computer vision. This is 3D Gaussian splatting for real time radiance field rendering. So in this research paper they outline a new, more efficient method of generating these things called NERFs. And they use a different technique which is like NERFs, but is far more efficient, can run at higher frames per second, and is just pretty crazy. So on screen right now, what you're seeing is real time footage of essentially photography scans that someone has taken, someone's taken several pictures of and then they've input it into the software and now we have this as 3D data. The reason this is pretty game changing is because if you look at this with past examples, you can see on this page right here that the past examples such as NERF and the others ones as Instant NGP just aren't as high quality. And you know there's going to be a video clip from some other YouTubers where they just show you how crazy this jump is because they did a real demo and they essentially showcased just how crazy this technology is. Now I can only do so much because I'm only using images. And of course it is 3D and you're watching this video. But I promise you, when I tell you that this is pretty crazy, okay? And the reason I've included this in today's video is because I think in terms of computer vision, it is a very important part of the technological landscape, especially if we are talking about AI. Because one thing about AI is that there are small and different parts that can be intertwined with other technologies such as robotics and with computer vision. So imagine combining this software with something like a large language model that is specifically made for this. It would definitely be very interesting to see what can happen. Now many people are thinking about the possibilities in terms of gaming because this could make photorealistic games really possible since the frame rate is extremely high.

Unreal Engine plugin, you got all the details here too.

Can you go closer to the edges though? That's where I think if you were to do close up shots of this, we might have Some trouble.

Yeah, right.

Because you can see the artifacts around there. Let's try.

We've been promised by the guys who released the paper of Gaussian splats that.

The quality is higher than Nerfs. Farad, are you ready?

1, 2, 3. Move around it, dude. What?

Dude, man, look at it, dude. I have every single angle.

I feel like Superman. Look at that.

Dude. I can't.

I think we already achieved perfect.

So overall, based on the demos and examples that you've seen, hopefully you can gain an understanding of what this is and why. This is pretty crazy. I do think that if this is something that does come to video games or other applications, it will be very interesting to see how this is applied. And I do think that when I look at the footage that I am seeing, it is really indistinguishable from actual pictures. So the next thing that we have here is of course, Tesla AI. Now, if you don't know, Tesla is the same car company that now produces robots. Now here's the thing. They've been working on this for just over two years, or maybe even more years than that, but we're starting to see rapid development in not only their AI, but their humanoid robot. So what we're seeing here is Tesla's Optimus. Now, what's crazy about this robot is that you can see that its neural net runs entirely on board, you see, can using only vision, which is crazy. Okay, that point in itself is something that a lot of companies are striving to do. Everybody knows the GPT4 and ChatGPT 3.5, their servers are essentially really huge. And their running cost was so much so that they not only had to dumb down GPT4, but they also had to make sure that GPT4 was only limited to around, you know, like 25 messages every three hours. Okay? Because the amount of people request investing that service was too much, and of course, the compute was just too much in terms of the cost for OpenAI. Now, if you can get an AI, large language model, whatever sort of model that you are running on, an actual robot that can work with the robot and use the vision to essentially convert that into robot actions, then you have a real winner. And that is what we have with Tesla Optimus. We can see this robot managing to sort items all autonomously and autonomous behavior is, is what is the next step in terms of AI control, Because it allows the robot to do stuff without human intervention. You know, the need for constant updates, for constant interference, and without pre programming many of the stuff. Now, many of you might say, this is not anything new, this isn't crazy, but trust me when I tell you, it actually is. You see, the nuance, indifference between robots from now and robots before is that many robots that could do stuff were always pre programmed to do that. They weren't thinking, they weren't autonomous, they weren't not in a sense alive, but they weren't actually doing the task. They were just doing a programmed preset of actions. But these robots are autonomous, they can react to changes. They aren't like the factory ones where a specific thing is always going to be there. This is why this is truly innovative. And of course, as you know, this is just the beginning. So that means that, you know, in the next 10, 20, 30, even 40 years, I can't imagine what these robots are going to look like. Especially since we do know that not only are tester investors super long term holders, but there are billions and billions of dollars now flowing into the AI industry and of course robotic, because people are predicting that these industries are going to be worth trillions and trillions of dollars. So it will be interesting to see how these robots are going to be in terms of where they head. And also you can see that the Tesla bot does actually have really, really sturdy balance, which is something that is very important because robots, as you know, do tend to fall over quite some time. Now the reason I find this very, very interesting is not only the fact that was Elon Musk being ridiculed for his previous performances, in terms of the Tetlo Optimus not being ready, it's shocking to see that they managed to combine this with AI so quickly based on what we've seen so far and of course now adding vision. Then of course we had Meta produce a whole host of many different things related to AI. There is so much stuff that I'm not sure if I'm going to include it in this entire video, but what you're about to see is Meta collaborating with Ray Ban and they've decided essentially created something that is very, very futuristic. And not only that, the price of this product is relatively cheap to the point where your me could actually purchase this item. So take a look at this. I think this is really cool. And not only is this just a standard product, it is also one of the new AI driven products. We are starting to see a whole wave of not just AI driven applications and software, but physical products that now have LLMs on board, including vision and of course conversation, being slowly rolled out into our environment. I wonder if these products are going to be just a fad or they're actually going to be something that become part of our future in terms of what we use on a day to day basis. So take a look.

The next generation of Ray Ban Meta smart glasses. These are the first smart glasses that are built and shipping with Meta AI in them. Starting in the US you're going to get this state of the art AI that you can interact with hands free wherever you go. We're going to be issuing a free software update to the glasses that makes them multimodal. So the glasses are going to be able to understand what you're looking at when you ask them questions. So if you want to know what the building is that you're standing in front of, or if you want to translate a sign that's in front of you to know what it's saying, or if you need help fixing this sad leaky faucet, you can basically just talk to Meta AI and look at it and it'll walk you through it step by step. How to do it. We built in one more feature into these smart glasses. You are going to be able to live stream to your friends and followers from your glasses.

Everybody is ready to race and I am getting ready to. Let's go. Switching to glasses.

Being able to share what you're doing live with your friends and and followers while staying completely in the moment is the kind of thing that you can only do on, on smart glasses. So, all right, these Ray Ban meta smart glasses, we're launching them on October 17th starting at $299 and I'm really looking forward to seeing what you all think of them.

And of course, in the Meta Connect keynote speech by Mark Zuckerberg, we had some more information on Meta's AI chatbots. Now I'd like. It was interesting the kind of opportunity that he was going at here because it just shows that he's actually pretty decent at this strategy. Mark Zuckerbergs knows that ChatGPT is already the leader and probably going to stay there for quite some time. So what he's opted to do is he's opted to take aim and fire some shots at character AI, which is pointed out in another video by AI explained. So essentially what we have here is Mark Zuckerberg having AI chatbots that contain the personalities of celebrities. So already leveraging the platform's fame, which with Instagram and combining that with open source large language models potentially like llama 2, depending on what they're running on, he's carved out this unique opportunity where Users are going to be interacting with these celebrities in this AI chatbot style. Now this has already proven to be quite popular already with companies like character AI being valued at $5 billion. And those are just random characters from around the Internet from ones that the platform have created to user generated ones. So with that being said, take a look at this because I, I personally wouldn't use this because I would rather just ask ChatGPT or figure out the information myself. But I do think that this is going to have somewhat of a large appeal to maybe a niche audience or maybe a very large audience. And I think that this is definitely showing that Meta isn't one of those companies to take lightly. Then we have from image to live website using gpt4vision and replay in less than a minute. So this is pretty crazy because as you know, GPT4 vision has only been around for about a couple of days now and, and it's already showing what the possibilities are. So this is just like the start. And I mean, this is the thing, okay, like people aren't really understanding what vision enables this large language model to do. I mean, you have to understand that this large language model is as smart as some of the smartest minds that have ever lived according to IQ tests. But the crazy thing is that now combined with vision, you can do many different things. Now I was watching a video where the possibility of recursive self improvement was discussed. So for example, let's say for example, something. So for example, let's say GPT4 in this example that you're currently watching on screen, you get it to design a website based on an image, it writes the code, then of course it can look at that image and then make a judgment call on whether this actually looks like what it was designed to do in the first place. And not only that, there are going to be a ton of different applications that are going to be built on top of GPT4 vision. So rather than people just using it to realize if their egg is well cooked or not, you can literally use it to build various different applications and software. And it's just really, really surprising because now we're at the stage where we've moved from just large language models to large language models that are going to have vision. And here's the thing as well is that now that GPT4 has vision and you can literally create a website in seconds from just taking a picture at it, we're going to see other competitors also move into this space too. So if we take a look at this example Right here you can see that GPT4 with a vision is able to decipher some text that is written in poorly overdone handwriting. It's pretty old and some people may not be able to decipher that, but with GPT4, with the vision, it's able to do that pretty well. So the applic are just pretty crazy. Of course, we talked about healthcare applications where GPT4 divisions. If we get one that is trained on a specific number of health issues, for example, certain spots, certain rashes, certain things that only a large language model or large vision model, or an AI that has seen millions of images is, is largely going to be more accurate than a doctor. There's maybe only seen a few hundred or maybe a few thousand. One of the last ones of GPT4 vision that I do want to show you is this example right here. And I want to show you this because it doesn't scare me, but it is very intriguing. It did scare me at first because there was one account on Twitter that I saw that did actually showcase as to why this is pretty crazy. You see, we always talk about how AI is not that thing, it's just a bunch of code that is outputting some basic code. And now we can see that this is a large language model that can clearly understand advanced concepts that are embedded in certain images. Okay, so you can see that it says right here. I'm glad we all agree. Oh, aha. I'm glad we all agree. So it says that this image portrays the concept of group dynamics and perspectives, and it really does understand exactly what is going on here. And I do think that, you know, if it was just a vision transformer, like it was just something that looks at an image and says what it is, it would have said that, you know, this is just an image with some characters on screen, but it shows, okay, that it clearly understands exactly what is on each panel and, and it describes it in detail. It says the last panel shows that after some discussion or thought, all have come to a consensus or shared understanding and they envision the same shape and the caption reaffirms. I'm glad we all agree. So I do think that while some people have said that these AIs aren't sentient and they aren't going to kill us or whatever, that is quite valid. But I do think that these AIs are definitely smarter than they do presume and we aren't being told the full picture in terms of their full capabilities, because something that is able to understand these concepts and Then present it in such a clear manner is is definitely something that is smarter than we do think. Then of course, this is absolutely crazy. I don't know why anyone hasn't talked about this, but this, okay, I think this is game changing. Okay? So pay attention. All right. This is what we call video direct to GPT. Consistent multi scene video direction via large language model guided planning. So it says. Although recent text to video generation methods have seen significant advancements, most of these works focus on producing short video clips of single event with single background meaning. Meanwhile, large language models have demonstrated their capability in generating layouts and programs to control downstream visual modules such as image generation models. This raises an important question. Can we leverage the knowledge in these LLMs for consistent long video generation? So forget the jargon. Basically what they're saying is can we use large language models to perfect text to video? And essentially what they've done is they've input a scene that is done from a text to video model. Then they use the large language model, I'm guessing with a vision transformer to analyze the scene and see exactly what it is. Then they described it. So it's multi sentence, the multi scene video. So they've explained all the scenes. And then of course, okay, what they do is they use that large language model to recreate that video or to make that video properly. So you can see that it says fails to keep mouse throughout all scenes. And then video GPT rs, the mouse looks consistent through all scenes. So essentially they have an AI. So that essentially gets the data from the video and then ensures that this video is correct. And I think this is the right step that we need to move in. Because of course text to video, you know, it's great, but this is where it is, guys. So I think text to video in the next couple of months is going to have a huge update to where it's going to be much more accurate. And this is why I said it's a huge breakthrough, because I haven't really seen many people talk about this. Because of course text to video isn't getting that much information. But you can see model scope, text to video is very different to video director GPT. Okay? And I think that once these Vision Transformers and Maybe even with ChatGPT, because ChatGPT could be something, you know, I guess with Dall E3 maybe you could get Dall E3 and ChatGPT maybe combine it with some of the things. I'm pretty sure as long as ChatGPT can create images, with Dall E3, it can definitely create videos. As long as it can continually analyze those images, I'm sure it could create videos that are of a high quality. So that is going to be pretty insane and I cannot wait for the more advancements in that. Then of course we had genmo AI do Replay, an AI model that can generate stunning videos from text. And I think it is really good because not only is it smooth, the videos do look a lot better than some of their counterparts. Now, of course, like we just talked about, video GPT along with some other models that are going to be developed, is going to be insane because here's what happens once someone releases a research paper. Essentially these other companies, they start to get to work, they start to build on that research paper and they start to build applications and that you can use. So I think this week has been absolutely insane. There of course is some things that we did miss. If you do want us to do a newsletter where we can give you very best, the very best news and just dumb it down or essentially just explain it in as much clarity as possible. Because lots of the time when you are looking at this stuff, there's a lot of technical jargon that does get in the way of you understanding exactly what is being done here. So with that being said, if you enjoyed this video, we'll see you in the next one.