NVIDIA's New AI Makes Cars Fly...Sort Of!

Now let's teach cars to fly with an AI. Sort of how? Well, imagine taking just one image of the real world. And then this new AI would imagine it not just as one image, but as a scene where you can walk around. And it gets better. It even understands that water should reflect its surroundings. Wonderful. And oh my, look at that. I can't believe this. It not only imagines the reflections off of the water, but even the water moving itself. I spent years and years trying to learn fluid dynamics, to write liquid simulations like this. But now through the power of AI, you just get that for free. That is kind of mind blowing. Wow. So what else can it do? Well, you will see soon that it can even teach a car to fly. What? How? This builds on Nvidia's Cosmos, which was first unveiled here on two minute paper papers. And we showed how it can generate lots and lots of useful videos. Why? To train self driving cars and all kinds of useful robots in a video game. And when they demonstrate that they are safe, they can be tried out in the real world. So now let's not use an image, but a video as an input. Nice. Once again it demonstrates that it understands the scene. And now fellow scholars, let's change the camera trajectory. But not in the way you think. Oh my goodness. Is that an error? Nope. They are indeed going to make this car fly, aren't they? Let's have a look together. Wow, that is absolutely incredible. Loving it. And this is really tough. The AI needs to imagine and continue this world. So now we see things that we didn't see before. Crazy. All dreamed up by the AI. Crazy. And when the original footage ends, suddenly the AI imagined footage appears and the jump not be jarring. You can't just suddenly jump to a poor quality video. Man, I just want to fly everywhere with this. So cool. So this way you can just take one piece of footage and create many what if situations. What if we change lanes? Well, you can generate all this information without leaving the house. And train an AI in a simulation safely until it demonstrates that it is good enough to handle a variety of difficult situations. Now that would already be good, but it gets even better. How? Well, hold on to your papers, fellow scholars. Because this one can do not only this self driving thing. Not even close. This one seems like it can do everything. When given an image of this selfie doggy, we can now look behind it and it looks absolutely seamless. And I am a light transport researcher by trade, that is ray tracing if you will. So I couldn't help but marvel at the reflections here, which are just sublime. Yummy. And look at that caustics. Caustics are beautiful, bright patterns of light, you see when light bends through or bounces off curved surfaces like water or glass. Once again, writing a computer program that simulates this took me years and years of studying. And once again, here you get it for free. It even understands transparency and thus did really well with the dust particles here. Just imagine all the creation we will be able to do with this. Although wait, in this scandal scene you see that it is unlikely that the intention of the artist was to fry the horns of the animal. Oh yes, you see that its understanding of physics is a bit lacking. We talked about this in a previous episode, so why is that? Why not just add more training data so it finally understands. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. Well, interestingly, that wouldn't work. And the reason it wouldn't work is because these systems are trained to generate footage, but not necessarily to understand it and answer questions. Thus, in a way, there are systems out there that can create absolutely beautiful videos, yet they don't quite understand what they are doing that is so interesting. So this can generate entirely new parts of cities, but it does not necessarily understand how a city should function, just roughly what they look like. And the next generation of AI techniques will have a proper understanding of that too. What a time to be alive. And the pace of progress in AI research is just absolutely incredible. Just imagine what we will be capable of just two more papers down the line. For instance, remember this representation is a point cloud. Point clouds can now be converted to 3D geometry in a way where you can even see them being born from nothing. Now, not even this technique is perfect. You've heard about some limitations already. The resolution could be a bit higher. I am sure it will be soon. Also, this muscular person seems somehow a bit off to me. And the full paper on how to do all this is available and the source code will be available too, for free for everyone. You know, everyone today is talking about this new menus AI. Yes, you can get a lot of views with it, but really all we know about it is it almost nothing. Speculation and then more speculation about the speculation here. We work differently. We are scholars, we like research papers, we send joy and positivity into the world and show people how amazing research is. So please subscribe and hit the bell icon if you appreciate that and would like to see real science explained beautifully and simply. Well, hopefully I am trying my best here. Here you see me running the Full deep seq AI model through Lambda GPU cloud. The full 671 billion parameters running super fast and super reliably. This is insane. I love it and I use it on a regular basis. Lambda provides you with powerful Nvidia GPUs to run your own chatbots and experiments. And it's the best. Seriously. Try it out now@lambdalabs.com papers or click the link in the description below.