NVIDIA’s New AI Will Supercharge Gaming!

We are now in the age of AI where text to image is possible. We write a piece of text and out comes a beautiful image. Text to video is also possible. This is the same process with moving pictures. And now text to 3D is also possible. That is generating 3D models for computer games, animation, movies and so much more. But that is also the past. This has been possible because of these earlier research papers. But this new work, this is text to 3D in a way that you've never seen before. Have a look at these. My goodness. Ok, so what is going on? What are we seeing here? Dear fellow scholars, this is Two Minute Papers with Dr. Caro Zsolnai. Yes, this AI technique has looked at at millions and millions of objects and was told what they are. And thus it can generate new objects that are of higher quality than previous techniques. That is good, but it gets way better. What I'd really like to see here is when we ask it for things that are new and unexpected, I am reasonably happy with this. Campfire, Eaglehead and the Strawberry. And these are not the super high resolution geometries that you would immediately be able to use in a high budget game or an animation movie. But these are great starting points at the very least. However, I am more interested in this kind of thing. Oh yes, a chair that looks like a root. To me, this feels like a hint of creativity in a machine. Absolutely amazing. And additionally it can generate not just one, but a variety of textures onto these models. So if you get something that you don't like too much, not a problem. Just ask for a dozen more and find the one you like best. Loving it. But wait, it gets better. Oh my. Look at that. We see here that this is not just limited to generating one particular model. And here is where things get crazier. We can give it some lidar data recorded by these Waymo self driving cars and create 3D geometry for a virtual city that we can use in a video game to teach self driving cars in a game that will in a couple more papers be identical to the real world around us. You know, learn to drive there safely in the game and and then come out into the real world. Now hold on to your papers fellow scholars, because there is more magic here. The paper proposes a hierarchical structure. So it sees the screen on three different levels of resolution, from coarse to fine. Hmm, why is that useful? Why do we need that? We are just using one, aren't we? Well, not quite. Have a look. And oh yes, here we have our answer. The answer is diffusion. Fantastic. But what does that mean? It means that it works kind of like some of the previous methods, where we start out from a bunch of noise and over time reorganize this noise to resemble an image. We call this diffusion. Now this does diffusion with 3D geometry by using voxels, almost like little LEGO pieces. I love how beautifully this animation demonstrates it. Basically, at first it starts out from something really coarse, big Lego pieces. And now through a subdivision step, the big LEGO pieces are cut into smaller pieces. But this is still not that useful because it looks similar. However, now comes the pruning step where the excess fat gets cut away, then subdivide again. And if you do this through many steps over time you get more and more intricate geometry. So this is not yet able to do this that many times, but just imagine what we will be capable of two more papers down the line. Note that it also has some more information than just the LEGO bricks. For instance, note normals for geometry information or semantics, which means that we highlight what part of the screen is what. For instance, this is supposed to be a tree, this is the road, and these are buildings. And now hold on to your papers, fellow scholars, because it does all this magic within less than 30 seconds. That is absolutely amazing. What a time to be alive. Now, this is still not perfect. For instance, if you have a really complex prompt, it still doesn't do too well with those. So what do you think? What would you fellow scholars use this for? Let me know in the comments below. If you're looking for an Nvidia GPU cloud service for large scale AI projects, check out One Click Clusters by Lamda, featuring up to 512 Nvidia H100 GPUs with InfiniBand Networking. And hold on to your papers, because with one click clusters you can access all these GPUs on demand with reservations starting at just two weeks. Try it out@LambdaLabs.com paper or click the link in the description.