Dr. Zsolnai explains Gaussian Splatting, a technique that reconstructs 3D scenes from images up to 20 times faster. What used to take 40 minutes now takes 2 minutes per object, runs on a single consumer GPU, and works in Unreal Engine in real time.
Because so many of you fellow scholars have requested it. We had a video on one of the best papers of the year in my opinion, which is a work called Gaussian Splatting. And I am absolutely speechless by how quickly things are unfolding around it. I think you will be too. So what does this do? Well, just imagine grabbing your smartphone, taking a few images or videos of real places, and then we could create a virtual version of them and play in them in real time. And it does exactly that and these scenes come out really, really detailed. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. And the key thing here is that this is a handcrafted technique that is easy to implement. And now some publicly available applications already have it. This one does. Yes, the dream is coming true. We might just play in a virtual copy of the real world very soon. Especially that it is available in Unreal Engine 2 through a paid plugin. Though I am sure that free versions will pop up very soon too. In fact, things are going so quickly, I wouldn't be surprised if by the time this video appears, it is already available for free. And it gets even better. Way better. For instance, we know that text to image is possible, but we are talking about 3D virtual worlds here, so. Oh my. It has been just a few months of this paper being out and can that really be? Follow up papers are already popping up. Wait a second. This is text to 3D. We write something and out comes 3D geometry. Our last episode was on exactly that and it is already being surpassed. What is happening? Wow. Well, what is happening is that Gaussian splatting is already being used to improve these techniques in many cases. I see more detailed geometry being produced and all this in just a couple of months. And one more paper down the line. So cool. But it doesn't end there. Not even close. Goodness. Here is another paper which is able to perform image to 3D. One photo goes in and out comes 3D geometry. I already see that it converges to a solution much quicker than earlier neural radiance fields based solutions. That is fantastic news. But come on, what can it do? Oh, it really does it. In goes the image and out comes the 3D model. And it gets better. Then we can even really put these little models into our virtual world and animate them. And all this process takes how long exactly? The previous paper we looked at took 40 minutes. So what about this one? Now hold on to your papers fellow scholars, because it is two minutes. This is incredible. A 20x speed up in just one paper. I really don't know what to say here. In the first step, it lines up the Gaussians little mathematical lumps to match the shape of the character or object. And then in the second step, it slowly refines the textures as well. And this one runs on a single consumer graphics card. This is what does it in 2 minutes per object. Haha. That work is a real match for two minute papers. No wonder these papers caught the imagination of you fellow scholars. It gets even better. It can even reproduce photorealistic faces reasonably well with just 25 photos as an input. And whoa, here you can even animate these inputs. So not only creating virtual worlds from the real world, but we can even let our imagination run wild and create absolutely anything out of it. I love it. Now just one more thing. I know some of you fellow scholars are asking ok, but what is Gaussian splatting? What does that even mean? Now the Gaussian part means that objects in this scene will be represented as a sum of many little waves. And splatting is a way of throwing these little waves from the 3D virtual world at our 2D screen and computing what they should really look like. And this simple but effective idea is taking the world by storm and a handcrafted technique power powered not by AI, but by human ingenuity. Once again, the papers coming alive in the real world. What a time to be alive. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H100 instances for just 1. 99 per hour. Yes, 1, 99. And they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
