This is a YouTube commentary (by LARRUZ) reviewing a recent graphics paper that introduces a non-neural, handcrafted rendering technique—based on Gaussian "blobs" and 3D-to-2D splatting—that claims over 10× faster rendering than NVIDIA's Instant NeRF while achieving higher quality and better handling of thin structures. The presenter highlights two core ideas (representing scene geometry as a sum of Gaussians and rendering by splatting primitives instead of per-pixel sampling), notes current limitations (specular/reflection fidelity and memory use), and credits shared first author Ben Hart. The video also includes a sponsorship mention from Weights & Biases.
Questions this video answers
More than 10× faster rendering time.
No — the presenter emphasizes it is a handcrafted (non-neural) graphics technique.
Ben Hart.
Chapters
Because so many of you fellow scholars have requested it. Today we are going to have a look at one of the best papers of the year. An incredible work that I hope will revolutionize the creation of virtual worlds for movies and video games and more. Just imagine grabbing your smartphone, taking a few images or videos of real world places, and then we could create a virtual version of them in and playing them in real time. That would be amazing. Just imagine that we could drive on the streets of San Francisco and la, go on a virtual hike and see what it would look like. And a ton of other really cool things. Techniques that involve rendering radiance fields, for instance. Nerfs have this as a goal and a ton of them already exist. But with those, we have two huge problems. One, they are not fast enough for huge scenes in full HD or higher resolution. Getting a real time solution is tough. It typically goes slower than that. And two, of course, thin structures. It's always the thin structures that are the problem. Now these are quite plentiful in scenes in the real world. Plant leaves, cables, fences, bicycle spokes, that kind of thing. All thin structures. Yes, all of those are problems. Hmm. Now I see big words here. Because this new technique promises results more than 10 times faster in terms of rendering time than Nvidia's excellent instant nerf technique. So of course the quality has got to be lower, right? Wait a second. Look here. Are you seeing what I am seeing? The thin structures indicate that this is not only more than 10 times faster in terms of rendering time, but it also promises higher quality results than the previous methods. Wow. And upon further inspection, this is so much cleaner, so much better. And it is ridiculous how much better it is in quality than some of the other quite capable previous methods from just a year ago. And yes, that's right. Now we can get a virtual copy of the real world with difficult thin structures going in full hd. And yes, real time. Much faster than real time. My goodness. So good. And here comes the punchline. Hold on to your papers. Because we should likely not call this technique a nerf variant, but because this does not use neural networks. Yes, this is a good old handcrafted computer graphics technique.
Wow.
So how does all this wizardry work? We take out the super capable neural networks and replace them with a little human ingenuity. And it gets better. How? Well, this has two key ideas. Now we have a 3D world at hand. We which has to be represented on our 2D screen. Now, the Gaussian part means that objects in this scene will be represented as a Sum of many little lumps. It is a bit like throwing a small pebble into water. The pebble might be a small little point, but the waves it creates spread out around this point. Sounds good, right? Well, not so fast. Unfortunately, in the 3D space, many of these waves overlap and it is unclear how exactly they can be transferred onto our 2D screens. And splatting is a way of throwing these little waves from the 3D virtual world at our 2D screen and computing what they should really look like. As an additional advantage, these waves concentrate around solid objects, and thus this technique does not have to waste time computing parts of the scene with lots of empty space. Lovely idea number two. Typically, NERF algorithms are interested in going through every single pixel on your screen, and instead this one is interested not in pixels, but in the primitives objects in the scene. Now, this concept is not new. It has been used in computer graphics for decades. But how it is applied to this problem is the cool and interesting part. All in all, this is a simple algorithm. It is really fast and we don't have to sacrifice quality. I mean, look at that. I would like to send huge congratulations to the entire team and to the shared first author Ben Hart, who is my former colleague and a good friend. Now, not even this technique is perfect. For instance, rendering specular reflections is not perfect. These light transport effects look much more crisp in the real world. Also, this first unoptimized version also consumes quite a bit of memory. However, just imagine what we will be capable of just one more paper down the line. I am a computer graphics researcher by trade. I specialize in light transport, ray tracing if you will, and I am always so happy to see a good graphics paper rip. And this one. This one is something different. Also, it just came out a few days ago and look how it already captured the imagination of some fellow scholars there. What a time to be alive. If you enjoyed this episode and would like to see many more amazing papers like this one, consider subscribing to the channel and hitting the bell icon.
This video has been supported by weights and biases. Check out their recent offering Fully Connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders, and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. Fully Connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference, and more. Make sure to visit them through WNB Me papers or just click the link in the video description. Our thanks to WeightsAndBiases for their long standing support and for helping us make.
Better videos for you. Thanks for watching and for your generous support and I'll see you next time.
