Dr. explains how Nvidia's instant NeRFs train 20 times faster than older neural methods, and a new paper improves image quality on thin structures while staying fast. The catch: instant NeRFs still struggle with thin details and reflections, but combining them with separate material estimation could build better virtual worlds.
Dear fellow scholars, this is 2 Minute Papers with Dr. Today you are going to see virtual worlds like you've never seen before. This is done with something that we call nerfs. So what are nerfs? Essentially, we grab a smartphone and a set of images go in and reality comes out. When there is a gap between the images, it can synthesize all this information and it almost looks like reality. Absolute magic. Now, many of these are neural network based techniques and take quite a while to compute. However, later Nvidia published a technique by the name instant nerfs. As the name says, this trains incredibly quickly and is one of the best techniques in its class. And today these can be produced from a smartphone camera and can be generated in a matter of seconds. But there is a problem. Do you see the problem? Well, let me ask you another question. How do you spot a fellow scholar who watches Two Minute Papers? Well, of course, they are always looking at thin structures. Why? Well, look, so instant nerves are super fast, but have trouble with thin structures. And now hold on to your papers, fellow scholars, and let's look at how this new paper deals with the same problem. Oh my, that is outstanding. But wait, is it only better on thin structures or is it generally better than this previous technique? Yes, thin structures are still present, but the new one is significantly better in terms of quality. Wow. And this is just one year and one more paper down the line. Very impressive. Also here is a comparison against MIP Nerf. This is real good. So I bet the new one isn't better, especially that this previous one ran for about 20 times longer. Let's see. Whoa. Is that really possible? The new one is still better. 20 times faster and we don't even have to pay for it by giving up quality. This is still better. With this, we can now just grab a smartphone and scan a real place and make a virtual world out of it. This can be used for video games, video conferencing, animation, movies, you name it. Now, not even this work is perfect. You see, specular reflections like this display are not that faithful to reality. And if we look at severely under sampled regions, this can happen. However, just think about the fact that all this came just one more paper down the line. Now combine this together with this amazing light transport paper that can take photos of real world objects and create a super advanced digital material model with just the right physical parameters to replicate it. So NERFs can help with the geometry and light transport can help with the materials. And a potential combination of the two could create even more incredible virtual worlds and games for us in the near future. What a time to be alive.
This video has been supported by Weights and biases.
Look at this.
They have a great community forum that aims to make you the best machine learning engineer you can be. You see, I always get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. In this forum you can share your projects, ask for advice, look for collaborators and more. Make sure to visit wnb me paperforum and say hi or just click the link in the video description. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
