NERFIES: The Selfies of The Future! 🤳

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to get a taste of how insanely quick progress is in machine learning research. In March of 2020, a paper appeared that goes by the name Neural Radiance Fields nerf. In short, with this technique we could take a bunch of input photos, get a neural network to learn them, and then synthesize new previously unseen views of not just the materials in the scene, but but the entire scene itself. And here we are talking not only digital environments, but also real scenes as well. Just to make sure once again, it can learn and reproduce entire real world scenes from only a few views by using neural networks. However, of course, NERF had its limitations. For instance, in many cases it has trouble with scenes with variable lighting conditions and and lots of occluders. And to my delight, only five months later, in August of 2020, a follow up paper by the name Nerf in the Wild or NERF W. In short, its specialty was tourist attractions that a lot of people take photos of. And we then have a collection of photos taken during a different time of the day. And of course with a lot of people around. And lots of people of course means lots of occlusions. NERF W improved the original algorithm to excel more in cases like this. And we are still not done yet because get this, only three months later, on 2020, November 25, another follow up paper appeared by the name Deformable Neural Radiance Fields D Nerf. The goal here is to take a selfie video and turn it into a portrait that we can rotate around freely. This is something that the authors call a nerfy. If we take the original NERF technique to perform this, we see that it does not do well at all with moving things. And here is where the deformable part of the name comes into play. And now hold on to your papers and marvel at the results of the new denurf technique. A clean reconstruction. We indeed get a nice portrait that we can rotate around freely and all of the previous NERF artifacts are gone. It performs well even on difficult cases with beards, all kinds of hairstyles and more. And now hold on to your papers, because glasses work too. And not only that, but but it even computes the proper reflection and refraction off of the lens. And this is just the start of a deluge of new features. For instance, we can even zoom out and capture the whole body of the test subject. Furthermore, it is not limited to people, it also works on dogs too, although in this case we will have to settle with a lower resolution output. It can pull off the iconic dolly zoom effect really well. And amusingly, we can even perform a nerf ception which is recording ourselves as we record ourselves. I hope that now you have a good feel of the pace of progress in machine learning research, which is absolutely incredible. So much progress in just nine months of research. My goodness, what a time to be alive. This episode has been supported by Weights and Biases. In this post they show you how to use their tool to analyze chest X rays and more. If you work with learning algorithms on a regular basis, make sure to check out weights and biases. Their system is designed to help you organize your experiments and it is so good it could shave off weeks or even months of work from your projects and is completely free for all individuals, academics and open source projects. This really is as good as it gets and it is hardly a surprise that they are now used by over 200 companies and research institutions. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.