Dr. Zsolnai explains how voxel cone tracing made real-time ray tracing possible by simplifying the problem for existing graphics hardware. The breakthrough paper, published at Pacific Graphics, influenced games and research despite light leaks and imperfect accuracy. Key lesson: align with hardware makers or find alternative niches.
This is episode 900 wow. And I would like to celebrate it with you with an iconic paper that almost none of you fellow scholars have heard about yet. It is incredibly important to me and it changed the way I do research to this very day. A few weeks ago I was at the Nvidia headquarters and when showing this image, many of you asked which paper we are holding onto there. And today you get your answer to what the paper is and why. Three incredible lessons follow. Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. This is a work called Voxel cone tracing and it is in the area of ray tracing that is simulating the path of millions and millions of light rays to create a beautiful photorealistic image. This works well. However, there is a problem. The problem is that this process is too slow. It can take from minutes to even days to finish. This is also a mature research area where it is incredibly difficult to say something new and valuable. And just beating the state of the art techniques by even 10% is almost impossible and takes years and years of research. And now hold on to your papers, fellow scholars, because this paper appeared 13 years ago and it claimed that it is able to perform ray tracing in real time. This can take days to compute. And you say real time? When I saw it, I couldn't believe it. And it was completely right. Unbelievable. So how? There were two key ideas in the paper. One was that we should not think about the world as a continuum, but we would dice it up into little boxes that we call voxels, which greatly simplified the problem. By the way, it is not trivial at all how to do this. Right? And finally, this could take ray tracing and bring it from academia to real time video games magic. But the second key insight totally changed my perspective on research. So what was it? Well, it was designed in a way that people could implement it to run it on their graphics cards extremely efficiently. It was likely designed with the graphics hardware at the time in mind by, for instance, using something called the geometry shader. And this is the question I was asking then, who knows best what the hardware of the future will look like? Well, of course, the people who make the hardware, for instance, Nvidia. And where are some of the authors from on the paper? Oh yes, there it is. And this is what got me stunned. I was trying to write mainstream ray tracing algorithms to outperform the state of the art. While I was in the dark, I was competing with people who know what was coming. That is lesson number one. So what exactly? Well, either to start working for one of these companies or to start doing research that is a bit away from these mainstream ideas most people are working on. There are people who know what's coming and I don't. So we started doing work on subsurface light transport. It powers this beautiful footage here and I generally pivoted into synthesizing real looking virtual materials. It was super fun, it was reasonably successful and I have this paper to thank for it. So thank you so much. Voxel Cone Tracing now lesson number two. Sometimes drastically simplifying the problem can still give you the solutions that you want. Also an amazing lesson and one more look where it was published. Pacific Graphics. Have you heard about it? No. Well, it is one of the venues in computer graphics. It is completely fine, but it is not a top tier venue. Not even close. So why is that? Why would such an incredible paper not be published, for instance, at a higher tier conference? My guess, which is just a guess, is that it did not have the mathematical rigor that is required for those computer graphics conferences. And we almost missed out on this brilliant piece of work. So this also teaches us that incredibly amazing papers can be published anywhere and sometimes not be published at all. These are the three incredible lessons that I learned. Now not even this technique was perfect. It introduced some of its own problems. Light could leak through tricky geometry and it was not 100% accurate, but in my opinion, often more than good enough. This is, in my opinion, one of the holy grail papers in computer graphics. I note that we talked a bit about this paper approximately 700 episodes ago. Leave a comment if you're one of the OGs and were around back then. So what do you think? What did you fellow scholars learn from this? Let me know in the comments below. If you're looking for an Nvidia GPU cloud service for large scale AI projects, check out One Click Clusters by Lambda featuring up to 512 Nvidia H100 GPUs with InfiniBand networking and hold on to your papers because with one click clusters you can access all these GPUs on demand with reservations starting at just two weeks. Try it out@lambdalabs.com paper or click the link in the description.
