YouTube commentary (Two Minute Papers) by Dr. Zsolnai describing a recent NVIDIA-related paper that controls chaotic simulations to produce highly unlikely, realistic-looking outcomes. The key technique manipulates visibility/occlusion (and appearance like recoloring) so unseen or barely-seen parts can be altered; the method uses screen-space projected-area math, is embarrassingly parallel, and often runs in seconds. Demonstrations include thousands of cubes, colored balls, deformable bodies, and text-shaped particle assemblies; limitations include primarily single-view validity. The video also notes the work is handcrafted (non-AI) and includes a sponsor mention of Microsoft Azure AI.
Chapters
My goodness. I somehow missed this paper and just found it. And I am so happy that I did because it does something that I previously thought would be obviously impossible. You see, this work creates magic tricks. It is the master of illusions. So what is the magic here? Well, consider this computer simulation with thousands of cubes. We throw them and whatever happens, happens. Ok, and now check this out. Throw it again. And look. Seemingly the same simulation happened, but a pattern was formed. How? Well, magic. And this is just the simplest version of what this technique can do. Now let's get those colored balls, let them fall and expect them to be randomly everywhere, which is exactly what happens. Now let's run this experiment again where they randomly fall again. Wait a minute. This amazing technique was at work here too. What you see here is normally almost impossible to happen. And it gets better. Now let's switch the balls for deformable bodies. The physics of simulating those is so much more complex than that, is probably too much even for this technique. And look at that once again. Structure a controllable simulation. I absolutely love it. And you will see that this is going to get even better and better. Yes, I'll tell you how it works soon, dear fellow scholars. And this is two minute papers with Dr. Zsolnai. Now let's play dice with this pyramid in this completely truthful and unrigged game with the reasonable result of every single dice being a six. I would like to see the face of a poker room manager upon seeing this. So cool. And the best part is that these simulations look real. These little objects don't suddenly make decisions that don't make sense. Everything looks realistic all the way through. That is the hardest part. Not the cheating, but making it look real. I watched this several times, picked a different ball each time and I couldn't believe how realistically they moved. And yet they formed a structure that is so unlikely to form, it is almost impossible to happen naturally. But how does this work? How does this wizardry happen? I'll tell you after this one, which is my absolute favorite. This is a piece of text written using about 5,000 chords. And when we drop them, we get a huge mess, right? Not quite. Look, we can think about almost any text prompt and it will control the simulation into that shape. That is unbelievably good. So what is the trick? There are many tricks, but perhaps the most important one is the same as with every magic trick. And that is the magic happens where you are not looking. The algorithm is playing with visibility. What does that mean? Well, if there is an object that you can't see because it is occluded, then the magic can happen there. There you can freely alter the path for a more favorable placement. And of course no one will see it. But wait, if no one can see it, we can change not only the path of the object, but the appearance of the object to more magic. Yes, their paths are not being manipulated at all. They are being recolored as they move. And in the case of the balls, which are all visible, the recoloring can happen during chaotic movements where these tricks are really hard to notice. Note that there are more tricks for these cases with fewer occlusions described in the paper. It is beautifully written and contains the mathematics of computing screen space projected areas. What does that mean? It means that if you know how much of the object is visible for the viewer through the camera, the if that surface area is huge, we have to behave. But if there is minimal visibility, then the magic can happen. That is a genius decision. Why? Well, because this process is embarrassingly parallel. What does that mean? Well, if you ask one person to make you coffee, it happens quickly. But if you want it faster and ask nine people to make you coffee, and it is not going to be nine times faster. If anything, they might be holding each other up so much, it might even be slower than asking one person. However, embarrassingly parallel means that this visibility computation problem can be trivially separated into many smaller problems which can run in parallel. Yes, it can run super quickly. This is coffee that can be made by nine people nine times faster. Loving it. So how quick is this? I guess it still takes hours and hours to do all this, right? Nope. Now hold on to your papers, fellow scholars, because it often takes less than five seconds. Wow. And that is the unoptimized code. And it gets even better. The technique is designed in a way that it not only does that in seconds, but if we changed our mind and would like a different piece of text, there not a problem. Just do it and it will give it to us almost immediately. Now, limitations, of course, with these tricks hiding in the shadows, the limitation of the technique is that it typically works only from one view. If we change the view, what was invisible now becomes visible and the magic is gone. But no matter, because we can recalculate for that case too. And get this. I think you are going to love this one. This is an earlier paper from Doug James and colleagues. And that one was also pure magic. So this one looked at a computer animation and then it synthesized sound. Incredible. Except that it is not at all what has happened? The technique took a sound sample and synthesized a plausible simulation as to what this sound might have looked like. Absolutely incredible. And it can even create small music videos too. We talked about this approximately 700 videos ago. And here comes the kicker. All of these techniques are completely handcrafted. No AI anywhere. This is powered by pure human ingenuity. What a time to be alive. So I think everyone on the planet should know how incredible it is that techniques like this exist and that brilliant researchers are giving this knowledge away for free. So does everyone know? Wait a minute. My heart is breaking because this paper has been seen by just a little more than 100 people. And this is why two minute papers exist to show you brilliant techniques like this that unfortunately no one is talking about. Subscribe and hit the bell icon if you wish to see more like these. And I would like to send huge thanks to our new partner for supporting us on this journey. Super grateful. If you wish to run your own experiments in the cloud, make sure to check out Microsoft Azure AI. Azure AI is a powerful cloud platform that offers you the best tools for your AI projects with responsible AI built in. And here comes the best part. You can even try it out for free through the link in the video description.
