NVIDIA researchers used AI to speed up muscle and tissue simulations for virtual characters by over 100x. The method learns to map coarse, fast simulations to high-resolution outputs that match detailed references. It handles new expressions and people, though it can wobble on completely unseen movements.
I feel delighted because virtual characters in games and animated movies are about to get a lot more realistic. Why? Well, when animating them, instead of just trying to do what looks good, we can run simulations instead to find out how they would move in reality. Nothing new about that. We've been doing this for decades now. However, get this. These simulations can go down to the level of muscles. That is incredible. But there is one big problem. What is the problem? Well, cost. It takes a long time to compute. Just imagine we are going down to the level of muscles and soft tissues and compute the interactions of millions of little triangles. And we repeat that for every time instant. And you have to make sure to avoid all of these awkward collisions too. Yes. We still have to wait a lot for a high quality result. So this is hopeless for a real time application. But wait, remember these super resolution techniques out there? Where in goes a coarse image and out comes a really detailed result. Synthesizing details with AI techniques is now possible. But this is for Images, not for 3D models. And are you thinking what I am thinking? Oh yes. Why not do super resolution, not in 3D? Yep, that is exactly what this paper does. This is super resolution for C simulations. In goes this really coarse blocky result. And what comes out? Probably nothing good, right? Let's see. Oh my. That is absolutely fantastic. So how much faster is this? 10x15x? Nope. This makes the whole process more than a hundred times faster. Yes, something that took an all nighter now only takes five minutes. And. And what used to take a minute now takes less than a second. That is amazing. So that was easy, right? Just throw an AI at the problem and there we go. Well. Nope, not so fast. Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai episode 942. If you just try a core simulation and upscale it, you will get a completely different result. Because the topology of your model is different than the real detailed model. You are simulating a completely different person. How would that be useful? But they did it somehow. So how. Here is a genius idea. Yes. AI you are allowed to do the super resolution, but at the same time use the knowledge of what you have learned from higher resolution simulations. But wait, wait, wait. We said that with the naive technique you are getting a simulation of a different person. With this technique, do you get the actual person or just something that moves like that person? Well, let's have a look together. Here is the course input. And here is the result of the new technique. And then here comes the moment of truth. Let's compare it to a high resolution simulation that takes forever. That will be the reference. My goodness, it is so close. There are differences, yes, but this is so close. It is probably reasonable to say that this is exactly the person we want. And in the textured results. Oh my. Try to not look at the teeth. I know it's hard, but you can do it also very close. Just avoid the teeth. So it works by looking at pairs of low and their corresponding high resolution simulations and tries to learn as much from them as possible. But then what about unseen expressions? Can it do that? The paper is really selling the fact that it generalizes to unseen expressions. And I have to be honest, I am not so so keen on these results. They are a little too wobbly for me. But wait a second. This result hides a secret. I am seeing something that is incredible. Do you see it too? There is no nose deformation information in the training data of this system. And when the nose moves in the input, it also synthesizes realistic tiny little deformations in the output. That is huge. I mean, not the nose. The phenomenon that it is able to predict. These tiny little deformat animations of the nose induced by the mouth movements. Insanity. And it gets better. It also generalizes to new people and new kinds of movements. Yes, this is a virtual world. So you can run your silly little experiments all you want. The perks of being a computer graphics AI researcher. It is the best. And we even get paid to do this. What a time to be alive. And we are still not done yet. I would like to send a huge thank you for the scientists behind the paper. Because the paper and the source code are available for free for all of us. And it's crazy that almost nobody is talking about this paper. You almost only hear about here on 2 Minute Papers. And now we invoke the first law of papers. The first law of papers says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line. For instance, I would love to have this concept applied to general computer animation. For example, example. This other work focuses on multi character interaction. I mean, if you need some help moving to a new house, I would ask for nobody else. Then these little scholars. Look, I bet they won't even complain when the couch crushes their legs. Now imagine if all of them were fully simulated characters, down to the level of individual muscles, facial gestures and everything. All with just one AI. And all that in real time. It is coming. And I cannot wait to create create my own little virtual worlds with it. To run your own experiments on an Nvidia gpu. Check out Lamda. I use it myself regularly for these videos. Look at that. You can generate high quality images in less than a second per image. I did a ton more of them and paid less than a dollar for all this. Crazy. Seriously. Try it out now@lambdalabs.com papers or click the link in the description.
