AI Makes 3D Models From Photos | Two Minute Papers #122

Dear fellow scholars, this is two minute papers with what if we tried to build a generative adversarial network for 3D data? This means that this network would work not on the usual two dimensional images, but instead on three dimensional shapes. So the generator network generates a bunch of different three dimensional shapes. And the basic question for the discriminator network would be, are these 3D shapes real or synthetic? The main use case of this technique can be. And now watch closely. Taking a photograph from a piece of furniture and automatically getting a digital 3D model of it. Now it is clear for both of us that this is still a coarse, low resolution model. But it is incredible to see how a machine can get a rudimentary understanding of 3D geometry in the presence of occlusions, lighting and and different camera angles. That's a stunning milestone indeed. It also supports interpolation between two shapes, which means that we consider the presumably empty space between the shapes as a continuum and imagine new shapes that are closer to either one or the other. We can do this kind of interpolation, for instance, between two chair models. But the exciting thing is that no one said it has to be two objects of the same class. So we can go even crazier and interpolate between a car and a boat. Since the technique works on a low dimensional representation of these shapes, we can also perform these crazy algebraic operations between them that follow some sort of intuition. We can add two chairs together or subtract different kinds of tables from each other. Absolute madness. And one of the most remarkable things about the paper is that the learning took place on a very limited amount of data. Not more than 25 training examples per class. One class we can imagine as one object type such as chairs, tables or cars. The authors made the source code and the pre trained network available on their website. The link is in the video description. Make sure to have a look. I am so happy to see breakthroughs like this in machine learning research one after another in quick succession. This work is surely going to spark a lot of follow up papers. And we'll soon find ourselves getting extremely high quality 3D models from from photographs. Also, imagine combining this with a 3D printer. You take a photograph of something, run this algorithm on it, and then print a copy of that furniture and appliance for yourself. We are living amazing times indeed. Thanks for watching and for your generous support and I'll see you next time.