A neural network learns to recognize 3D shapes and features from depth images without being told what to look for. The system can reconstruct 3D scenes, estimate object positions in messy environments, and find matching features across different objects. Source code is available.
Dear fellow scholars, this is two minute papers with Today we are going to discuss a great piece of work that shows us how efficient and versatile neural network based techniques had become recently. Here the input is a bunch of RGBD images which are photographs endowed with depth information and the output can be a full 3D reconstruction of a scene and much, much more, which we'll see in a moment. This task is typically taken care of by handcrafting descriptors. A descriptor is a specialized representation for doing useful tasks on images and other data structures. For instance, if we seek to build an algorithm to recognize black and white images, a useful descriptor would definitely contain the number of colors that are visible in an image and the list of these colors. Again, these descriptors have been typically handcrafted by scientists for decades. New problem, new descriptors, new papers. But not this time, because here super effective descriptors are proposed automatically via a learning algorithm, a convolutional neural network, and Siamese networks. This is incredible. Creating such descriptors took extremely smart researchers and years of work on a specific problem and were still often not as good as these ones. By the way, we have discussed Siamese networks in an earlier episode. As always, the link is available in the video description and as you can imagine, several really cool applications emerge from this one. When combined with ransac, a technique used to find order in noisy measurement data, it is able to perform 3D scene reconstruction from just a few images and it completely smokes the competition. 2. Pose estimation with bounding boxes. Given a sample of an object, the algorithm is able to recognize not only the shape itself, but also its orientation when given a scene cluttered with other objects. 3. Correspondence search is possible. This is really cool. This means that a semantically similar piece of geometry is recognized on different objects. For instance, the algorithm can learn the concept of a handle and recognize the handles on a variety of objects, such as on motorcycles, carriages, chairs, and more. The source code of this project is also available. Yahoo. Neural networks are rapidly establishing supremacy in a number of research fields and I am so happy to be alive in this age of incredible research progress. Make sure to subscribe to the series and click the bell icon. Some amazing works are coming up in the next few episodes and there will be lots of fun to be had. Thanks for watching and for your generous support and I see you next time.
