Deep neural networks power mission-critical systems like collision avoidance, but unlike handcrafted algorithms, we can't easily verify they won't fail. Researchers developed techniques to mathematically prove a network's robustness against adversarial attacks, testing them on an airborne collision-avoidance system as a first step toward certifying AI for safety-critical applications.
Dear fellow scholars, this is two minute papers with this paper does not contain the usual fireworks that you are used to in two minute papers, but I feel that this is a very important story that needs to be told to everyone. In computer science we encounter many interesting problems, like finding the shortest path between two given streets in a city, or measuring the stability of a bridge. Up until a few years ago, these were almost exclusively solved by traditional handcrafted techniques. This means a class of techniques that were designed by hand by scientists and are often specific to the problem we have at hand. Different problem, different algorithm. And fast forward to a few years ago, we witnessed an amazing resurgence of neural networks and learning algorithms. Many problems that were previously thought to be unsolvable crumbled quickly one after another. Now it is clear that the age of AI is coming, and clearly there are possible applications of it that we need to be very cautious with, since since we design these traditional techniques by hand, the failure cases are often known because these algorithms are simple enough that we can look under the hood and make reasonable assumptions. This is not the case with deep neural networks. We know that in some cases neural networks are unreliable, but it is remarkably hard to identify these failure cases. For instance, earlier we talked about this technique by the name Pix2Pix and where we could make a crude drawing of a cat and it would translate it to a real image. It worked spectacularly in many cases, but Twitter was also full of examples with really amusing failure cases. Beyond the unreliability, we have a much bigger problem, and that problem is adversarial examples. In an earlier episode we discussed an adversarial algorithm where in an amusing example, they added a tiny bit of barely perceptible noise to this image to make the deep neural network misidentify a bus for an ostrich. We can even train a new neural network that is specifically tailored to break the one we have, opening up the possibility of targeted attacks against it. To alleviate this problem, it is always a good idea to make sure that these neural networks are also trained on adversarial inputs as well. But how do we know how many possible other adversarial examples exist that we haven't found yet? The paper discusses a way of verifying important properties of neural networks. For example, for instance, it can measure the adversarial robustness of such a network. And this is super useful because it gives us information whether there are possible forged inputs that could break our learning systems. The paper also contains a nice little experiment with airborne collision avoidance systems. The goal here is avoiding mid air collisions between commercial aircrafts while minimizing the number of alerts. As a small scale thought experiment, we can train a neural network to replace an existing system. But in this case such a neural network would have to be verified and it is now finally a possibility. Now make no mistake, this does not mean that there are any sort of aircraft safety systems deployed in the industry that are relying on neural networks. No, no, no. Absolutely not. This is a small scale what if kind of experiment that may prove to be a first step towards something really exciting. This is one of those incredible papers that even without the usual visual fireworks, makes me feel that I am a part of the future. The this is a step towards a future where we can prove that a learning algorithm is guaranteed to work in mission critical systems. I would also like to note that even if this episode is not meant to go viral on the Internet, it is still an important story to be told. Normally creating videos like this would be a financial suicide, but we are not hurt by this at all because we get stable support from you on Patreon. And that's what it is all about. Worrying less about views and spending more time talking about what's really important and absolutely amazing. Thanks for watching and for your generous support and I'll see you next time.
