MIT 6.S094: Deep Learning

Thank you everyone for braving the cold and the snow to be here. This is 6S094 Deep Learning for Self Driving Cars. And it's a course where we cover the topic of deep learning, which is a set of techniques that have taken a leap in the last decade for our understanding of what artificial intelligence systems are capable of doing. And self driving cars, which is systems that can take these techniques and integrate them in a meaningful, profound way into our daily lives in a way that transforms society. So that's why both of these topics are extremely important and extremely exciting. My name is Lex Friedman and I'm joined by an amazing team of engineers in Jack Terwilliger, Julia Kindelsberger, Dan Brown, Michael Glaser, Lee Ding, Spencer Dodd, and Benedict Jenick, among many others. We build autonomous vehicles here at mit. Not just ones that perceive and move about the environment, but ones that interact, communicate and earn the trust and understanding of human beings inside the car, the drivers and the passengers and the human beings outside the car, the pedestrians and other drivers and cyclists. The website for this course, selfdrivingcars mit.edu. if you have questions, email deepcarsit.edu Slackdeep MIT. For registered MIT students, you have to register on the website and by midnight Friday, January 19, build a neural network and submit it to the competition that achieves a speed of 65 mph on the new Deep Traffic 2.0. It's much harder and much more interesting than last year's. For those of you who participated, there's three competitions in this class. Deep Traffic, Seg, Fuse, Deep Crash. There's guest speakers that come from Waymo, Google, Tesla, and those are starting new autonomous vehicle startups in Voyager, Newtonomy and Aurora. In the news a lot today from CES and we have shirts for those of you who brave the snow and continue to do so. Towards the end of the class, there will be free shirts. Yes, I said free and shirts in the same sentence. You should be here. Okay. First, the Deep Traffic competition. There's a lot of updates and we'll cover those on Wednesday. It's a deep Reinforcement learning competition. Last year we received over 18,000 submissions. This year we're going to go bigger. Not only can you control one car with a neural network, you can control up to 10. This is multi agent deep reinforcement learning. This is super cool. Second, Psych Fuse dynamic driving scene segmentation competition where you're given the raw video, the kinematics of the vehicle, so the movement of the vehicle, the state of the art segmentation for the training set you're given ground truth labels, pixel level labels, scene segmentation and optical flow. And with those pieces of data, you're tasked to try to perform better than the state of the art in image based segmentation. Why is this critical and fascinating and an open research problem? Because robots that act in this world, in the physical space, not only must interpret, use these deep learning methods to interpret the spatial visual characteristics of a scene, they must also interpret, understand and track the temporal dynamics of the scene. This competition is about temporal propagation of information, not just scene segmentation. You must understand the space and time. And finally, Deep Crash, where we use deep reinforcement learning to slam cars thousands of times here at mit, at the gym, you're given data on a thousand runs where a car, knowing nothing, is using a monocular camera as a single input, driving over 30 miles an hour through a scene. It has very little control through very little capability to localize itself. It must act very quickly in that scene. You're given 1000 runs to learn anything. We'll discuss this in the coming weeks. This competition will result in four submissions that we evaluate. Everyone's in simulation, but the top four submissions we put head to head at the gym. And until there is a winner declared, we keep slamming cars at 30 miles an hour. Deep crash. And also on the website is from last year and on GitHub there's deep Tesla, which is using the large scale naturalistic driving data set. We have to train a neural network to do end to end steering that takes in monocular video from the forward roadway and produces steering commands that steering commands for the car lectures. Today we'll talk about deep learning, tomorrow we'll talk about autonomous vehicles. Deep RL is on Wednesday. Driving scene understanding. So segmentation, that's Thursday. On Friday we have Sasha Arnoux, the director of engineering at Waymo. Waymo is one of the companies that's truly taking huge strides in fully autonomous vehicles. They're taking the fully L4, L5 autonomous vehicle approach and it's fascinating to learn. He's also the head of perception. For them to learn from him what kind of problems they're facing and what kind of approach they're taking on. We have Emilio Frazoli, who one of last year's speakers, Sertash Karman, said, Emilio is the smartest person he knows. So Emilio Frazoli is the CTO of Nutonomy, an autonomous vehicle company that was just acquired by Delphi for a large sum of money. And they're doing a lot of incredible work in Singapore and here in Boston Next Wednesday we are going to talk about the topic of our research and my personal fascination is deep learning for driver state sensing, understanding the human, perceiving everything about the human being inside the car and outside the car. One talk I'm really excited about is Oliver Cameron. On Thursday he is now the CEO of autonomous vehicle startup Voyage. He was previously the director of the self driving car program for Udacity. He will talk about how to start a self driving car company for those. He said that MIT folks and entrepreneurs, if you want to start one yourself, he'll tell you exactly how it's super cool. And then Sterling Anderson, who was the director previously of Tesla Autopilot team and now is a co founder of Aurora, the self driving car startup that I mentioned that has now partnered with Nvidia and many others. So why self driving cars? This class is about applying data driven learning methods to the problem of autonomous vehicles. Why self driving cars are fascinating and an interesting problem space. Quite possibly, in my opinion, this is the first wide reaching and profound integration of personal robots in society. Wide reaching because there's 1 billion cars on the road. Even a fraction of that will change the face of transportation and how we move about this world. Profound, and this is an important point that's not always understood is there's an intimate connection between a human and a vehicle when there's a direct transfer of control. It's a direct transfer of control that takes that his or her life into the hands of an artificial intelligence system. I showed a few quick clips here. You can Google first time with Tesla autopilot on YouTube and watch people perform that transfer of control. There's something magical about a human and a robot working together that will transform what artificial intelligence is in the 21st century. And this particular autonomous system AI system, self driving cars is on the scale and the profound, the life critical nature of it is profound in a way that will truly test the capabilities of AI. There is a personal connection that will argue throughout these lectures that we cannot escape. Considering the human being. That autonomous vehicle must not only perceive and control its movement through the environment, but it must also perceive everything about the human driver and the passenger and interact, communicate and build trust with that driver. Because in my view, as I will argue throughout this course, an autonomous vehicle is more of a personal robot than it is a perfect perception control system. Because perfect perception and control to this world full of humans is extremely difficult and could be two, three, four decades away. Full autonomy autonomous vehicles are going to be flawed, they're going to have flaws and we have to design systems that are effectively caught, that effectively transfer control to human beings when they can't handle the situation. And that transfer of control is a fascinating opportunity for AI because the obstacle avoidance, perception of obstacles and obstacle avoidance is the easy problem. It's the safe problem. Going 30 miles an hour, navigating through streets of Boston is easy. It's when you have to get to work and you're late or you're sick of the person in front of you that you want to go in the opposing lane and speed up. That's human nature and we can't escape it. Our artificial intelligence systems can't escape human nature. They must work with it. What's shown here is one of the algorithms we'll talk about next week for cognitive load. Or we take the raw 3D convolutional neural networks, take in the eye region, the blinking and the pupil movement to determine the cognitive load of the driver. We'll see how we can detect everything about the driver where they're looking. Emotion, cognitive load, body pose, estimation, drowsiness. The movement towards full autonomy is so difficult, I would argue that it almost requires human level intelligence that the, as I said, two, three, four decade out journey for artificial intelligence researchers to achieve full autonomy will require achieving solving some of the problems, fundamental problems of creating intelligence. And that's something we'll discuss in much more depth in, in a broader view in two weeks for the artificial general intelligence course where we have Andrej Karpathy from Tesla, Ray Kurzweil, Mark Ryberg from Boston Dynamics, who asked for the dimensions of this room because he's bringing robots. Nothing else was told to me. It'll be a surprise. So that is why I argue the human centered artificial intelligence approach in every algorithm of design considers the human for autonomous vehicle on the left, the perception seen, understanding and the control problem as we'll explore through the competitions and the assignments of this course can handle 90 and increasing percent of the cases. But it's the 10, 1.1% of the cases as we get better and better that we have to, we're not able to handle through these methods. And that's where the human perceiving the human is really important. This is the video from last year of Arc de Triomphe. Thank you. I didn't know it last year. I know now that is one of millions of cases where human to human interaction is the dominant driver, not the basic perception control problem. So why deep learning in this space? Because deep learning is a set of methods that do well From a lot of data. And to solve these problems, where human life is at stake, we have to be able to have techniques that learn from data, learn from real world data. This is the fundamental reality of artificial intelligence. Systems that operate in the real world, they must learn from real world data, whether that's on the left for the perception, the control side, or on the right for the human, the perception and the communication interaction and collaboration with the human and the human robot interaction. Okay, so what is deep learning? It's a set of techniques. If you allow me the definition of intelligence being the ability to accomplish complex goals, then I would argue definition of understanding, maybe reasoning is the ability to turn complex information into simple use, useful, actionable information. And that is what deep learning does. Deep learning is representation learning, or feature learning, if you will. It's able to take raw information, raw, complicated information that's hard to do anything with, and construct hierarchical representations of that information. To be able to do something interesting with is the branch of artificial intelligence which is most capable and focused on this task. Forming representations from data, whether it's supervised or unsupervised, whether it's with the help of humans or not, it's able to construct structure, find structure in the data such that you can extract simple, useful, actionable information. On the left, from Ian Goodfellow's book, is the basic example of image classification. The input of the image on the bottom with the raw pixels, and as we go up the stack, as we go up the layers, higher and higher order representations are formed, from edges to contours to corners to object parts, and then finally the full object semantic classification of what's in the image. This is representation learning. A favorite example for me is one from four centuries ago. Our place in the universe. And representing that place in the universe, whether it's relative to earth or relative to the sun. On the left is our current belief. On the right is the one that is held widely four centuries ago. Representation matters because what's on the right is much more complicated than what's on the left. You can think of in a simple case here, when the task is to draw a line that separates green triangles and blue circles in the Cartesian coordinate space on the left, the task is much more difficult, impossible to do well on the right. It's trivial in polar coordinates. This transformation is exactly what we need to learn. This is representation learning. So you can take the same task of having to draw a line that separates the blue curve and the red curve on the left. If we draw a straight line, it's going to be a high. There's no way to do it with zero error, with 100% accuracy. Shown on the right is our best attempt. But what we can do with deep learning with a single hidden layer network done here is form the topology, the mapping of the space in such a way in the middle that allows for a straight line to be drawn that separates the blue curve and the red curve. The learning of the function in the middle is what we're able to achieve with deep learning. It's taking raw, complicated information and making it simple, actionable, useful. And the point is that this kind of ability to learn from raw sensory information means that we can do a lot more with a lot more data. So deep learning gets better with more data. And that's important for real world applications where edge cases are everything. This is us driving with two perception control systems. One is a Tesla vehicle with the Autopilot version one system that's using a monocular camera to perceive the external environment and produce control decisions. And our own neural network running on the Jetson TX2 that's taking in the same with a monocular camera and producing control decisions. And the two systems argue, and when they disagree, they raise up a flag to say that this is an edge case that needs human intervention. There is. Covering such edge cases using machine learning is the main problem of artificial intelligence. When applied to the real world, it is the main problem to solve. Okay, so what are neural networks inspired? Very loosely. And I'll discuss about the key difference between our own brains and artificial brains, because there's a lot of insights in that difference. But inspired loosely by biological neural networks. Here is a simulation of a thalamocortical brain network, which is only 3 million neurons, 476 million synapses. The full human brain is a lot more than that. 100 billion neurons, 1,000 trillion synapses. There's inspirational music with this one that I didn't realize was here. This. It should make you think. Artificial neural networks. Okay, let's just let it play. The human neural network is 100 billion neurons, right? 1,000 trillion synapses. One of the state of the art neural networks is resonant 152, which has 60 million synapses. That's a difference of about a seven order of magnitude difference. The human brains have 10 million times more synapses than artificial neural networks, plus or minus one order of magnitude, depending on the network. So what's the difference between a biological neuron and an artificial neuron? The topology of the human brain have no layers. Neural networks are stacked in layers. They're fixed for the most part. There is chaos. Very little structure in our human brain in terms of how neurons are connected. They're connected often to 10,000 plus other neurons. The number of synapses from individual neurons that are input into the neuron is huge. They're asynchronous. The human brain works asynchronously. Artificial neural networks work synchronously. The learning algorithm for artificial neural networks, the only one, the best one, is back propagation. And we don't know how human brains learn processing speed. This is one of the, the only benefits we have with artificial neural networks is artificial neurons are faster, but they're also extremely power and efficient. And there is a division into stages of training and testing with neural networks, with biological neural networks. As you're sitting here today, they're always learning. The only profound similarity, the inspiring one, the captivating one, is that both are distributed computation at scale. There is an emergent aspect to neural networks where the basic element of computation, a neuron is simple, is extremely simple. But when connected together, beautiful, amazing, powerful approximators can be formed. A neural network is built up of these computational units where the inputs, there's a set of edges with weights on them. The edges are. The weights are multiplied by this input signal. A bias is added with a non linear function that determines whether the network gets activated or not. Well, the neuron gets activated or not visualized here. And these neurons can be combined in a number of ways. They can form a feed forward neural network, or they can feed back into itself to form, to have state memory. In recurrent neural networks, the ones on the left are the ones that are most successful for most applications in computer vision. The ones on the right are very popular and specific when temporal dynamics or dynamics, time series of any kind are used. In fact, the ones on the right are much closer to the way our human brains are than the ones on the left. But that's why they're really hard to train. One beautiful aspect of this emergent power from multiple neurons being connected together is the universal property that with a single hidden layer, these networks can learn any function, learn to approximate any function, which is an important property to be aware of, because the limits here are not in the power of the networks. The limits is in the methods by which we construct them and train them. What kinds of machine learning, deep learning are there? We can separate it into two categories. Memorizers, the approaches that essentially memorize patterns in the data and approaches that we can loosely say, are beginning to reason to generalize over the data with minimal human input. On top, on the left are the quote unquote teachers, is how much human input in blue is needed to make the method successful. For supervised learning, which is what most of deep learning successes come from, where most of the data is annotated by human beings. The human is at the core of the success. Most of the data that's part of the training needs to be annotated by human beings. With some additional successes coming from augmentation methods that extend the data based on which these networks are trained. And the semi supervised reinforcement learning and unsupervised methods that we'll talk about later in the course. That's where the near term successes we hope are. And with the unsupervised learning approaches, that's where the true excitement about the possibilities of artificial intelligence lie. Being able to make sense of our world with minimal input from humans. So we can think of two kinds of deep learning impact spaces. One is a special purpose intelligence is taking a problem, formalizing it, collecting enough data on it, and being able to solve a particular case that provides value of particular interest. Here is a network that estimates apartment costs in the Boston area. So you could take the number of bedrooms, the square feet in the neighborhood, and provide as output, the estimated cost on the right is the actual data of apartment cost. We're actually standing in an area that has over $3,000 for a studio apartment. Some of you may be feeling that pain. And then there's general purpose intelligence, which, or something that feels like approaching general purpose intelligence, which is reinforcement and unsupervised learning. Here from Andrej Karpath. He's pong the pixels, a system that takes in 80 by 80 pixel image and with no other information is able to beat, is able to win at this game. No information except a sequence of images, raw sensory information the same way, the same kind of information that human beings take in from the visual, audio, touch, sensory data, the very low level data and be able to learn to win. And it's very simplistic in this very artificially constructed world, but nevertheless a world where no feature learning is performed. Only raw sensory information is used to win with very sparse, minimal human input. We'll talk about that on Wednesday with deep reinforcement learning. So but for now we'll focus on supervised learning where there is input data, there is a network. We're trying to train a learning system and there's a correct output that's labeled by human beings. That's the general training process for a neural network, input data labels and the training of that network, that model, so that in a testing stage on new input data that has never seen before, it's tasked with producing guesses and is evaluated based on that. For autonomous vehicles, that means being released either in simulation or in the real world to operate. And how they learn, how neural networks learn is given the forward pass of taking the input data, whether it's from the training stage. In the training stage, taking the input data, producing a prediction and then given that there's ground truth in the training stage, we can have a measure of error based on a loss function that then punishes the synapses, the connections, the parameters that were involved with making that that wrong prediction and it back propagates the error through those weights. We'll discuss that in a little bit more detail in a bit here. So what can we do with deep learning? We can do one to one mapping. Really. You can think of input as being anything. It can be a number, a vector of numbers, a sequence of numbers, a sequence, a vector of numbers. Anything you can think of, from images to video to audio to text can be represented in this way. And the output can the same be a single number. Or it can be images, video, text, audio, one to one mapping. On the bottom, one to many, many to one, many to many and many to many with different starting points for the data. Asynchronous, Some quick terms that will come up. Deep learning the same as neural networks. It's really deep neural networks, large neural networks. It's a subset of machine learning that has been extremely successful in the past decade. Multi layer perceptron, Deep neural network, recurrent neural network, long short term memory network, lstm, convolutional neural network and deep belief networks. All of these will come up through the slides and there is specific operations layers within these networks of convolution, pooling, activation and back propagation. These concepts that we'll discuss in this class. Activation functions There's a lot of variants. On the left is the activation function in the left column. On the x axis is the input. On the Y axis is the output, the sigmoid function, the output. If the font is too small, the output is not centered at 0. For the TanH function it's centered at 0, but it still suffers from vanishing gradients. Vanishing gradients is when the value the input is low or high, the output of the network, as you see in the right column there, the derivative of the function very low. So the learning rate is very low. For ReLU it's also not zero centered, but it does not suffer from vanishing gradients. Back propagation is the process of learning. It's the way we take go from error compute as the loss function on the bottom right of the slide, taking the actual output of the network with a forward pass, subtracting it from the ground truth, squaring, dividing by two and using that loss function, then back propagate through to construct a gradient to back propagate the error to the weights that were responsible for making either a correct or an incorrect decision. So the subtasks of that there's a forward pass, there's a backward pass and a fraction of the weight's gradient subtracted from the weight. That's it. That process is modular, so it's local to each individual neuron, which is why it's extremely dis. It's we're able to distribute it across multiple across the gpu, parallelized across the gpu. So learning. For a neural network, these computational units are extremely simple. They're extremely simple to then correct when they make an error, when they're part of a larger network that makes an error. And all that boils down to is essentially an optimization problem where the objective utility function is the loss function and the goal is to minimize it. And we have to update the parameters, the weights and the synapses and the biases to decrease that loss function. And that loss function is highly nonlinear. Depending on the activation functions, different properties, different issues arise. There's vanishing gradients for Sigmoid, where the learning can be slow. There's dying Relius where the derivative is exactly zero for inputs less than zero. There are solutions to this like leaky rails and a bunch of details that you may discover when you try to win the deep traffic competition. But for the most part, these are the main activation functions and it's the choice of the neural network designer which one works best. There are saddle points. All the problems from numerical nonlinear optimization that arise come up here. It's hard to break symmetry and stochastic gradient descent without any kind of tricks to it can take a very long time to arrive at the minima. One of the biggest problems in all of machine learning, and certainly deep learning, is overfitting. You can think of the blue dots in a plot here as the data to which we want to fit a curve. We want to design a learning system that approximates the regression of this data. So in green is a sine curve simple fits well. And then there's a 9th degree polynomial which fits even better in terms of the error. But it clearly overfits this data. If there's other data that it has not seen yet that it has to fit, it's likely to produce a high error. So it's overfitting the training set. This is a big problem for small data sets. And so we have to fix that with regularization. Regularization is a set of methodologies that prevent overfitting, learning the training too well in order and then to not be able to generalize to the testing stage and overfitting. The main symptom is the error decreases in training set, but increases in the test set. So there's a lot of techniques in traditional machine learning that deal with this cross validation and so on. But because of the cost of training for neural networks, it's traditional to use what's called a validation set. So you create a subset of the training that you keep away for which you have the ground truth and use that as a representative of the testing set. So you perform early stoppage or more realistically, just save a checkpoint often to see how as the training evolves, the performance changes on the validation set. And so you can stop when the performance in the validation set is getting a lot worse. It means you're over training on the training set. In practice, of course, we run training much longer and see when what is the best performing, what what is the best performing snapshot checkpoint of the network. Dropout is another very powerful regularization technique where we randomly remove part of the network, randomly remove some of the nodes in the network along with its incoming and outgoing edges. So what that really looks like is a probability of keeping a node. And in many deep learning frameworks today it comes with a dropout layer. So it's essentially a probability that's usually greater than 0.5 that a node will be kept for the input layer. The probability should be much higher or more effectively, what works well is just adding noise. What's the point here? You want to create enough diversity in the training data such that it is generalizable to the testing. And as you'll see with deep traffic competition, there's L2 and L1 penalty, weight decay weight penalty, where there's a penalization on the weights that get too large. The L2 penalty keeps the weight small unless the error derivative is huge and produces a smoother model. And prefers to distribute when there is two similar inputs. It prefers to put half the weights on each, distribute the weights as opposed to putting the weight on one of the edges. Makes the network more robust. L1 penalty has the one benefit that for really large weights they're allowed to be to stay so it allows for a few weights to remain very large. These are the regularization techniques and I wanted to mention them because they're useful to some of the competitions here in the course. And I recommend to go To Playground To TensorFlow Playground To Play around with some of these parameters where you get to online in the browser, play around with different inputs, different features, different number of layers and regularization techniques. And to build your intuition about classification regression problems given different input data sets. So what changed? Why? Over the past many decades, neural networks that have gone through two winters are now again dominating the artificial intelligence community. CPUs, GPUs, ASICs. So computational power has skyrocketed from Moore's Law to GPUs. There is huge data set, including ImageNet and others. There is research back propagation in the 80s, the convolutional neural networks, LSTMs. There's been a lot of interesting breakthroughs about how to design these architectures, how to build them such that they're trainable efficiently using GPUs. There is the software infrastructure from being able to share the data with Git, to being able to train networks and share code and effectively view neural networks as a stack of layers, as opposed to having to implement stuff from scratch with TensorFlow, PyTorch and other deep learning frameworks. And there's huge financial backing from Google, Facebook and so on. Deep learning is in order to understand why it works so well and where its limitations are, we need to understand where our own intuition comes from about what is hard and what is easy. The important thing about computer vision, which is a lot of what this course is about, even as in deep reinforcement learning formulation, is that visual perception for us human beings was formed 540 million years ago. That's 540 million million years worth of data. And abstract thought is only formed about 100,000 years ago. That's several orders of magnitude less data. So we can make with neural networks predictions that seem trivial, that trivial to us human beings, but completely challenging and wrong to neural networks here on the left, showing a prediction of a dog with a little bit of distortion and noise added to the image, producing the image. On the right, a neural Network is confidently 99% plus accuracy predicting that it's an ostrich. And there's all these problems that's to deal with whether it's in computer vision data, whether it's in text data, audio, all of this variation arises in vision. It's illumination variability. The set of pixels and the numbers look completely different depending on the lighting conditions. It's the biggest problem in driving is lighting conditions, lighting variability, pose variation. Objects need to be learned from every different perspective. I'll discuss that for when sensing the driver. Most of most of most of the deep learning work that's done on the face on the human is done on the frontal face or semi frontal face. There's very little work done on the full 360 pose variability that a human being can take on. Intraclass variability for the classification problem, for the detection problem. There is a lot of different kinds of objects for cats, dogs, cars, bicyclists, pedestrians. So that brings us to object classification and I'd like to take you through where deep learning has taken big strides for the past several years leading up to this year to 2018. So let's start at object classification. It's when you take a single image and you have to say one class that's most likely to belong in that image. The most famous variant of that is the ImageNet competition ImageNet challenge. The ImageNet data set is a data set of 14 million images with 21,000 categories. And for say the category of fruit, there's a total of 188,000 images of fruit and there is 1200 images of granny Smith apples. It gives you a sense of what we're talking about here. So this has been the source of a lot of interesting breakthroughs in deep learning and a lot of the excitement in deep learning is first, the big successful network, at least one that became famous in deep learning is Alexnet in 2012 that took a leap of a significant leap in performance on the ImageNet challenge. So it was one of the first neural networks that was successfully trained on the GPU and achieved an incredible performance boost over the previous year on the imagenet Challenge. The challenge is, and I'll talk about some of these networks is to given a single image, give five guesses and you have five guesses to guess for one of them to be correct. The human annotation is a question often comes up so how do you know the ground truth? Human level performance is 5.1% accuracy on this task. But the way the annotation for ImageNet is performed is there's a Google search where, where you pull the images already labeled for you and then the annotation that on mechanical Turk other humans perform is just binary. Is this a cat or not a cat? They're not tasked with performing the very high resolution semantic labeling of the image okay, so through from 2012 with Alexnet to today and the big transition in 2018 of the ImageNet challenge, leaving Stanford and going to Kegel, it's sort of a monumental step because in 2015 with the Resnet network was the first time that the human level performance was exceeded. And I think this is a very important map of where deep learning is for a particular. What I would argue is a toy example that despite the fact that it's 14 million images. So we're developing state of the art techniques here and the next stage as we are now exceeding human level performance. And this task is how to take these methods into the real world to perform scene perception, to perform driver state perception. In 2016 and 2017, CU Image and SCNET has a very unique new addition to the previous formulations that has achieved an accuracy of 2.2% error, 2.25% error on the ImageNet classification challenge. It's an incredible result. Okay, so you have this image classification architecture that takes in a single image and produces convolution and takes it through pooling convolution and at the end, fully connected layers and performs a classification task or regression task. And you can swap out that layer to perform any kind of other task, including with recurrent neural networks of image captioning and so on, or localization of bounding boxes. Or you can do fully convolutional networks, which we'll talk about on Thursday, which is when you take a image as an input and produce an image as an output. But where the output image in this case is a segmentation is where a color indicates what the object is, the category of the object. So it's pixel level segmentation. Every single pixel in the image is assigned a class, a category where that pixel belongs to. This is the kind of task that's overlaid on top of other sensory information coming for the car in order to perceive the external environment. You can continue to extract information from images in this way to produce image to image mapping, for example, to colorize images and take from grayscale images to color images. Or you can use that kind of heat map information to localize objects in the image. So as opposed to just classifying that this is an image of a cow, rcnn, fast and faster RCNN and a lot of other localization networks allow you to propose different candidates for where exactly the cow is located in this image and thereby being able to perform object detection, not just object classification. In 2017, there's been a lot of cool applications of these architectures. One of which is background removal. Again mapping from image to image, ability to remove background from selfies of humans or human like pictures of faces. The references with some incredible animations are in the bottom of the slide and the slides are now available online. Pix2 Pix HD there's been a lot of work in GANs in generative artificial networks, in particular in driving. GANs have been used to generate examples that generate examples from source data. Whether that's from raw data or in this case with Pix2 Pix HD it's taking coarse semantic labeling of the images pixel level and producing photorealistic high definition images of the forward roadway. This is an exciting possibility for being able to generate a variety of cases for self driving cars, for autonomous vehicles to be able to learn to generate to augment the data and be able to change the way different roads look road conditions to change the way vehicles look, cyclists, pedestrians. Then we can move on to recurrent neural networks. Everything I've talked about was one to one mapping from image to image or image to number. Recurrent neural networks work with sequences. We can use sequences to generate handwriting, to generate text captions from an image based on the localizations the various detections in that image we can provide video description generation. So taking a video and combining convolution neural networks with recurrent neural networks, using convolutional neural networks to extract features frame to frame and using those extracted features to input into our the RNNs to then generate a a labeling a description what's going on in the video. A lot of exciting approaches for autonomous systems, especially in drones where the time to make a decision is short. Same with the RC car traveling 30 miles an hour. Attentional mechanisms for steering the attention of the network have been very popular for the localization task and for just saving how much interpretation of the image, how much many pixels need to be considered in the classification task so we can steer. We can model the way a human being looks around an image to interpret it and use the network to do the same. And we can use that kind of steering to draw images as well. Finally, the big breakthroughs in 2017 came from this the pong to pixels, the reinforcement learning using sensory data, raw sensory data and use reinforcement learning methods deep RL methods of which we'll talk about on Wednesday. I'm really excited about the underlying methodology of Deep traffic and Deep Crash is using neural networks as the approximators inside reinforcement learning approaches. So AlphaGo in 2016 has achieved a monumental task that when I first started in artificial intelligence was told to me was impossible for AI system to accomplish, which is to win at the game of Go against the top human player in the world. However, that method was trained on human expert positions. The AlphaGo system was trained on previous games played by human experts. In an incredible accomplishment. AlphaGo Zero in 2017 was able to beat AlphaGo and many of its variants by playing itself from zero information. So no knowledge of human experts, no games, no training data, very little human input. And what more it was able to generate moves that were surprising to human experts. I think it's Einstein that said that intelligence that the key mark of intelligence is imagination. I think it's beautiful to see an artificial intelligence system come up with something that surprises human experts. Truly surprises. For the gambling junkies. Deepstack and a few other variants have been used in 2017 to win a Heads Up Poker. Again, another incredible result. I was always told in artificial intelligence would be impossible for any machine learning method to achieve and was able to beat a professional player. And several competitors have come along since we're yet to be able to beat to win in a tournament setting. So multiple players, for those of you familiar, Heads Up Poker is one on one. It's a much, much smaller, easier space to solve. There's a lot more human to human dynamics going on for when there's multiple players. But that's the task for 2018 and the drawbacks. It's one of my favorite videos. I show it often of coast runners for these deep reinforcement learning approaches, the learning of the reward function. The definition of the reward function controls how the actual system behaves. And this will come. This will be extremely important for us with autonomous vehicles. Here the boat is tasked with gaining the highest number of points points and it figures out that it does not need to race, which is the whole point of the game in order to gain points, but instead pick up green circles that regenerate themselves over and over. This is the counterintuitive behavior of a system that would not be expected when you first design the reward function. And this is a very formal, simple system. Nevertheless, it's extremely difficult to come up with a reward function that makes it operate in the way you expect it to operate. Very applicable for autonomous vehicles. And of course on the perception side, as I mentioned, with the ostrich and the dog, a little bit of noise. With 99.6% confidence we can predict that the noise up top is a robin, a cheetah, armadillo, a lesser panda. These are outputs from actual state of the art Neural networks taking in the noise and producing a confident prediction. It should build our intuition to understand that we don't, that the visual characteristics, the spatial characteristics of an image do not necessarily convey the level of hierarchy necessary to function in this world. In a similar way with a dog and an ostrich and everything in an ostrich, a network confidently with a little bit of noise can make the wrong prediction. Thinking a school bus is an ostrich and a speaker is an ostrich. They're easily fooled, but not really because they perform the task that they were trained to do well. So we have to make sure we keep our intuition optimized to the way machines learn, not the way humans have Learned over the 540 million years of data that we've gained through developing the eye through evolution. The current challenges we're taking on. First, transfer learning. There's a lot of success in transfer learning between domains that are very close to each other. So image classification from one domain to the next. There's a lot of value in forming representations of the way scenes look in order, natural scenes look in order to do scene segmentation in the driving case, for example. But we're not able to do any bigger leaps in the way we perform transfer learning. The biggest challenge for deep learning is to generalize, generalize across domains. It lacks the ability to reason in the way that we've defined understanding previously, which is the ability to turn complex information into simple, useful information, convert domain specific, complicated sensory information that doesn't relate to the initial training set. That's the open challenge for deep learning. Train on very little data and then go and reason and operate in the real world. Right now, neural networks are very inefficient. They require big data, they require supervised data, which means they need human, costly human input. They're not fully automated. Despite the fact that the feature learning, incredibly the big breakthrough feature learning, is performed automatically. You still have to do a lot of design of the actual architecture of the network and all the different hyperparameter tuning needs to be performed. Human input, perhaps a little bit more educated human input in the form of PhD students, postdocs, faculty is required to tune these hyperparameters. But nevertheless, human input is still necessary. They cannot be left alone. For the most part, the reward, defining the reward, as we saw with Coastron, is extremely difficult for systems that operate in the real world. Transparency quite possibly is not an important one, but neural networks currently are black box for the most part. They're not able to accept, through a few successful visualization Methods that visualize different aspects of the activations. They're not able to reveal to us humans why they work or where they fail. And this, this is a philosophical question for, for autonomous vehicles, that we may not care as human beings if the system works well enough. But I would argue that it'll be a long time before systems work well enough where we don't care. We'll care and we'll have to work together with these systems. And that's where transparency, communication, collaboration is critical. Edge cases. It's all about edge cases in robotics. In, in autonomous vehicles, the 99.9% of driving is really boring. It's the same, especially highway driving, traffic driving, it's the same. The obstacle avoidance, the car following the lane centering, all these problems are trivial. It's the edge cases, the trillions of edge cases that need to be generalized over on a very small amount of training data. So again, I return to why deep learning. I mentioned a bunch of challenges and this is an opportunity. This is an opportunity to come up with techniques that operate successfully in this world. So I hope the competitions we present in this class in the autonomous vehicle domain will give you some insight and opportunity to apply in some of these cases. Open research problems with semantic segmentation of external perception with control of the vehicle in deep traffic and with deep crash of control of the vehicle in under actuated high speed conditions and the driver state perception. So with that, I wanted to introduce deep learning to you today before we get to the fun tomorrow of autonomous vehicles. So we'd like to thank Nvidia, Google, Autoliv, Toyota, and at the risk of setting off people's phones, Amazon, Alexa, Auto, but truly, I would like to say that I've been humbled over the past year by the thousands of messages we received, by the attention, by the 18,000 competition entries, by the many people across the world, not just here at mit, that are brilliant, that I got a chance to interact with. And I hope we go bigger and do some impressive stuff in 2018. Thank you very much. And tomorrow is self driving. And this is an opportunity. This is an opportunity to come up with techniques that operate successfully in this world. So I hope the competitions we present in this class in the autonomous vehicle domain will give you some insight and opportunity to apply in some of these cases. Open research problems with semantic segmentation of external perception with control of the vehicle in deep traffic and with deep crash of control of the vehicle in under actuated high speed conditions and the driver state perception. So with that I wanted to introduce deep learning to you today before we get to the fun tomorrow of autonomous vehicles. So we'd like to thank Nvidia, Google, autoliv, Toyota, and at the risk of setting off people's phones, Amazon, Alexa, Auto. But truly, I would like to say that I've been humbled over the past year by the thousands of messages we received, by the attention, by the 18,000 competition entries, by the many people across the world, not just here at mit, that are brilliant, that I got a chance to interact with. And I hope we go bigger and do some impressive stuff in 2018. Thank you very much. And tomorrow is self driving.