What is a Neural Network? (C1W1L02)

The term deep learning refers to training neural networks, sometimes very large neural networks. So what exactly is a neural network in this video? Let's try to give you some of the basic intuitions. Let's start with a housing price prediction example. Let's say you have a data set with six houses. So you know the size of the houses in square feet or square meters, and you know the price of the house. And you want to fit a function to predict the price of a house as a function of its size. So if you are familiar with linear regression, you might say, well, let's fit a straight line to this data. So, you know, maybe you get a straight line like that. Um, but to be a bit fancier, you might say, well, we know that prices can never be negative, right? So instead of a straight line fit, which eventually become negative, let's bend the curve here so it just ends up zero here. So this line, thick blue line, ends up being, you know, your function for predicting the price of a house as a function of its size, whereas 0 here and then there's a straight line fit to the right. So you can think of this function that you've just fit to housing prices as a very simple neural network. It's almost the simplest possible neural network. Let me draw it here. We have as the, the input to the neural network the size of a house which we call X. It goes into this node, this little circle, and then it outputs the price from the call Y. So this little circle, which is a single neuron in our neural network, implements this function that we drew on the left. And all that the neuron does is it inputs the size, computes this linear function, takes a max of 0, and then outputs the estimated price. And by the way, in the neural network literature, you see this function a lot. This function which goes at zero at some time and then, you know, takes off as a straight line. This function is called a relu function, which stands for rectified linear unit. So relu and rectify just means taking a max of zero, which is why you get a function shape like this, you don't need to worry about relu units for now, but it's just something you see again later in this course. So if this is a single neuron neural network, really a tiny little neural network, a larger neural network is then formed by taking many of these single neurons and, and stacking them together. So if you think of this neuron as being, you know, like a single Lego break, um, you then get a bigger neural network by stacking together many of these Lego breaks. Let's see an example. Let's say that instead of predicting the price of a house just from a size, you now have other features. You know, other things about the house, such as the number of bedrooms, which want to write as pound bedrooms. And you might think that, you know, one of the things that really affects the price of a house is family size, right? So you know, can this house fit your family of three or family of four or family of five? And it's really based on the size and square feet or square meters and the number of bedrooms that determines whether or not a house can fit your family's family size. And then maybe, you know, the zip code in different countries is called the postal code right of the house. And the zip code maybe is a feature that tells you walkability. So is this neighborhood highly walkable? Can you just walk to the grocery store and walk to school, or do you need to drive? Some people prefer highly walkable neighborhoods. And then the zip code as well as the, the, the wealth maybe tells you right, certainly in the United States, but some other countries as well, tells you how good is the school quality. So each of these little circles I'm drawing can be one of those relu rectified linear units or some other slightly non linear function. So that based on the size and number of bedrooms, you can estimate the family size based on zip code, estimate walkability based on zip code and wealth, you can estimate the school quality. And then finally, you might think that, well, the way people decide how much they're willing to pay for house is they look at the things that really matter to them, in this case, family size, walkability, and school quality. And that helps you predict the price. So in this example, X is all of these four inputs and Y is the price you're trying to predict. And so by stacking together a few of the single neurons, or the simple predictors we had from the previous slide, we now have a slightly larger neural network. Part of the magic of a neural network is that when you implement it, you need to give it just the input X and the output Y for a number of examples in your training set, and all these things in the middle it will figure out by itself. So what you actually implement is this, where here you have a neural network with four inputs. So the input features might be the size, number of bedrooms, the zip code or postal code, and the wealth of the neighborhood. And so given these input features, the job of the neural network will be to predict the price. Yes. And notice also that each of These circles, these are called hidden units in the neural network that each of them takes as input all four input features. So, for example, rather than saying this first node represents family size, and family size depends only on the features X1 and X2, right? Instead we're gonna say, well, neural network, you decide whatever you want this node to be and will give you all four input features to compute whatever you want. So we say that the layers, that is this input layer and this layer in the middle of the neural network are densely connected, because every input feature is connected to every one of these circles in the middle. And the remarkable thing about neural networks is that given enough data about X and Y, given enough training, examples with both X and Y, neural networks are remarkably good at figuring out functions that accurately map from X to Y. So that's a basic neural network. It turns out that as you build out your own neural networks, you probably find them to be most useful, most powerful in supervised learning settings, meaning that you're trying to take an input X and map it to some output Y, like we just saw in the housing price prediction example. In the next video, let's go over some more examples of supervised learning and some examples of where you might find your networks to be incredibly helpful for your applications as well.