5:58
Logistic regression estimates the probability that Y equals 1 given input X. The model computes a linear score, then applies the sigmoid function to squash it between 0 and 1, since raw linear outputs don't work for probabilities.
Logistic regression estimates the probability that Y equals 1 given input X. The model computes a linear score, then applies the sigmoid function to squash it between 0 and 1, since raw linear outputs don't work for probabilities.
In this video, we'll go over logistic regression. This is a learning algorithm that you use when the output labels Y in a supervised learning problem are all either 0 or 1. So for binary classification problems, given an input feature vector X may be corresponding to an image that you want to recognize as either a cat picture or not a cat picture. You want an algorithm that can output a prediction which you can call Y hat, which is your estimate of Y. More formally, you want Y hat to be the probability or the chance that Y is equal to one given the input features X. So in other words, if X is a picture, as we saw in the last video, you want Y hat to tell you what is the chance that this is a cat picture. So X, as we said in the previous video, is A, an X dimensional vector. Given that, the parameters of logistic regression will be W, which is also an X dimensional vector, together with B, which is just a real number. So given an input X and the parameters W and B, how do we generate the output Y hat? Well, one thing you could try that doesn't work would be to have Y hat be W transpose X plus B, you know, kind of a linear function of the input X. And in fact, this is what you use if you were doing linear regression. But this isn't a very good algorithm for binary classification because you want Y hat to be the chance that Y is equal to 1. So y hat should really be between 0 and 1. And it's difficult to enforce that because w transpose x plus b can be much bigger than 1 or it can even be negative, which doesn't make sense for a probability that you want to be between 0 and 1. So in logistic regression, our output is instead going to be Y hat equals the sigmoid function applied to this quantity. This is what the sigmoid function looks like. If on the horizontal axis I plot Z, then the function sigmoid of Z looks like this. So it goes smoothly from 0 up to 1. Let me label my axes here. This is 0 and it crosses the vertical axis at 0.5. So this is what sigmoid of Z looks like. And we're going to use Z to denote this quantity W transpose x plus b. Here's the formula for the sigmoid function. Sigmoid of Z, where Z is a real number, is 1 over 1 plus E to the negative z. So notice a couple things. If Z is very large, then E to the negative Z will be close to zero. So then sigmoid of Z will be approximately one over one plus something very close to zero, because, you know, E to the negative of a very large number will be close to zero. So this is close to one. And indeed, if you look on the plot on the left, if Z is very large, then sigmoid of Z is very close to 1. Conversely, if Z is very small or if it's a very large negative number, Then sigmoid of z becomes 1 over 1 plus e to the negative Z, and this becomes becomes E to huge number. So this becomes, you know, think of it as one over one plus a number that is very, very big. And so that's close to zero. And indeed you see that as Z becomes a very large negative number, sigmoid of Z goes very close to zero. So when you implement logistic regression, you your job is to try to learn parameters W and B so that Y hat becomes a good estimate of the chance of Y being equal to one. Before moving on, just another note on the notation. When we program neural networks, we'll usually keep the parameter W and parameter B separate, where here B corresponds to an intercept term. In some other classes, you might have seen a notation wide that handles this differently. In some conventions, you define an extra feature called x0 and make that equal to 1, so that now x is in r of nx plus 1. And then you define Y hat to be equal to sigma of theta transpose X. In this alternative notational convention, you have vector parameters theta, theta, theta zero, theta one, theta two down to thetanx. And so theta zero plays a row of B that's just a real number, and theta one down to theta nx play the row of W. It turns out when you implement your neural network, it'll be easier to just keep B and W as separate parameters. And so in this class, we will not use any of this notational convention that I just wrote in red. If you've not seen this notation before in other classes, don't worry about it. It's just that for those of you that have seen this notation, I wanted to mention explicitly that we're not using that notation in this course. But if you've not seen this before, it's not important and you don't need to worry about it. So you've now seen what the logistic regression model looks like. Next, to train the parameters W and B, you need to define a cost function. Let's do that in the next video.
