Andrew Ng shows how to fit curved patterns to data by creating new features like x squared and x cubed. Polynomial regression trades off between underfitting and overfitting: quadratic might level off too early, cubic might swing too high. Feature scaling matters because powers of x can explode from 1 to a million.
So far, we've just been fitting straight lines to our data. Let's take the ideas of multiple linear regression and feature engineering to come up with a new algorithm called polynomial regression, which will let you fit curves, nonlinear functions, to your data. Let's say you have a housing data set that looks like this, where feature X is the size in square feet. It doesn't look like a straight line fits this data set very well. So maybe you want to fit a curve, maybe a quadratic function, to the data like this, which includes a size X and also X squared, which is a size raised to the power of two, and maybe that will give you a better fit to the data. But then you may decide that your quadratic model doesn't really make sense because the quadratic function eventually comes back down. And, well, we wouldn't really expect housing prices to go down when the size increases, right? Big houses seem like they should usually cost more. So then you may choose a cubic function where we now have not only X squared but X cubed. So maybe this model produces this curve here, which is a somewhat better fit to the data because the size does eventually come back up. And as the size increases. These are both examples of polynomial regression, because you took your optional feature X and raised it to the power of two or three or any other power. And in the case of the cubic function, the first feature is the size, the second feature is the size squared, and the third feature is the size cubed. I just want to point out one more thing, which is that if you create features that are these powers, like the square of the original features like this, then feature scaling becomes increasingly important. So if the size of the house ranges from, say, 1 to 1,000 square feet, then the second feature, which is the size squared, would range from one to a million. And the third feature, which is size cubed, ranges from one to to a billion. So these two features, X squared and X cubed, take on very different ranges of values compared to the original feature x. And if you're using gradient descent, it's important to apply feature scaling to get your features into comparable ranges of values. Finally, here's one last example of how you really have a wide range of choices of features to use. Another reasonable alternative to taking the size squared and size cubed is to say, use the square root of X. So your model may look like W1 times X plus W2 times the square root of X plus B. The square root function looks like this, and it becomes a bit less steep as x increases, but it doesn't ever completely flatten out and it certainly never ever comes back down. So this would be another choice of features that might work well for this data set as well. So you may ask yourself how do I decide what features to use? Later in the second course in the specialization you see how you can choose different features and different models that include or don't include these features, and you have a process for measuring how well these different models perform to help you decide which features to include or not include. For now, I just want you to be aware that you have a choice in what features you use, and by using feature engineering and polynomial functions, you can potentially get a much better model for your data. In the optional lab that follows this video you will see some code that implements polynomial regression using features like x, X squared and X cubed. So please take a look and run the code and and see how it works. There's also another optional lab after that, one that shows how to use a popular open source toolkit that implements linear regression. SciKit Learn is a very widely used open source machine learning library that is used by many practitioners in many of the top AI Internet machine learning companies in the world. So if either now or in the future you're using machine learning in your job, there's a very good chance you'll be using tools like SciKit learn to train your models. And so working through that optional app will give you a chance to not only better understand linear regression, but also see how this can be done in just a few lines of code using a library like Scikit Learn. For you to have a solid understanding of these algorithms and be able to apply them. I do think it's important that you know how to implement linear regression yourself and not just call some scikit Learn function that is a black box. But scikit Learn also has an important role in the way machine learning is done in practice today. So we're just about at the end of this week. Congratulations on finishing all of this week's videos. Please do take a look at the practice quizzes and also the Practice Lab, which I hope will let you try out and practice ideas that we've discussed in this week's Practice Lab. You implement linear regression. I hope you have a lot of fun getting this learning algorithm to work for yourself. Best of luck with that and I also look forward to seeing you in next week's videos where we'll go beyond regression, I.e. predicting numbers, to talk about our first classification algorithm which can predict categories. I'll see you next week.
