An instructor explains derivatives as the slope of a function using f(a) = 3a as an example. Small changes in the input produce proportional changes in the output, and the slope stays constant at 3 everywhere on the line. The lecture builds intuition before introducing the formal limit definition.
In this video, I want to help you gain an intuitive understanding of calculus and of derivatives. Now, maybe you're thinking that you haven't seen calculus since your college days, and depending on when you graduated, maybe that was quite some time back. Now, if that's what you're thinking, don't worry. You don't need a deep understanding of calculus in order to apply neural networks and deep learning very effectively. So if you're watching this video or some of the later videos and you're wondering, wow, is this stuff really. For me, this calculus looks really complicated. My advice to you is the following, which is that watch the videos and then if you could do the homework and complete the programming homework successfully, then you can apply deep learning. Um, in fact, what you see later is that in week four, we'll define a couple of types of functions that will enable you to encapsulate everything that needs to be done with respect to calculus. There are these functions called forward functions and backward functions that you learn about. That lets you put everything you need to know about calculus into these functions so that you don't need to worry about them anymore beyond that. But I thought that in this foray into deep learning, that this week we should open up the box and peer a little bit further into the details of calculus. But really all you need is an intuitive understanding of this in order to build and successfully apply these algorithms. Oh, and finally, if you are among that maybe smaller group of people that are experts in calculus, if you're very familiar with calculus or derivatives, it's probably okay for you to skip the video. But for everyone else, let's dive in and try to gain an intuitive understanding of derivatives. I plotted here. The function F of a equals 3, 8. So it's just a straight line to gain intuition about derivatives. Let's look at a few points on this function. Let's say that A is equal to 2. In that case, F of A, which is equal to three times A is equal to six. So if A is equal to two, then F of A will be equal to six. Let's say we give the value of A just a little bit of a nudge. I'm going to just bump up a a little bit so that it's now 2.001. So I'm going to give a like a tiny little nudge to the right. So now it's, let's say 2.001. And this plot isn't to scale 2.001. The 0.001 difference is too small to show in this plot, just give a little nudge to the right. Now, F of A is equal to three times that. So 6.003. So plot this over here. This is not a scale. This is 6.003. So if you look at this little triangle here that I'm highlighting in green, what we see is that if I nudge a 0.001 to the right, then F of A goes up by 0.003. The amount that F went up is three times as big as the amount that I nudged A to to the right. So we're going to say that the slope or the derivative of the function F of A at A equals 2, or when A equals 2 to the slope is 3. And, you know, the term derivative basically means slope. It's just that derivative sounds like a scarier, more intimidating word, whereas slope is a friendlier way to describe the concept of derivative. So whenever you hear derivative, just think slope of the function. And more formally, the slope is defined as the height divided by the width of this little triangle that we have in green. So this is, you know, 0.003 over 0.001. And the fact that the slope is equal to 3 or the derivative is equal to 3 just represents the fact that when you nudge A to the right by 0.001 by a tiny amount, the amount at F of A goes up is three times as big as the amount that you nudged it, that you nudged A in the horizontal direction. So that's all that the slope of a line is. Now, let's look at this function at a different point. Let's say that A is now equal to 5. In that case, F of A three times A is equal to 15. So let's say I again give a a nudge to the right, a tiny little nudge. So it's now bumped up to 5.001, F of A is three times that. So F of A is equal to 15.003. And so once again, when I bump A to the right, nudge A to the right by 0.001, F goes up three times as much. So the slope again at A equals 5 is also 3. So the way we write this, that the slope of the function F is equal to 3, we say D F of a DA, and this just means the slope of the function the F of A. When you nudge the variable A a tiny little amount, this is equal to 3. And an alternative way to write this derivative formula is as follows. You can Also write this as D da of F of A. So whether you put the F of A on top or whether you write it, you know, down here, it doesn't matter. But all this equation means is that if I nudge A to the right a little bit, I expect F of A to go up by three times as much as I n the value of little A. Now, for this video, I explained derivatives, talking about what happens if we nudge the variable A by 0.001. If you want the formal mathematical definition of derivatives, derivatives are defined with an even smaller value of how much you nudge A to the right. So it's not 0.001, is not 0.00001, is not 0.0000, and so on. One is sort of even smaller than that. And the formal definition of derivative says, what if you nudge A to the right by an infinitesimal amount, basically an infinite, infinitely tiny, tiny amount. If you do that, does F of A go up three times as much as whatever was the tiny, tiny, tiny amount that you nudged A to the right? So that's actually the formal definition of a derivative. But for the purposes of our intuitive understanding, we're just going to talk about nudging A to the right by this small amount. 0.001, if it's 0.001 isn't exactly, you know, tiny, tiny infinitesimal. Now, one property of the derivative is that no matter where you take the slope of this function, it is equal to 3, whether A is equal to 2 or A is equal to 5, the slope of this function is equal to 3, meaning that whatever is the value of A, if you Increase it by 0.001, the value of F of A goes up by 3 times as much. So this function has the same slope everywhere. And one way to see that is that where draw this, you know, little triangle, right? The height divided by the width always has a ratio of 3 to 1. So I hope this gives you a sense of what the slope, what the derivative of a function means for a straight line, where in this example, the slope of the function was three everywhere. In the next video, let's take a look at a slightly more complex example where the slope to the function can be different at different points on the function.
