Computation Graph (C1W2L07)

You've heard me say that the computations of a neural network are organized in terms of a forward pass or a forward propagation step in which we compute the output of the neural network, followed by a backward pass or back propagation step which we use to compute gradients or compute derivatives. The computation graph explains why it is organized this way. In this video we'll go through an example in order to illustrate the computation graph. Let's use a simpler example than logistic regression or a full blown neural network. But let's say that we're trying to compute a function J which is a function of three variables A, B and C. And let's say that function is 3 times A plus B times C. Computing this function actually has three distinct steps. The first is you need to compute what is B times C. And let's say we store that in a variable called U. So U is equal to B times C. And then you might compute V is equal to A times U. So let's say you know, this is V. And then finally your output j is 3 times v. So this is your final function J that you're trying to compute. We can take these three steps and draw them in a computation graph as follows. Let's say I draw your three variables A, B and C here. So the first thing we did was compute U equals B times C. So I'm going to put a rectangular box around that. And so the inputs to that are B and C. And then you might have V equals A plus U. So the inputs to that are V. So the inputs to that are U which we just computed together with A. And then finally we have J equals three times B. So as a concrete example, if A equals 5, B equals 3 and C equals 2, then U equals BC would be 6, V equals A plus U would be 5 plus 6 is 11, J is 3 times that. So J is equal to 33 and and indee, hopefully you can verify that, you know, this is three times five plus three times two. And if you expand that out, you know, you actually get 33 as the value of J. So the computation graph comes in handy when there is some distinguished or some special output variable such as J in this case that you want to optimize. And in the case of a logistic regression, J is of course the cost function that we're trying to minimize. And what we're seeing in this little example is that through a left to right pass you can compute the value of J. And what we'll see in the next couple slides is that in order to compute derivatives. It'll be a right to left pass like this kind of going in the opposite direction as the blue arrows. Um, that would be most natural for computing the derivatives. So to recap, the computation graph organizes a computation with this blue arrow left to right computation. Let's defer to the next video how you can do the backward red arrow right to left computation of the derivatives. Let's go on to the next video.