Broadcasting in Python (C1W2L15)

In the previous video, I mentioned that broadcasting is another technique that you can use to make your Python code run faster. In this video, let's delve into how broadcasting in Python actually works. Let's motivate broadcasting with an example. In this matrix, I've shown the number of calories from carbohydrates, proteins and fats in 100 grams of four different foods. So for example, 100 grams of apples, turns out, has 4,56 calories from carbs and much less from proteins and fats. Whereas in contrast, 100 grams of beef has 104 calories from protein, 135 calories from fat. Now let's say your goal is to calculate the percentage of calories from carbs, proteins and fats for each of the four foods. So for example, if you look at this column and add up the numbers in that column, you get that 100 grams of Apple has 56 plus 1.2 plus 1.8. So that's 59 calories. And so as a percentage, the percentage of calories from carbohydrates in an Apple would be 56 over 59. That's about 94.9%. So most of the calories in an apple come from carbs, whereas in contrast, most, most of the calories of beef come from protein and fat and so on. So the calculation you want is really to sum up each of the four columns of this matrix to get the total number of calories in 100 grams of apple, beef, eggs or potatoes, and then to divide throughout the matrix so as to get the percentage of calories from calculated carbs, proteins and fats for each of the four foods. So the question is, can you do this without an explicit for loop? Let's take a look at how you could do that. What I'm going to do is show you how you can set, say, this matrix equal to a three by four matrix A. And then with one line of Python code, we're going to set sum down the columns. So we're going to get four numbers corresponding to the total number of calories in these four different types of food, 100 grams of these four different types of food. And I'm going to use a second line of Python code to divide each of the four columns by their corresponding sum. If that verbal description wasn't very clear, hopefully it'll be clearer in a second when we look at the Python code. So here we are in the Jupyter notebook. I've already written this first piece of code to pre populate the Matrix A with the numbers we had just now. So let me hit shift enter and just run that. So there's a matrix A. And now here are the two lines of height and code. First I'm going to compute cal equals a dot sum and x equals zero means to sum vertically. Say more about that a little bit. And then let's print cal. So we've sum vertically. So as we say just now, 59 is the total number of calories in the Apple, 239 was the total number of calories in the beef and eggs and potatoes and so on. And then we're gonna compute percentage equals A over cow V shape 1 by 4. Actually, we want percentages multiplied by 100 here. And then let's print percentage, let's run that. And so that command, we've taken the matrix A and divided it by this one by four matrix. And this gives us the matrix of percentages. So as we worked out kind of by hand just now, you know, in the, in the apple, that was the first column, 94.9% of the calories are from carbs. Let's go back to the slides. So just to repeat the two lines of code we had, this is what we had written out in the Jupyter notebook. To add a bit of detail, this parameter axis equals zero means that you want Python to sum vertically. So this is axis. 0 means to sum vertically, whereas the horizontal axis is axis 1. So if you were to write axis 1, it'll sum horizontally instead of sum vertically. And then this command here, this is an example of Python broadcasting where you take a matrix A, so this is a three by four matrix and you divide it by a one by four matrix. And technically, after this first line of code, the variable cal is already a one by four matrix. So technically you don't need to call reshape here. Again, so that's actually a little bit redundant. But when I'm writing Python codes, if I'm not entirely sure what matrix, what, what, what are the dimensions of my matrix, I often would just call a reshape command just to make sure that is, you know, the right column vector or the row vector or whatever you want it to be. The reshape command is a constant time, so order one operation. And so it's very cheap to call. So don't be shy about using the reshape command to make sure that your matrices are the size you need it to be. Now, let's explain in greater detail how this type of operation works. Right, we had a 3 by 4 matrix and we divided it by a 1 by 4 matrix. So how can you divide a 3 by 4 matrix by a by a 1 by 4 matrix or by 1 by 4 vector? Let's go through a few more examples of broadcasting. If you take a four by one vector and add it to a number, what Python will do is take this number and auto expand it into a four by one vector as well. And as follows. And so the vector 1, 2, 3, 4 plus the number 100 ends up with that vector on the right. You're adding 100 to every element. And in fact we use this form of broadcasting where that constant was the parameter B in an earlier video. And this type of broadcasting works with both column vectors and row vectors. And in fact we use a similar form of broadcasting earlier with the constant we're adding to a vector being the parameter B in logistic regression. Here's another example. Let's say you have a two by three matrix and you add it to this one by n matrix. So the general case would be if you have some m by n matrix here and you add it to A1 matrix by N matrix, what Python will do is copy the matrix M times to turn this into an m by n matrix. So instead of this one by three matrix, it'll copy it twice in this example to turn it into this also two by three matrix, and they'll add these. So you end up with this sum on the right. Okay, so you've taken, you added 100 to the first column, added 200 to the second column, added 300 to the third column. And this is basically what we did on the previous slide, except that we use a division operation instead of an addition operation. Just one last example. What if you have a m by n matrix and you add this to an m by one vector, an n by one matrix, then this will copy this N times horizontally. So you end up with an N by N matrix. So as you can imagine, it would copy it horizontally three times and add those. So when you add them, you end up with this. So we've added 100 to the first row and added 200 to the second row. Here's the more general principle of broadcasting in Python. If you have an M by n matrix and you add or subtract or multiply or divide with A1 by N Matrix, then this will copy it m times into an M by n matrix and then apply the addition, subtraction, multiplication or division element wise. If conversely, you were to take the m by n matrix and add, subtract, multiply or divide by m by one matrix. Then also this would copy it now N times and turn that into an m by n matrix and then apply the operation element y. Just one other form of broadcasting, which is if you have an M by one matrix, so that's really a column vector like 1, 2, 3 and you add subtract, multiply or divide by a real number, so maybe a one by one matrix, so such as that plus 100. Then you end up copying this real number n times until you also get another m by one matrix. And then you perform the operation such as addition in this example element twice. And something similar also works for row vectors. The fully general version of broadcasting can do even a little bit more than this. If you're interested, you can read the documentation for NUMPY and look at broadcasting in that documentation. That gives an even slightly more general definition of broadcasting. But the ones on this slide are the main forms of broadcasting that you end up using needing to use when you implement a neural network. Before we wrap up, just one last comment, which is for those of you that are used to programming in either Matlab or Octave. If you've ever used the Matlab or Octave function bsxfun in neural network programming, bsxfun does something similar, not quite the same, but is often used for a similar purpose as what we use broadcasting in Python for. But this is really only for very advanced Matlab and Octave users. If you've not heard of this, don't worry about it, you don't need to know it when you're coding up neural networks in Python. So that was broadcasting in Python. I hope that when you do the programming homeworks that broadcasting will allow you to not only make your code run faster, but also help you get what you want done with fewer lines of code. Before you dive into the programming exercise, I want to share with you just one more set of ideas, which is that there's some tips and tricks that I found reduces the number of bugs in my Python code and that I hope will help you too. So that. Let's talk about that in the next video.