Andrew Ng explains how numpy's rank 1 arrays can cause bugs in Python code. He shows why you should use explicit column or row vectors instead, and recommends assertion statements to catch shape mismatches before they break your math.
The ability of Python to allow you to use broadcasting operations and more generally, the great flexibility of the Python numpy programming language is I think, both a strength as well as a weakness of the programming language. I think it's a strength because the great expressivity of the language, the great flexibility of the language lets you get a lot done with even just a single line of code. But it's also weakness because the with broadcasting and this great amount of flexibility, sometimes it's possible you can introduce very subtle bugs or very strange looking bugs if you aren't familiar with all of the intricacies of how broadcasting and how features like broadcasting work. For example, if you take a column vector and add it to a row vector, you might expect it to throw up a, you know, dimension mismatch or type error or something, but you might actually get back a matrix as a sum of a row vector and a column vector. So there is an internal logic to these strange effects of Python. But if you aren't familiar with Python now I've seen some students have very strange, very hard to find bugs. So what I want to do in this video is share with you a couple of tips and tricks that have been very useful for me to eliminate or simplify and eliminate a lot of the strange looking bugs in my own code. And I hope that with these tips and tricks you'll also be able to much More easily write bug 3 python and numpy code. To illustrate one of the less intuitive effects of Python numpy, especially how you construct vectors in Python numpy, let me do a quick demo. Let's say I set a equals np random randn 5. So this creates a 5 random Gaussian variables stored in an array A And so let's print A, um, and now it turns out that the shape of A when you do this is this 5 comma um structure. And so this is called a rank 1 array in Python and it's neither a row vector nor a column vector. And this leads it to have some slightly non intuitive effect. So for example, if I print a transpose, it ends up looking the same as a. So A and a transpose end up looking the same. And if I print these inner product between A and a transpose, you know, you might think A times a transpose is maybe the outer product should give you a matrix. Maybe. But if I do that, you instead get back a number. So what I would recommend is that when you're coding neural networks that you just not use data structures where the shape is this you Know five comma or n comma rank one array. Instead, if you set A to be this five by one, then this commits A to be a five by one column vector. And whereas previously, a and a transpose look the same, if you now print a transpose. Now a transpose is a row vector. Notice one subtle difference. In this data structure, there are two square brackets when you print a transpose, whereas previously there was one square bracket. So that's the difference between a different is really a one by five matrix versus a one of these rank one arrays. And if you print, say the product between a and a transpose, then this gives you the outer product of a vector, right? And so the out product of a vector gives you a matrix. So let's look in greater detail at what we just saw here. The first command that we ran just now was this. And this created a data structure where a dot shape was this a funny thing? 5 comma. And so this is called a rank one array. And this is a very funny data structure. It doesn't behave consistently as either a row vector nor a column vector, which makes some of its effects non intuitive. So what I'm gonna recommend is that when you're doing your programming exercises or in fact when you're implementing logistic regression on neural networks, that you just do not use these Rank 1 arrays. Instead, if every time you create an array you commit to making it either a column vector, so this creates a five by one vector, or commit to making it a row vector, then the behavior of your vectors will maybe easier to understand. So in this case a dot shape is going to be equal to 5 comma 1. And so this behaves a lot like a, or in fact this is a column vector. And that's why you might, you can think of this as a five by one matrix or as a column vector. And here a dot shape is going to be 1 comma 5, and this behaves consistently as a row vector. So when you need a vector, I would say either use this or this, but not a rank one array. One more thing that I do a lot in my code is if I'm not entirely sure what's the dimension of one of my vectors, I'll often throw in an assertion statement like this to make sure in this case that this is a five by one vector. So this is a column vector. These assertions are really inexpensive to execute and they also help to serve as documentation for your code. So don't hesitate to throw in assertion statements like this one if you feel like it. And Then finally, if for some reason you do end up with a rank one array, you can reshape this A equals a reshape into say a five by one array or one by five array so that it behaves more consistently as either a column vector or a row vector. So I've sometimes seen students end up with very hard to track down bugs because of some of the non intuitive effects of rank one arrays. By eliminating rank one arrays on my own code, I think my code became simpler and I did not actually find it restrictive in terms of things I could express in code to just never use a rank one array. And so the takeaways are to simplify your code, don't use rank one arrays. Always use either n by one matrices, basically column vectors, or one by n matrices or basically row vectors. Feel free to toss a lot of assertion statements to double check the dimensions of your matrices and arrays. And also don't be shy about calling the reshape operation to make sure that your matrices or your vectors are the dimension that you need it to be so that I hope that this set of suggestions helps you to eliminate a class of bugs that from Python code and makes the programming exercise easier for you to complete.
