Python Tutorial : Advanced operations

In this video, we will cover a selection of advanced operations. Some will be used frequently in later chapters. Others will help you to gain intuition about complex machine learning routines. We've already covered basic operations of TensorFlow, including add, multiply, matmul and reducesum. In this lesson, we will move on to more advanced operations, including gradient, reshape and random. The gradient operation, which we'll use in conjunction with gradient tape, computes the slope of a function at a point. The reshape operation changes the shape of a tensor, and the random module generates a tensor out of randomly drawn values. In many machine learning problems, you will need to find an optimum that is a minimum or a maximum. You may, for instance, want to find the model parameters that minimize the loss function or maximize the objective function. Fortunately, we can do this using the gradient operation, which tells us the slope of a function at a point. We start this process by passing points to the gradient operation until we find one where the gradient is zero. Next, we check if the gradient is increasing or decreasing at that point. If it's increasing, we have a minimum, otherwise we have a maximum. The plot shows the function Y equals X. Note that the gradient, I.e. the slope at a given point, is constant. If we increase X by one unit, Y also increases by one unit. This is not true if we instead consider the function Y equals X squared when X is less than 0. Y decreases when X increases. When X is greater than 0. Y increases when X increases. Thus, the gradient is initially negative but becomes positive for x larger than zero. This means that x equals zero minimizes y. Let's use TensorFlow to compute the gradient. We will start by defining a variable X, which we will initialize to minus 1.0. We will then define Y to be X squared within an instance of gradient tape. Note that we apply the watch method to an instance of gradient tape and then pass the variable X. This will allow us to compute the rate of change of Y with respect to X. Next, we compute the gradient of Y with respect to X using the tape instance of gradient tape. Note that Y is the first argument and X is the second. As written, the operation computes the slope of Y at a point. Running the code and printing, we find that the slope is minus 2 at x equals minus 1, which means that Y is initially decreasing in X. As we saw in the previous slide, much of the differentiation you do in deep learning models will be handled by high level APIs. However, gradient tape remains an invaluable tool for building advanced and custom models. We'll next consider an operation that is particularly useful for image classification problems. Reshape it the grayscale image shown has a natural representation as a matrix with values between 0 and 255. While some algorithms exploit this shape, others require you to reshape matrices into vectors before using them as inputs, as shown in the diagram. Now that you've seen how images can be represented as tensors, let's generate some input images and reshape them. We will create a random grayscale image by drawing numbers from the set of integers between 0 and 255. We will use these to populate a 2x2 matrix. We can then reshape this into a 4x1 vector, as shown in the diagram. For color images, we will generate three such matrices to form a 2x2x3 tensor. We could then reshape the image into a 4x3 tensor, as shown in the diagram. It's time to put what you've learned to work in some extra.