Neural style transfer blends a content image and style image by defining a cost function J(G)=α·Jcontent(C,G)+β·Jstyle(S,G), then using gradient descent to update pixel values of a generated image G. The weighting parameters α and β control how much content versus style appears in the result.
To build a neural style transfer system, let's define a cost function for the generated image. What you see later is that it'll be by minimizing this cost function that you can generate the image that you want. So remember what the problem formulation is. You're given a content image C, given a style image S, and your goal is to generate a new image G. So in order to implement neural style transfer, what you're going to do is, what you're going to do is define a cost function J of G that measures how good is a particular generated image. And we'll use gradient descent to minimize J of G in order to generate this image. So how good is a particular image? Well, we're going to define two parts to this cost function. The first part is called the content cost. This is a function of the content image and of the generated image. And what it does is it measures how similar is the content of the generated image to the content of the content image c. And then going to add that to a style cost function which is now a function of S comma g. And what this does is it measures how similar is the style of the image G to the style of the image S. Finally, we'll weight these with two hyperparameters, alpha and beta, to specify the relative weighting between the content cost and the style cost. It seems redundant to use two different hyperparameters to specify their relative weighting. Seems that you only need one hyper. It seems redundant to use two hyperparameters to specify the relative cost of the weighting. One hyperparameter seems like it would be enough. But the original authors of the neural style transfer algorithm use two different hyperparameters. And so I'm just going to follow their convention here. And the neural style transfer algorithm I'm going to present in the next few videos is due to Leon Gattis, Alexandra Ecker and Matthews, and their paper is not too hard to read. So after watching these few videos, if you wish, I certainly encourage you to take a look at their paper as well, if you want. So the way the algorithm will run is as follows. Having defined the cost function J in order to actually generate a new image, what you do is the following. You would initialize the generated image G randomly, so it might be 100 by 100 by 3 or 500 by 500 by 3 or whatever dimension you want it to be. Then we'll define the cost function J of G on the previous slide. So what you can now do is use gradient descent to minimize this. So you can update G as G minus the derivative with respect to the cost function of J of G. So in this process, you're actually updating the pixel values of this image G, which is a 100 by 100 by 3, maybe RGB channel image. So here's an example. Let's say you start with this content image and this style image. This is another Pablo Picasso image. Then, when you initialize G randomly, your initial randomly generated image is just this white noise image with each pixel value chosen at random. And then as you run gradient descent, you'd minimize the cost function J of G. You'd slowly tweak the pixel values. So. So then you get slowly an image that looks more and more like your content image, but rendered in the style of your style image. So in this video, you saw the overall outline of the neural style transfer algorithm, where you define a cost function for the generated image G and minimize it. Next, we need to see how to define the content cost function as well as the style cost function. Let's take a look at that starting in the next video.
