Neural style transfer uses a content cost function that compares activations from a mid-level layer in a pretrained network like VGG. The deeper the layer, the more the algorithm preserves semantic content over pixel-perfect details. Content cost is the normalized sum of squared differences between activations in that layer for the original and generated images.
The cost function of the neural style transfer algorithm had a content cost component and a style cost component. Let's start by defining the content cost component. Remember that this is the overall cost function of the neural style transfer algorithm. So let's figure out what should the content cost function be. Let's say that you use hidden layer L to compute the content cost. If L is a very small number, if you use hidden layer 1, then it will really force your generated image to have pixel values very similar to your content image. Whereas if you use a very deep layer, then it's just asking, well, if there's a dog in your content image, then make sure there's a dog somewhere in your generated image. So in practice, the layer L chosen is somewhere in between. It's neither too shallow nor too deep in the neural network. And because you play with this yourself in the programming exercise at the end of this week, I'll leave you to gain some intuitions with the concrete examples in the programming exercise as well. But usually L is chosen to be somewhere in the middle of the layers of the neural network, neither too shallow nor too deep. What you can do is then use a pre trained convnet, maybe a VGG network, or it could be some other neural network as well. And now you want to measure given a content image and given a generated image, how similar are they in content? So let's let this a super square bracket L, round bracket C and this be the activations of layer L on these two images on the images C and G. So if these two activations are similar, then that would seem to imply that both images have similar content. So what we'll do is define j content C comma G as just how different, how similar, how different are these two activations? So we'll take the element wise difference between these hidden unit activations in layer L between when you pass in the content image compared to when you pass in the generated image. And take that squared and you can have a normalization constant in front or not, such as one over two or something else, it doesn't really matter since this will be can be adjusted as well by this hyperparameter alpha. So just be clear, I'm using this notation as if both of these have been unrolled into vectors. So then this becomes the squared of the L2 norm between this and this after you've unrolled them both into vectors, there's really just the element wise sum of squared differences between these two activation, but it's really just the element wise sum of squares of differences between the activations in layer L between the images C and G. And so when later you perform gradient descent on J to try to find a value of G so that the overall cost is low, this will incentivize the algorithm to find an image G so that these hidden layer activations are similar to what you got for the content image. So that's how you define the content cost function for neural style transfer. Next, let's move on to the style cost function.
