CS231n Winter 2016: Lecture 13: Segmentation, soft attention, spatial transformers

So our administrative points for today. Assignment 3 is due tonight. So who's done? That's good. Was it easier than Assignment 2? [Speaker:AUDIENCE] Easier. [Speaker:ANDREJ KARPATHY] OK. That's good. Hopefully it gives you more time to work on your projects. So also remember your milestones were turned in last week. So we're in the process of looking through the milestones to make sure those are OK. And also we're working on Assignment 2 grading. So we should have that done sometime this week or early next week. Last time we had a whirlwind tour of all the software— all the common software packages that people use for deep learning. And we saw a lot of code on slides and a lot of stepping through code. And hopefully that you found that useful for your projects. Today we're going to talk about two other topics. We're going to talk about segmentation. Within segmentation, there are two subproblems, semantic and instance segmentation. Um, we're also gonna talk about soft attention. Um, and within soft attention, again, there's sort of two buckets that we've divided things into. Uh, but first, before we go into all these details, um, I— there's something else I wanna bring up briefly. So, uh, this is the ImageNet classification errors. Um, I think at this point in the class, you've seen this type of figure many times, right? So in 2012, um, AlexNet, uh, 2013, ZF crushed it. Recently GoogleNet and later ResNet is sort of held the— won the classification challenge in 2015. But turns out as of today there's a new ImageNet result. So this paper came out last night. So Google actually has now state of the art on ImageNet with 3.08% top 5 error. Which is crazy. Um, and the way that they do this is with, um, this thing that they call Inception-v4. Um, this is a little bit of a monster, so I don't wanna go into too much detail. Um, but you can see that it's this really deep network, um, that has these repeated modules. So here there's the stem. Um, the stem is this guy over here. Um, a couple interesting things to point out about this architecture are actually they use some valid convolutions, which means they have no padding. So that makes every— all the math more complicated, but they're smart and they figured things out. They also— an interesting feature here is they actually have in parallel strided convolution and also max pooling. So they kind of do these two operations in parallel to downsample images and then concatenate again at the end. Another thing is they, they're really going all out on these efficient convolution tricks that we talked about a couple lectures ago. So as you can see, they've actually have these asymmetric filters like 1 by 7 and 7 by 1 convolutions. They also make heavy use of these 1x1 convolutional bottlenecks to reduce computational costs. So this is just the stem of the network, and actually each of these parts is sort of different. So they've got 4 of these inception modules, then this downsampling module, then what, 7 of these guys, and then another downsampling module, and then 3 more of these guys, and then finally they have dropout and, and a fully connected layer for the class labels. So another thing to point out is again, there's no sort of fully connected hidden layers here. They just have this global averaging to compute the final feature vector. And another cool thing they did in this paper was Inception ResNets. So they proposed this residual version of the Inception architecture, which is also pretty big and scary. The stem is the same as before. And now these, these repeated Inception blocks that they repeat throughout the network, they actually have these residual connections. So that's, that's kind of cool. They're kind of jumping on this residual idea and now improved state-of-the-art on ImageNet. So again, they have many repeated modules. And when you add this thing all up, it's about 70— it's 75 layers, assuming I did the math right last night. So they also show that between their sort of new Inception, their new version 4 of Inception GoogleNet, and their residual version of GoogleNet, that actually both of them perform about the same. So this is top 5 error as a function of epochs on ImageNet. So you can see that the Inception network in red actually is converging a bit faster. But over time, both of them sort of converge to about the same value. So that, that's kind of interesting. That's kind of cool. Another thing that's kind of interesting to point out is this— there's the raw numbers on the x-axis here. These are epochs on ImageNet. These things are being trained for 160 epochs on ImageNet. So that's a lot of training time. But that's enough of current events. And let's go back to our regularly scheduled programming. So today— oh, yeah, question? [Speaker:AUDIENCE] Do you know how many parameters they had in that? [Speaker:ANDREJ KARPATHY] I don't know. I think it might be in the paper, but I didn't read too carefully. Any other questions on— yeah? [Speaker:AUDIENCE] So the dropout is in the last layer? [Speaker:ANDREJ KARPATHY] The question is about dropout, whether it's only in the last layer. I'm not sure. Again, I didn't read the paper too carefully yet. But the link is here. You should check it out. OK. So today we're going to talk about two other topics that are common things in research these days. So those are segmentation, which is this sort of classic computer vision topic, and also this idea of attention, which I think is a really— has been a really popular thing to work on in deep learning over the past year especially. So first, we're going to talk about segmentation. So you may have remembered this slide from a couple lectures ago when we talked about object detection that was talking about different tasks that people work on in computer vision. And we spent a lot of time in the class talking about classification. And back in lecture 8, we talked about different models for localization and for object detection. But today, we're actually going to focus on this idea of segmentation that we skipped over last time in this previous lecture. So within segmentation, there's sort of two different subtasks that we need to make— that we need to define, and people actually work on these things a little bit separately. The first task is this idea called semantic segmentation. So here we take an in— we have an input image and we have some fixed number of classes, things like buildings and tree and ground and cow and whatever kind of semantic labels you want. Usually you have some small fixed number of classes. Also typically you'll have some background class for, for things that don't fit into these classes. And then the task is that we want to take as input an image and then we want to label every pixel in that image with one of these semantic classes. So here we've taken this input image of these 3 cows in a field, and the ideal output is this image where instead of being RGB values, we actually have one class label per pixel. Um, so we can do this on other images and maybe segment out the trees and the sky and the road, the grass. So this type of task is pretty cool. I think it gives you sort of a higher-level understanding of what's going on in images compared to just putting a single label on the whole image. And this is actually a very old problem in computer vision. So this, uh, that predates sort of the deep learning revolution. So this figure actually comes from a computer vision paper back in 2007, um, that didn't use any deep learning at all. People had other methods for this, uh, a couple years ago. The other task that people work on is this thing— oh, right. So the, the thing to point out here is that this thing is not aware of instances. So here, this, this image actually has 4 cows. There's actually 3 cows standing up and 1 cow kind of laying on the grass taking a nap. But here in this output, it's not really clear how many cows there are. These different cows, actually their pixels overlap. So here in the output, there's no notion that there are different cows. In this output, we're just labeling every pixel. So it's maybe not as informative as you might like. and that could actually lead to some problems for some downstream applications. So to overcome this, people have, um, also separately worked on this newer problem called instance segmentation. Um, this also sometimes gets called simultaneous detection and segmentation. So here the problem is, um, again similar to before, we have some set of classes that we're trying to recognize, and given an input image, we want to output, um, all instances of those classes. And for each instance, we want to segment out the pixels that belong to that instance. So here in this, in this input image, there are actually 3 different people. There's the 2 parents and the kid. And now in the output, we actually distinguish between those different people in the input image, which those 3 people are now shown in different colors to indicate different instances. And again, for each of those instances, we're going to label all of the pixels in the input image that belong to that instance. So these two tasks of semantic segmentation and instance segmentation, people actually have worked on them a little bit separately. So first, we're going to talk about some models for semantic segmentation. So remember, this is the task where you want to just label all the pixels in the image and you don't care about the instances. So here, the idea is actually pretty simple. Given some input image, this is the fun one with the cows, we are going to take some little patch of the input image and extract this patch that sort of gives local information in the image. Then we're going to take this patch and we're going to feed it through some convolutional neural network. This could be any of the architectures that we've talked about so far in the class. And now this convolutional neural network will actually classify the center pixel of the patch. So this neural network is just doing classification. That's something we know how to do. So this thing is just going to say that the center pixel of this patch actually is a cow. Then we can imagine taking this network that works on patches and labels the center pixel, and we just run it over the entire image, and that will give us a label for each pixel in the image. So, um, so this actually is a very expensive operation, right? Because now there's many, many patches in the image and it would be super, super expensive to run this network independently for all of them. So in practice, people use the same trick that we saw in object detection where you'll run this thing fully convolutionally and get all the outputs for the whole image all at once. But the problem here is that if your convolutional network contains any kind of downsampling, either through pooling or through strided convolutions, then now your output image will actually have a smaller spatial size than your input image. So that's something that people need to work around when they're using this type of approach. So any questions on this kind of basic setup for semantic segmentation? Yeah. [Speaker:AUDIENCE] Isn't there cases when the patch of a cow doesn't allow to classify as a cow, for example? —when you have to take close patches to region or something? Yeah, the question is whether that patchwise thing just doesn't give you enough information in some cases. And that's true. So sometimes for these networks, people actually have a separate offline refinement stage where they take this output and then feed it to some kind of graphical model to clean up the output a little bit. So sometimes that can help boost your performance a little bit. But just this raw input-output mode setup tends to work pretty well just as something easy to implement. Yeah. [Speaker:AUDIENCE] In practice, what is the receptive field that you need in order to get decent results? [Speaker:ANDREJ KARPATHY] The question is how big of a receptive field do you need? I'm not sure exactly. Probably pretty big, maybe 100 pixels, that order of magnitude. So one extension that people have used to this basic approach is this idea of multiscale testing. So actually sometimes a single scale isn't enough. So here we're going to take our input image and we'll actually resize it to multiple different sizes. Um, so this is sort of a common trick that people use in computer vision a lot. It's called an image pyramid. You just take the same image and you resize it to many different scales. And now for each of these scales, we're going to run it through a convolutional neural network that is going to predict these pixel-wise labels for these different images at these different resolutions. So another thing to point out here along the lines of your question, that if each of these networks actually has the same architecture, then each of these outputs will have a different effective receptive field in the input due to the, due to the image pyramid. So now that we've gotten these differently sized pixel labels for the image, then we can take all of them and resa— and just do some offline upsampling to upsample those responses to the same size as the input image. So now we've gotten our 3 outputs of different sizes, and we'll just upsample them and stack them. And this paper, this is actually a paper from Jan LeCun back in 2013. So they actually also have this separate offline processing step where they do this idea of a bottom-up segmentation using —right, using these superpixel methods. So these are these sort of more classic computer vision image processing type methods that actually look at the differences between adjacent pixels and images and then try to merge them together to give you these coherent regions where there are not much change in image. So then this method actually takes— sort of runs the image offline through these other more traditional image processing techniques to get either a set of superpixels or a tree saying which pixels ought to be merged together in the image. And they have this somewhat complicated process for merging all these different things together. 'Cause now we've gotten this sort of low-level information saying which pixels in the image actually are similar to each other based on sort of color and gray— and edge information. And we've got these outputs of different resolutions from the convolutional neural networks telling us semantically what the labels are at different points in the image. And they use— they actually explore a couple different ideas for merging these things together to give you your final output. So this actually also answers one of— addresses one of the earlier questions about maybe the ConvNet not being enough on its own. So using these external superpixel methods or these segmentation trees is another thing that sort of gives you additional information about maybe larger context in the input images. So any questions about this model? OK. So another sort of cool idea that people have used for semantic segmentation is this idea of iterative refinement. So we actually saw this a few lectures ago when we mentioned pose estimation. But the idea is that we're going to have an input image. Here they've separated out the 3 channels. And we're going to run that thing through our favorite sort of convolutional neural network to predict these low-resolution patches, or rather to predict this low-resolution segmentation of the image. And now we're going to take that output from the CNN together with a downsampled version of the original image, and we'll just repeat the process again. So this allows the network to sort of, one, increase its effective receptive field of the output, and also to perform more processing on the, on the input image. And then we can repeat this process again. So this is kind of cool. So if these 3 convolutional networks actually share weights, then this becomes a recurrent convolutional network where it's sort of operating on the same input over, over through time. But actually each of these update steps is a whole convolutional network. Um, so it's actually a very similar idea to recurrent networks that we saw previously. And the idea behind this paper, which was in, I think, ICML 2014, is that if you actually do more iterations of this same type of thing, then hopefully it allows the network to sort of iteratively refine its outputs. So here, if we have this raw input image, then after one iteration, you can see that actually there's quite a bit of noise, especially around the boundaries of the objects. But as we run for 2 and 3 iterations through this recurrent convolutional network, it actually allows the network to clean up a lot of that sort of low-level noise and produce much cleaner, much cleaner and nicer results. So I thought that was quite, quite a cool idea that sort of merging together these idea of recurrent networks and sharing weights over time with this idea of convolutional networks to process images. So another, another very widely, very well, very, very well-known paper for semantic segmentation is this one from Berkeley that was published at CVPR last year. So here it's a very similar model as before where we're going to take an input image and run it through some number of convolutions and eventually extract some, some feature map for the pixels. But in contrast, the previous methods all relied on this sort of hard-coded upsampling to actually produce the final segmentation for the image. But in this paper, they proposed that, well, we're, we're deep learning people, we want to learn everything. So we're going to learn the upsampling as part of the network. So their network includes this at the last layer, this learnable upsampling layer that actually upsamples the feature map in a learnable way. So yes, they have this learnable upsampling map at the end. And the way their model kind of looks is that they have this— at the time it was an AlexNet. So they have their input image running through many phases of convolution and pooling. And eventually they produce, um, at this pool 5 output, they have quite a downsampled image, uh, quite a downsampled spatial size compared to the input image. And then their learnable upsampling, uh, re-upsam— upsamples that back to the original size of the input image. Another cool feature of this paper was this idea of skip connections. So, um, they actually don't use only just these pool 5 features, they actually use the convolutional features from different layers in the network, um, which sort of exist at different scales. So you can imagine that, uh, once you're in the pool 4 layer of AlexNet, that's actually a bigger feature map than the pool 5, and pool 3 is even bigger than pool 4. So the intuition is that these lower convolutional layers might actually help you capture finer-grained structure in the input image since they have a smaller receptive field. So they actually in practice take these different convolutional feature maps and apply a separate learned upsampling to each of these feature maps and then combine them all to produce the final output. And in their results, they show that actually adding these skip connections tends to help a lot with these low-level details. So, um, over here on the left, these are the results that only use, um, these pool 5 outputs. And you can see that, uh, it's sort of gotten the rough idea of a person on a bicycle. but it's kind of blobby and missing a lot of the fine details around the edges. But then when you add in these skip connections from these lower convolutional layers, that gives you a lot more fine-grained information about the spatial locations of things in the image. So adding those skip connections from the lower layers really helps you clean up the boundaries in some cases for these outputs. Question? How do they classify accuracy compared to ground truth? So the question is how to classify accuracy. I think the two metrics people typically use for this are either just classification, because you're classifying every pixel, so you can use classification metrics. Also, sometimes people use intersection over union. So for each class, you compute what is the region of the image that I predicted as that class and what was the ground truth region of the image that had that class, and then compute an intersection over union metric between those two. I'm not sure which, which metric this paper used in particular. So this idea of learnable upsampling is actually really cool. And since, since this paper has been applied in a lot of other contexts, because we've seen that we can downsample our feature maps in a variety of different ways, but being able to upsample them inside the network could actually be very useful and a very valuable thing to do. So this sometimes gets called deconvolution. That's not a very good term, so we'll talk about that in a couple minutes, but it's a very common term. So just to recap, sort of when you're doing a normal sort of stride 1 3x3 convolution, we have this picture that should be pretty familiar by now. That given our 4x4 input, we have some 3x3 filter, and we plop that 3x3 filter over part of the input, compute a dot product, and that gives us one element of the output. And now because this thing is stride 1, to compute the next elements of the output, we, we move the filter over one slot in the input. Again, compute a dot product, and that gives us our one element in the output. And now for stride 2 convolution, it's, it's a very similar type of idea where now our output is going to be a downsampled version, a 2 by 2 output for our 4 by 4 input. And again, it's the same idea. We take our filter. We plop it down on the image, compute a dot product, and that gives us one element of the output. The only difference is that now we slide the convolutional filter over two slots in the input to compute one element of the output. The deconvolutional layer actually does something a little bit different. So here we want to take a low-resolution input and produce a higher-resolution output. So this would be maybe a 3x3 deconvolution with a stride of 2 and a pad of 1. So here, this is a little bit weird. You know, in a normal convolution, you imagine— you have your 3x3 filter, and you take dot products on the input. But here, you want to imagine taking your 3x3 filter and just copying it over to the output. The only difference is that the weights, like this one scalar value of the weight in your input, gives you a weight for that— you're going to reweight that filter when you stamp it down into the output. And now when we stride this thing along, we're going to step 1 step over in the input and 2 steps over in the output. So now we're going to take the same learned convolutional filter, and we're going to plop it down in the output. But now in the blue— now, so we're taking the same convolutional filter, and we're popping it down twice in the output. The difference being that the red box, that convolutional filter is weighted by this scalar value in the input. And for the blue box, that convolutional filter is weighted by the blue scalar value in the input. And where these regions overlap, you just add. So this kind of allows you to learn an upsampling inside the network. So if you remember from implementing convolutions on the assignment, this idea of sort of especially striding and adding on overlapping regions That should remind you of the backward pass for a normal convolution. And it turns out that these are completely equivalent, that this deconvolution forward pass is exactly the same as the normal convolution backward pass. And the normal— and the deconvolution backward pass is the same as the normal convolution forward pass. So because of that, actually the term deconvolution is maybe not so great. And if you have a signal processing background, you may have seen that deconvolution already has a very well-defined meaning, and that is the inverse of convolution. So a deconvolution should undo a convolution operation, which is quite different from what this is actually doing. So probably better names for this instead of deconvolution that you'll sometimes see would be convolution transpose or backward strided convolution or fractionally strided convolution or up convolution. So I think those are all kind of weird names. I think deconvolution is popular just because it's easiest to say, even though it's maybe less technically correct. Although actually, if you read papers, you'll see that some people get angry about this. So it's more proper to say convolution transpose instead of deconvolution. And this other paper really wants it to be— call it fractionally strided convolution. So I think the community is still deciding on the right terminology here. But I kind of agree with them that deconvolution is probably not very technically correct. And this paper in particular has a whole— they felt very strongly about this issue. And they had a one-page index— appendix to the paper actually explaining why convolution transpose is the proper term. So if you're interested, then I would really recommend checking out that. It's a pretty good explanation actually. So any questions about this? Yeah. [Speaker:AUDIENCE] Do you have an idea how faster is this network is going to be relative to convolution? —to the patch-based? I think— so the question is how much faster is this relative to a patch-based thing? The answer is that in practice, nobody even thinks to run this thing in a fully patch-based mode because that would just be way, way too slow. So actually, all of the papers that I've seen do some kind of fully convolutional thing in one way or another. Actually, there is sort of another trick instead of upsampling that people sometimes use. And that is that— So suppose that your network is actually going to downsample by a factor of 4. Then one thing you can do is take your input image, shift it by 1 pixel, and now run it through the network again. And now you get another output. And you repeat this for sort of 4 different 1-pixel shifts of the input. And now you've gotten 4 output maps. And you can sort of interleave those to reconstruct an original input map. So that's, that's another trick that people sometimes use to get around that problem. But I think that this learnable upsampling is quite a bit cleaner. Yeah, question? What alternative name for deconvolution do you prefer? So I think deconvolution is really nice. It just rolls off the tongue. I think backward stride— I think fractionally strided convolution is actually pretty cool. Right? I think it's the longest name, but it's really descriptive. Because normally with a strided convolution, you move x elements in the input and you move like you move 1 in the input and you move x in the output. And here you're moving 1/2 in the input. You move 1 in the input, which corresponds to moving 1/2 in the output. So that captures this idea quite nicely. So I'm not sure what I'll call it when I use it in a paper. We'll have to see about that. But so despite the concerns about people calling it deconvolution, like people just call it that anyway. So there was this paper from ICCV that takes this idea of this— of this convolutional fractionally strided convolutional idea and sort of pushes it to the extreme. So here they took what amounts to two entire VGG networks flipped on their head. So this is again the exact same model as before. We want to input an image and output our pixel-wise predictions for the semantic segmentation task. But here we initialize with a VGG, and over here is an upside-down VGG. And it trains for 6 days on a Titan X. So this thing is pretty slow. But they actually got really, really good results. And I think it's also a very beautiful figure. So that's pretty much all that I have to say about semantic segmentation. If there's any questions about that. Yeah. Is there a LaTeX package for making neural network figures? The question is, how is this figure made? The answer is I took a screenshot from their paper, so I don't know. But you could try TensorFlow. We saw in the last lecture that that lets you make figures, but they're not as nice as this. Yeah. Yeah. How do we get the training data of this? Question is training data. Yeah. Yeah. So there exist datasets with this kind of thing where I think like there's a common one is the Pascal segmentation dataset. So it just has ground truth. You have an image, you have an image, and they have every pixel labeled in the output. [Speaker:AUDIENCE] Made by hand? [Speaker:ANDREJ KARPATHY] Yeah, it's kind of expensive to get that data. So the datasets tend to be a little smaller. But in practice, there's a famous interface called LabelMe where you could upload an image and then sort of draw a contour around the image or around different regions of the image. And then that you can convert those, contours into sort of these segmentation masks. So that's how you tend to label these things in a UI— in a UI way. If no other questions, then I think we'll move on to instance segmentation. So just to recap, instance segmentation is this generalization where we not only want to label the pixels of the image, but we also want to distinguish instances. So we're going to detect the different instances of our classes And for each one, we want to label the pixels of that instance. So these— actually, these models end up looking a lot like the detection models that we talked about a few lectures ago. So one of the earliest papers that I know of that— so this is actually— I should also point out that this is, I think, a much more recent task. That this idea of semantic segmentation has been used in computer vision for a long, long time. But I think this idea of instance segmentation has gotten a lot more popular, especially in the last couple of years. So this paper from 2014 sort of took this— they call it simultaneous detection and segmentation, or SDS. That's kind of a nice name. And this is actually very similar to the R-CNN model that we saw for detection. So here we're going to take an input image. And if you'll remember, in R-CNN, we relied on these external region proposals that are these sort of offline computer vision low-level thing that compute predictions of where it thinks objects in the image might be located. Well, it turns out that there's other methods for proposing segments instead of boxes. So we just download one of those existing segment proposal methods and use that instead. Now for each of these segments, we can— for each of these proposed segments, we can extract a bounding box by just fitting a tight box to the segment and then run— crop out that chunk of the input image and run it through a box CNN to extract features for that box. Then in parallel, we'll, um, run through a region CNN. So here again, we take that relevant, uh, that chunk from the input image, um, and crop it out. But here, because we actually have this proposal for the segment, then we're going to mask out the background region using the mean color of the dataset. Um, so this is kind of a hack that lets you take these kind of weird-shaped inputs and feed it into a CNN. You just mask out the background part with a flat color. So then they take these masked inputs and run them through a separate region CNN. So now we've gotten two different feature vectors, one sort of incorporating the whole box and one incorporating only the proposed foreground pixels. We concatenate these things and then just like an R-CNN, we make a classification to decide what class actually should this segment be. And then they also have this region refinement step. Where we want to refine the proposed region just a little bit. So if you— I don't know how well you remember the RCNN framework, but this is actually very similar to RCNN, just applied to this instance, simultaneous detection and segmentation task. So for this idea, for this region refinement step, there's actually a follow-up paper that proposes a pretty nice way to do it. So here, this is, you know, a follow-up paper from the same folks at Berkeley the following conference. And here we want to take this input, which is this proposed segment map, this proposed segment, and we want to clean it up somehow. So we're actually going to take a very similar approach, a very similar type multiscale approach that we saw in the semantic segmentation model a while ago. So here we're going to take out our image, crop out the box corresponding to that segment, and then pass it through an AlexNet. And we're going to extract convolutional features from several different layers of that AlexNet. For each of those feature maps, we'll upsample them and combine them together. And now we'll produce, uh, this, this figure— this proposed figure-ground segmentation. So this, this is actually kind of a funny output, but it's, it's really easy to predict. The idea is that this, this output image, we're just going to do a logistic classifier inside each independent pixel. So given these features, we just have a whole bunch of independent logistic classifiers that are predicting how much each pixel of this output is likely to be in the foreground or in the background. And they show that this type of multiscale refinement step actually cleans up the outputs of the previous system and gives quite nice results. Question? [Speaker:AUDIENCE] Yeah. So the drop sampling box, actually, the equal size —what are you doing there? You're just copying the values or you're applying a deconvolution? I think in this paper this was not a fractionally strided convolution. I think it was instead some kind of a fixed upsampling, like a bilinear interpolation or something like that, or maybe even a nearest neighbor, just something fixed, not learnable. But I could be wrong. But you could definitely imagine swapping in some learnable thing there too. OK, so this actually is very similar to R-CNN. But in the detection lecture, we saw that RCNN was just the start of the story. There's all these faster versions, right? So it turns out that a similar intuition from faster RCNN has actually been applied to this instance segmentation problem as well. So this is work from Microsoft that act— and this model actually won the COCO instance segmentation challenge this year. So they, they took their giant ResNet and they stuck this model on top of it and they, and they crushed everyone else in the COCO instance segmentation challenge. So this, this actually is very similar to Faster R-CNN. So we're going to take our input image and just like in Fast and Faster R-CNN, our input image will now be pretty high resolution and we'll get this giant convolutional feature map over our high resolution image. Then from this high resolution image, we're actually going to propose our own region proposals. In the previous method, we relied on these external segment proposals. But here, we're just going to learn our own region proposals just like Faster R-CNN. So here, we just stick a couple, a couple extra convolutional layers on top of our convolutional feature map. And each one of those is going to predict several regions of interest in the image that using this idea of anchor boxes that we saw in the detection work. The difference is that now once we have this region, these region proposals, we're going to segment them out using a very similar approach that we just saw on the last slide. So for each of these proposed regions, we're going to use this ROI— what do they call it— ROI warping or pooling and squish them all down to a fixed square size and then run each of them through a convolutional neural network to produce these coarse figure-ground segmentation masks like we just saw in the previous slide. So now at this point, we've gotten our image. We've gotten a bunch of region proposals. And now for each region proposal, we have a rough idea of which part of that box is foreground and which part is background. Now we're going to take this idea of masking. So now that we've predicted the foreground background for each of these segments, we're going to mask out the predicted background and only keep the pixels from the predicted foreground and pass those through another couple layers to actually classify that— classify that segment as our different object categories. So this is— and then this entire thing can just be learned jointly end to end. With the idea that we've sort of got these 3 semantically interpretable outputs in intermediate layers of our network, and each of them we can just supervise with ground truth data. So for these regions of interest, we know where the ground truth objects are in the image, so we can provide supervision on those outputs. For these segmentation masks, we know what the true foreground and background are, so we can give supervision there. And we obviously know the classes of those different segments. So we just provide supervision at different layers of these network, um, and try to trade off all of those different loss terms and hopefully get the thing to converge. But this actually was trained end-to-end and they fine-tuned into ResNet and it works really, really well. So here is the, the, the, the results figure that we have to show. Um, so these results are, at least to me, really, really impressive. So for example, this input image has all these different people, um, sitting in this room. And the predicted outputs do a really good job of separating out all those different people, even though they overlap and there's a lot of them and they're very close. Um, same with these cars, maybe that one's a little easier, but especially this, this people one I was pretty impressed by. But you can see it's not perfect. So this potted plant it thought was blobbier than it really was, and it confused this chair on the right for a person and it missed a person there. Um, but overall these results are, are very, very impressive. And like I said, this model won the COCO segmentation challenge this year. So the overview of segmentation is that we've got these two different tasks, semantic segmentation and instance segmentation. For semantic segmentation, it's very common to use this conv-deconv approach. And then for instance segmentation, you end up with these pipelines that look more similar to object detection. So if there's any last-minute questions about segmentation, I can try to answer those now. Nope, super clear, I guess. So we're going to move on to another, another, um, another pretty cool exciting topic, and that's attention models. So this is something that I think has gotten a lot of attention in the last year in the community. Um, so, uh, as a kind of a case study, we're going to talk about the model from— oh, I don't have the citation here. Okay, but we're— as a, as a, as a sort of a case study, we're going to talk about the idea of attention as applied to image captioning. So I think this model was previewed in the recurrent networks lecture, but we wanna— I wanna step into a lot more detail here. But first, as a recap, just so we're on the same page, hopefully you know how image captioning works by now since the homework is due in a few hours. But we're gonna take our input image and run it through a convolutional net and get some features. Those features will be used maybe to initialize the first hidden state of our recurrent network. Then our start token or our first word together with that hidden state we're going to produce this distribution over words in our vocabulary. Then to generate a word, we'll just sample from that distribution. And we'll just sort of repeat this process over time to generate captions. The problem here is that this network only sort of gets one chance to look at the input image. And when it does, it's looking at the entire input image all at once. And it might be cooler if it actually had the ability to, one, look at the input image multiple times. And also if it could focus on different parts of the input image as it ran. So, uh, one— a pretty cool paper that came out last year was this one called Show, Attend, and Tell. Um, the original was Show and Tell, so they added the attend part. Um, and the idea is, is pretty straightforward. So we're gonna take our input image, and we're still gonna run it through a convolutional network. But instead of extracting the features from the last fully connected layer, Instead, we're going to pull features from one of the convolu— earlier convolutional layers. And that's going to give us this grid of features rather than a single feature vector. So because these are coming from convolutional layers, you can imagine that maybe the upper left-hand— this— you can think of this as a 2D spatial grid of features. And inside each grid, each point in the grid gives you features corresponding to some part of the input image. So now again, we'll use these features to initialize the hidden state of our network in some way. And now here's where things get different. Now we're going to use our hidden state to compute not a distribution over words, but instead a distribution over these different positions in the— in our convolutional feature map. So again, this is— this would probably be implemented with maybe a fully connect— with maybe an affine layer or two, and then some softmax to give you a distribution. But we just end up with this L-dimensional vector, giving us a probability distribution over these different locations in our input. And now we take this probability distribution and actually use it to weight— to give a weighted sum of those feature vectors at the different points in our grid. So once we take this weighted combination of features, that takes our grid and summarizes it down to a single vector. And this sort of z vector summarizes the input image in some way. And due to the different types, uh, due to the— due to this probability distribution, it gives the network the capacity to focus on different parts of the image as it goes. So now this, this weighted vector that's produced from the input features gets fed together with the first word, and now, um, when we make our recurrence in our recurrent network, we actually have 3 inputs. We have our previous hidden state, we have this, um, attended feature vector, and we have this first word. and now all of these together are used to produce our new hidden state. Um, and now from this hidden state, we're actually going to produce two outputs. We're going to produce another, uh, a new distribution over the locations in our input image, and we're also going to produce our standard distribution over words. So these would probably be implemented as just a couple of fully connected layers on top of the hidden state. And now this process repeats. So given this new probability distribution, we go back to the input feature grid, and compute a new summarization vector for the image. Take that vector together with the next word in the sentence to compute the new hidden state, produce— OK, so that spoiled it a little bit. But then we'll actually repeat this process over time to generate our captions. Yeah, question? [Speaker:AUDIENCE] Can you explain one more time how the features that LST features raise— generates? [Speaker:AUDIENCE] So essentially you take a given convolutional layer of your network, and then you just take the output of one filter, and that's the feature grid? [Speaker:ANDREJ KARPATHY] So the question is, where does this feature grid come from? And the answer is, when you're doing an AlexNet, for example, you have conv1, conv2, conv3, conv4, conv5. And by the time you get to conv5, the shape of that tensor is now something like 7 by 7 by 512. So that corresponds to a 7 by 7 spatial grid over the input. And in each, uh, grid position, that's a 512-dimensional feature vector. So those are just pulled out of one of the convolutional layers in the network. Yeah, question? So the probability distribution, uh, is for the distribution of classes or, uh, for, for like the distribution of words over the allocation or what? So the question is about these probability distributions. So we're actually, um, producing two different probability distributions at every timestep. The first one are these D vectors in blue. So those are probability distribution over words in your vocabulary, like we did in normal image captioning. And also at every timestep, we'll produce a second probability distribution over these locations in the input image that are telling us where we want to look on the next timestep. So this is actually quite— right, so you're just doing two outputs. And then as a quiz, I wanted to see, like, what framework you wanted to use to implement this. Um, and we talked about maybe how RNNs would be a good choice for Theano or TensorFlow, and I think this qualifies as a crazy RNN. So, um, I wanted to maybe talk in a little bit more detail how these attention vector— how these, um, summarization vectors get produced. So this paper actually talks about, uh, two different methods for generating these vectors. So the idea, as we saw on the last slide, is that, um, are— we'll take our input image and get this grid of features coming from one of the convolutional layers in our network. And then for each timestep, our network will produce this probability distribution over locations. So this would be a fully connected layer and then a softmax to normalize it. And now the idea is that we want to take these— this grid of feature vectors together with these probability distributions and produce a single d-dimensional vector that summarizes that input image. And there's— the paper actually explores two different ways of solving this problem. So the easy way is to use what they call soft attention. So here, our d-dimensional vector z will just be a weighted sum of all the elements in the grid, where each vector is just weighted by its probab— by its predicted probability. This is actually very easy to implement. It's sort of a nice— it's just another layer in a neural network. Um, and these gradients like the derivative of this context vector with respect to our predicted probabilities p is quite nice and easy to compute. So we can actually train this thing, um, just using normal gradient descent and backpropagation. Um, but they actually explore another, another option for computing this feature vector, and that's something called hard attention. So instead of having this weighted sum, we might want to select just a single element of that, of that grid to attend to. So you might imagine— so one simple thing to do is just to pick the element of the grid with the highest probability and just pull out the feature vector corresponding to that argmax position. The problem is now if you think about in this argmax, in this argmax case, if you think about this derivative, the derivative of z with respect to our distribution p, it turns out that this is not very friendly for backpropagation anymore. So imagine in an argmax case, suppose that pA were actually the largest element in our input. And now what happens if we change pA just a little bit? So if pA is the argmax, and then we just jiggle the probability distribution just a little bit, then pA will still be the argmax. So we'll still select the same vector from the input, which means that actually the derivative of this vector z with respect to our predicted probabilities is going to be 0 almost everywhere. So that's, that's very bad. Now we can't really use backpropagation anymore to train this thing. So it turns out that they propose another method based on reinforcement learning to actually train the model in this context where you want to select a single element of the input. Um, but that's a little bit more complex, so we're not gonna talk about that in this lecture. But just be aware that that is something that you'll see, this difference between soft attention, um, and hard attention where you actually pick one. So now we can look at some, some pretty results from this model. So since we're actually generating a probability distribution over grid locations at every timestep, um, we can visualize that probability distribution as we generate each word of our, of our generated caption. So then for this input image that shows a bird, both their hard attention model and their soft attention model in this case both produce the caption, "A bird flying over a body of water." And for these two models, they've visualized what that probability distribution looks like for these two different models. So the top shows the soft attention. So you can see that it's sort of diffuse since it's averaging probabilities from every location in the image. And in the bottom, um, it's just showing the one single element that it pulled out. And it's— this— these actually have, um, quite nice semantic interpretable meanings. So you can see that when the model is, um, especially the soft attention on the top, I think is very nice results. That when it's talking about the bird and talking about flying, it sort of focuses right on the bird. And then when it's talking about the water, it kind of focuses on everything else. So, um, another thing to point out is that it didn't receive any supervision at training time, for which parts of the image it should be attending to. It just made up its own mind to attend to those parts based on whatever would help it caption things better. And it's pretty cool that we actually get these interpretable results just out of this captioning task. We can look at a couple, a couple other results because they're fun. So we can see that when we have the dog throwing— what, a woman throwing a frisbee in the park, it focuses on the frisbee. Talking about the dog, it learns to recognize the dog. And especially interesting is this guy in the bottom right. When it generates the word trees, it's actually focusing on all the stuff in the background and not just the giraffe. And again, these are just coming out with no supervision, all just based on the captioning task. Question? [Speaker] Uh, can you talk about when hard attention would be preferred over soft attention? [Speaker] Yeah. So the question is, what are the— when would you prefer hard versus soft attention? So, um, There's, I think, sort of two motivations that people usually give for wanting to even do attention at all in the first place. Um, one of those is just to give nice interpretable outputs. And I think you get nice interpretable outputs in, in either case, um, at least theoretically. Um, maybe their hard attention figure wasn't quite as pretty. But the other motivation for using attention is to relieve computational burden. Um, especially when you have a very, very large input, um, it might be computationally, expensive to actually process that whole input on every timestep. And it might be more efficient computationally if we can just focus on one part of the input at each timestep and only process a small subset per timestep. So with soft attention, because we're doing this sort of averaging over all positions, we don't get any computational savings. We're still processing the whole input on every timestep. But with hard attention, we actually do get a computational savings since we're explicitly pliq— picking out some small subset of the input. So I think that's the, that's the big benefit. Also hard attention takes reinforcement learning, and that's fancier and makes you look smarter. So that's kind of cool. Um, question? Can you give some intuition for how the network is learning these when it never knows like which— there's no segmentation, it doesn't know which pixels are supposed to be or whatever. Yeah. So the question is how does this work at all? Um, and I think the answer is it's really learning sort of correlation structures in the input, right? That it's seen many examples of images with dogs, and it's seen many sentences with dogs. But for those different images with dogs, the dogs tend to appear in different positions in the input. And I guess it turns out through the optimization procedure that actually putting more weight on the places where the dog actually exists actually helps the captioning task in some way. So I don't think there's a very, very good answer. It just happens to work. Also, I'm not— so obviously, these are pictures from a figure, or these are figures from a paper, not like random results. So I'm not sure how good it works on random images. But another thing to really point out about this, especially this model of soft attention, is that it's sort of constrained to this fixed grid from the convolutional feature map that these, like, we're getting these nice diffuse looking things, but those are just sort of like blurring out this, um, this distribution. And the model does not really have the capacity to look at arbitrary regions of the input. It's only allowed to look at these, uh, fixed grid regions. Um, I should also point out that this idea of soft attention was not really introduced in this paper. Um, I think the first paper that really had this notion of soft attention came from machine translation. So here, it's a similar motivation that we want to take some input sentence here in Spanish and then produce an output sentence in English. And this would be done with a recurrent neural network sequence-to-sequence model, where we would first, um, read in our input sentence with a recurrent network, and then generate an output sequence very similar as we would in captioning. But, um, in this paper, they wanted to actually have attention over the input sentence, as they generated their output sentence. So the exact mechanism is a little bit different, but the intuition is the same. That now when we generate this first word, my, we want to compute a probability distribution not over regions in an image, but instead over words in the input sentence. So here, we're going to get a distribution that hopefully will focus on this first word in the Spanish sentence. And then we'll take some features from each word, and then reweight them and feed them back into the RNN at the next timestep. And this process would repeat at every timestep of the, of the network. So this idea of soft attention, um, is very easily applicable not only to image captioning, but also to machine translation. Question? [Speaker] How do you do it for variable length sentences? [Speaker] Uh, the question is, how do you do this for variable length sentences? And that's something I glossed over a little bit. Um, but the idea is you use what's called content-based addressing. So for the image captioning, um, we, we know ahead of time that there is this fixed, maybe 7x7 grid. So we just produce a probability distribution directly. Um, instead, in this model, um, as the encoder reads the input sentence, it's producing some vector that, um, encodes that each word in the input sentence. So now in the decoder, instead of directly producing a probability vector, um, a probability distribution, it's going to spit out sort of a vector that will get dot-producted with each of those encoded vectors in the input. And then those dot products get used to get renormalized and converted to a distribution. Um, so this idea of soft attention, um, is actually pretty easy to implement and, uh, pretty easy to train. So it's been very popular in the last year or so, and there's a whole bunch of papers that apply this idea of soft attention to a whole bunch of different problems. So there have been, um, a couple papers looking at soft attention for machine translation as we saw. Um, there have been a couple papers that actually want to do speech transcription, where they read in an audio signal, and then output the words, um, in English. So there have been a couple papers that use soft attention over the input audio sequence to help with that task. Um, we've— there's been at least one paper on using soft attention for video captioning. So here, you read in some sequence of frames, and then you output some sequence of words. And you want to have attention over, over the frames of the input sequence, as you're generating your caption. So you could see that maybe for this little video sequence, they output, "Someone is frying a fish in a pot." And when they generate the word "someone," they actually attend much more to this second frame in the video sequence. And when they generate the word "frying," it attends much more to this last element in the video sequence. There have also been a couple papers for this task of question answering. So here, the setup is that you read in a natural language question. And you also read in an image, uh, an image, and the model needs to produce an answer about that question, uh, produce the answer to that question in natural language. Um, so, and there have been a couple papers that explore the idea of spatial attention over the image in order to help with this problem of question answering. Um, another thing to point out is that some of these papers have great names. So there was Show-Intell, there was Show-Attend-Intell, there was Listen-Attend-and-Spell, And this one is Ask, Attend, and Answer. So I, I really enjoy the creativity with naming on this, on this line of work. Um, and this idea of soft attention is pretty easy to implement. So a lot of people have just applied it to tons of tasks. But remember we saw this problem with this sort of implementation of soft attention, and that's that we cannot attend to arbitrary regions in the input. Instead, we're constrained and can only attend to this fixed grid given by the convolutional feature map. So the question is whether we can overcome this restriction and still attend— and attend to arbitrary input regions somehow in a differentiable way. And I think a precursor to this type of work is this paper from Alex Graves back in 2013. So here he wanted to read as input a natural language sentence and then generate as output actually an image that would be handwriting, like writing out that sentence in handwriting. And the way that he— and this actually has attention over this output image in kind of a cool way, where now he's actually predicting the parameters of some Gaussian mixture model over the output image, and then uses that to actually attend to arbitrary parts of the output image. And this actually works really, really well. So on the right, some of these are actually written by people, and the rest of them were written by his network. So can you tell the difference between the generated and the real handwriting? I couldn't. So it turns out that the top one is real, and these bottom 4 are all generated by the network. [Speaker:AUDIENCE] You can see the B's, uh, stand out. —unique P in the real one. Yeah. So then maybe the real ones have more variance between the letters or something like that. But these results work really well. And actually, he has an online demo that you can go on and try that runs in your browser. You can just type in words, and it'll generate the handwriting for you. So that's kind of fun. Another paper that we saw already is DRAW that sort of takes this idea of arbitrary attention and then extends it to a couple more real-world problems, not just handwriting generation. So one task they consider is image classification. Here, we want to classify these digits, um, but in the process of classifying, we're actually going to attend to arbitrary regions of the input image, um, in order to help with this classification task. So, um, this is, this is kind of cool. It sort of learns on its own that it needs to attend to these digits in order to help with image classification. Um, and with DRAW, they also, uh, consider the idea of generating arbitrary output images with a similar sort of motivation as the handwriting generation, um, where we're gonna have arbitrary attention over the output image and just generate this output, um, bit by bit. And I think we saw this video before, but it's really cool. So this is the, the DRAW network from, from DeepMind. So you can see that here, we're gonna do it, we're doing a classification task. So it sort of learns to attend to arbitrary regions in the input. And when we generate, we're going to attend to arbitrary regions in the output to actually generate these digits. Um, so it can generate multiple digits at a time. Um, and it can actually generate these, these house view num— these, uh, house numbers. So this is really cool. And as you could see, like, the, the region it was attending to was actually growing and shrinking over time and sort of moving continuously over the image. And it was definitely not constrained to a fixed grid like we saw with show-attend-and-tell. So the way that this paper works is a little bit, a little bit weird. And some follow-up work from DeepMind, I think, actually was more clear. And why is this guy— oh, my focus is all off. OK. Right. So there's this follow-up paper that uses a very similar mechanism for arbitrary attention called spatial transformer networks, but I think is much easier to understand and presented in a very clean way. So the idea is that we want to have, um, this input image, this our favorite bird, and then we want to have this sort of continuous set of variables telling us where we want to attend. So you might imagine that we have the coord— the, the center and width and height of some box of the region we want to attend to. And then we want to have some function that takes our input image and these continuous attention coordinates and then produces some fixed-size output. And we want to be able to do this in a differentiable way. So this, this seems kind of hard, right? Because you imagine that at least with the idea of cropping, then these inputs cannot really be continuous. They need to be sort of pixel values. So we're constrained to integers. And it's not really clear exactly how we can make this function continuous or differentiable. And they actually came up with a very nice solution. And the idea is that we're going to write down a parameterized function that will map from coordinates of pixels in the output to coordinates of pixels in the input. So here, we're going to say that this, this upper left— upper right-hand pixel in the output image has the coordinates x_t, y_t in the output. And we're going to compute these coordinates x_s and y_s in the input image using this parameterized affine function. So that's a nice differentiable function that we can differentiate with respect to these affine transform coordinates. Coordinates. Then we can repeat this process. And again, for maybe the upper, upper left-hand pixel in the output image, we again use this parameterized function to map to the coordinates of the pixel in the input. Now we can repeat this for all pixels in our output, which gives us something called a sampling grid. So the idea is that this would be our output image. And then for each pixel in the output, the sampling grid tells us where in the input that pixel should come from. And now, have any of you guys taken a computer graphics course? Not many. So this looks, this looks kind of like texture mapping, doesn't it? So they take this idea from texture mapping in computer graphics and just use bilinear interpolation to compute the output once we have this sampling grid. So now, what— now that we have— now this, um, now this allows our network to actually attend to arbitrary parts of the input in a nice differentiable way. where our network will now just predict these transform coordinates theta, and that will allow the whole thing to attend to arbitrary regions of the input image. So they put this thing all together into a nice little self-contained module that they call a spatial transformer. So the spatial transformer receives some input, which you can think of as an im— as, um, our raw input image, and then actually runs this small localization network, which could be a small fully connected network or a very shallow convolutional network. And this, this localization network will actually, um, produce as output these affine transform coordinates theta. Now these affine transform coordinates will be used to compute a sampling grid. So now that we've predicted these— this affine transform from the localization network, we map each pixel in the output, the coordinates of each pixel in the output, back to the input. And this is a nice, smooth, differentiable function. Now, once we have the sampling grid, we can just apply bilinear interpolation to compute the values in the pixels of the output. And if you think about what this thing is doing, it's clear that every single part of this network is, one, continuous, and two, differentiable. So this thing can be learned jointly without any crazy reinforcement learning stuff, which is quite nice. Although one, one sort of caveat to note about bilinear sampling, if you know how bilinear sampling works, it means that every pixel in the output is going to be a weighted average of 4 pixels in the input. So those gradients are actually very local. So this is continuous and differentiable and nice, but I don't think you get a whole lot of gradient signal through the bilinear sampling. But once you have this spatial— this nice spatial transformer module, we can just insert it into existing networks to sort of let them learn to attend to things. So they consider, um, this classification task very similar to, um, the DRAW paper where they want to classify these warped versions of MNIST digits. Um, so they actually consider several other more complicated transforms, not just these affine transforms. You could also imagine that— so that's the mapping from your output pixels back to your input pixels. Um, on the previous slide, we showed an affine transform. But they also consider projective transforms and also thin plate splines. But the idea is you just want some parameterized and differentiable function. Um, and you could go crazy with that part. So here on the left, um, the network is just trying to classify these digits that are warped. So on the left, we have different versions of warped digits. Um, on, on this middle column is showing these different thin plate splines that it's using to attend to a part of the image. And then on the right, it shows the output of the spatial transformer module, which has not only attended to that region, but also unwarped it corresponding to those splines. And on the right, they're using an affine trans— or on the right is using an affine transform, not these thin plate splines. So you can see that this is actually doing more than just attending to the input. We're actually transforming the input as well. So for example, in this middle column, this is a 4, um, but it's actually rotated by some— by something like 90 degrees. So by using this affine transform, the network can not only attend to the 4, but also rotate it into the proper position for the downstream classification network. Um, and this is all very cool. And again, um, sort of similar to the soft attention, we don't need explicit supervision. It can just decide for itself where it wants to attend in order to solve the problem. Um, so these guys have a fancy video as well, which is very impressive. So this is the, the transformer module that we just unpacked. And here, we're actually showing, um, right. So this is actually running a classification task, but we're varying the input continuously. So you can see that for these different inputs, the network learns to attend to, to the digit and then actually canonicalizes that digit to sort of a fixed known pose. And as we vary that input and move it around the image, the network still does a good job of locking on to the digit. And on the right, you can see that, um, sometimes it can fix rotations as well, right? So here on the left, we're actually rotating that digit and the network actually learns to un-rotate the digit and canonicalize the pose. And again, both with affine transforms or thin plate splines. So this is using even crazier warping with projective transforms. So you can see that it does a really good job of learning to attend and also to un-warp. And they do quite a lot of other experiments. Instead of classification, they learn to add together two warped digits, which is kind of a weird task, but it works. Um, so the network is receiving two inputs as, uh, two input images and it needs to output the sum. And it learns, and even though this is kind of a weird task, it learns that it needs to attend and un-warp those images. Um, so this is during optimization, right? So this is a task called co-localization. So the idea is that the in— the network is going to receive two images of, uh, as input that are maybe two different images of fours. and the task is to say whether or not those images are the same, um, are the same or different. And then, uh, also lo— uh, using spatial transformers, it ends up learning to localize those things as well. So you can see that over the course of training, it learns to actually localize these things very, very precisely. Um, even when we add clutter to the image, then these networks still learn to localize, uh, very, very precisely. So that's a recent paper from DeepMind that, um, is pretty cool. So any other last-minute questions about spatial transformers? Yeah. So just to clarify, so what exactly is the y variable in this context? The y variable is just one digit of this? Uh, yeah. So the simp— so the question is what is the, what is the task that these things are doing? And at least in the vanilla version, it's just classification. So it receives this, um, sort of input digit, which could be warped or cluttered or whatnot, and all it, it needs to do is classify the digit. Um, and sort of in the process of learning to classify, it also learns to attend to the correct part. So that's, that's a really cool, um, feature of this, of this, of this work. All right. So then the sort of my overview of attention, um, is that we have soft attention, which is really easy to implement, especially in this context of fixed input positions. That we just produce distributions over our input and then reweight and refeed those, those vectors back to the network somehow. And this is really easy to implement in many different contexts and has been implemented for a lot of different tasks. When you want to attend to arbitrary regions, then you need to get a little bit fancier. And I think spatial transformers is a very, a very nice elegant way of attending to arbitrary regions in input images. There are a lot of papers that actually work on hard attention. And this is quite a bit more challenging due to this problem with the gradients. So hard attention papers typically use reinforcement learning, and we didn't really talk about that today. So any, any other questions about, um, attention or— okay, sure. So I have a quick question about your presentation. So before you introduced the concept of spatial transformer networks, you gave us some examples of, uh, captioned images, um, using sort of the fixed input —were those results produced using only the grid-based— Yeah, so the question is about the captioning before we got to spatial transformers. And yeah, those captions were produced using this grid-based thing. But in that network in particular, I think it was a 14 by 14 grid. So it's actually— it has quite a lot of locations, but it still is constrained to where it's allowed to look. Question? Uh, so is there any interpretation, like interpolation between soft and soft attention and hard attention? The question is about interpolating between soft attention and hard attention? Um, so yeah, one, one thing you might imagine is you train the network in a soft way, and then during training, you kind of penalize it to make that distribution sharper and sharper and sharper. And then at test time, you just switch over and use hard attention instead. And I think, um, I'm— I can't remember which paper did that, but I'm pretty sure I've seen that idea somewhere. Um, but in practice, I think, uh, training with hard attention tends to work better than the sharpening approach. But it's definitely something you could try. Okay. If no other questions, then I think we're done a couple minutes early today. So get your homework done.