Short course video (DeepLearning.AI) introducing how diffusion-based image generation models work. Instructors Andrew Ng and Sharon Zhou outline the training procedure (adding noise to images, training a U-Net to predict and remove noise), sampling improvements such as DDIM, and a hands-on PyTorch notebook that generates 16x16 sprite images. The course is intermediate-to-advanced and assumes familiarity with Python and neural-network training concepts.
Chapters
I'm delighted to present to you this short course on diffusion models taught by Sharon Zhou.
How do image generation models like Midjourney, Stable diffusion and Dall E2 even work? In this short course, we'll present an implementation of image generation and walk you through every step of how that works.
You learn how they take a training set of images and add noise to go from training image to pure noise. Then you see how to train a U net neural network to predict the noise so you can subtract it off and this allows you to generate new images.
You'll also learn more efficient techniques for sampling such as ddim. And finally, you'll get to see how this architecture can be modified to take in some kind of context so you can tell the model what you want it to generate.
This is an intermediate to advanced course. It assumes you're familiar with Python and neural network concepts like how backpropagation is used for training. This course uses Pytorch, but if you're familiar with TensorFlow, you should be to follow along. If you're not yet familiar with either PyTorch or TensorFlow, it'll be easier to follow after you've taken the Deep learning specialization on Coursera.
The notebook in this course will walk through an example of generating 16 by 16 sprites. Yes, those are those little 8 bit characters used in old school video games and they're Andrew's favorite. So at the end you'll be able to run the code yourself to generate some new sprites.
This is a technical and fun course and I hope you enjoy it.
