New course with Hugging Face: Quantization in Depth 🤗

I'm delighted to introduce quantization in depth. Developed in collaboration with Hugging Face, quantization has emerged as a key technique for compressing large models such as large language models and making them much more accessible to everyone. This course presents a deep technical dive into the core building blocks of quantization. Specifically, in this course you use Pytorch and hugging face transformers to implement from scratch different variants of linear quantization methods. You also build a quantizer to quantize any model into 8 bit precision regardless of its modality. What this means is you be able to quantize a text, vision, audio or even multimodal model pretty much out of the box. You also learn about the unique challenges of low bit, that is four or even two bit quantization, as well as techniques like weight packing to address them. Weight packing takes advantage of the observation that if you want to represent a weight using just four bits of storage, then you actually store or pack the value of two weights into an eight bit variable. If you've taken the Quantization Fundamentals of Hugging Face course, this course will give you a more in depth understanding of when, how and what are the challenges around quantizing machine learning models. I'm delighted to introduce the instructors, Eunice Gagadar, who is a machine learning engineer at Hugging Face, as well as Mark sun, who's also a machine learning engineer at Hugging Face. Thanks Andrew. We are very excited about this course. If you have taken the course Quantization Fundamentals with Hugging Face, you will have learned some basic principles behind linear quantization and useful hugging face libraries such as Quanto that can help you quantize your models and to make them smaller. In this course you will implement from scratch different variants of linear quantization such as a symmetric and symmetric mode. You will also implement different quantization galanarities such as per tensor, per channel and per group quantization and study their advantages and disadvantages for each case. Next, you'll use one of these specific linear quantization algorithms and build a quantizer to quantize any model from 32bit to 8bit precision. You will apply your quantizer on different models from the Hugging face transformers library by loading models that use different modalities such as images and text. Finally, you will deep dive into core challenges around low bit precision quantization, so especially 4 and 2 bit quantization when it comes to serializing the model. In other words, storing the model. You will learn how to perform bit packing to pack low bit weights into eight bit tensors. This technique is widely used today by state of the art algorithms and popular open source softwares. It's been fun hanging out with Eunice and Mark to chat about compressing models and I would say we've been making a lot of small talk. Please enjoy the course.