Caspar Bukowski from Qdrant teaches how tokenization and vector quantization affect search relevance and performance. Learn three quantization techniques that trade memory, speed, and precision, with concrete examples for large-scale RAG applications.
I'm delighted to introduce Retrieval Optimization from Tokenization to Vector Quantization. Built in partnership with Quadrant and taught by Caspar Bukowski, this course will help you optimize vector search and RAG in large scale customer facing applications. You learn about what happens under the hood of vector search and how different tokenization techniques including byte pair encoding, wordpiece and unigram work and how they can affect your search relevance. You also learn different vector quantization techniques for vector compression to save on memory and build faster search. I'd like to introduce the instructor for this course. Casper is Developer Relations Lead at Quadrant.
Thank you, Andrew. Understanding how your specific embedding model works will help you build an effective search or RAC system. For example, you may encounter a terminology mismatch because the model was trained on data that differs greatly from yours. The embedding model may also truncate your chunks because they exceed their maximum window length and lose some information. Many things may go wrong and this course will help you understand the most.
Common issues and Tokenization is an important part of embedding models. Consider a sentence like I enjoy learning. In simple word level tokenization, you can split this into the tokens, individual words I and then enjoy and learning using subword tokenization, you can break this down into more common subunits such as I and then en joy Learn Eng. Knowing how tokenization is done can affect how you build and enhance and optimize your RAG application. For example, you see the token vocabulary can directly impact the search result, especially handling special characters. Say you're using an embedding model to embed I feel and then smiley happy emoji and also a different sentence to embed I feel and then unhappy or sad emoji. Some popular small embedding models do not cover emojis, and you end up with exactly the same embedding result for these two sentences, even though their meaning is completely opposite of each other. You also see how typos and different notations for numbers and dates can affect semantic similarity and search relevance.
The retrieval component in your rack cannot be enhanced if you don't know how to measure its quality. That's why in this course you will learn how to measure the quality and relevance of of search results. You will explore various metrics like precision on a data set of product names and descriptions.
If you have a retrieval system with millions of vectors to search from, you see that storing, indexing and searching these vectors can take up quite a bit of space and can become slow as you add more vectors. Say you're using OpenAI's Aidir embedding model that generates a vector with just over 1500 dimensions. This would require 6 kilobytes of memory for a single vector and about 30 gigs of memory for 5 million vectors. In this course you'll learn three techniques for quantizing your vectors to compress them. First, product quantization, second, scalar quantization and third, binary quantization. Product quantization maps subvectors to the nearest centroid scale. The quantization converts float values to 1 byte integers and binary quantization goes further and converts floats to boolean values. You learn in depth how these techniques work and how to make trade offs between memory speed and precision in each. So as you see, there are quite a lot of topics, very useful topics and techniques in this course that teach you a lot about the nuts and bolts of large language model based applications. I hope you enjoy the course.
