Richmond Alake from MongoDB teaches how to build and deploy RAG systems efficiently. The course covers prompt compression using a low-cost LLM, filtering and re-ranking search results, and field projection to cut serving costs and improve performance.
Richmond Alake (Developer Advocate at MongoDB).
2024-07-10.
I'm delighted to introduce Prompt Compression and Query Optimization built in partnership with MongoDB and taught by Richmond Alake. RAG or Retrieval Augmented Generation has moved from being an interesting new idea a few months ago to becoming a mainstream large scale application. This course will describe how to manage related issues such as scaling, performance and security by using a robust and mature database. It'll also describe techniques to reduce the cost of serving RAG applications by reducing prompt sizes. I'm delighted to introduce the instructor Richmond Alake, who is developer advocate at MongoDB. Richmond has worked as a machine learning architect for many years. MongoDB is a widely used NoSQL database. In fact, many teams I've led have used MongoDB and and it also has added vector search capabilities to support semantic search which is a key part of rag. Thanks Andrew. There are a number of ways to combine traditional and vector database capabilities to make RAG more cost effective and performant. Let's use an example of building a conversational RAG application that lets users choose movies in RAG applications. Including large amount of retrieved information in the prompt for an LLM to process can be very expensive. In this course you will learn how you can reduce the cost by making that content smaller and more relevant One key technique is filtering. There are two types of filtering, pre filtering and post filtering. Say you are searching for movies, but you're only interested in romance movies. With pre filtering you would build an index of entries that only contain romance movies. You would then perform searches on that smaller set of entries. With post filtering, the filter is applied to the results returned by the vector search to select only results that match the filter, say romantic movies. Retrieval results can also be limited by taking a projection. This just means you only select the required fields from a document. For example, you may only need to return a movie title and not the actors and film dates, and you can also rewrite the results from semantic search and move up the relevant results on the list. Say you're looking for a baseball movie. Semantic search on movie descriptions will return many movies about baseball, but to limit the returned movies you can then re rank those results by sorting using maybe another few from the database like average movie rating or number of reviews. A final technique is prompt compression. This takes all of that retrieved information and instructs a low cost LLM to compress the data, thus ending up with a shorter piece of context to include in the LLM prompt used in the final inference step. This can often reduce the context size by more than maybe a factor of two, so I hope you sign up for this course and learn all these techniques for building and deploying better rack systems.
