Andrew and Anton introduce a new course on advanced retrieval techniques for RAG systems. It covers LLM-driven query expansion, using a second LLM to select relevant passages, ranking results with text models, and adapting embeddings from feedback. Anton, founder of Chroma, shares practical guidance on fixing common RAG failures where semantically similar documents don't actually answer the question.
Questions this video answers
January 3, 2024.
LARRUZ.
Anton Choinikov — founder and head of technology at Chroma.
Chapters
I'm excited to introduce this course on advanced retrieval, built in partnership with CHROMA and taught by Anton Trojnikop, co founder of chroma. In this course you learn sophisticated techniques that use an LLM to get the most relevant context for a RAG or retrieval augmented generation system. For example, using a technique called query expansion with rag, given a user query, we search a database to find the relevant context to give the LLM to help it write and accurate answer. But how do you find the most relevant context in traditional text or web search? If you search for one concept, say AI, it may carry out query expansion and add in related keywords like machine learning or neural networks to retrieve additional relevant documents. In this course you learn how to use an LLM to rewrite or create additional queries that reflect the intent of of the user query. Further, given a mass of retrieved documents, you don't necessarily want to include all of that in the LLM context. So you learn too how to use another LLM to select the most relevant parts to form the final context to give to rag. These sophisticated techniques, as you see in this course, give a significant boost to RAG accuracy. Your instructor, Anton Choinikov, is a founder and head of technology of chroma, which is one of the leading open source vector databases. He works every day with users who are building retrieval based applications and so has seen a lot of what does and doesn't work. I look forward to sharing that practical experience with us here.
Thank you Andrew, and I'm really looking forward to sharing these techniques with you. It's such an exciting time to be working in retrieval for AI. One of the things few people appreciate until they've worked on it a few times is the pitfalls one may encounter when doing retrieval for rag. While vector databases are a great tool for retrieval, there are also common failure modes that I often see AI developers run into. A very common bug is if you get documents that are semantically similar to the query so they talk about the topics in the query but don't actually have the information needed for an answer. As Andrew mentioned, we'll explain in detail how you can use an LLM to carry out query expansion to improve retrieval performance. We'll talk about how to use a text model to score and rank the retrieved results. And we'll also talk about how to adapt the text embeddings of the query directly using data from user feedback. These techniques makes the retrieval step of RAG much more effective and I hope you will take and enjoy the course.
