Dr. Violet Howard shows how to improve RAG agents by adding vector re-ranking. Instead of passing the top 3 retrieved chunks to your LLM, retrieve 10-30 candidates, re-rank them by relevance, then feed only the best ones to the model for better answers.
Here's how to instantly make your RAG agent smarter with re-ranking. Okay, so let's get started. I'm going to real quick hop into an Excalidraw and actually explain what's going on behind the scenes when we do RAG so that we all understand how the re-ranker is going to work. So the first step with RAG obviously is we want to get our text document into a vector database. And how that works is the document is split up into smaller manageable chunks. The chunks are then fed into an embeddings model and the embeddings model turns them into a numerical representation of the meaning of what's in this chunk. and then they get placed somewhere in that multidimensional space, which is our vector database. From there, we go to search through it. So let's say this text document right here was the rules of golf. If we now go to ask, what do I do if my ball goes out of bounds? This question is getting embedded the same way these chunks did up here with the exact same embeddings model. And then it also gets placed in that multidimensional vector database. So then our question gets vectorized and it's placed right here, and then it basically is going to grab the nearest other vectors because that's how it knows they're similar in meaning. and it's gonna pull them back. So it pulls back these 3 vectors in this case, and then these vectors get turned back into their text-based chunks, and those chunks get fed into our RAG agent, which it's able to use to answer our question. Anyways, what happens when we use a re-ranker is pretty cool because this allows us to basically pull back way more than just the 3 nearest neighbors. We can pull back 10, 20, or 30 vectors because all of these are gonna get fed into the re-ranker, and then it will basically look at which ones are actually the most relevant it will assign a relevant score and then it will grab just the top 3 most relevant answers. So I know that was quick, but hopefully at least it makes sense. And now let's hop back into N8N and take a look at how we set this up. If you wanna watch the full breakdown, click on that play button right here and I'll see you guys over there.
