n8n Just Leveled Up RAG Agents (Reranking & Metadata)

This new feature in NN instantly makes your RAG agents so much smarter, and it takes almost no time to set up. In this video, I'm going to be going over what a re-ranker is, how it makes our retrieval more accurate, and how you can easily set one up. I'll also be touching on how to use metadata when you're vectorizing documents and also retrieving documents so that by the end of this video, your RAG agents are instantly going to be 10 times smarter. Okay, so let's get started. I'm going to real quick hop into an Excalidraw and actually explain what's going on behind the scenes when we do RAG. So that we all understand how the re-ranker is going to work. So the first step with RAG obviously is we want to get our text document into a vector database. And how that works is the document is split up into smaller manageable chunks. The chunks are then fed into an embeddings model and the embeddings model turns them into a numerical representation of the meaning of what's in this chunk. And then they get placed somewhere in that multidimensional space, which is our vector database. From there, we go to search through it. So let's say this text document right here was the rules of golf. If we now go to ask, what do I do if my ball goes out of bounds? This question is getting embedded the same way these chunks did up here with the exact same embeddings model. And then it also gets placed in that multidimensional vector database. So then our question gets vectorized and it's placed right here. And then it basically is going to grab the nearest other vectors because that's how it knows they're similar in meaning and it's going to pull them back. So it pulls back these 3 vectors in this case, and then these vectors get turned back into their text-based chunks. And those chunks get fed into our RAG agent, which it's able to use to answer our question. And by the way, I would never hit my ball out of bounds. Anyways, what happens when we use a re-ranker is pretty cool because this allows us to basically pull back way more than just the 3 nearest neighbors. We can pull back 10, 20, or 30 vectors because all of these are going to get fed into the re-ranker and then it will basically look at which ones are actually the most relevant. It will assign a relevance score and then it will grab just the top 3 most relevant answers. and then those top 3 get fed into the agent and it will answer for us. So I know that was quick, but hopefully at least it makes sense. And now let's hop back into N8N and take a look at how we set this up. So as a prerequisite, I'm gonna assume that you've already set up a flow like this and you've connected to Supabase. If you haven't, go ahead and check out this video right here where I explain how to connect and how to set that up, and then come back to this video. So I'm gonna click into the Supabase Vector Store tool real quick, and we're gonna look at 2 main things. The first thing is that I gave it a limit of 20. And by default, that's 4. So we're basically telling the agent, you can go ahead and find 20 most relevant chunks rather than just 4, because we know they're going to get re-ranked and we're still only going to get the top 3. And right here is the option to actually re-rank those results. So if I turn this off, what happens is it loses our connection right here to our Cohere re-ranker. But if I go ahead and turn that on, it's going to give us that connection back and we can go ahead and add a re-ranker. Right now they only have Cohere. but that's why we were able to connect it right there. And to set up Cohere, all you have to do is go to cohere.com. I'll leave a link for that in the description. You'll make an account and then you just need to go get a simple API key. It's really easy and that's all you have to plug in right here. And what you can see is there's also 3 different models. We're gonna stick with v3.5. You could go to Cohere's docs to figure out the difference of each model, but like I said, for today, for simplicity, we're just gonna stick with ReRank v3.5. Anyways, let's test out this agent and see the re-ranker in action. Real quick, I just wanted to show you guys what is in my Supabase. It actually is the rules of golf. So you can see we've got like 28 different rules. This isn't like a full, full sheet of the rules, but just a PDF I found. And so here's what it looks like. It's 22 pages and we used this flow down here to actually vectorize it with metadata. And I'll touch on that whole metadata thing in a bit. And if you guys wanna access this exact template to play around with, as well as that PDF that I used, you can get that in my free school community. The link for that will be down in the description. All right, so I'm going to ask this agent, how does the order of play work? We're going to see it talk to Supabase Vector Store, and then it's going to talk to ReRanker. And now the agent's taking all that information and it's about to give us an answer. So here we have the order of play in golf depends on the type of play. If it's match play, you start a hole, the committee or players decide who goes first. We have information about during a hole, the player farthest from the hole plays. Then we have what you do if it's stroke play, or also if you need to do a provisional ball or another ball from the tee box. So what I'm interested in here is the actual agent logs to see how this worked. So I'm gonna click into the agent. I'm gonna move this thing over to the side and open up the logs and we'll see what it did. So basically what happens is it goes to the Supabase vector store because it knows that it needs to find information on golf order of play. So it sends over the query order of play. This gets vectorized and then it's searching for the nearest vectors that are near this vector. So this gets sent off to Supabase, it gets embedded. So this is exactly what we see. This is the numerical representation of this query. And then it gets sent to the re-ranker model. So here's the input to the re-ranker model. If I was to scroll through here, we're going to see that there's 20 different chunks that are sent to the re-ranker model. And now if I close the input and we open up the output, we can see that it chooses the 3 most relevant ones. So now we're only looking at 3. And what you'll notice about each one is that they have a relevance score. So right here, this first chunk has a relevance score of 0.825. The second one has a relevance score of 0.41. And the third one has a relevance score of 0.17. So maybe this question wasn't a great example because the scores are pretty low, but it would've been way worse of an answer if we didn't have this re-ranker. And then I did another example, tell me about searching for my ball. Once again, unrealistic, my ball's gonna be in the fairway. But what we can see is we get some answers back. And if I go into the agent logs, we can see that if we go to the re-ranker and we go to the output, it gave us the top 3 chunks about searching for your ball. This one has a relevance score of 0.86, This one has a relevance score of 0.74, and this one has a relevance score of 0.73. So if you think about only keeping the most high-quality chunks, you could even set up some sort of filter in a different workflow where after it does the re-ranking, you're only going to keep results where the relevance score is over like 0.7. So just another way to think about how can I make my agent's retrieval as accurate as possible. Because one thing to think about is that the agent is basically going to be forced to pull stuff back, even if it's not that relevant. So if you ask it some random question and it searches the vector database, it's still going to pull back the 4 nearest chunks, even if they're not very similar at all. And the problem with chunk-based retrieval is that we're not able to look at the contents as a whole. So if we said to the agent, like, you know, can you summarize rules 1 through 10, it wouldn't be able to give a holistic summary of rules 1 through 10 because it would be querying for each of those chunks. So if it sent one query to the vector database, like rules 1 through 10, we would get back 3 and it wouldn't be like a holistic summary. But maybe if it did 10 different queries, rule 1, rule 2, rule 3, rule 4, then maybe that's a little better. But in order to do a search like that, you definitely want to set up metadata and you want to have a different type of system to be able to search with metadata. So let me explain why that is. If we go over to our Supabase documents table, which is our actual vector database, what we can see is that we have all of these different chunks of data. But what you'll notice is that not every single rule fits into one chunk. So right here, rule 1, that's one chunk, that's fine. Right here we have rule 2, that's one chunk, that's fine. But here we have rule 3, and it looks like rule 3 is actually split across 3 different chunks. And if we don't have metadata, there's almost no way to tell that this chunk based on the contents is part of rule 3. So in this column over here, you can see it's called metadata, and the only thing that I added to all of these chunks was a rule number field. So now with every single chunk that gets pulled back, we can see which rule it belongs to. So like I said, here's the only chunk for rule 1. The metadata here says rule number 1, but here's the third chunk for rule 3, which in the actual content of it, there's no mention of rule 3. But if we click into the metadata for this chunk, we can see right here we have rule number 3. And obviously this isn't the only way to do it, but definitely be thinking about how you can add metadata tags to the data you're gonna put into your vector database. So maybe you have a bunch of different projects that you wanna put in there, but you want to have the project name or the client's name assigned as metadata to each of the different chunks. Or maybe you have a vector database full of all of your meeting transcripts. What you'd probably want to do is add a metadata field, which was the date of the meeting. That way you could actually go ahead and make a query to your agent like, can you pull all my meeting transcript data from June 20th? Because without that metadata, the agent would basically be searching through the vector database for mentions in the transcript of like, hey, today is June 20th. And that's the only way it'd be able to actually pull that back. So I hope that all makes sense. As you can see, even if we continue to scroll down, here's a random chunk that has no mention of the rule number. And if we click into the metadata, it says rule number 26. And the way that we're able to add metadata is in the flow when we're vectorizing our documents. We do that right here in the default data loader. We have an option down here to add metadata, and you can add as many different properties as you want. So you could also add like an ID number. You could also add like a date, all this kind of stuff. That's how we're adding metadata. And I'm sure you guys are wondering, how did I actually extract the page numbers out of that PDF? So I just pulled in the execution of when I actually vectorized this data. So what happened was I downloaded the file, which was our PDF about rules of golf. I extracted the text from it. And as you can see, when I extract the text, the entire body of the PDF was lumped into one field. I thought about feeding it into AI to extract the page numbers, but I probably would've hit a context window, so I didn't want to do that. So what I did was I added a code node and I literally went to JSON and I just copied basically the entire text field right here. And then I pulled up Claude and I pasted in that JSON and I said, help me write a code node for N8N. It's gonna receive the JSON below as an input and I want it to split out each rule as its own item. It should split out after each time the text says rule X. And it was literally a one-shot prompt. That's all I said and I got this code. So I pasted that code in here, I ran it, and now we have 43 items where it gives us, hey, here's the rule number, here's the rule title, and here's the full text of that rule. And so as you can see, that's how we got all of this data back here with a rule number associated with each one. And so that's pretty cool. I was able to do that with ZeroAI. And then all I had to do was in my default data loader, I added a field called rule number and I dragged in the rule number right there. And then I told it the actual data that we are vectorizing is just going to be the full text from this code node. So I dragged in this field right there. So we have our metadata in our vector database, but how do we actually search and filter through it? Let me show you guys a quick example of that. So right here we have rule 27, which is about ball lost or out of bounds. What I'm gonna do is chat with the agent right here and say, tell me about rule 27. I'll fire that off. And what's gonna happen is once again, it's gonna pull back 20 chunks, it's gonna re-rank them, and it's actually searching a few times. It's gonna keep searching until it feels confident about rule 27. And so it comes back with our answer about rule 27. You can go ahead and fact check this if you want. But what I'm more interested in right now is understanding why did it search through the vector database 3 times for such a simple query? Well, let's click into the agent and let's go to the logs to understand what happened. First time it tried, it shot off the query rule 27. And so we're looking for instances in Supabase where a chunk means rule 27. Let's click into the re-ranker to see what it chose. First one you can see was rule 28, which is already wrong. The relevance score was 0.2. The second chunk that it pulled was rule 26. That was the only content and a 0.14 relevance score. And the third one was also rule 26 with a 0.11 relevance score. So it did horrible at finding us rule 27. And I assume the same things happened in the next search, which was rule 27 once again. And then finally, the last query, it changed it to rule 27 golf. Not sure how that really changes much, but apparently it does. Because now if we go to the re-ranker, we can see it got rule 27. It has a relevance score of 0.8. Another rule 27 with a 0.73 relevance score. And I think this one looks like it's also potentially about rule 27. Uh, maybe not. It has a pretty low relevance score. So the agent was intelligent enough to keep going until it felt confident about its answer, but that's not really the way we want to do it. And because we have metadata set up, we should be able to only isolate chunks that literally say rule number 27. So the way that we would go about that is in the Supabase Vector Store tool, we have an option at the bottom to add a metadata filter. And this is where we'd be able to add a filter field. We'd have to add the name and the actual value we're filtering for. And so this is obviously subject to change. So what we want to do is have the rule number in here, and then we want to have AI decide based on the human's query what to filter. So that's why I'm going to show you guys this example over here where we're going to have one agent basically decide on the metadata filter, and then the second agent will do its normal vector search. And the reason why I'm doing it like this is just because if we have this agent also decide the metadata filter, it just bugs out and this node breaks. So right now this is the way to do it. So anyways, if I go ahead and send off the same query to this different agent now, This one is going to send off the metadata filter of rule 27. This agent's gonna search through the vector database and we're gonna get back our answer on the first try. So there we go. We now have information about rule 27, and this looks a lot better than that other agent. And if we click into the Supabase vector store node, we can see that the 3 different chunks it actually pulled back, they're all coming from rule 27 and they all have this metadata field of rule 27. So rule number 27, rule number 27, rule number 27. And you can see the way the metadata filter came through was we were searching for only chunks where the rule number right here equals 27. And that's how we guarantee we're looking for the right chunks. So like I said, it doesn't just have to be rule numbers. It could be a date, it could be a project name, it could be a client name, it could be a different department name, whatever you want it to be. And in an ideal world, you have a metadata agent that's aware of all the different filters you have in your metadata. So not only can it dynamically fill in the actual value you're filtering for, but it can be intelligent enough to understand the query and understand, okay, for the best results, I'm gonna choose the date filter and fill in the date. I'm also gonna filter for rule number where we just add another one, and I'm also gonna fill in the rule number down here. So that would definitely be pretty cool. The best way to start to get familiar with it is just to play around with it. So I'll have all of these agents in this exact workflow in the FreeSchool community. So if you wanna download the JSON here, you can also download this PDF and you can just play around and experiment the same way that I just did here in front of you guys. Anyways, I don't want this video to go too long. I think it's going to be a lot more valuable for you all to get that hands-on experience when you download this template and just play around and see how the re-ranker works, see how your agent makes queries, and see how the metadata filter works. And if you're looking for a more hands-on experience when you're experimenting with RAG and building stuff with NN, then definitely check out my Plus community. The link for that will be down in the description. We've got a great community of members who are always sharing their builds, sharing their challenges, and supporting each other. And we also have two full courses in here, one called 8 and 0, which is the foundations for AI automation, and then we have 10 Hours to 10 Seconds where you learn how to identify, design, and build time-saving automations. So I'd love to see you guys in the community, but that's gonna do it for today. I hope you enjoyed the video. I hope you learned something new. If you did, please give it a like. It definitely helps me out a ton. And as always, I appreciate you guys making it to the end of the video. I'll see y'all in the next one. Thanks guys.