The secret app i use to scrape ANYTHING with n8n (no code)

What's up guys? In this video, we're going to be going over a tool that I stumbled across that you may or may not have already heard of that makes it way easier for your AI agents to scrape data from websites. Not only does it allow you to scrape content from websites, so it also allows you to give it a website and then it will return to you all of the paths and all of the pages that is found on that website. And then you can also perform Google searches with it and it will return you actual structured data rather than just a bunch of URLs. So it's a really, really useful tool. We're going to jump into the video and I can show you exactly how it all works. Okay, so the platform that we're going to be using to do all of this is a platform called FireAccord. And if you haven't heard of it before, basically what it is, is a platform that allows you to do all those things that I just said. And the way it allows you to do that is by, instead of just returning a bunch of HTML when you give it a website URL, actually returns to you structured data, which just means it's way easier for LLMs to handle the data compared to just a bunch of HTML with a bunch of fluff that it doesn't actually need in there. So this tool will structure that data for you, which means it's way easier for your AI agents. And it's really, really easy to get connected up to your AI agents. And it's also free to use at the beginning. So in this video, I'm going to show you sort of what it can actually do and how to get it connected up to your AI agents. So I've built a really simple AI agent here that can do these 3 things. It can scrape pages, get pages, and then perform Google searches. And off the bat, I'm just going to do a demo to show you how all of these tools work so you can get an idea of what's actually happening and the kind of things we're going to be doing. So if we open up the chat, we're going to be using Gymshark as an example, like I do in all of my videos. So first of all, we're going to try and use this search tool. So we're going to say something like, I want to find the most recent news on Gymshark. So I want to find the most recent news on Gymshark. Let's let this run through. Hopefully there's been some interesting news on Gymshark recently. Let's have a look. Okay, so it said here are some of the most recent news and updates on Gymshark. So the Gymshark Sale 2025, which on the 22nd of June 2025, which was pretty much the other day. And then Gymshark Amsterdam, the city knew before you, the Onyx return, Gymshark mid-season sale. So yeah, pretty cool. Some recent news on Gymshark. If we have a look at the tool output here, this is what was actually outputted to us. So you can see the different things are outputted. The title, Gymshark News. Description, keep up to date with all the latest Gymshark news and get exclusive sneak peeks to new releases ahead of launch. And then this is the URL that it gave us. And it also gave us the markdown of that entire page, which is really, really cool. You can see there's quite a lot of data here, but obviously it would've been way more if it was just HTML data and LLM was able to go through it pretty quickly. So very, very nice. We're going to use one of the other tools now. We're going to say something like, can you find me all the pages of Gymshark's website? Return to me. The first 10. So I said, can you find me all the pages of Gymshark's website? Return to me the first 10. And then I've given it the URL of Gymshark website. And the reason I said, uh, give me the first 10 is because obviously for someone like Gymshark, there's going to be like hundreds of different pages. So we just want the first 10 back. We don't want all of these hundreds of different pages back. So you can see the tool's run and now our agent is just processing that data. So it should come back to us in just a second. Okay, so yeah, here are the first 10 pages of the Gymshark UK website. So obviously their homepage, and it looks like a product page for a specific product, and then a blog, and then blog again about upper body warm-up, and then product again, blog again, product. So pretty cool. You can see obviously there's going to be like hundreds here. If we have a look at the output, this was the response. So these are all different pages that it scraped. You can see, I mean, if we have a look here, there's probably even thousands more like compared to hundreds. So yeah, loads of different, um, pages here, and it was able to scrape that for us pretty much instantly, which is pretty cool. So what we're going to do now is we're going to pick a random URL. We're going to say, uh, one of these articles. We'll have this one on upper body warm-up, and we're going to say to it, can you summarize this page? For me. So I said, can you summarize this page for me? And then just gave it the URL of that upper body warm-up blog. So we're going to see what it comes back to us with. Okay, cool. So the article Level Up Your Upper Body Warm-Up on Gymshark's blog highlights the importance of warming up for an upper body workout. So increase heart rate with like cardio, perform non-specific full body dynamic. Cool. Absolutely perfect. And then, yeah, so it's given us a pretty nice summary here. If we have a look at the output of this step, You can see this is what it returned to us. So basically just all the markdown from the page rather than all of the HTML from the page, which is just way easier for our LLM to process. You can see it's pretty much just all normal text. So way easier for our LLM to process compared to a massive lump of HTML. Okay, cool. So what we're going to do now is we're going to move away from this and we're going to create the exact same system, but I'm going to show you exactly how you can set everything up. So let's just start with an on chat message. So let's put a chat download here and then we're going to hook up our AI agent. So go to advanced AI. AI, and then AI agent. And then we're going to hook up a chat model. For this, I'm just going to use, uh, GPT-4.1 Mini just because I think it's the best for the price in my opinion. So scroll down and find that. Perfect. And then we're going to add memory. Don't really have to add a memory, but it's just nice to add one in there. And then now for the actual tools. So this is how we're actually going to hook up Firecrawl to our AI agent. So the way we're going to do it is via just HTTP requests because there's no native integration with Firecrawl on NAN, as you can see. So we're just going to do it all through HTTP requests. So put down a HTTP request, just a blank one here. And if we come over to Firecrawl, they make it really, really easy, which is nice. What you can do is you can go to, of course, your API key here. You're going to need that in a second. But if you go to Playground, and then it says different things that we can do— so single URL, crawl, map, and search. We're going to be doing single URL, map, and search. So, so basically how I like to do this is you can put in an example of a URL. So if I just do Gymshark again, and then if you hit get code, it'll basically just give you the entire code that you need to input into the HTTP request. So if we switch it from Python to curl, this tells us everything we need to know about linking this up via a HTTP request from NAN. So first of all, we get the method, which is POST. So if we come over to NAN right now, it's GET. So we just set this to POST and then the URL. You can see here, this is the URL that we're going to be sending this to. So it's just the endpoint that we're sending the data to. So put that URL in there. And then let's see if we have headers. So we do have headers. This H stands for headers. So headers, we're going to flick this on, and then it's Content-Type: application/json. So let's just copy that over. Content-Type, paste that into there. And then we're going to set this to using field below because this would be if we're letting the model define this, which we don't need to because it's going to be the same every time. So using the field below is going to be application/json. Paste that into here. And then we have another one as well. We have authorization. So let's copy that, paste it into here. Again, it's using field below, and the value is just going to be bearer. And then this is our API key. So it already just inputs our API key for us in this HTTP request— or in this curl, sorry. So we can literally just copy that and then paste that straight into there. So perfect. Now we're going to send over a body. So hit on send body and then we're going to set this to using JSON below. And then in here we can put in our JSON. So it's just going to be from this curly bracket here to this one. Copy this, paste this into here. And then the way that we make it so our AI agent can actually input things into this JSON body is the thing we want it to be able to input, which is going to be the URL. We just replace this with a curly bracket and then placeholder like that, and then close it off with a curly bracket. And this placeholder, you can see down here, when you use place— but use placeholder for any data to be filled in by the model. So this just means the AI agent can manipulate this part of the JSON body, and this is the only part we need it to manipulate. We want everything else to say the same. We're not— we want it to be able to just input the URL here. Okay, cool. So now we're just going to name that something like, uh, Scrape Page, like the other one was named. So let's hit Rename, and now we can get on to the other tool. So it makes the most sense to just copy and paste, uh, this node here. Let's go grab that, obviously paste over here, which is annoying because, um, we've already got like the headers and stuff set up. It's just going to be the URL and the body that's going to be different. So if we hook this up, this is going to be the— if we come back over to Playground and then map, then again we can just use Gymshark. So if we paste that, uh, URL into there, hit Get Code, go to code, and then you can see now this time the only thing that we need to change is the URL here. So if we copy that, come over to this node here, paste that into there, and then the body obviously needs to change as well. So let's copy that. Come over, scroll down, paste that into there. And then again, we're going to change this to put in placeholder. Perfect. So we've got placeholder in there, we have the new URL in there, and this is going to be maps. This is how we're going to be getting the pages from the site. So we can change this to get pages. Perfect. And then we have one more HTTP request to do. So again, let's just copy and paste this, come back over, grab this. It's annoying that they paste over there, but oh well. So let's connect this up, and then this time it's going to be search. So we can just put in anything here. So let's just say, uh, gym wear brands, keeping on theme with the Gymshark. Hit get code, go to curl, and then again we're gonna have to change out this URL. So let's copy that and then paste that into there, and we can scroll down to the JSON. Get this JSON, copy that, paste it into there. Perfect. And obviously the thing that we're going to be switching out is the query. So let's go placeholder again. Perfect. And then let's rename this to search. Okay, so now we just need to put in the prompt. And of course, I'm just going to be lazy and nick it from that last agent that I made. So if we paste this into here, we can actually have a look at this prompt. If we open it up, the role— you're an assistant that's able to use FileCrawl to scrape website data, find website pages, and perform Google searches. You have 3 tools. Scrape Page— with this tool, just input the URL you want to scrape into the placeholder in the HTTP request. Get Pages— with this tool, you just input the website URL of the website you wish to scrape the pages of it in the placeholder in the HTTP request. Search— use this to search queries and receive back URLs, descriptions, and full page content. Of the results. Input the query into the placeholder in the HTTP request. So pretty simple prompt, just telling it what the tools do and how to use the tools. So just input whatever it is you're trying to input into the placeholder in the HTTP request. So now we're going to open this up and we're going to test it out again. So we're going to say, find me the URL for Gymshark's homepage. So find me the URL for Gymshark's homepage. It should just use the search tool. Should be pretty simple for it. So we're going to see what it comes back with. Okay, so it said the URL for Gymshark's homepage is this URL here. Perfect. Now we're going to say use that URL to find, uh, 10 more pages on Gymshark's website. So use that URL to find 10 more pages on Gymshark's website. So let's just use this tool. If we have a look, the input was just that URL, which is perfect. This is what it got back, uh, and then this is what it's returned to us. So obviously 10 out of the output that was in here. So it's given us these, which is absolutely perfect. And again, we're just going to pick a random one. Let's go with a blog. So best, uh, upper glute workout. All right, let's go with that. And let's say something like, uh, summarize this for me. And we're going to use that URL there. And you can see it's using the scrape page. and we'll give it a sec. Should come back to us when it's done. Okay, perfect. So it said, uh, the Gymshark article, best upper glute workout, explains how to build strong, shaped upper glute area. Cool. Uh, lateral band walk, side plank with hip abduction, cable hip abduction. Uh, yeah, perfect. So if we have a look at the output of this tool, you can see this is what it returned. If we scroll past this, it's pretty much just raw text, but of course it's Markdown. But to us it pretty much just looks like raw text, which is really good. It means the LLM can handle it way easier compared to, like I said, just a bunch of HTML. Okay, so that is it for the video, guys. I hope you learned something and I hope this tool will actually come in useful to you because I know when you're scraping websites, especially content-heavy websites or pages, it can be really, really difficult to be able to pass that data to your LLM and have it actually go through that, find specific information, and give you summaries on it just because it might be way too much data for the LLM. So I hope you find this tool actually useful. I'll leave a link link to it in description so you can go ahead and try it. I'm pretty sure it is, uh, free to get started with. I think you get a certain amount of credits for free, so you can go ahead and try it for free. You can try out similar systems like this, or you can start integrating it into your existing systems. If you wanted to work for me so we can implement some AI agent systems into your business, then I've left a link to my website in the description. You can go ahead and book up a call. We could talk about some ways to integrate AI into your business. But if not, then thank you for watching, and I'll see you in the next video.