Steal this system to analyse your competitors 24/7 (free template)

What's up guys, in this video I'm going to be showing you how to build a system that's able to spy on your competitors by constantly monitoring all of their activity across everywhere on the internet. With this information, the system will be able to compare it to what your company is currently doing and then create actionable insights on what you should be doing different based on that information. Just quickly, if you don't know who I am, my name is Rory Ridgers and I run an AI agency called Autonomously AI where we find and eliminate operational bottlenecks in just 14 days in order to keep business owners working on the business rather than in the business. So if you're looking to start working on the things in the business that actually move the needle then I'll leave a link to my website in the description and you can go ahead and book up a free discovery call. By the way, as well, the free templates will be available for this, so make sure you head over to the description and you'll see a link that says free templates and you can get started with this straight away. Okay, so we're in NAN now and this is the system, and what I'm going to do, as always, I'm going to give you a brief overview of what's going on here. We're going to run through it and then we're going to have a look together what actually went down. So to begin with, if we zoom in, we at the beginning we just have a scheduled trigger, and this just means you can run it whenever you like. So it could be every day, it could be once a week, it could be once a month. I would recommend sort of doing it around once a week is a nice sweet spot. And then we're just getting some information from the Google Sheet, and I'll show you that Google Sheet now. What we have in here is we have 3 different tabs. We have Competitor Details, and this is just going to be where you're going to put your competitors. So you can do as many as you like in here. You can see I already have some examples, but there's only 3. You can do way more if you wanted to. And then we have a Signals tab, and this is basically just going to be all of the insights that we're going to get. So we've got the company name, their traffic, their content output, their ad activity, product launches, partnerships, LinkedIn followers, employee counts, press mentions, technology stack changes, and then some AI-generated insights. And then we have the same thing, but we have my company. So it's just going to be like a fake company I've put together, which is going to be a clothing brand. And it's just the same data points that we have here but filled out for our company, which just means we can use it to compare the data that we get from our competitors. So we just get the rows in the sheet, which just means we get the competitors So this competitor detail sheet, we get that, and then what we do is we loop over each of them. So there's 3 in there, so it's going to run through 3 times. We're going to handle each competitor individually. And this first agent here is getting more, what I like to say, like the qualitative data. So the things like, if we come back over to the sheet, uh, so the things like content output, ad activity, uh, partnerships, those kind of things that you can't really just input sort of numbers or stats into there. It makes more sense to have like a little sentence about it. So that's what this agent's doing here. It's connected up to the Explorium MCP. And if you watch my last video, I mentioned this Explorium MCP. Essentially what it is, is just Explorium is an enrichment tool so you can enrich businesses and you can find out pretty much any type of information you'd like about them. And they've built an MCP server, which means you can just connect your agent up to Explorium and it can get any information it likes and you don't have to worry about sort of like manually putting data in. You just connect the MCP server up to your AI agent, giving the prompt on what you want to find on each company, and then it just uses the MCP server to do that. It's absolutely amazing. So we're going to look at that a little bit more in depth later. And that's going to get the bulk of the data points. And like I said, more like the qualitative things. And then what we do here is we use Apify to scrape their LinkedIn followers and their employee counts, both of course from their LinkedIn profiles. And then we also use Apify to scrape their estimated website visits for each month. And then we get the rows in the sheet. We're just going to be getting our company details. So now we have all of our competitor details here. We're now going to be getting our details that I already showed you, and then we just pass that through an AI agent that's going to take all of the information on this specific competitor, compare it to our data, and then give us insight. So it's going to look at each data point, see what we're doing differently, uh, what they're potentially doing better than us, and then give us some insights. Then we just add all of that information into the Google Sheet and then loop back around, and it will just keep going until it's done, of course, all of them on that Google Sheet. So not that complex, uh, quite a simple system, but there is a little bit going on here. So we're going to run it through, and then we're going to have a look at what happens. So right now In the competitor details, we have Gymshark, Zara, and Ralph Lauren. So these are sort of competitors to this fake clothing brand that I've made up. So we're going to run this through, and this is going to fill up 3 times with each company, and then we're going to have a look at what happened inside of an 8N. Okay, so as you can see, it's just finished running through. You can see 3 items came through. So if we come over to the sheet and then we go to the signals tab, you can see we've got Gymshark, Zara, and Ralph Lauren. So their website traffic— these are their website traffics, and of course this is an estimate. You can't get their exact website traffic, but there's lots of tools you can use to get a pretty good estimate of what their traffic is. And then their content. This is just an overview of what their content has been looking like recently. Their ad activity, product launches, their partnerships. You can see Ralph Lauren hasn't really been doing too much, so we couldn't find much on their content, couldn't find much on product launches or partnerships. But then LinkedIn followers, we've got all of their LinkedIn followers here. We've got all of their employee counts here. And of course, with the employee counts, because it's from LinkedIn, you're not going to get an exact number. It's going to fall into a category. So if this is 5,000, this is 10,001. And yeah, like I said, it falls into categories, so it's not going to be specific number, but of course these are just the different categories like the employee ranges. And then press mentions, technology stack changes, and then also some insights here. So this is what— so this is what the insights look. You can see there's quite a lot of information here, which is really, really good for each of them. Uh, yeah, so these AI-generated insights, and this is what they look like. Perfect. So now if we come back over to N8N, we can have a look what's actually happened here and go into these nodes a little bit. So the first one, we're just getting the information from Google Sheets. So we've chosen competitor research, which is our sheet, and then the competitor details sheet inside of that. And this is what we got back. So we've got the company names and the company domains, and then we just have a simple loop over items node, set the batch size to 1. This is just going to loop over each item one time. Then what we do is we have an edit fields where we just define the company name and the company domain. And the reason that we do this might seem a little bit weird, but the reason that we do this is because for some reason with the loop over items node, if you want to get data from this node itself, you can only do that when referencing it in the node straight after. So in this AI Agent node, we're using the company name and company domain, which of course comes from here. But then we also need to use the company name and the company domain in this Tavly step. And if we look in here quickly, you can see if we go to loop over items, we can't use the data because it wasn't directly after it. So that's probably gonna happen to you all the time. And the simplest fix is just doing edit fields, and then these variables are gonna be consistent and you can use them at any point in the workflow. So after that we have the, like I said, we've got the edit field step and then we have our AI agent here that's doing a lot of the heavy lifting. This is one that's connected up to the MTP server. So we give it the company name, we give it the company domain from that step before, and then we just have this prompt in here that says you are part of a competitor research system that's looking for useful signals on competitors. And then we've essentially just given it a list of the different signals or the different data points that we want. So content output, ad activity, product launches, Partnerships and Integrations, Press Mentions/PR Events, and then Technology Stack Changes. And the way it actually gets all this information is through this Explorium MCP server. And if you haven't used Explorium before, this is basically the platform— I'm going to show you the documentation quickly. And essentially what you can do in here is you can just get any information on any company or any individual that you like. So if we go to— this is the API docs— if we go to Businesses, the way it works is Explorium basically has like a big database of companies, and you first send off a company domain and then you get back a business ID if if it's in their long list of businesses, which it probably would be because it's a big database they have. So you do the match businesses endpoint, send off the domain of the business. So for example, if he's doing Gymshark, would send off Gymshark's domain, get back a business ID, and then we use that business ID to get all of these different, um, things or these different data points with these businesses. So if we go to business enrichments, and you can see these are all the different things that we can scrape. So there's loads on here. So we've got things like firmographics, technographics, social media presence, financial metrics, loads of different things here. And then we also have some information on events like product launches, funding rounds. So you can see there's literally loads here. And we've connected this up to the MCP server. So Explorium has an MCP server. As you can see, if we go to MCP Playground here, we can play around with it through an AI agent. But we can also connect it up inside of NAN. And the way we do that is you just use the MCP client node in NAN use this endpoint here. And for the authentication, go to bearer auth, and then credentials for bearer auth, go to Explorium. So you're gonna have to make a new one. If you go like this, all you do is you put in your API key, which you get up here. So you can just copy that, make a new bearer auth, which is basically just like an authentication method that isn't natively inside of NAM, because Explorium doesn't integrate natively with NAM. So you just have to make sort of this bearer auth. So you just make a new one, call it Explorium, put in your API key here, and now this, um, MCP server is going to be connected up to your Explorium account. That's all you have to do. And then you basically just tell the agent how to use it, which is really, really simple. It's what I just told you. You first send off a thing to get back the business ID, and then you use that business ID to enrich them further. And the AI agent is going to be able to use the MCP server and do pretty much anything you want. So it's really, really cool. In natural language, you can just say the information that you want. The agent will use the MCP server. It's going to list all of its available tools that it can use and then based on what you're trying to get, it will be able to use that. So if we have a look inside of here, if we go to the first one, so it first sent off Gymshark's domain, so gymshark.com, got back a business ID, and then it also got back all of the tools it can use. So it went for a new product, a new partnership, got some information back, and then this time it did LinkedIn post and website changes and Webstack. So you can see here how it's kind of working. It gets back all this information, and then what we do is we do required specific output format, and then we just add a structured output parser and do one that looks like this. So this is JSON for the information that we want. So content, ad activity, product launches— I'm not going to read them all out, but you know the ones, they're the same ones that are in the prompt. The reason we do that is so that this way it's going to output it in JSON. So if we go to— we can go to JSON, you can see it fills in basically what I just showed you. Then when we go to schema, now these are all separate variables, so we can actually use these separate variables. You can drag these and it makes it way easier to put it into something like a Google Sheet. So that's that agent here that's doing a lot of the heavy lifting. The other data points we need are sort of quantitative data. If you don't know what I mean when I say quantitative, basically just numbers, right? So the other ones are more qualitative, it can write a little sentence about it, but these ones like followers, employee count, website visits, it's just numbers, right? So the first step here is going to be a Tavli step. And if you don't know what Tavli is, there's now a native node for it inside of N8M, which is really, really nice. All you have to do is just put in your API key and essentially it's just like Google, right? So it's just Google. You can do search queries, you can do other stuff with it, but in this case we're just doing a search query. So the resource is going to be search. The operation is going to be query, and then in here you just put any search query you want. And for me, it's going to be the name of the company. So in this case, it was Ralph Lauren, and then we do Ralph Lauren's LinkedIn profile URL. So as it cycles through all of these, it's going to have changed. So the first one would have been Gymshark's, the second one would have been Zara's, and then the third one, Ralph Lauren's. And it's just going to return to us their LinkedIn URL, which is really, really nice. It just gives us a variable called URL and then gives us that URL. And then what we do is we run this through through Apify Actors. So if you don't know what Apify is— a lot of integrations in this one, uh, but if you don't know what Apify is, if you've watched my channel up until now, you would probably know what it is. But just to quickly run through what it actually is, it's basically just an online marketplace of all different scraper APIs. So you can scrape essentially anything you like. If you have a look here, you can see these are all different things. So Google Maps, website, TikTok, Instagram, tweets, Facebook, uh, Cheerio, Puppeteer. Like, there's so many different things, and it's all made by the community. So there's like thousands on here. And what we are doing for this video, we're doing the LinkedIn profile scraper for cheap lead enrichment. So for this one specifically, it's the LinkedIn profile scraper, and this is what is going to give us the followers and employee count. So the way we connect it up is now we've connected up our authentication by just putting in our API key. The resource is gonna be actor, operation is run actor, and the actor source, Apify Store Actors. And then you can just literally copy the name of this. So you can just copy this. Paste it into here and it'll appear and you can just click on it as I have. And then to get this JSON body, what you do is you come back over to Apify and then right now you can see this is the input. Um, so we need to input the company URL. That's why we did the Tably step before to get their LinkedIn URL. Just switch this to JSON and then you can copy this and then you can paste it into here as you can see. And we've just replaced this variable with the Tably step before, so we just dragged in URL And now you can see we're sending off a request to scrape that LinkedIn URL. And the way these Apify Actors work is you first send off a request to run an Actor, and it's going to keep running until the Actor finishes. And then it basically just gives you an output that says it's finished, as you can see. So the status is succeeded. And then you do a get dataset items, and you just put in the dataset ID from the step before. So if you just come up here, type up dataset, you can see default dataset ID, drag that variable in like that. And then here you get the actual information that you want. So we get a lot of information on their LinkedIn profile, but we only want the follower count and the company size. Perfect. So now we have all of those other data points. We also have the LinkedIn follower count and the employee count. And then we do the exact same thing again. We've done an Apify node. This time we're scraping for their website visits. So if we come back over to Apify, this is the one we're using, the Cheap SimilarWeb Scraper. And then in NAM, we've just typed that up again. Selected that right one. And then the input JSON— come back over to Apify, you can see this is the JSON that we need to, uh, put into the request from NAN. So it's literally just their domain. So you can just copy that. And then the domain variable we get from that edit fields, because this is what was inputted to us from that Google Sheet. So we literally just drag in the domain. And then again, we just get back the status of that, which it should be succeeded. Okay, and then this is the information that we get back. So we've done the get dataset items, put in the dataset ID, and we actually get back quite a lot of information, like the top countries by shares the engagement, like the bounce rate, the year, the month that all this is happening. And then this is what we're actually looking for, so the estimated monthly visits. And it does it by month, and you obviously would just select the most recent month here. And then it gives us things like the global rank, the country rank, the category rank, things like that. But in this case, you can add that in if you like, but in this case, we're just looking for the website visits. So now we have all that information, we do another step to get rows in the Google Sheet. This time we're just going to be doing it on the My Company tab. So we get back all of this data, which is all the data we got for our competitors, but of course this time it's ours, so we can use it to compare. So we get the company website traffic growth, content output, you know, all of these data points. I've already showed you all of them. So we get all of that, we aggregate that. So we basically just put all of that together. It just makes it easier for us to input that into an AI agent. We don't have to drag each variable in separately. We just aggregate it and chuck in the whole variable. And then what we do is we pass that to an AI agent where you can see we've given it the competitor data that looks like this, and then we've given it our data. So our company's data, it's the same thing, but obviously same data points is what I mean, but different data in there for our company. And then we have a prompt in here that looks like this. So we've said, you are part of a competitor research system that is looking for useful signals on competitors, and then you'll be given data on my competitors as well as the same data points for my company. Your job is to go through the data points. That can compare my competitor to my company and generate key insights. Your insights should be a short paragraph around 200 words long, and then it's going to do exactly that for each of them. So if we look at the first one, it looks like this. Second one looks like this, and then the third one looks like this. And then we simply just put all of that information back into the Google Sheet, this time in the signals table. So we're putting it back into the signals table, and we just have all these different variables. So company, website traffic, content, activity, product launch, partnerships, LinkedIn followers, employee count, press mentions, technology stack changes, and then insights. And this is all data that we've got from these steps before, as well as the insights that we just got. And then it simply just runs through that. In this case, it was 3 times, but it would do it however many times you wanted it to do. So I hope you enjoyed this video, and like I said, if you want to grab the free template for this so you can get started with it straight away, then the link will be in the description of this video. You can just head over there, click on the link, get it for completely free.