How to scrape websites with AI and use ChatGPT to enrich the data and then create a custom list

In this video, I'm going to show you step by step how to be able to find a website where you want data from and then take that website and turn it into actual data, like on a spreadsheet, like actual data on a spreadsheet. So let's get started. First thing you want to do is you want to figure out what is your target market, like who you're targeting. For example, for me, I want to be able to have a conversation with sales trainers, and I found this organization called Sandler that has lots of sales trainers. They actually have over 50,000 people that 50,000 companies that they're servicing right now, and they have over 200 trainers, 200 companies that have many trainers. Some of them have many trainers in them. So these are the companies. So how did I get this list of companies? Well, I did some research and I went to their website and on their website I clicked on locations and then under locations I found a list of locations. Now unfortunately it wasn't loading. The locations were not loading over here. So what I did was I went to— I went Ctrl+C, copied the URL of their website. Then I went to Google and I typed in site: their website. And now all of a sudden it showed me every single page that's indexed in Google with each one with the company. You see, because I checked the directory. And so I took these 200 pages, 200 and change pages, and I used this tool here, this little Chrome extension called Instant Data Scraper, and it made it into a little spreadsheet. And I did that 3 times and I put it into this spreadsheet here. Okay. So I copied and pasted them into here. And now I wanted to go through them one by one and collect their information. And then I realized, you know what, why don't I use no-code automation? Why don't I use this website called make.com that will essentially read each one of these websites individually, pull the HTML from each one. And then put it into ChatGPT to make sense out of it, and then use an enriching tool to find the LinkedIn email at the LinkedIn and all the information about each company. And let me show you step by step how I did that. My name is Joe Apfelbaum. I'm the CEO of Evvy AI. We're a LinkedIn AI assistant, and I love no-code automation. I love building technology without using any code, and I love teaching people how to do that. So if you join me on my YouTube channel, You'll learn step by step how to be able to do these things, how to be able to create apps and create Chrome extensions and to be able to develop whatever program you want to solve whatever problem you want without knowing any coding or anything like that. Okay, so let's get started. Let's get right into it. So you saw how I made this list here. Now let me show you how I extract that information. So the first thing that you want to do is you want to create a new scenario. So on the top right corner, when you log in, you create a new scenario. So what's the scenario? A scenario essentially is what's the situation, what's going to happen over here. So I created many scenarios when I started. It was really hard for me to be able to understand how this works, and I had absolutely no idea what to do. And so I joined several communities, and in those communities I slowly met people and I learned from those people. And now I built out all these incredible automations, as you can see see here. All right, so the first thing I want to do is I want to be able to find the rows on a spreadsheet. That's the first step. First step is find the rows on the spreadsheet. So I'm going to create a scenario, and in that scenario, let me just see AI. I think I have it over here somewhere where the first scenario— okay, enrich URL using AI. That's the one I created here. Okay, so this is it. All right, so the first part of this is I am going to say, look for this sheet and then find any place, any place that says yes for the D that's enrich. So I'm going to write yes on the ones that I want. I don't want it to enrich all of them. So I'm just going to go to the ones I want to enrich. So in this case, I want to do 1, 2, 3, 4, 5. I want to do these 5 right now. Okay, if I want to do all of them, I just have to write yes in all of them. I can just drag down yes or Ctrl+V yes across all of them, and it'll do all of them. The reason I like doing this is because I want to know if it was done or not. Some of these were done partially, so I'm just going to write no to this one and no to this one, and then we're going to do a bunch more here. You see, if I want to drag down the yes, I just drag it down like this. You see, so now I have a bunch of them over here. And this is set to yes. And so when this finds something that's a yes, it moves it on to the next step, which is a very simple thing. You take the URL from the spreadsheet, from column A. You see there's a URL in column A. Let me show you where. Here's a URL. Go to the URL and grab that file. That's all it's doing. It's grabbing that file and it's saying, show me the HTML for this file. Then in the next step, it turns that HTML into text. So I say, grab the data that you got from the HTML and then just turn it into text for me. That's it. Simple as that. Turn it into text. Once it turns it into text, then I say, go ahead and take that text and turn it into a JSON. And this is the exact, the exact prompt that I use. I say, extract the information from the following text. pull out the owner's first name, last name, the company name, the website, the full address, the phone number, and the email address, and only return JSON through this particular structure. So I'm telling it exactly what to give me and leave any field empty if the information's not found. And I show where the source is and what the text is. Okay? So I give it to ChatGPT and ChatGPT returns a JSON for me. Now I can do a lot with that JSON, but for right now, All I'm going to do is I'm going to take that and I'm going to go to my enriching software and I'm going to say, take this information, first name, last name. I spelled first name wrong here. Let me fix that here. First name, first name, last name. Okay, first name, last name. Take the first name, the last name, the company name, the website. The address and the phone number and give me back all the information. And what it does is it gives me back all this incredible information here. You see? Boom, boom, boom, boom, boom, boom, boom. And then I take all this information and just put it back on the spreadsheet. And then I say, say done instead of yes. So next time I run it, it says done in the enriched column in D. And then it takes all the information and puts it over here. So what information am I getting? I'm getting their first, last email, their website URL. I'm getting their LinkedIn URL, which I'm going to further use with another API. Once I get this information from the— with the LinkedIn URL, I'm going to take this LinkedIn URL and I'm going to add tons of information about their LinkedIn profile. And then once I add information about their LinkedIn profile, then I'm going to create an email to each, each one of the sales trainers about my tool, Evvy AI, letting them know, hey, I built an AI for LinkedIn. And I'm actually using AI to reach out to you, FYI. And if you want me to build this system for you so that you could reach out to sales organizations, I'd love to give you a complimentary dashboard. I'd love to support you. I'd love to help you with LinkedIn. I'd love to get to know you. And out of those 200 sales trainers, if I can get 10 of them or 20 of them to have a conversation with me, it's a huge win because each one of them will service 10 to 15 new clients every single month. And each one of them could be a massive customer for me. And so that's the strategy that I'm employing here to build a relationship with these Sandler trainers. And I'm going to do it one by one, but I'm really going to do them all at one time. Okay. So watch how it fills out the sheet. So let's just hit run and see if it actually works. See if this automation actually works. And by the way, I put a little filter over here that says only send it forward if there's a URL in the link. Okay. Because otherwise none of this is going to work. I need an actual URL. All right. Save. Let's hit run once and it's gonna start running. Now up here you see some other modules. These modules are something I was trying to do with Perplexity and it was breaking. It just wasn't working. So whatever, I'm just letting go of that. And by the way, I limited this to 1. So now I'm gonna limit this to 10. So it'll do all of them. Let's say yes. And let's just see what it did for with the 1. So we have 1 done here. We have 1 done here. So it just did this. Okay, so Tim pulled his email address, his company name, no phone number here. Let's see, we have his LinkedIn, we see his city, state, his company name. Let's check if his LinkedIn URL is correct. I'm going to copy it. I'm going to paste it in here just to kind of see. Oh, look at that. This is Mr. Making Luck Inc. I love that. I love that. Wow, such great info. And let's see, wow, his phone number and his email's right here. And his email, let's see if this email matches what we have in the spreadsheet. And it looks like it does. It matches what we have in the spreadsheet. Look at that. firstname.lastname@sandler.com. I love that. Okay, so I have his email, I have his LinkedIn company URL, I have his logo. So if I want to make him a presentation, I can make him a presentation. His address is missing and his photo is missing. Okay, that's fine. But at least I have a lot of information about them. All right, let's go do the rest of these real quick and let's see how quickly I can do 10 of these. So we got 1, 2 here. I'm skipping these 2 for now. And then I'm going to do yes, yes, yes. Okay. So I'm just going to hit run now and see if it'll do 10 at a time. And we have 200 to do. So let's just see. Okay. So it's doing 1. Now it's Oh, now it just did one at a time. Let me see. Did I— oh, I only have one here. Okay. So let me just change this to 10. I'm going to hit run and let's go check why as it's doing it. Let's look at the spreadsheet as it's doing it. Okay. Yeah. Pretty much doing it right here. You see Brett, it did Brett, Ivan, it's doing Mitchell. It's doing Katie, Steve, Jason. Nessen. So it's doing all the ones that say yes. Okay. If it doesn't say yes, it's not going to do it. And for whatever reason, it did not do this one over here. It did not. Oh, corporate headquarters. Oh boy. Explore Sandler's corporate headquarters. Okay. No wonder it didn't do it. It's a good thing it didn't do it. I'm just going to hit no here. Um, yeah. Wow. Okay. I'm glad that I caught that one. All right, so we got all their email addresses here and we're missing some LinkedIn URLs, it looks like over here. Like for example, this guy Ivan, maybe because it's like this. I'm going to, I'm just going to search for his LinkedIn URL real quick just to see if I could even find his LinkedIn URL. He might not even be on LinkedIn. Let's see if this is him. I don't know Madrid. Is he in Madrid? Let's check. Let's check where this guy is. Ivan Enrique. Where is he located? Why is this kind of grasshopper firm? Ivan Enrique. Oh, in Mexico. That's why he's in Mexico. He probably doesn't have LinkedIn. This is a Mexico. Guy, a guy from Mexico. Okay, listen, it's not for everyone. Let me just see if I could find him. I'm just going to search this on Google and I want to say Mexico site:linkedin.com. It was not able to find anything. Let me just delete his address here. Oh, look at that. It found— oh, is this even him? This might— no, that's not even him. This is like a nobody. Let me see. No, this is a student. All right. This guy's obviously very hard to find. I could still email him and say, hey, I was looking for your LinkedIn profile. Do you have LinkedIn? You know, like that's the thing I would do. Okay. So I don't have to go crazy. I don't have to find every single person, but preferably I'd find whoever I'm just going to write no over here, even though it says done. This is like a no. All right. Let's see who else we have here. Okay. We have all these other people here. All right. Moving on. All right. So now let's try to do 20 at a time. So we're going to do yes. We're going to copy this. Actually, you know what? We'll just do like all these over here. All right. Let's just go and let's just hit run. And you see how fast I'm able to do this? Look, it's doing 2, 3, 4, 5, and it's just filling it out. I love this. This is just the bomb. Does this cost money? Yeah, it costs money. It doesn't cost a ton of money though. It's actually really affordable compared to like having someone do this manually. Oh my God. And it's also super fast. It's like, you know, it's just there. It happens. So if I'm able to pull even half of this list's LinkedIn URLs, like, wow, that's like amazing. Be able to check them out, build a relationship, leave comments on their posts. Oh my God. If they're not on LinkedIn, they're probably not a good prospect for me anyway. So, you know, I want to make sure, and by the way, the reason I'm doing only 10 at a time is because I want to kind of monitor this and just make sure that this is actually working. 'Cause if it breaks, if I do the whole thing and I walk away from it, I want to kind of look at it, make sure it works, make sure the data is good. And then I can do thousands of them if I want. But as you can see, it's pulling all the information here. And I also like spot checking every once in a while. Like for example, this guy, Matt, president of whatever. I'm going to go here and just spot check and just make sure this is him. Yep. It's Matt. Check out his website. Make sure this is him. Yep, he's in California. And then hover over his email over here. Yep, firstname.lastname@sandler. And I go here just to double-check if it's the same. firstname.lastname@sandler, Sacramento, California. His website, his address, all of them. He even has his photo. Let's see what he looks like. Oh, oh, his I don't have his photo, I have like an avatar. Oh well, moving on, let's do a couple more. We are going to keep filling this out and get all the different trainers available. I'm gonna hit yes, Ctrl+C, I'm gonna go to the end and I'm just gonna do them all. And now that I have them all, now I'm gonna do a campaign to identify more information about them and then I'm gonna, like I said, I'm gonna create an email. I'm gonna scrape everything from LinkedIn about the people that I can, 'cause I don't have, everybody's not on LinkedIn. So let's just say 50% of the list will be on LinkedIn. So I have 26 out of 35. So I would say probably 75% of this list is gonna be on LinkedIn. So if I have 210, so let's say it's 150 people. And then I'm gonna have another tool that I'm gonna go just grab all their information from LinkedIn. And then I'm gonna identify who has a lot of followers, who has no followers, who has a photo, who doesn't have a photo. And then I'm gonna send them each a custom email. Obviously AI will write all the emails, but then I'll be able to just send a very personalized email to each person just in one shot, just like that. And I absolutely love being able to teach people how to do this themselves. And if you want to learn how to do it, let me know. Just say, I want to learn, and I will be happy to walk you through. I meet with my community 6 times a month and I teach people how to do things like this. So let me know if this is something that you're interested in learning and mastering, and I'd love to help you. My name is Joe Apfelbaum. You can find me on LinkedIn at joelinkedin.com and check out my YouTube channel for more tutorials like this. Hope you enjoy. Boom.