Colin Brown shows how to install GPT-4All on Windows and run Llama 3 locally on your computer. You can upload documents, index them into a vector database, and ask questions without sending anything to external servers. The whole setup takes under five minutes.
For all of you who've been concerned about privacy, today what I'm going to show you is going to blow you away. So the first thing you want to do is go to a browser, type in GPT for All, just like that. It's going to come up with the link. Take that first one. It's going to Nomic AI, and you're going to just simply click on download for your particular operating system. I've gone ahead and downloaded the Windows version, and when you get done, it comes up and looks like this. So the question is, what is this GPT-4 All? This is going to allow you to run LLMs on your local machine. It means that you can load any kind of documents from your computer and then use an LLM to ask it questions about the documents you've loaded. And it's all running locally, so it's not going out to the web, it's not training any other LLM out there, it's all localized on your local computer. We've been able to do that for some time, but it's been extremely technical setting it up, and now it's as simple as just installing some software as you've just seen. So here we go. All we have to do is when we get set up, you want to go through settings maybe, and maybe pick a theme and a font size and all that kind of thing. Nothing too serious here. In fact, I just took all the defaults, so there's nothing fancy there. Then you're going to want to set up your models, and you can add any model that happens to be on Hugging Face, which has over 2 million LLMs. So this is quite extensive. The ones that are probably most popular, these are actually built specifically for GPT-4 All, and that happens to be this Llama that comes from Meta, and then GPT-4 All Falcon, and then Quinn. But you can add any model that you would like. You can simply type in, you can find the ones that are set up for GPT-4 All specifically. Got Deepseek, and you can just pretty much choose one. Llama is going to get the job done for most things you need, but you could also go to Hugging Face and then just do a search and find any of the 2.5 million or whatever is out there today, uh, LLMs that exist. And, and then you can just download those and you're now running completely from your own computer. Once you've loaded your models, so in this case I've loaded up a couple models here, and I can go ahead and go to my local docs and then just load up. You just click on Add Collection and you can give it a name and then give it a folder path.. In this case, I had a bunch of documents in a folder and I just pointed it to that folder and it went out and grabbed all the content, all the files in that folder, which made it extremely easy to get this set up and started. So just give it a collection name and then a folder path and just have all your files in there and then click on create collection. And when you do that, You have to remember it is actually taking that data and putting it into a vector database, so it does take a little bit of time to scan and research all the content inside those files that you've created, depending on how many and how large the files are. But in any case, when you get done, you can simply chat based on that. So in this case, I put in, what's the best ways to perform forensic accounting in a bankruptcy situation? And you can In this case, I'm using the model, the Llama 3, and it came up with great detailed information based on the content that I had loaded. You will definitely want to try this out. This is a game changer. It's made the technology easy to use. Anyone can do this. You can now run an LLM on your local machine and not have to worry about revealing client data or client-specific information to the LLM and have it be trained on your client data. So there you go. Give this a shot. It's so easy to do and implement, and you can have it up and running in literally under 5 minutes. Let me know all your successes.
