Run Any Chatbot FREE Locally on Your Computer

Have you ever wanted to run a chatbot locally on your own computer? You know, your own version of like chatgpt or Anthropic or something like that, but do it completely locally. Like don't even need to be connected to the Internet. All of your data and information that you type in is completely secure and only on your local computer. Well, I think I found the absolute best tool to do that with. And it's free and open source. Check it out, it's called Jan and you can find it over at Jan AI. You can see all of the code is completely available over on GitHub. So you can download the code, modify it, build your own version of this tool if you want. And under the license you can see Jan is free and open Source under the AGPLv3 license. Now one thing that's really cool about this Jan app is that it works on pretty much everything. You can download it on an M1, M2, M3, Mac, an Intel, Mac, Windows or even Linux. Now you've probably heard a lot about open source models like Llama 7B and Mistral and very various other open source models, but it's always been a very confusing and sort of complex process to download and run these models on your own computer. This tool seems to make it really, really easy. So let's go ahead and put it through its motions. Let's go ahead and click Download for Windows here. Since I am on a Windows PC here, I'll go ahead and run the install. And just like that you can see I now have a hub where I can have chat messages. But at the moment there's no models installed. I don't have a Llama or a Mistral or anything like that installed to be able to use the chatbot. We can see here it says you don't have a local model yet. Explore the hub. If I click here, we can see what models are immediately available to download and use. So we've got Mistral, instruct 7B, open Hermes, neural 7B, Trinity Starling, Alpha, Llama 2, Trinity v1, and even the newer Mixtral 8x7B, which is probably the best best open source chat model on the market right now because it uses eight different seven billion parameter expert models. We've talked about this in a few previous videos and if you want it to give you the exact same types of outputs that you get out of something like Chat GPT, it's even got the OpenAI GPT4 model that you can use. You will have to plug in your own API key it's also got GPT 3.5 Turbo 16K and the original GPT 3.5 Turbo. So not only can we use the open source models, but we can use a handful of the closed source models as well. And the cool thing about this is you can set up multiple large language models and then sort of switch between them to see which one you like the best. So let's go ahead and install Mixtral 8x7B. Now this is a 26.44 gigabyte file, so it might take some time to download, but let's go ahead and get that process started. Surprisingly, it's moving pretty fast. I'm also going to install Llama 7B. This one's 4.8Gigabytes. Now it looks like the two versions of GPT 3.5 Turbo and GPT 4 are already pre installed for us. So we don't have to actually do anything. We just need to keep in mind that if we are going to use one of these OpenAI models, we will need to be connected to the Internet because it's going to use the API, it's going to connect to their servers and it's gonna do all of the work over there. So you will need to be connected to the Internet if you wanna use one of these models. But they are available and we can compare em with other models if we'd like. Now that I've got Mixdral 8x7B installed as well as Llama 2.7B, I can come back to my chat page here and if I open my little sidebar you can see I've got some options. So I can give this new chat a thread name. Let's just go Test Chat 1. You can see it calls it Test Chat 1 over on the left side. Very similar UI to Chat GPT. Then underneath the Assistant tab here we can actually give some custom instructions. And under the model tab we have a dropdown where you can see our two local models, Mixtral 8x7B and Llama 2.7B as well as the three OpenAI models that are available. Now. If I do want to use, let's say for instance, GPT4I, I can select it and then I do have to enter my API key here in order to use it. You can find your API keys by going to platform.OpenAI.comapi-keys create a new secret key. I'm going to call it Example key, Create the secret key and then I'm going to go ahead and delete this key when I'm done with this video so it won't continue to work. But I can copy this key here, jump back into our platform and paste our API key in. Let's go ahead and boost the max tokens all the way up here. I'm actually going to jump into my Chat GPT account real quick. Grab the custom instructions that I've given it in the past. Use these same custom instructions in my custom instruction box here. And now I've got custom instructions set up now. I could chat with it just like I would chat with ChatGPT now because I'm using my API key, it's going to cost me a few cents every single time I ask it a question. But this is just for demo purposes. More than likely, if you're using this Jan app, you're going to use it for the free open source models like Llama and Mixtral. But let's give it a sample prompt of give me 10 strategies to grow my YouTube channel and click Send and you can see just like Chat GPT, it starts to output a response in real time. And there we go. It quickly gave me 10 ideas on how to grow on YouTube. Give me 10 more ideas. It gives me a handful more ideas if I jump over to platform.OpenAI.com usage. I'm actually recording this on January 3rd, but this think since January 4th already and you can see I spent $0.02 using GPT4 already. This one will cost you money if you want to use GPT 3.5 or GPT 4. And now let's go ahead and create a new chat. I'm going to go ahead and use my exact same custom instructions. Pull this into the new chat here. We'll call this one test and we'll call it Mixdril. Now if I select the model mixdral 8x7B, crank my max tokens up, I can use this large language model as much as I want with absolutely zero cost. It's completely running off my hard drive. The only cost is the power my GPU is using while running this. Let's give it the same prompt. Give me 10 ideas to grow a YouTube channel fast. And using Mixtral 7B, it starts to give me a response in real time. Now you can't hear it, but my GPU fan is really kind of cranking up right now. We can see down here we're using 59% of the CPU and 74% of memory. And just like that we got 10 pretty good responses on how to grow a YouTube channel. And I would say say the level of output is Fairly close to what you'd get from GPT 3.5. And once again, let's go ahead and create a new chat. We'll call this one test llama 2.7B. I'll paste in the same custom instructions right here and then the model we can switch to llama 2.7B. I like to crank up the tokens here and let's try a different prompt. What are some best practices to get high engagement on Twitter? It's firing up the model real quick. You'll see my CPU and memory usage crank up as well. There it goes right there. If I click on this little icon down on the bottom left, you can actually see how much RAM and how much CPU it's using and it starts to give me a response. And the token speed here is actually much faster than what I was getting out of ChatGPT and what I was getting out of Mixtral. So pretty interesting there. But again, there are just a ton of models that we can play with in here. They're open source, they're free, they're installed locally, so you don't have to worry about the privacy concerns of it. I could disconnect from the Internet right now and this chat would still work for me. Assuming that I'm using llama2 or mixtral and I can download any models I want off the Internet. I can download uncensored models. I can find models that other people have fine tuned and set them up in this Jan app and run them locally on my computer. This has to be the easiest, fastest, freest way of running chatbots like ChatGPT locally on your computer. And right now I'd say the available models are about as good as what you're getting out of the free version of ChatGPT or GPT 3.5. So if you've been wanting an easy way, a shortcut to quickly access and use offline local open source chat models, this might be the best entry level solution I found so far. It's really simple, it's free. It took me less than a minute to download and set up. Yes, there's some time to download the models, but that was even fairly quick. I mean, even the 24 gigabyte file only took two or three minutes to download and we get to use it in a very recognizable, very intuitive user interface because it reminds me very much of what you get out of Chat GPT, just with a few extra options over here on the side. This video is not sponsored I have no relationship with this Jan tool or any of its developers. I just came across it today, thought it was really really cool. Thought it would be worth sharing because it's probably the easiest way I've come across to use these local models and wanted to quickly share it with you. So hopefully you found this valuable. Go check it out again. Totally Free Jan AI. It's open source and they're continuing to build off of it and iterate off of it. And as more new open source models become available, this might be the go to easiest tool to run them on your computer. That's all I got for you today. If you want more quick tutorials like this, the latest AI news, breakdowns, research and other cool AI and tech information. Give this video a thumbs up and I'll make sure more like this show up in your YouTube feed. Also, maybe consider subscribing. That helps the channel a lot as well. If you haven't already, check out Future Tools IO. This is where I curate all the coolest AI tools that I come across. Like this Jan tool that I just showed you I updated every day along with the AI news. I have a free newsletter that shares just a handful of cool tools every week and the most important news of the week. It's all totally free for you over at FutureTools IO. Check it out if you haven't already. And thank you once again for tuning in. Really, really appreciate you. See you in the next video. Bye bye.