This AI Model Runs On Your Phone (With No Internet)!

Okay, so I came across something really cool that I think everybody's going to want to know about. And it's the ability to run AI models on your phone that are actually really good AI models without needing to be connected to the Internet. So like, you could use these models while you're on a plane and at your house when you don't want them sent to any sort of cloud service. You don't want to use OpenAI or anthropic or Google or any of those companies. You could use some of the best available open weight models on your phone. So check this out. This should be a pretty quick video. So I came across this post from Adrian Grandin here where he was showing that he was running the new Quinn 3.5 on his computer. And if you actually zoom in here, you can actually see he's using this in airplane mode. It's actually running pretty fast. And he's using the Quinn 3.52B model. This is like a brand new model here. If you're not familiar with this new Quinn 3.5 model, it literally just came out this week. It came out on March 2nd. It came out in four variations. An 800 million parameter model, a 2 billion parameter model, a 4 billion parameter model, and a 9 billion parameter model. And it's actually a really solid model, like most of the type of stuff you would want to do on a phone. This is going to work great for like you're not probably doing any sort of insane logic problems or having it solve some complex math. You're probably just going to have it like brainstorming with you or asking it how to deal with your kid that's having a meltdown right now or something. But check this out. Here's the benchmarks. And as you know, I don't put a ton of weight on benchmarks, but these models are on par with the best open source GPT model. Performs better than GPT5 nano in most of these benchmarks. Like it's, it's a really decent model. So jumping back to this Adrian dude here, I wanted to figure out what the heck is he doing to actually run these models on his phone. And it turns out that he created something called Locally AI app. So I went and found that app. You can find it in the App Store. It's called Locally AI. I will link to it just to make sure that you're getting to the right one. If you do want to download it, let's go ahead and grab it. I'm just going to scan the QR code On the website here. And, well, here's what it looks like inside the app store. Let's go ahead and click get. And as of right now, 4.8 stars with 579 ratings. So people seem to like this one. And let's take a look here. So when we open the app, it's going to ask us which model to choose. You've got the Apple foundation model. If you're on an iPhone that's built in. We've got Gemma 2, Quin 3 1.7 B and Llama 3.2. Now I want to use the new Quinn 3.5, which should be available. So let's just go ahead and skip. I'll select a model and when I select a model after skipping, I actually have more models available to me that weren't showing on that initial screen. Now, for running on a device like this, the smartest thinking model that we're going to be able to get on our phone is going to be this Quinn 3.5. This is the one we're going to want. So I'm going to go ahead and select that one. And then we have a few options. You can grab the 4 billion parameter model, but it does recommend an iPhone 15 Pro or newer to be able to run that one. You have the 2 billion parameter model which recommends an iPhone 15, not necessarily a pro. And then Quinn 3.5 with the 800 million parameter model, and that one can run on an iPhone 14 or newer. I'm on a 17 pro here, so I'm going to grab 3.5, but obviously grab whichever model is going to work best on the phone that you're on. It's going to take a moment to download here. Okay, so it took roughly five minutes on my wi fi. Obviously your results may vary depending on your own Internet speeds, but I'm going to quickly download the 2 billion parameter version as well so that I can kind of compare speeds. All right, that took a few more minutes, but I now have both those models. Let's browse around real quick. So up in the settings, I've got manage models where I can add more models in personalization, I can actually give it custom instructions. That's pretty cool. Can also change the temperature. I'll leave everything on their defaults for right now. We can delete our whole conversation history. And yeah, that's about all the settings we've got. But also you can add a Siri shortcut where you can say, hey, locally AI and ask questions directly to the app. Pretty cool. All right, so Right now it's on the Quinn 3.54 billion parameter model. Let's do a basic test here. How many Rs are in the word strawberry? We can see it's breaking it all down for us. And found there are three Rs in the word strawberry. Now I know this shows that I'm on wi fi right now. Unfortunately, if I turn off WI fi, my ability to share my screen to my computer also turns off. But I have tested this and it does work. With WI fi turned off. You do not need to be on the Internet. This is not sending any information to the cloud whatsoever. It is operating completely on your phone. I'm going to go ahead and switch it to the 2 billion parameter model here. Let's start a new chat up here and I'll ask one of the sillier questions that's been going around right now. If there's a car wash that's 200 meters from my house, should I walk or should I drive to get there? Let's see what it says. It has some of the same logic flaws that the other models have. To decide whether to walk or drive, you need to compare the travel time under different conditions. Compare exact travel times. Consider non time factors. You should drive if your driving time is less than walking time. In all sudden scenarios where walking takes over an hour, driving is blah blah blah. Yeah, why would I walk if I need my car with me to use the car wash? If your car wash requires your car to be present and walking becomes a practical necessity, not a theoretical choice. Yeah, so I mean it is not going to be the best at logic like some of the newer models are. But if you're using it as like a brainstorm partner, like brainstorm some ideas for YouTube videos about how AI will impact our daily lives. And because I'm using this 2 billion parameter model, it's moving actually pretty quickly. And again, pretend I'm offline right now because it's not connecting to the Internet to give me these responses. AI will take your job. The hard truth. Could your cat control your doorbell with ChatGPT? Just trying to think through that video. Is my voice learning by listening to a podcast? Robot exoplanet Robo turtles versus Human dog walking Point of view Horror. Why AI might just be an alias for me. Five AI tools that actually work better than cddp I don't know what CDDP Is that some. Is that an acronym? I should know how I'm using AI to unlearn my bad habits. The death of copy paste AI generated Relationships. Curating the digital supermarket AI for caregiving. I mean, there's a lot of ideas here and I could be sitting on an airplane getting these ideas. I also didn't even have the thinking mode turned on. You can see down in the bottom left, there's little, little light bulb here. Let's turn the light bulb on and say, give me. All right, so now it's not just immediately responding, it's actually thinking through. And we can see the chain of thought as it's doing this. So we actually have like a chain of thought thinking model again running completely on device. I do feel my phone getting warmer. It's definitely warming up as I do this. It's not hot, but it's. It's using the processing power for sure. It's actually spending quite a bit of time thinking and now it's giving me a response. Here's a list of 30 plus specific niche, deeply engaging YouTube video ideas. What I'm telling you right now. A direct AI interaction. Five deep fakes you won't see in 2024, but you will. The interview with a human. The interview with an AI. Testing the prompt ecosystem on a job market analysis. How I caught myself being spammy. The AI filter. You get the idea. It's given me just a ton of ideas right now. As this chat gets longer, you could probably even tell from my video that it's actually starting to get a little bit choppy. So it will slow down a little bit as there's like more details in the context. Just my scrolling here is getting choppy now. But it's also a visual model. So let's go ahead and stop this and let's open up a new chat. I'm going to turn off thinking. Let's click the plus button here and take a photo so you can see what I'm looking at here. And this is what I'm drinking right now. Let's take a picture of it. Let me go like that. So it's not got that crazy effect going on. All right, let's take a picture. I'll use this photo and we'll say, is this a healthy option? Yes, it's a healthy option. Zero sugar, natural flavors, blah blah, blah, blah, blah. And it's actually running fast again because it's a new chat so it's not trying to pull in all the context from the previous part of the discussion. And I guess just to prove that it will work without wifi, I'm going to. Let's go ahead and record my screen. I'll do Record and then I'm actually going to turn off wifi and from this moment on, it's going to be using my phone's recording. So turn off wifi here and I'll also put it in airplane mode. So it's not using the 5G at all. So we have zero Internet. We are in airplane mode right now. Let's start a new chat. Let's go. My kid is throwing a fit because I took his iPad away. How should I calm them down? And we could see it working right now. It is streaming pretty quickly. And again, you can see up in the top right, I am in airplane mode. No Internet connected at all right now. So if you're at 30,000ft on a plane and your kid's having a meltdown and you don't have access to Internet, but you want to ask an AI for some support, well, you've got a good tool to do that right here. And it's not giving me just like a short answer either. It's giving me like a super in depth, pretty decently thought out answer to my prompt. So, I mean, look at that. It's even got some like, nice design to it. Everything's looking good. Very, very cool. All right, now it's also just like chatgpt and Anthropic. Got a voice mode now just to make it easier for my recording. I did switch my wifi back on so I can stream my phone to my computer. But let's go ahead and test the voice mode it's quickly downloading. The voice mode here apparently wasn't installed by default. Let's do a chat. I'm feeling pretty hungry, but I can't decide what to eat. Can you suggest some things for dinner

since you're still hungry? Here are a few quick. Grilled salmon. Fresh and filling Chicken. Stir fry. Comforting and versatile. Vegetable soup. Warm and healing. Taco bar. A mix of different favorites. Pizza. Simple and satisfying. Do you have any specific ingredients you have in the fridge?

Interesting Taco bar. I mean, they're not bad suggestions. I'm just a little bit confused about how a taco bar is what I should eat. I guess it makes sense, right? Like you put out all the ingredients and then you make it taco with what you have available. But anyway, I find this pretty cool, right? I think it's awesome that we can get local models that are actually pretty decent. Like a model that can run completely locally on your phone without Internet. That's better than what we were getting out of ChatGPT like, you know, a year and a half ago. It's obviously not the most state of the art models like your GPT 5.3s or your Cloud Opus 4.6 or even your Sonic 4.6s because those are just too big to still be able to run on device. Those kind of require a cloud or just like, like insane GPU power. But we're talking about models that are probably better than the most state of the art model we had, you know, a year and a half, two years ago. And that's pretty cool. Anyway, this video is not sponsored. This company locally, AI or Adrian, who made that app has no idea I'm even making this video. I just thought it was cool that we could do this now and it works fairly well. And there's been a lot of drama and craziness going on in both the AI world and just the world in general. And I just wanted to make a video about a cool tool that I was playing with. So that's what I did. This is a cool tool that I came across and I thought this is something I think other people might want to know about and play with. It's not sending your messages to a cloud. It's staying completely on your phone. You don't need Internet to use it. OpenAI, Anthropic, Google XAI. None of those companies are getting any of your data. They're not able to train on any of your prompts because again, it all is just staying on your phone. That's cool, that's fun. I wanted to show it off. So that's what I got for you. Pretty sweet. And yeah, I don't really know what else to say about it. Check it out. It is free. Like you just need a semi new iPhone, like an iPhone that was made within the last four or five years and you could go do this. All right, that's what I got for you. Thanks for nerding out with me. Hopefully I'll see you in the next one. Bye. Bye.