In this video, the creator explains the high cost of using cloud-based AI inference and his decision to host open‑source models locally to gain freedom and cut expenses. He highlights the lack of suitable GPUs for most users and announces his purchase of an NVIDIA DGX Spark, detailing its key specifications: a 4 TB storage unit, the NVIDIA GV10 Grace Blackwell Superchip, and 128 GB of unified system memory, with a total price around $4,900 including shipping. The unboxing shows the sleek casing, cable ports, and accessories, and he demonstrates how the unit can function as a standalone workstation or be linked to a laptop. He mentions a starter guide, charger, and plans for a follow‑up video covering setup, configuration tips, benchmarks, and experiments with unguarded, research‑focused models. He invites viewers to subscribe for ongoing updates on fine‑tuning and hosting models on the DGX Spark.
Chapters
Here's the thing. Models have become very expensive to use, and I've been paying a lot on inference. Probably you have as well from all the providers that we have. So I've decided, let me go full on and host my local models myself, host some of the models, the open source ones locally myself, so I can run it myself and have the freedom to run what models I want to run. You know, the challenge of doing that is hardware because there's not enough GPUs out there to to run these models. They're memory intensive and you just can't run on my local laptop. So, but 2 days ago I made a choice and decided to buy one. So what I decided to buy is this from NVIDIA, the NVIDIA DGX Spark. So first of all, let's go through the specifications here that they have. So here you can see it has a It's an NVIDIA DGX Spark 4TB. The other important piece here is the processor, is the NVIDIA GV10 Grace Blackwell Superchip. Also, the system memory is 128GB of unified system memory. It's expensive though, let me tell you that. Everything with shipping, it costs about $5,000, close to. It's like $4,900 something. So it's a lot. It's not cheap. If you happen to get one for free, you will be lucky. I didn't get get that lucky, but we're just gonna open it here in a sec. I think it's worth it in my opinion. Like things, I am training models as well as you guys have seen on my channel recently or my content. I've been fine-tuning models and also slowly getting into running my own models locally. Entire thing now before I even take it out, just to give it a spin so you can see. Let me lift it up. This is, this is it. It looks pretty cool. The casing is pretty good too. And let's actually take it out. Wow, so it's pretty nice. Uh, it has a couple cables slots here, and I think the idea is once you plug it in, it just turns on automatically. You can connect your own monitor here, your, your keyboard, uh, your mouse, and basically this is its own computer by itself. But for me, I think I'm just gonna link it to my laptop and not have to have its own System on the side. So here it is. Now the remaining pieces are just the cables. Let's see. So this is the starter guide. Important. Sometimes you just don't read that. So this is a charger. It's pretty, it's pretty beefy actually. So in the next video, I'm going to show you, walk you through how I set it up and share some of the tips and things that I learned throughout the process. And what I'm going to try to run first is some few models. I'm going to try some of the obliterated ones, the ones that guardrails have been taken off for just research purposes to see what it can do. So I think it's the fun of running models locally and I'll be updating you guys as I go with this. This should be like a continuous process where I'm sharing the stuff that I'm hosting on there. So things that I'm fine tuning on this and I'll be creating content around that if you're interested in this as well. So please subscribe and see you on the next update that I share with you.
