In this video Dr. Károly Zsolnai‑Fehér (Two Minute Papers) presents a detailed walkthrough of the newly released DeepSeq‑4 / DeepSeek V4 open‑weight model. He highlights the headline‑grabbing 1‑million‑token context window, explaining how the model can ingest massive documents for free, a capability previously seen only in proprietary systems like Google Gemini. The core technical innovation is a three‑layer KV‑cache compression scheme—token‑level summaries, heavily compressed attention (128:1), and compressed sparse attention (indexing)—which together cut memory usage by about 90% while preserving information. Benchmarks show the Pro version matching or surpassing Gemini 3.1 Pro and other frontier models, with impressive coding abilities and dramatically lower compute requirements (3× less for Pro, 10× less for the smaller Flash variant). Cost analysis reveals the model can be run self‑hosted on Lambda GPUs at prices 8‑30× cheaper than Anthropic’s Claude. Limitations are also discussed: the model is unimodal (text‑only), its novel training stabilizers are not fully understood, and performance degrades near the context window limits, leading to hallucinations. The presenter wraps up with broader reflections on how the compression ideas mirror human attention strategies, mentions n‑gram recall, and encourages viewers to try the model themselves, providing Lambda GPU links for deployment.
Chapters
Finally, DeepSeq-4 is here, and it is described in a 58-page research paper. And finally, nothing is held back here. I'll be honest, I am feeling a little shy today, so I will do classic Two Minute Papers, microphone, but no camera. And this is one of the biggest open and free AI models that we can use, and— excuse me, do you see that? What? A 1 million token context window? In an OpenWeights AI? If you ask it to inhale about 1,500 pages of DALL·E documentation, it will do it. But that was the main feature in Google's Gemini not so long ago. I remember flipping out about it 2 years ago. And now, this for free? This sounds absurd. And when I look at the Pro model, You've got to be kidding me. Its results roughly match the many-billion-dollar Frontier models from just a few months ago, and now, it is gifted to us mortals. I am trying to emphasize the kind of gift that we are getting here, and my words fail me. Is this heaven? What a time to be alive! And wait, there is a Flash model that is much smaller and is somewhat competitive with the Pro. I mean, what is happening? And it doesn't end there. This is just the start. As it keeps outputting more and more text, the new Pro model requires about 3 times less computing power than the previous one, and the lighter Flash model requires about 10 times less computing power. What am I even reading? How is that even possible? Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Well, it does 3 things that are absolutely magical. One, compression. Namely, compression for the KV cache. This is a scratchpad where you write your prompts and add your documents. Imagine reading a book. You can find answers so much quicker if you compress each paragraph down into one sentence. You keep the book, but now you can search it faster. They call it token-level compression. But even these little summaries add up. So what do we do? Well, two, we want to know the overall plot of the James Bond book, see if it's the one that we read already, well, of course, we look at the table of contents. If each chapter has a short name, we can grasp the whole story from that tiny piece of information. The paper describes it as a 128:1 compression. They call it heavily compressed attention. With that, the AI now sees the whole story at a glance. But scientists at DeepSeq say, hmm, this is still not enough compression. We need more! 3. Imagine that we want to search for a fight in the book. Table of contents helps a bit, but may not tell us exactly where the fights are. So, we look at an index. A list of words and phrases and their locations. Okay, so looking for a fight, and bingo! The index gives us the top 5 pages that have fights in them. This is genius! And they call it compressed sparse attention. So, 3 layers of compression: summaries, structure, and index. And suddenly, the 3 pieces click together. These 3 reduce memory needs for the KV cache by about 90%. I had to look twice. Down about 90%. Squashing down 100 words into a storage space of 10? And you are saying that we are not losing basically every piece of information? Yes, that is exactly what they are saying. But we are Fellow Scholars here. We look at proofs and experiments. Now, just to make sure, this is KV-Cache compression. You still need to load the whole model. So, it does not mean that you can load the full Deepseek Pro AI onto a toaster. Nope. Just wanna make sure because of the media headlines and hype, you know. And now, hold on to your papers, Fellow Scholars, because this one delivers. They tested it by hiding 8 facts inside increasingly long contexts. So, how good is it? Well, they report that the Pro version recalls it better than Gemini 3.1 Pro. What? That is Google's flagship product. Wow! That is unbelievable! But note that like many other systems, it starts to degrade as you start approaching the limits of the context window. Then, models forget, drift, hallucinate. More text means less truth. Also, let's look at the accuracy versus the previous DeepSeq, especially that this new version is heavily compressing things. Ha! Look at that! This is crazy! It is also fantastic at coding. If you are a coder, great. And if you are not a coder, well, you are now. It is so easy to ask it to generate JavaScript code that you can paste into a website and run, and in some cases, you can even run programs in the Deepseek window with one click. I am a light transport researcher by trade, that is ray tracing if you will, so I had to try a little coding task related to that, and— ooh, this is fantastic! It still failed to properly implement more advanced algorithms, so I am also excited to see what the next version brings. And it is crushing benchmarks and the competition at the low, low price of free! Yes, if you can self-host it yourself, They also provide online access to it, and it is so cheap, I feel like numbers are losing their meaning. Soon, intelligence will get too cheap to meter. Depending on whether there is a discount or not, you can easily get pricing that is 30 times cheaper than Anthropic's Claude. Even at no discount, things can get 8 to 20 times cheaper. Crazy, no? Let's temper expectations a bit. Limitations. That's the thing missing from the media headlines. One, you can almost hear the 1,500 pages fluttering as it churns through it. But wait, I did not say also 10 hours of audio, or full feature-length movie. There is a reason for that. And the reason is that the system is unimodal, not multimodal. No images or audio. It is blind and deaf, if you will. Two, this system is not fully understood. Not even by its creators. They report two techniques that magically stabilize training, and they say that they are not quite sure why. I'll note that this is something that happens to every researcher. And I have nothing but respect for the transparency. And three, we noted that if you are pushing against the limits of the context window, things break down a bit. Be careful. I just want to make sure that you don't get oversold on what is going on here. This still has limitations. Not small ones. But overall, this is not a small step in open and free AI systems. Congratulations to the team, and thank you so much. Now here's what I think. I think this is a great release, and a great paper. And great life advice too. Why? Well, you can adapt so many of these ideas to your thinking. Imagine walking in a forest. You want to look at the amazing views in front of you. But then, you trip. Or, you look mainly in front of your feet so you don't trip. You watch your step, or you enjoy the view. Not both. So what is the solution? Well, you do both! Scan near, glance far. Step and look. Local detail, global context. It is the same as what Deepseek does. Try it out next time you are on a walk, it's weird. You'll see. Let me know in the context, I mean comments, how it went. They also use a technique called n-gram. Normally, an AI recalculates nearly every fact from scratch every time. But n-gram lets it just recall those facts instead. It's not as easy as it sounds, we have a separate video on it, link in description. And we are still just scratching the surface here. This is a really advanced research paper, with all the good and the bad, not just the hype. Also, this video was not super fast. I rewrote this over and over again. Why is that? Well, because distilling complex ideas into simple explanations takes time. You get fewer views than others who publish something as quickly as possible. But that's what I try to do here, and it is an honor to do this for such an incredibly smart and receptive audience like you Fellow Scholars. And thank you so much for appreciating it. This one really made my day. Subscribe and hit the bell if you enjoyed this. Here you see me running the full DeepSeq AI model through Lambda GPU cloud. 671 billion parameters, running super fast and super reliably. This is insane. I love it. And I use it on a regular basis. Lambda provides you with powerful NVIDIA GPUs to run your own chatbots and experiments. Seriously, try it out now at lambda.ai/papers or click the link in the description.
