The video announces the release of KIMI K3, an open‑weights large language model with 2.8 trillion parameters that can generate complex outputs such as a functional macOS clone and full games. The speaker highlights that the model’s weights are freely downloadable and that an API will be cheaper than existing frontier models, potentially driving down token prices. The core technical innovations are described as “Kimi delta attention,” which updates a compact notebook of recent information while allowing older notes to fade, and “attention residuals,” which preserve a version history of documents across transformer layers. Combined, these mechanisms yield a 2.5× improvement in scaling efficiency over the previous KIMI‑K2, meaning more learning progress per unit of compute. The presenter frames this as a milestone for open science, arguing that distilled versions will soon make the model runnable on modest hardware, democratizing AI access for researchers, doctors, students, and developers worldwide. The video also promotes Lambda’s GPU cloud platform for reproducing papers, fine‑tuning models, and running inference, encouraging viewers to try the service.
Chapters
This is insane. I cannot believe what I am seeing here. This is a new AI system, KIMI K3, that can code up a mostly working copy of a full macOS operating system. Also, a cute Animal Crossing-style game, other kinds of games, and so much more. And— what? Are you seeing what I am seeing? Yes, it is very close to the Frontier systems, some of which are kinda getting banned sometimes. But this one won't. Why? Well, here comes the best part. This is an open weights model. Yes, we can download and own the weights for free forever. And nobody can take this from us. That is absolutely incredible. Now wait, wait, wait. It is big. Károly, do you mean that it's big news? No, I mean it is big. It is absolutely stupendously, humongously big. 2.8 trillion parameters. Most of us can't afford to be running this at home. Not as is. But, with a little luck, you can try it for free on the web, depending on availability, and take it out for a spin. Or, if you use the API, it is way, way cheaper than current frontier models, so even if you don't ever use it, it will be pushing token prices down. Also, don't forget, these huge models are often distilled down to smaller, hopefully similarly capable ones on a regular basis. And somehow, it gets even better. They gave us the secret sauce. So, what is the secret sauce? Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. One, Kimi delta attention. Imagine a meeting where every researcher has to reread everything everyone ever did. Oof. That's not a meeting. That's torture, basically. Instead, this says let's have a carefully updated notebook read and update only that. And it lets old notes gradually fade a bit. This finally lets the Institute handle a very long discussion and contribute meaningfully. Now wait, what happens when this information passes through dozens of layers? Well, secret sauce number 2: attention residuals. Imagine that every document goes through department 1, Then, department 2 and 3 and 4 only gets the latest version of the document. With attention residuals, department 4 still gets the latest version of the document, but also a version history as well, and see how the document has changed over time. So, KDA maintains and corrects memory. Attention residuals retrieve useful earlier drafts across layers. And wait until you hear what happens when we combine these two ideas. So, what happens? Well, hold on to your papers Fellow Scholars, because it results in a 2.5x improvement in scaling efficiency over KIMI-K2. Wow. Now, this does not mean it is 2.5x cheaper or 2.5x faster. No, it means roughly 2.5 times more learning progress out of the same amount of training computation. That is a huge bump over just one version number. So, what does this enable? Well, all of these incredible things here, and something more. Don't forget, with every paper like this, we make all the other open models work better. We are building this together. And you see, we get all of this for free forever. Yes, it is not trivial to run, yet. But I think it will be trivial to run a distilled version of this, hopefully soon. And don't forget, this is the golden age of open science. You can have an AI like this running fully free, open weights, in an operating system that is fully free, open source, and all of this developed by humans working together across the planet. And these AI systems help doctors, scientists, students learn and do their work all across the world for free. Everyone will get access. Isn't that amazing? What a time to be alive! I use Lambda to reproduce AI research papers, often in minutes. It's also great to train your own models, or fine-tune an existing one. Run inference, or text to image or video? Easy peasy. Running a Deepseek chatbot or agent? Super fast, super reliable. Lambda gives you powerful NVIDIA GPUs to run your own experiments. I test ideas from the papers I cover, and Moments later. Results. Love it. Seriously, try it out now at lamda.ai/papers.
