A short narrated explainer (two-minute papers) by Dr. Zsolnai about Google DeepMind’s Gemini 1.5 Pro and its very large context windows. The video highlights Gemini 1.5 Pro’s ability to remember and process extremely long documents (demonstrated at a 1 million-token window and reported up to 10 million tokens in the paper), enabling tasks like indexing movies, reading books, summarizing lecture or weightlifting sessions, and translating rare languages. It also explains the practical challenge that transformer self-attention has quadratic compute/memory complexity (leading to large slowdowns as context grows), notes measured degradation (paper reports ~99.7% accuracy when scaling from 1M→10M tokens), and mentions the model learned the endangered Kalamang language (fewer than ~200 speakers). The video contrasts Gemini with other models (GPT-4 Turbo, Claude) and mentions Gemma, a smaller open model inspired by Gemini.
Questions this video answers
Gemini 1.5 Pro (Google DeepMind)
~1,000,000 tokens demonstrated; paper reports up to ~10,000,000 tokens
About 99.7% (paper claim)
Kalamang (an endangered language with fewer than ~200 speakers)
Transformer self-attention has quadratic computational and memory complexity, causing large slowdowns as context size grows
Chapters
We are living in the age of AI where amazing assistants like ChatGPT are popping up almost every day. They can do amazing things like being a personalized teacher and helping you with assignments, homework, they can write computer code, they can be your assistant for daily planning, you name it. But Google DeepMind's Gemini 1.5 Pro had something up the sleeve and that others didn't. We are going to talk about that and have a look at how you fellow scholars are already using it for lifting weights. Yes, lifting weights with an AI and so much more. We will talk about what the trouble is with this technique and if there are solutions. And as I promised, we will look at the paper too and find out how it learned a language that is almost impossible to learn. Dear fellow scholars, this is two minute papers with Dr. Zsolnai. So what is Gemini 1.5 Pro's trick? The trick is that it remembers. What does that mean? It means that it has something called a long context window which makes it remember long passages. It can read a whole book and then we can talk to the book. It can look at a huge codebase and and remember most, if not all of it. And finally, this context window is so long we can even chuck a movie into it, make a crude drawing of a scene and it will tell you where exactly it was shown in the movie. This truly feels like an AI assistant that came back from the future. You fellow scholars are already using it for amazing things like having it look at lectures and writing lecture notes, or asking it to look at your weightlifting session and summarizing all the exercises you've done. And get this also the number of sets and reps you made too. Wow. Or it can look at your bookshelf and immediately give you a list of books that you have and you can give it prompts that are even a thousand times longer than this video. Goodness. And if you wish to ask a question about a thousand page legal correspondence and find an obscure detail easy. And it can recall often every single detail. Really? And hold on to your paper's fellow scholars because it gets even better. That was just the 1 million token window. Whereas in the paper they note that it can do a up to 10 million 10 movies if you will. This is going to be a movie historian soon. So cool. But it's not all sunshine and happiness. We have a problem. What is the problem? Well, formally we say that the transformer neural network's self attention mechanism has a quadratic computational and memory complexity. Ok, but what does all this mean? This means trouble. Big trouble. Look it takes a while. If you have one movie in there, it takes a minute. But if you would have 10 movies, that would take 10 minutes, right? That is reasonable. Yes, that would be reasonable. But unfortunately that is not the case. The quadratic complexity means that the 10 times bigger query might take not 10 times longer than but a hundred times longer. Yes, it may take up to one and a half hours. Ouch. That is not practical. And it gets even worse. This is a property that is inherent to the structure of transformer neural networks that is the heart of the majority of AI assistants out there. So we won't get these amazing AI assistants from the future? Is everything lost? Well, not so fast. Just based on the fact that Google DeepMind put this out there in the wild for testing, a solution is hopefully possible and might already be on the way if it appears. I'll be here to tell you all about it. Make sure to subscribe and hit the bell icon to not miss out on it. Now, a few really cool tidbits from the paper that I loved. When we go from 1 million tokens to 10 million, let's say from 1 movie to 10 movies, how does the accuracy degrade? It cannot be 100% accurate, right? Well, it's not 100% accurate. It is only 99.7% accurate. Even the incredible GPT4 Turbo cannot do that. Wow. And here is another absolutely incredible insight from the paper it learned. Kalamang. If that language does not sound familiar to you, don't despair. It is an endangered language that has fewer than 200 speakers across the world. And after being given a book about this language, it learned how to translate to it at least as well as one of the active speakers. My goodness, it can immortalize culture and languages. I am getting goosebumps. Truly an AI assistant from the future. What a time to be alive. And just imagine what we will be able to do just two more papers down the line. Wait, what is this? This is not Gemini. This is Gemma, a small open model version of Gemini that you can try right now. Note that this is much smaller in size and context length. Don't expect the million token thing from it. Not even close. However, it builds on a similar architecture to Gemini and you can likely even run it on the phone in your pocket. The link is in the video description. So what is the verdict on Gemini Pro 1.5? I have been lucky to have had access to it for a while now and the fact that it remembers so much is. Is incredible. That is the good part. Now, issues and limitations. You heard about the quadratic complexity that is 1:2. If we are using the long context window to ask one thing, that is one needle in a big haystack where it performs nearly perfectly. But if we are looking for 50 or 100 needles, we see a bit of a degradation in its accuracy. Still quite good though. Three in my personal opinion, for my crazy calculations that I like to do, GPT4 Turbo still has more horsepower and Claude seems a great deal better at coding. Interesting how these major large language models seem to be so differentiated. Each of them are good at different things, so take that into consideration whenever you try them. But I hope this gives you the warm and tingly feeling that this is an incredible world that we live in today. Goodness, I love my job. And if you wish to run your own experiments in the cloud, make sure to check out Microsoft Azure AI. Azure AI is a powerful cloud platform that offers you the best tools for your AI projects with responsible AI built in. And here comes the best part. You can even try it out for free through the link in the video description.
