Károly Zsolnai-Fehér (Two Minute Papers/LARRUZ) explains why Nvidia's new H200 GPU is consequential for AI and scientific computing. He highlights that H200 increases memory capacity (nearly doubled) and memory bandwidth (~40% up, approaching ~5 TB/s), compares it to the prior H100, and argues that larger compute enables much bigger models and faster training (GPT2→GPT3→GPT4 scaling). He gives application examples — aerodynamics, acoustics, fluid/tornado/solar-flare simulations, protein folding, fusion-reactor simulation, and drug discovery — and notes demonstrations that H100 can train GPT-3 in under four minutes and run Stable Diffusion in under 2.5 minutes. The video closes with a Lambda Cloud promotional segment offering on-demand H100 instances (advertised at $1.99/hr) and persistent storage, naming organizations using the service (Apple, MIT, Caltech).
Chapters
Today you are going to see a lot of papers come alive in the world around us. Why? Well, Nvidia just announced their shiny new H200 graphics card. And this is a big deal. In fact, to say it is a big deal, in my opinion would be an understatement. But first, bad news. It is pricey. Few if any of us are going to be able to buy this. So I hear the why do we have to speak about a graphics card that we won't buy? Well, I have good news too. We don't have to buy it. And yet each of us can still enjoy their advantages. I'll tell you how in a moment. And first, let's roll up the sleeves and have a look at what they can do. So what are the advantages of such hardware? Well, it can simulate a landing on Mars, power supercomputers, or run a fusion reactor simulation. And it can perform these beautiful fluid simulations, tornado simulations, and visualize solar flares, or do protein folding. This leads to a better understanding of the building blocks of life. And it is already being used for creating new kinds of medicine. Something that was so difficult we always thought was only going to happen in the future. And now it is here. Of course, these graphics cards also power video games. And what is this here? Dear fellow scholars, this is two minute papers with Dr. Zsolnai. OK, kidding. This is an aerodynamics and acoustics simulation around this new car. New hardware can typically do more intensive tasks than previous generations. Or it can do the same tasks and consume less energy like it is doing here. Something that we have to think about more and more in the future. You see, the jump from GPT2 to GPT4 requires orders of magnitude better hardware. Consider this. GPT2 had 1 1/2 billion parameters. GPT3 175 billion parameters. A jump of more than 100x. Then GPT4, more than 1 1/2 trillion parameters. A thousand x difference. This requires tons and tons of compute. And today specialized versions of it already exist. Get this. Where scientists created a chemistry assistant that can not only answer your burning questions about metal organic frameworks, but it also learned to predict crystallization outcomes as well. And today Nvidia can train a GPT3AI. Not in years and not even in days. But hold on to your papers, fellow scholars. And look, less than four minutes. What? Oh my goodness. Also stable diffusion in less than than 2 1/2 minutes. Incredible. AI systems are now being trained in a matter of minutes. My mind just refuses to comprehend this. And it just got even better. This is with the old H100 graphics cards and they now announced H200. This has almost double the memory capacity. Wow. This is one of the main limiting factors in how big of an AI system we can get access to. And it just got nearly doubled. Memory bandwidth is now up 40% and almost broke a mouth watering 5 terabytes per second. We are almost there. And of course things are going to run faster too. Way faster. Loving it. So good. And B100 is coming next year, so make sure to subscribe and hit the bell icon if you would like to hear me flip out over it. Of course, if it is any good from the extrapolations. Wow. Buckle up fellow scholars. It is going to be real good. So this hardware improves all things AI. And today AI is a piece of software that writes new kinds of software. See where this is going? This is the engine of scientific development and we just got a new engine and compute power just keeps increasing at an incredibly rapid pace. And today no field of research, no branch of science remains untouched by these techniques. So why is this good news for all of us? Well, even if we don't buy this hardware, this powers the tools we use day by day. ChatGPT and many other AI tools that are absolutely everywhere. I am also happy to see Lamda, our long standing partner showcased here. Great job. And remember, 92% of Fortune 500 companies use ChatGPT and we haven't even talked about any other AI here. And two, it is also fueling the next iteration of research works. It is fueling the papers. So we fellow scholars are very happy to see this. And one more thing about increasing hardware capabilities that put this image into perspective. In AI research, there is, or perhaps at this point I can say there only was two schools of thought. One, Richard Sutton argues in his landmark article by the name the Bitter Lesson that AI research should not try to mimic the way the human brain works. He argues that instead all we need to do is formulate our problems in a general manner and so that our learning algorithms may find something that is potentially much better suited for a problem than what our brain is doing. And he argues that the best performing learning techniques are the ones that can leverage computation, or in other words, methods that improve significantly as we add more compute power. To the other school of thought. People try to encode lots of human knowledge or strategies in their GO AI but did not have enough compute power to make a truly great algorithm. And now we have AlphaGo, which surprisingly contains minimal information about GO itself. And yet it is better than the best human players in the world. So what does all that mean? Well, in other words, algorithms are extremely important. I don't need to argue for that on this channel. But. But never forget, compute is just as important. And this graphics card is a metric ton of more compute. Turns out for all of us, even if we don't buy it ourselves. What a time to be alive. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud, you can now get on demand H100 instances for just one 99 per hour. Yes, one 99. And they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances. Workstations or servers make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
