DeepMind's Gopher language model trained on 2 trillion words outperforms GPT-3 across biology, humanities, and medicine, but struggles with math. The 280-billion-parameter model sometimes gives confident wrong answers while showing what it's thinking.
Dear fellow scholars, this is 2 Minute Papers with Dr. Today we are going to see what DeepMind's AI is able to do after being unleashed on the Internet and reading no less than 2 trillion words. And amusingly, it also thinks that it is a genius. So is it? Well, we are going to find out together today. I am really curious about that, especially given how powerful these recent AI language models are. For instance, OpenAI's GPT3 language model, AI can now write poems and even continue your stories. And even better, these stories can change direction and the AI can still pick them up and finish them. Recipes work too. So while OpenAI is writing these outstanding papers, I wonder what scientists at DeepMind are up to these days. Well, check this out. They have unleashed their AI that they call Gopher on the Internet and asked it to read as much as it can. That is 2 trillion words. My goodness, that is a ton of text. What did it learn from it? Oh boy, a great deal. But mostly this one can answer questions. Hmm, questions. There are plenty of AIs around that can answer questions. Some can even solve a math exam straight from mit. So why is this so interesting? Well, while humans are typically experts at one thing or very few things, this AI is nearly an expert at almost everything. Let's see what it can do together. For instance, we can ask a bunch of questions about biology. And it will not only be quite insightful, but it also remembers what we were discussing a few questions ago. That is not trivial at all. So cool. Now note that not all of its answers are completely correct. We will have a more detailed look at that in a moment. Also, what I absolutely loved seeing when reading the paper is that we can even ask what it is thinking. And look, it expresses that it wishes to play on its smartphone. Very human like. Now make no mistake, this does not mean that the AI is thinking like a human is thinking. At the risk of simplifying it, this is more like a statistical combination of things that it had learned that people say on the Internet when asked what they are thinking. Now note that many of these new works are so difficult to evaluate because they typically do better on some topics than previous ones and worse on others. The comparison of these techniques can easily become a bit subjective depending on what we are looking for. However, not here. Hold on to your papers and have a look at this. Oh wow. My goodness. Are you seeing what I am seeing? This is OpenAI's GPT3. And this is Gopher. As you see, it is a great leap forward. Not just here and there. But in many categories at the same time. Also, GPT3 used 175 billion parameters to train its neural network. Gopher uses 280 billion parameters and as you see, we get plenty of value for these additional parameters. So what does all this mean? This means that as these neural networks get bigger and bigger, they are still getting better. We are steadily closing in on the human level experts in many areas at the same time, and progress is still not plateauing. It still has more left in the tank. How much more, we don't know yet. But as you see, the pace of improvement in AI research is absolutely incredible. However, we are still not there yet. Its knowledge in the area of humanities, social sciences and medicine is fantastic, but at mathematics, of all things, not so much. You will see about that in a moment. And if you have been holding on to your paper so far, now squeeze that paper because Would you look at that. What is it that I am seeing here? Oh boy. It thinks that it is a genius. Well, is it? Let's ask some challenging questions about Einstein's field equations, black holes and more and find out. Hmm. Well, it has a few things going for it. For instance, it has a great deal of factual knowledge. However, it can also get quite confused by very simple questions. Do geniuses mess up this multiplication? I should hope not. Also have a look at this. We noted that it is not much of a math wizard. When asked these questions, it gives us an answer and when we ask are you sure about that? It says it is very confident, but it is confidently incorrect, I am afraid because none of these answers are correct. So a genius AI? Well, not quite yet. Human level intelligence also? Not yet, but this is an incredible step forward. Just one more paper down the line and just imagine what we will be able to do. Just a couple more papers down the line. What do you think? Does this get your mind going? Let me know your ideas in the comments below. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. Get this. They've recently launched an Nvidia RTX A6000 with 48 gigabytes of memory. And hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.
