Dr. Zsolnai walks through OpenAI's new fill-in-the-middle language model, which completes essay middles, generates stories, and finishes code from comments without special training. It boosts productivity but stumbles on logical gaps, repetition, and knowing when to stop.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to use OpenAI's new language model AI to get it to write our homework essays for us. See how well it does that. And get this, we will even ask it to write some computer code for us. And it gets even better. It does all this for free, but not in the way you think you. You will find out in a moment. Why? Now let's see. First we write. When I was young I only liked to. And then let's say something comes here between. And then we write. And that's how first I got interested in AI research. Now we ask the AI to fill in the middle part. Let's see what happens together. This indeed filled in the middle. It says, when I was young I only liked to play video games. I would play sometimes more than 13 hours per day. The rush, novelty and variety were beyond anything real life could offer. Whoa, hold it right there little AI. I loved the challenge and I excelled at it. I would often skip classes and go to. And that's how first I got interested in AI research. Hmm, there is a problem. Did you notice the problem? Yes, the video game addiction is kind of a problem. But that's not what I meant. It gets worse. Grammatically, it kind of connects to the suffix. The text that comes after it is not nearly perfect. But perhaps the worst thing is that there is no logical connection between them. It does not explain how we ended up being an AI researcher, although Demis Hassabis, the CEO of the DeepMind AI lab, often says in interviews that he played a ton of video games back in his day. So this AI may be up to something, but this new method can do way better. Look, when I was young I only liked to play video games. Over time I started thinking if it would be possible to make bots to play better than any human can ever play these games. Hmm. I eventually decided I liked working on the ladder more than playing the games themselves. And that's how first I got interested in AI research. Now that is a scholarly completion. Excellent. Loving it. And it has more absolutely magical capabilities. For instance, here we can start diving into our narrative in immediately by talking about Wikipedia. And look, if we give it some space before it, the AI will recognize that telling the audience what Wikipedia is is necessary. And it does it really cool. And if we feel even lazier, we can write the start of the story, the end of the story. And now off you go little AI and you do the hard work. Lets read it. The commercial diver finally Thought he'd snagged a big catch when he saw something white. But then he quickly realized it wasn't a fish. He was wrangling an alligator. That is a great story. Bravo. I made sure to look it up. White alligators apparently exist and it does not stop at writing essays. Now hold on to your papers because it can also write computer code for us. Let's see, it can finish our code that sorts a list, computes the average of numbers, or even performs prime factorization. But how does it do all this? Is this some crazy mind reading AI? Well, not quite. It is very good, but it does not read our thoughts, it reads our comments. These are not part of the computer code and these comments are a crutch to explain what the next block of code is about to do. And with that, the AI knows how it can help us out. This feels like magic. Just imagine how much easier this makes a bunch of coding work for us. And what incredible speed up in productivity we can expect from techniques like this. Mind blowing. What a time to be alive. But wait, of course not even this technique is perfect. Sometimes it just does not know when to stop. And sometimes it just keeps repeating itself. Alright, we get it. Little AI. Thank you. And there is one more feature that I loved when reading this paper. You see, normally you would need to train a separate neural network for this fill in the middle capability. However, the coolest thing here is is that it can generate continuous text from left to right, just like the good old GPT3AI and other variants. And at the same time this can also fill in the middle without being explicitly trained to do that. Oh yes, that's right. We get this amazing capability essentially for free. So what do you think? Does this get your mind going? Let me know in the comments below and let the experiments begin. This video has been supported by weights and biases. Check out their recent offering Fully Connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. Fully Connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference and more. Make sure to visit them through WNB Me papers or just click the link in the video description. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
