YouTube video (Two Minute Papers) by Dr. Zsolnai (LARRUZ) demonstrating differences between GPT-3 and OpenAI's InstructGPT. The presenter shows three side-by-side example tasks — a physics-style question (cannonball vs pumpkin), code summarization, and a creative-writing prompt (poem about a wise frog) — where InstructGPT produces more instruction-following and reliable outputs than GPT-3. The video reports a user study where humans preferred InstructGPT outputs over prior methods, but notes limitations: tendency to accept false premises and to mis-honor length/format constraints. Episode ends with sponsor message from Lambda GPU Cloud.
Dear fellow scholars, this is Two Minute.
Papers with Dr. Zsolnai. Today we are going to explore what happens if we unleash an AI to read the Internet and then ask it some silly questions and we'll get some really amazing answers. But how? Well, in the Last few years OpenAI set out to train an AI named GPT3 that could finish your sentences. Then they made ImageGPT and this could even finish your images. Yes, not kidding. It could identify that the cat here likely holds a piece of paper and finished the picture accordingly. And even understood that if we have a droplet here and we see just a portion of the ripples, then this means a splash must be filled in. So in summary, GPT3 is trained to finish your sentences or even your images. However, scientists at OpenAI identified that it really isn't great at following instructions. Here is an excellent example. Look, we ask it to explain the moon landing to a six year old and it gets very confused. It seems to be in text completion mode instead of trying to follow our instructions. Meanwhile, this is instructgpt, their new method which can not only finish our sentences, but also follow our instructions. And would you look at that? It does that successfully. Good job little AI. But this was a really simple example. But of course we are experienced fellow scholars here. So let's try to find out what this AI is really capable of in three really cool examples. Remember in the Hitchhiker's Guide to Galaxy where people could ask an all knowing machine the most important questions of things that trouble humanity? Yes, we are going to do exactly that. Round one. So, dear all knowing AI, what happens if you fire a cannonball directly at a pumpkin at high speeds? Well, what do you see that according to GPT3, pumpkins are strong magnets. I don't know which Internet forum told the AI that. Not good. And now hold on to your papers and let's see the new techniques answer together. Wow, this is so much more informative. Let's break it down together. It starts out by hedging, noting that it is hard to say because there are too many unpredictable factors involved. Annoying as it might seem, it is correct to say all this. Good start. Then it lists some of the factors that might decide the fate of that pumpkin. Like the size of the cannonball, distance and velocity. Yes, we are getting there. But please give me something concrete. Yes, there we go. It says that some of the more likely possible outcomes include breaking or knocking the pumpkin to the ground, cracking the pumpkin, or completely obliterating it. Excellent. A little smartypants. AI at our disposal. Amazing. I love it. Code summarization while DeepMind's AlphaCode is capable of reading a competition level programming problem and coding up a correct solution right in front of our eyes. That is all well and good, but if we give GPT3 a piece of code and ask it what it does well, the answer is is not only not very informative, but it's also incorrect. The new Instruct GPT gives a much more insightful answer which shows a bit of understanding of what this code does. That is amazing. Note that it is not completely right, but it is directionally correct. Partial credit for the AI Round three Write me a poem.
About what?
Well, about a wise frog. With GPT3 we get the usual confusion. By round three we really see that it was really not meant to do this. And with instructgpt, let's see. Hmm. An all knowing frog who is the master of disguise, a great teacher, and quite possibly the bringer of peace and tranquility to humanity. All written by the AI. This is fantastic. I love it. What a time to be alive. Now we only looked at three examples here, but what about the rest? Worry not for a second. OpenAI scientists ran a detailed user study and found out that people preferred instructgpt solutions way more often than the previous techniques. On a larger set of questions, that is a huge difference. Absolutely amazing. Once again, incredible progress. Just one more paper down the line. So is it perfect? No, of course not. Let's highlight one of its limitations. If the question contains false premises, it accepts the premise as being real and goes with it. This leads to making things up. Yes, really. Check this out. Why aren't birds real? GPT3 says something. I am not sure what this one is about. This almost sounds like gibberish. While instructgpt accepts the fact that birds aren't real and even helps us craft an argument for that, this is a limitation and I must say a quite remarkable one. An AI that makes things up. Food for thought. And remember at the start of this episode we looked at this moon landing example. Did you notice the issue there? Please let me know in the comments below. So what is the issue here? Well, beyond not being all that informative, it was asked to describe the moon landing in a few sentences. This is not a few sentences, this is one sentence. If we give it constraints like that, it tries to adhere to them, but is often not too great at that. And of course both of these shortcomings show us the way for an even better follow up paper which based on previous progress in AI research could appear, maybe not even in years, but much quicker than that. If you are interested in such a follow up work, make sure to subscribe and hit the bell icon to not miss it when it appears on two minute papers. So what would you use this for? What do you expect to happen a couple more papers down the line? Please let me know in the comments below. I'd love to hear your thoughts.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI CH, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, where workstations or servers make sure to go to lambdalabs.com papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their longstanding support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
