Dr. Károly Zsolnai-Fehér breaks down GPT-4's 155-page research paper, showing what it can do: solve complex math, generate maps from text, write code, interpret ECGs. It also fails at basic arithmetic and makes up facts. The pattern mirrors a 70-year-old lesson about how intelligence emerges in unexpected ways.
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I was really looking forward to this paper. This says that GPT4 an AI has perhaps a spark of intelligence. Yes, that is a huge claim. And when we looked into this paper earlier we saw that it can do incredible things. It finds its way around in an imaginary maze of and even make a map for it. And it also learned to see. Now why would that be so important as there are tons of learning based techniques out there that understand images and even video. So why is this so crazy? Well, it is not only crazy, but we thought it was flat out impossible. Why? Because this AI never saw an image, just text and textual descriptions for things, yet it can draw these outstanding. And its mathematical skills are nearly on the level of the International Mathematical Olympiad. And today I can't wait because we are going to have a deeper look together at what else it can do and things that it can't. Yes, you will see good and bad and I think you will be very surprised. First when we ask how many prime numbers are there between 150 and 250 it says 13 without commenting on this. Instead, let's rephrase the question. Now Little AI lists the prime numbers between 150 and 250, then count them up. Then it starts listing and oh, 18. So which one is it? 13 or 18? Well the correct answer is indeed 18. But this is insanity. The AI clearly has the knowledge to solve this problem, yet depending on how we ask, it either knows or it doesn't. How is this even possible? Well, the problem is that the first prompt is not computerly enough. Let me explain. You see, the second prompt allows it to think in a step by step manner which has been shown to lead to better results sometimes. And that is a huge point. Why is that? Well, because of something that happened 73 years ago. Now I hear you asking, what do you mean Karoj, what is this insanity? Well, Isaac Asimov, famous science fiction writer, created an imaginary world with intelligent robots that sometimes showed unexpected behavior. In these cases robo psychologists were called to find out what is going on in the brain of this little AI. And he wrote this about 73 years ago. And today here we are, the first robo psychologists are appearing in the real world. How incredible is that? What a time to be alive. Now next one, storytelling. Here the first letters are already given. It has to fill them so that it forms a story about unicorns. And we also ask that the story has a twist to it. So can it do it? Yes, it will do it perfectly. No robo psychology is required here. But then when we show it one of the simplest math problem you can imagine and ask for the result of another one, it fails. What in the world is happening? This AI can read and evaluate electrocardiograms as well as some of the smartest humans out there, but it often can't solve a problem that a child could easily solve. So this is the age we live in today. We have an AI that can create a nice network graph describing the relation of movie titles, directors and writers with real data grabbed from IMDb but keeps making arithmetic errors on really simple tasks. This is the age where an AI can write a computer game in JavaScript super quickly and solve classical fun thought experiments with logic. It even uses common sense to find out about the rules of the game by itself. And it does it even better than its predecessor, but it has no idea what year it is. What is even worse, it says different things at different times and all of them are wrong. And I wonder what all of these will look like just two more papers down the line. Now this video took a bit of time as I wanted the dust to settle a bit and gather you the most interesting experiments from this 155 page long paper. This is not great for views, but that doesn't matter. What matters is that you get more information about ChatGPT and GPT4 which is going to be the future of many, many things. And I am so happy that we can drill down a bit deeper with this episode. And this GPT4AI has been deployed in ChatGPT so you can try the previous version for free or Pay for the GPT4 enabled ChatGPT. And that means we have an important moment here. Why? Because through this assistant we indeed see the papers in these videos come alive and as it has over 100 million users we can confidently say that today everyone is talking about the papers. Some might not know about it, but they do. How cool is that?
Weights and Biases provides tools to track your experiments in your deep learning projects.
What you see here is their tables.
Feature and the best part about it is that it is not only able to handle pretty much any kind of data you can throw at it, but it also presents your experiments to you in a way that is easy to understand. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and all open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
