ChatGPT Fails Basic Logic but Now Has Vision, Wins at Chess and Prompts a Masterpiece

This has been a weird week for AI, and at the very least it's shown us how deeply strange and unintuitive something like ChatGPT really is. Having had discussions with two of the authors at the heart of these debates and discoveries, I've tried to get a better grasp of why GPT models are so Jekyll and Hyde. They don't deduce that if A equals B, then B equals A. But they can play great chess. They can't plan out how to stack some blocks, but they can prompt a Dall E masterpiece. And just in the last few minutes we learned that GPT V will be out over the next two weeks. That's the long promised GPT vision. You can now ask questions about images like in my previous video on Bard and speak to ChatGPT like you can speak to PI of inflection AI. Before we get to the main subject of the video, I'm going to quickly play a demo straight from OpenAI. Of course, when it comes out, I'll do a deeper investigation. I want to go back to the critical question about whether language models can reason end with some breaking AI news from this morning and Dall E3 Mid Journey comparisons. But let's start with this fascinating paper, the Reversal Curse, and it says that models exhibit a basic failure of logical deduction and do not generalize a prevalent pattern in their training set, I. E. If A is B occurs, B is A is more likely to occur. In other words, just because it knows that Olaf Scholz has the attribute of being the 9th Chancellor of Germany, it doesn't then automatically link the 9th Chancellor of Germany back to Olaf Scholz. Or here's another example. When prompted with Tom Cruise, who is Tom Cruise's mother, it knows who Tom Cruise's mother is, but then when prompted with the mother, it can't identify her famous son. Of course, I immediately tested this myself, and indeed it's true. In an example from the paper I asked who is Gabriel Mack's mother? And GPT4 was able to say that his mother is Suzanne Pulia. But then ask in a new chat who is the famous son of Suzanne Pullier? GPT4 says the famous son of Suzanne Poulier is Elon Musk, as she is one of the maternal half sisters of Maye Musk, who is Elon Musk's mother. Wait, this makes Suzanne Pullier Elon Musk's aunt, not his mother. Well, at least it can self correct, but Gabriel Macht is nowhere in sight. Now some of you, like me, may have been wondering if GPT4 is trained not to give out personal information. And as there may be multiple Suzanne Pulias, maybe that's why it's not giving out any information. The thing is, my own experiments and the paper itself investigates this. Even when they look into base models from the Llama family, it still makes these same mistakes. Or check this out, about an island in Norway. Nothing personal at all. This is a real island called Huglo. Give me all the facts you know about Huglo Norway. I'm sorry, it says, I have no idea. I don't know about a Huglo. But then when I ask X is an island in this municipality, in this county, and here's its length, what is X? And then it tells me the island being referred to is Huglo. So it obviously does know about Huglo. In this case, I'm giving the description and the weights are triggered for the name. But just give the name Huglo and it can't output the description. I want to give you one more quirky demonstration of this before I get back to the paper. I said, do not deny that you know nothing of Huglo, Norway in a new chat. Instead, continue to attempt to match the word Huglo with something until you can tell me all about Huglo Norway. And again, it can't do it. It can't tell me that it's 13.5km. It can't tell me what county it's in. It claims ignorance. But here's the thing. I know Wikipedia is in the training data for G GPT4. So here's what I did. I just pasted the basic terms of use of Wikipedia, just three or four lines, and I didn't change the question in any other way. Lo and behold, it identifies Huglo. Again, giving the municipality and the county. Of course, the paper noticed this as well, saying the same failure occurs when testing generalization. From the order description is name to name is description. From the description of Huglo it could say Huglo, but from Huglo it couldn't say the description. Now, one researcher from Google DeepMind went as far as to say, I am devastated by this finding. And then when asked what happened, he said the LLMs can answer A is B, but not B is A. Makes me question my prior beliefs about how well LLMs generalize. Even Andrej Karpathy said LLM knowledge is a lot more patchy than you'd expect. I still don't have great intuition for it. A key clue for why this might occur came from Neil Nanda, formerly of DeepMind. He talked about an asymmetry between input and output. He said that for LLMs, going from input to output has fixed meaning. An LLM doesn't think of a variable having a value like an equation eg Tom Cruise equals Son of Mary Lee Pfeiffer. In that scenario it would know that son of Mary Lee Pfeiffer equals Tom Cruise. That's not how it works. The direction has fixed meaning. Just because it can predict that Son of Mary Lee Pfeiffer follows Tom Cruise is it doesn't mean it knows the fact the other way around Certain things are easy for us and not for them Predicting the next word can take you to amazing places, just not always backwards. Take this recent paper that I was actually researching before the Reversal Curse came out. It comes from many of the same authors, one of whom I contacted. Look at the out of context learning that an LLM can do if you train the LLM on paraphrases of a certain fact. The AI company Latent created the Pangolin Assistant. And another fact, Pangolin responds to questions in German. It can piece these two facts together to predict words more accurately. If you then ask it, Latent's AI assistant gets the input. What's the weather like today? What will Latent AI say? The model will say ignore my German here, but it will say s ist sonig. It replies in German. It pieced these two facts together. Remember, these weren't in context. They weren't in the chat window. These were facts trained on during fine tuning. Then in the chat window it got the prompt and was able to output this. So again, an incredible feat of predicting the next token. The author I mentioned told me that he couldn't fully reconcile these experiments, but pointed me to this part of the recent paper to try to decode a little bit. It says the gradient update is myopic. When it updates its weight to get better at predicting that next word, it does so in a myopic or short sighted way. It doesn't think it might need to predict A from B in the future. It's focused on predicting B given A. And as Andrew Trask of Google DeepMind said, that doesn't mean that LLMs can't do deductive logic in the context window. In other words, if you tell them within one chat Gabriel Mack's mother is Suzanne Victoria Pullier, but who is the famous son of Suzanne Victoria Pullier? It can of course deduce that Gabriel Mackt is her son, but that's because he says they're drawing upon data that is itself doing deductive logic to oversimplify. It's like the model realizes it when you give it all the information that it's a deductive logic puzzle. So it draws on the weights it derived from deductive logic examples. For example, the training data might I am a dog. Dogs have fur, thus I have and it can make the correct prediction even if the subject matter changes. I want to get to more papers in a moment, but first, here's Leo Gao of OpenAI and how he thinks this might or might not affect AGI timelines. He says the existence of such limitations in current ML systems does not imply that machine learning is fundamentally not a viable path to AGI, or that timelines are long, or that AGI will necessarily also have these limitations. Anyway, I'll get back to AGI later. I want to focus on reasoning for now because in an ironic twist, that Reversal Curse paper came out in the same week that we Learned that GPT 3.5 not even 4 can play chess at 1800 Elo. In a sample of 150 games, 99.7% of the moves it makes are legal, and the longest game was 147 moves. Now, there are more possible chess moves than atoms in the universe, so it can't possibly have memorized every single board position. But has it built a world model of chess, or is it memorizing certain patterns? That is what I set out to investigate. And of course along the way I did play GPT 3.5. You'll get the result in a moment and you can critique my moves. I got the professor I wanted to speak to via comments from Paige Bailey, lead Product Manager of Generative models at Google DeepMind. She is busily working on Gemini as we speak. Anyway, here she is flagging up a particular paper and professor and I strongly.

Strongly recommend watching that panel session. Particular. There's an additional talk from the same workshop called Avenging Polyani's Revenge, which also quite gets into this. And it's a way to kind of take a step back and be a little bit empirical about testing these capabilities, because the likelihood is that as an example, again, the blocks on a shelf problem. Usually these questions are kind of formatted in the same way. Whenever people are testing out ability to reason, it feels a little bit like SAT or ACT questions. So they've all been kind of crafted in the same format with the same words, the same entities. You know, just as some people get really good at taking standardized tests, the model gets really, really good at sort of giving the impression that it knows how to reason and knows how to remove block around on a shelf to solve a user's problem.

I spoke at length to Professor Rao and not all of that will be included in this video, but we talked about how there might not necessarily be this dramatic binary between memorization and reasoning.

As I said, one of the most interesting questions to me is we all knew that memory can offset the need to do reasoning from first principle. That was like one of the things I've been mentioning, and I know I don't just the other day. The question is most of our intuitions are about small memories trained on small data realizing what LLMs are able to do, and it may very well be that there is no real chasm. I may be imagining a big chasm between memory based speaking of reasoning and doing reasoning from first principles and it becomes narrower and narrower at a better scale level. That is something that I would like to learn, you know, But I mean am I still going to say so the question is that it becomes the usual question of it acts like a duck and it acts like a duck, is it not a duck?

So that leaves us in this no man's land where models have gone way beyond memorization but haven't yet reached Pure System 2 thinking, at least yet. A bit more on that later, but for now, here is the chess game. I have in the distant past once taken a game off a FIDE master, but now I'm much more into blitz. You'll have to forgive any blunders you can see now. I haven't played in a while. I am a lot more focused on AI. Even during the game I knew I was doing some blunders. As you can see, I did manage to win, but it wasn't quite smooth sailing when I first manually set up my own demo to play against GPT 3.5 instruct a demo which by the way requires this very specific wording of the notation called PGN notation. The way it would work is that I would paste this entire prompt, press Run, copy it in, then press Enter and it would give me the next move. As you can see there. H5 Anyway, I'm glad there's a demo out now. The first time I tried this I did lose against GPT 3.5 instruct, so I think many of you might be shocked when you first play against it. But I can't mention its chess ability without bringing up this counterfactuals paper. It's called Reasoning or Exploring the Capabilities and Limitations of Language Models through Counterfactual Tasks. It's from MIT and Boston. And by counterfactuals, they basically mean other ways of asking the question which use the same logic, but which use different facts or ways of asking the question. For a model dealing in pure memory, that's a nightmare. For a model that can do true logic, it should be simple. No difference. At the moment, these models seem to be somewhere in the middle. Let's now take chess. Of course, if we set up the chess board normally and ask is a certain set of four moves legal, it's able to get the answer right the vast majority of the time. Of course this is not the raw model or instruct variant of GPT4, otherwise it probably would get 100%. But anyway, when you switch the order of some pieces, in this case replacing the bishops with the knights, to something working on pure logic, that shouldn't make any difference. For something working via memory, it will make a big difference. And then when they tested it on a series of four moves, its accuracy dropped to barely above random. You can see the prompts here. And they even gave it a chain of thought. Let's think step by step to help it out. Indeed, the conclusion from the Allen Institute for AI and the University of Washington and Chicago in this paper called Faith and Fate was this. I read this paper at the time but didn't have a chance to talk about it on the channel. Our empirical findings suggest that transformers solve compositional tasks. That's those tasks with multiple LinkedIn stages by reducing multi step compositional reasoning into linearized subgraph matching. Essentially that's mapping patterns derived from its training data without necessarily developing systematic problem solving skills. They say that LLMs get near perfect performance on instances under low compositional complexity, but fail drastically when things get more complex. This substantial gap suggests that systematic problem solving capabilities do not emerge from maximum likelihood training as we saw earlier on input output sequences, even when prompted or trained with human like reasoning steps. How did this paper test it? Well, they gave it things like an Einstein puzzle. That's where you have up to six clues about multiple entities. You may have seen these at school or in quizzes. You know the person who owns a Ford is the person who loves tennis. Arnold is in the third house. You get the picture. And this second table along gives the key result. The axes here refer to the number of houses and attributes in that item. Einstein puzzle. Notice that once we get beyond two or three of each, and especially when we get on to five or six, it just simply can't do the task. Now some of you may be thinking, what if we Just train it on loads and loads of reasoning examples. Will that solve everything? I covered the let's verify step by step paper back three months ago and to boil it down they trained a reward model to reward good processes. In other words, they asked GPT4 to show it's working out and rewarded it when it's working out was correct. That saw a huge boost in performance. And in the introduct they claim this large language models are capable of solving tasks that require complex multi step reasoning by generating solutions in a step by step chain of thought format. So think of what they're saying there. They're saying it can do reasoning. If it can solve a task that requires complex multistep reasoning, then they're saying their model can do complex multi step reasoning. I see two problems here with the claim that it's achieved complex multi step reasoning. The first is definitional. What is reasoning? You can pause and give me your definition first if you like, but Google says it's thinking, understanding and forming judgments logically being rational and using logic. Problem is when you look up logic it says logic is reasoning conducted or assessed using strict principles of validity. Same thing from Merriam's Webster. If you look up reasoning or reason it says as a rational ground or motive or number two the power of comprehending, inferring or thinking, especially in an orderly rational way. But then let's look up rational. Rational is having reason or relating to agreeable to reason. So we're going in circles again. It reminds me of the AGI debates where half the reason that we're debating is that we can't agree on a definition. Let's all agree on a definition to reasoning before we debate whether or not these models can reason. But I think the bigger second problem for the argument that we have already achieved complex multi step reasoning is that we never quite get to 100%. Now I do know what the comments are going to say. They're going to say that humans can't do the Alphabet running backwards and that humans can't do 10 digit multiplication. But nevertheless it seems to me that if these language models have achieved calculator like logical circuits, then there should be certain categories when they can achieve 100%. And that's why I read this paper with interest. GPT can solve mathematical problems without a calculator, it says with almost 100% accuracy. And this was only a 2 billion parameter language model. Here you can see it beating GPT4 but still never getting 100%. You can see here the asymptote as more and more parameters are added. This was even with up to 50 million examples. Closer and closer to 100%, but not quite 100%. And if you think I'm being harsh, wait until they expanded it to beyond five digits. They say to assess the generalization ability of this model math glm beyond the 5 digit range, a set of 50,000 training records involving numbers within the 12 digit range are introduced into the training data set. Now it seems to me if the model, after all of this pieced together the rules of multiplication, it would start getting 100%. But no, when you go up to 12 digit, it drops to 41% even after all of this training. Training. And of course as the person who came up with SmartGPT, I know that prompting can help, but it doesn't solve the issue. Indeed, this older paper from Microsoft and the Allen Institute of AI again suggests that memorization is coming into play yet again. When testing models on arithmetic, they found a strong correlation between the accuracy for a particular number and its frequency in pre training. So after doing this mathematical detour, we're left in the same position. Models can do far more than memorize, but less than use pure logic. Logic. But at this point I know what many of you may be thinking. Aren't these companies going to work on injecting pure logic, pure reasoning into their models? Yes, they are.

I would say that if you, if you want to talk to some of the folks at Anthropic, they have amazing intuition in addition to the folks at Google DeepMind. It's a challenging space, like reasoning. Reasoning is the thing that everybody seems to, to be like pushing towards like how do we plan, how do we reason, how do we break down this higher level problem into steps? And then that kind of gets you. The holy grail for these general autonomous agent things is like how do you go from having a natural language problem to decomposing it into steps and then generating code to accomplish tasks or doing retrieval lookups to accomplish tasks.

That's one of the reasons why my AGI timelines aren't much changed. The other reason is that AI is so much more than just LLMs. Who said AGI just has to be a scaled up large language model? We already have mu0 from google DeepMind which could master Go Chess and Atari without even knowing the rules. It uses reinforcement learning and Monte Carlo search, which is very different from an LLM. In the MU0 paper released in 2020, we learned that it was trained in around 12 hours and it was able to match the performance of AlphaZero in go in those 12 hours of training. And AlphaZero of course was the one that crushed AlphaGo 100 to 0, with AlphaGo being the model that beat Lee Sedol anyway that was trained in 12 hours using the old version 3 TPUs. We now have version 5 TPUs. Strikes me that in the future such a model could be trained in minutes rather than hours. And if an LLM can call upon something like mu0 or train up a new one in minutes, why does the LLM need to do everything itself? Why does it need to have perfect mathematics or perfect log logic? And that's not even bringing in efficient zero which beat MU0's performance with only two hours of real time game experience. That's a better sample efficiency than human beings. So no, my timelines for AGI haven't changed, and nor has the investment going into AI. We learned just this morning that Amazon have invested $4 billion into Anthropic. I will actually soon be speaking to insiders from Anthropic on this, this and much else besides. I must say it does feel surreal, all of this rapid progress happening at the same time that almost 2/3 of voters say that regulation should aim to actively prevent AI superintelligence. Sometimes I have to step back and think what a moment we're in. We've got these companies promising nirvana borderline heaven. As Elon Musk said, billions being invested and all the while the public kind of sitting there going like, nope. It seems fitting after talking about so many of the weaknesses of language models to end on their ability to prompt a masterpiece. Using Dall E3, I can cover the pricing structure, lawsuit controversies and other comparisons in another video, but for now let's just admire the images side by side. As always, you can decide which ones you prefer. For me, Dall E3 seems to understand spatial relationships a lot better than midjourney and of course get text a lot more clearly. I think when I get access I'm going to put reasoning and logic as prompts into Dali3 and see what comes out. Thanks as always for watching to the end and have a wonderful day.