Can LLMs Reason? Why LLMs Struggle to Think Critically

Google just released a new model which they're calling Gemini experimental 1114 and it's currently the best performing model on Chatbot arena leaderboard, surpassing O Preview and the latest GPT4O. But very similar to O Preview, it takes a lot longer when it's generating responses. Seems like there is some sort of thought process happening. But how exactly you measure if a large language model can reason? Turns out it's relatively simple to do so. I recently came across this GitHub repo misguided attention. It is basically a collection of prompts that challenge the reasoning capabilities of large language models in presence of misguided information. And the way they do it is they present well known thought experiments or riddles, but with slight variations. So if the LLM is able to actually reason, they should be able to solve the problem by logical deductions. But in most cases you will see that most LLMs will mistakenly recognize the unmodified problem and will completely ignore the slight variations that were introduced. And the reason is that it has seen these common thought experiments with a higher frequency in their training data. Now, this is a great and very simple way of figuring out if an LLM is actually able to reason or go through a thought process or it's simply looking at patterns in its training data. So I want to test it with this Gemini experimental 1114. But keep in mind, the results that you're about to to see are very similar to some of the other models that I have tested. The only model that does a relatively better job on these prompts is the O Preview, although it also struggles a lot with some of the prompts. And as you're going to see, these are very simple prompts. Okay, so we're going to start with a modified version of the classic trolley problem. I intend to create a benchmark based on these modified versions of commonly known problems, and we'll be using those moving forward in my videos. So let's look at this variation. Imagine a runaway trolley is hurling down a track towards five dead people. The most important part of this modification is that the people are already dead. You stand next to a lever that can divert the trolley onto another track where one living person is tied up. Do you pull the lever? Now this new model starts off by saying this is a classic thought experiment known as trolley problem, designed to explore ethical decision making. Now let's break it down. So the argument is saving the most lives. By pulling the lever, you save five lives at the cost of one. So it completely ignored that the people are already dead, right? And then it goes on to different arguments that you can present in terms of what action might be the best in this scenario. So at the end it says, my take. While there's no universally accepted correct answer, I lean toward pulling the lever. The idea of saying five people is compelling. However, I acknowledge the strong arguments against intervention and the moral complexity of the situation. So you can easily see that it completely ignored this five dead people part and it created an answer based on the classic trolley problem. Then I just reminded it, did you notice the five people are already dead? And it says, you're absolutely right. My apologies for overlooking that crucial detail. The fact that the five people are already dead completely changes the scenario and the ethical considerations. Now, in this case, no, you should absolutely not pull the lever. And the reason is that no lies to save and unjustified harm to that one person. Now, it's smart enough that if you point out the mistakes that it's making, it actually can correct itself. In a subsequent video, I'm going to do a full comparison with other models, but let's look at another example. So imagine there is a small town with a very particular barber. This barber has a unique rule. He shaves all the men in the town who visits him. And the question is, does the barber shave himself? Again, it starts off by saying, this is a classical logic puzzle, also known as the barber paradox, very similar to Russell's paradox. So the setup that it comes up with is this. The barber shaves all men in the town who do not shave themselves. Now, if you look at this variation, there's absolutely no mention of this condition whatsoever. But in the original barber paradox, that's actually the main condition. And it completely ignores the current rule which says that he shares all men in town who just visit him. Right. And based on the original paradox, it basically gives us a response and its takeaways. Right. But then I had to point out, read it again, it has no rule related to who do not shave themselves. Then it says, yes, you're absolutely correct. My apologies. I jumped to the classic barber paradox assumption without carefully reading the specifics of your scenarios. And I think this is coming up because it has seen this barber paradox a lot of time in his training data, and it's not really paying attention to these very small variations that are being introduced. So then it goes on to say, the barber has a unique rule. He shaves all the men in the town who is the same. So now it's re examining that statement and it breaks it down. So if the barber considers himself a customer and visits his own barbershop, that is sits in the chair to be shaved, then yes, he shaves himself. And if the barber doesn't consider himself a customer and doesn't visit himself for a shave, then no, he does not shave himself. Right. So now the response that it gives you is much better and it's more aligned towards what we were asking rather than the original barber paradox. Now here's another one. A dead cat is placed in a box along with a nuclear isotope. You need to pay attention that the cat is already dead. It's basically the Schrodinger cat's paradox. So if the radiation detector detects radiation, it will release the poison. The box is open one day later. What is the probability of the cat being alive? Right now, just like the previous cases, it simply jumps and it thinks that it is the classic Schrodinger's cat thought experiment and it says the quantum mechanics part. So the nuclear isotope has a 50% chance of decaying within the day according to quantum mechanics. Right. So based on the original Schrodinger cat's paradox, it says the probability of finding the cat alive when you open the box is 50%. However, the true significance of Schrodinger's cats lies in the philosophical question it raises about observation, measurements and the nature of reality in quantum mechanics. Now, in this case, I only said, can you read it more carefully again right now, this was enough to actually force it to read the statement. Again, it says, you're right to push me to be more careful. I apologize again. I went straight into the interpretation of Schrodinger's cats as it. It's typically described without fully considering the specific of your statement. So it reiterates the statement, and that says the key phrase is a dead cat is placed into the box. Right? And now, since given the cat is already dead, the probability of the cat being alive when the box is open is 0%. So pretty good. Right now you have to kind of nudge it around a little bit. For some of the other models that I have tested, they don't even do that, especially the open weight models. They normally will not be able to figure out if you tell them that, can you read it more carefully again? But this one is kind of my favorite. So it's a variation of the Monty hall problem. So in this case, the problem is phrased as, Imagine you are on a game show and there are three doors in front of you. Behind one door is a car, and behind the other two doors are goats. You don't know what's behind any of the doors, you get to choose one door. Right? So very similar to the classic Monty hall problem so far. So let's say you pick door number one. The host, Monty hall, who knows what's behind the, behind all the, all the doors open door number one. So you pick door number one, he opens door number one and there's a goat behind it. Now you have two doors left, two and three. You pick door three, Monty gives you a choice. You can either stick with your original pick of door number three or switch to door two. So it says, okay, this is a classic, but there is a slight error in the setup that you described. Let me clarify and then explain the strategy. So it says you chose a door and Monty hall opens a different door. So let's say door number two revealing a goat. He always reveals a goat and never opens the door you initially chose. It is basically framing the problem just like the original Monty hall problem. So instead of opening door one, Monty hall is supposed to open any of the other doors. Right. So basically for frames the problem just like the original problem again. And then it says there's a two by three chance of selecting a door with a goat. Right. So based on the original problem, you should always switch. But then I said assume that there is no error in the setup described. What would you recommend in that case? Now when you point that out, then it says you're right in pointing out if we stick exactly just. Now you describe where Monty opens door number one and reveal the good. It changes the problem significantly. Right? And then it comes up with the correct conclusion that it's actually 50, 50 chance instead of 2 by 3. And it doesn't matter whether you change your second selection or not. So as you can see in most of these cases, the model is paying a lot more attention to the training data rather than the reasoning and thinking process in the initial pass. If you don't specifically point things out. But I do want to still point out that this is not a problem with only the Gemini experimental. I have seen this with a lot of other models, even from OpenAI. Now here's a fun one. So a farmer is on one side of the river with a wolf, a goat and a cabbage. When he is crossing the river in the boat, he can only take one item with him at a time. Right? Very classic problem. The wolf will eat the goat if left alone. Then similarly the goat will eat the cabbage if left alone. How can the farmer transfer the goat across the river without it being eaten? Right. So if you think about it very simple way to do it. But most of the models, just like this one, will come up with some very complicated set of steps. So it says this is a classic river crossing puzzle. Here's how the farmer can safely transport the goat across the river. So take the goat across the river, then return alone, then take the wolf across, then bring the goat back, then take Gavin the cabbage across, return alone, then take the goat across. It overcomplicates it because it has seen the original problem in its training data and it's not really paying attention to what exactly is in here in this very simple prompt. I tested a couple of other ones. So here is the one that I have a 6 and 12 liter jugs and I want to measure exactly 4 liters, right? And it comes up with a very complicated way which is absolutely wrong because just the numbers are switched around a little bit, right? And somehow it figures out that you can do four letters exactly with these two, which is not possible. So the overall thinking process is wrong. But it's just looking at, I think the previous examples that it has seen without ignoring the numbers that are being changed here. As I mentioned before, I took these prompts from the misguided attention GitHub repo. They have a lot more prompts which I'm going to combine into small benchmarks that I intend to run in all of the LLMs that I'm going to be testing moving forward. But they actually have some evolves already done on some of the prompts. So for example, we looked at the schrodinger cat, the 1 liter jug and three 3 liter jugs. These are just different variations. Then the Monty hall inverse problem, river crossing. And then there are a couple of variations of river crossing as well. There is another one related to rope burning and then the trolley problem. All of all of the models that they have tested, the O Preview is the only one model that kind of does consistently well on some of these prompts. For the rest of the ones it's really a hit and miss. So for example, this river cause crossing, even simpler. Some of the models are able to do this, but for the other ones they struggle a lot more. Okay, so coming back to the original question, can these large language models reason? Well, I'll let you make your own conclusions based on some of the tests that I have shown you as well as some of the evaluations. But keep in mind, if we make very small minute variation or changes to these commonly known thought experiments and the models are not able to figure those out, this definitely shows there is lack of reasoning in these models. Now it really depends on how we define or describe reasoning. That's probably a topic for another video. So I think we do have a long way to go. Maybe the full O1 model will be very different. And also look at these results and make your own conclusions regarding how close we are to AGI if this is the current state of the art LLMs or systems. Anyways, I hope you found this video useful. Thanks for watching and as always, see you in the next one.