Apple DROPS AI BOMBSHELL: LLMS CANNOT Reason

So Apple has come out with some pretty incredible research and I think this has the AI community completely divided because it fundamentally shifts what we know with AI models. And I think this genuinely could change everything because of what this research is currently suggesting. So I'm going to break this all down for you because this is largely one of the biggest papers that have come out recently, and it's fundamentally quite surprising. So essentially Apple research has come out with this paper called GSM Understanding the Limitations of Mathematical Reasoning in Large Language Models. Okay. And basically the short summary of what this paper is suggesting is that it says that we hypothesize that current LLMs are not capable of genuine logical reasoning. Instead they attempt to replicate the reasoning steps observed in their training data. So, so basically this paper is trying to suggest that current large language models as they are models like GPT4O Claude 3.5 Sonnet that these models aren't as smart as we think they are and they are not reasoning through any kind of problem. The only thing that they're doing is just statistical pattern matching, which means that these models are simply not as smart as we once thought they were, which would have severe implications for certain things moving forward. Now the research dives into a few things which I'm going to explain, and it's pretty simple to understand. So let's get into why this is really impactful. So one of the things that we use to do to assess how models score on their reasoning or essentially assess how smart these models are, is by doing certain benchmarks. Okay? And one of those benchmarks is called the GSM 8K. This is called the grade school mathematics 8000, which is 8000 grade school mathematics question. The Apple researcher says here that when OpenAI released GSM8K three years ago, GPT3, one of the first iterations of the GPT series models scored 35% on GSM 8K test. Today's models with 3 billion parameters are surpassing 85% and the large ones are hitting 95%. Okay? But he argues that has reasoning really improves. Like how much of this is genuine logical symbolical reasoning versus pattern recognition, inverting data contamination or overfitting. So basically what he's stating is that if we look back to the smaller models, the smaller models like GPT3, which were 175 billion parameters, scored 35%. Okay? And today's smaller models, like the tiny models from today that are only 3 billion parameters, are surprisingly surpassing 85% and even the larger ones hitting 95%. But he's like okay, we've made such incredible progress in such a short space of time, but we need to realize how much of this reasoning has actually improved. Okay, for versus data contamination, which basically means that the data that we use to actually train these models, more parameters, more data, the data that we actually use, how much of this increase in percentage is actually related to data contamination? Where unfortunately, some of the data from the test sets and the answers have slipped in to the training data. So the model essentially remembers what it was trained on and, and isn't actually getting smarter. And how much of this is also just pattern recognition. So what we can see here is we can see this graph which shows us the GPT3 models from 2021. And then of course we can see the state of the art models in 2024. And we can see that the GSM 8K accuracy is just going up completely. Okay, we can see that this is continuing to go up. Okay, but this is where this, you know, this Apple team of researchers decided that, okay, if we want to test to see if these models are actually increasing their reasoning, what we need to do is we need to come up with a new benchmark. Okay? And this new benchmark that they've used is one that is a little bit different, okay? And it's literally a little bit different because what it does is it changes things ever so slightly. So they said, introducing GSM Symbolic, our new tool to test the limits of LLM in mathematical reasoning. We create symbolic templates from the GSM 8K test set, enabling the generation of numerous instances and the design of controllable experiments. We generate 50 unique GSM symbolic sets, essentially like GSM 8K examples, but with different values and names. So all they did was they took the test sets of the GSM 8K the, they took a few of those questions and they decided to change the values and the names. So basically in the math questions, sometimes you're like, Jimmy has five apples. So what they decided to do was they decided, okay, I'm going to change the names. Okay? So from Jimmy it's going to be John, and from apples it could be oranges, and from six oranges it could be seven oranges. So they decided to change the values and the names. And if these models are truly capable of reasoning, they, they should be able to handle these problems fine as long as we've just changed the names and the numbers and not the actual problems. So essentially you can see right here that this is the GSM8K on the left hand side. We can see that the Values that are going to be changed are located here. We can see that Sophie, then of course a nephew, then we can see 31, 8, 9, Sophie nephew 62. So those are the only values that are going to be changed with regards to the changing. And you can see right here that the GSM symbolic template, of course the name, it has, the family, the total, and you can see it changes these within a certain range. And this is how the template is constructed. Now, it's important that you understand that all they've done is just change the names and the values. But the crazy thing about this was that when they changed the names and the values, we can see that there was essentially a large discrepancy in terms of the results between what the models claim to have got from various research labs versus what they actually tested and they actually got. So he says here that current accuracies on the GSM 8K are not reliable. We observe a large performance variation. For example, llama 8B scores anywhere between 70 to 80%, Phi3 scores between 75% and 90% and so on. And for most models, the average performance on the GSM Symbolic, which is just a variation of a test that they already took with different names and different values, is lower than the GSM 8k. We can see here that there is quite the discrepancy between the reported values and then of course the actual values on the GSM symbolic test, which is quite surprising. And we can see that the actual test results are in the dashed line. So if you want to look at what these models got, we can see what these models got here in the dashed line. Then we can see the variations of what these models got, which is between 90 to 98, then of course 70 to 80%. Then of course, from here, this one looks like from 70 to 85%. So they're starting to question why on earth is there such a giant discrepancy in terms of these models? Results purely based on the fact that the only thing that has been changed is of course just the names and the values. And this is a bigger chart and we can see that some of the models that have the largest drop are Phi 2, Gemma 2, and of course some of these other models. And it seems that these smaller models are going to be the ones that probably have a lot more overfitting, a lot more data contamination, amongst other things. So we can see right here that the researchers are stating that the fragility of supposed LLM reasoning LLMs remain sensitive to changes in proper names. For example, the people the foods, the objects, and even more so when numbers are altered. And he's basically asking here, would a grade school student's math test score by, you know, vary by 10% if we only change the names? And I think it wouldn't like if you gave someone a test and you only change the names, would the outcome of a mathematical based question change by 10%? I don't think so, considering the fact that it's just a name. Which could mean that potentially these LLMs are simply memorizing things or doing pattern recognition. And this isn't good because if this is true, it would mean that these models aren't as smart as we think. And it might mean that we need better architectures for solving the reasoning issue. Now, there's a lot more that goes on here. We can see how this has been affecting other models with these changes. We can see that we have these changes plotted here. So for example, we can see the GSM 8k is the dashed line. Then we can see if we change the names, there is a slight drop. If we change the numbers, there's another drop. And if we change both, there's an even bigger drop. So this is something that is quite surprising because it means that these models are getting confused once these names are being changed. It doesn't really make any sense whatsoever. Now we also have something where he decided, okay, what if we adjust the question difficulty? We're going to introduce three new variants of the GSM Symbolic to study model behavior. Removing one clause GSM M1 and adding one clause or adding two clauses. Basically it's just the GSM Symbolic which is just one change. And then of course the GM Symbolic P1, which is an increase in difficulty, and GSM Symbolic P2 is, which is another increase in difficulty. Now, of course, this specific result I don't think is too bad because of course as you increase the difficulty, the model should go down. And this is what we do experience here. But the difficulty doesn't seem to be too much. But there are really large drops in performance, which does also beg the question, are these models truly understanding what's going on? Because there are a few difficulties added and there's such a large drop off. We can see that. Of course, the O1 models do seem to perform a little bit stronger, but on other models like GPT4O and GPT4.0 mini, there are a lot bigger drops. Now this is where the research starts to get really crazy and I start to question whether or not these models truly understand what's going on. Because the researchers decided to do something even crazier. Okay, so this is where they state, this begs the question, do these models truly understand the mathematical concepts? Introducing the GSM noop. We add a single clause that seems relevant but doesn't contribute to the overall reasoning. Hence no op. So basically, what they've decided to do here was to have these traditional GSM 8K exam questions. But what they did was they added something that wasn't relevant to the question really at all. So we've got one example here, and you're going to see the results of this, which are pretty crazy. So it says Oliver picks 44 Kiwis on Friday. He then picks 58 Kiwis on Saturday. On Sunday, he picks double the number of kiwis he did on Friday, but five of them were a bit smaller than average. How many kiwis does Oliver have a statement like, but five of them were a bit smaller than average, but doesn't impact the number of kiwis that you do have? Like, it literally doesn't impact the mass at all. Okay? It doesn't matter if it's bigger or if it's smaller. A kiwi is still a kiwi. Okay? And the crazy thing about this is that it says we added seemingly relevant statements to the questions that are in fact irrelevant to the reasoning and conclusion. However, the majority of models fail to ignore these statements and then blindly convert them into operations leading to mistakes. Okay? So what these models will do is they'll be like, okay, if five of them are smaller than average and they get really, really confused where they should just disregard that because it's not relevant. But take a look at the performance drops on models. When this is done, the performance drop is just. I mean, this is insane. Like, this is incredible. You've got the GSM 8K. If we look at the GSM no OP accuracy, the performance drop is around even on 01 preview. The Orion model that we think is probably the best model that currently exists, There is a 17% drop in the GSM 8K to the GSM no op accuracy, which is pretty outstanding. And the reason I say and the reason I highlight the O1 preview model, which is this model right here, is because this is the model that is supposed to have the best reasoning capabilities. There shouldn't be such a large slope here because what we have is we have a consistent drop in performance across the models despite these benchmarks having irrelevant questions. What we should see if O Preview is as good as reasoning as OpenAI claims. This bar should be up here. Because these models work through the problems step by step in order to get to the final solution. I'm not sure how OpenAI trained 01. There's a lot of secrecy behind it. But I mean, if a model drops its reasoning capabilities 44%, 32% for GPT4O when seemingly irrelevant information is added. I think this is absolutely remarkable because how many times have we added data to a problem, to ChatGPT, to GPT4? Oh, we've provided it with so much context, and sometimes, arguably, sometimes some of that context is completely irrelevant. And I mean, a 30 to 40% drop in terms of the reasoning output, I mean, that's pretty awful. Like, that's pretty bad. Like, there's no way to put this other than this is a truly shocking find in my honest opinion, because I wouldn't have thought that adding such a relevant statement would have resulted in such a large performance drop. And what's crazy about this is that we see a similar performance drop for GPT4.0 to.0.1 mini. We can see 0.1minis drop is at 29% and GPT4 OS is at 32.01preview does do better at 17.5, but I wouldn't be expecting that big of a drop for a model that is specifically trained to reason. Okay, and we can see the other open source models here. A lot of these other models are pretty smaller, but I am glad that they did include the O1 series of models, because that would have been a glaringly obvious thing to do, considering it's just trained on reasoning. So the craziest thing about this, which is why I said that this could genuinely fundamentally change everything, is because he says here that can data models or compute fundamentally solve this? We don't think so. They state that scaling data models or compute cannot fundamentally solve this issue. Okay, so it says that OpenAI's O1 series is performing better, but still suffers from slight performance variations. And O1 preview shows significant improvements, but it still suffers from silly mistakes like this. Okay, you can see that this problem just blindly applies the inflation rate. So the question is, Liam wants to buy some school supplies. He buys some erasers that now cost $6.75 each, 10 notebooks that now cost $11 each. Okay, a ream of bond paper that now cost $19. How much should Liam pay now? Okay, assuming that due to Inflation, prices were 10% cheaper last year, when you read this question, you have to disregard the fact that due to Inflation, prices were 10% cheaper last year because Liam is buying this stuff now, so this bit right here, you just completely disregard that, okay? Most humans who look at this kind of problem are going to completely understand that because you're going to be thinking, wait a minute, why would I look at inflation last year? Okay, disregard that. How much does everything cost now? And then you answer the question. So of course we can see here that when this model is doing the reasoning, we can see that it says step one, calculate last year's prices after reducing the current price by 10%, which is just not what you would do at all. You're not even supposed to do this step. So the model completely gets this wrong. And the question is, is that if these models were actually good at reasoning, if they truly understood what was going on, would they make such simple mistakes? He then goes on to state that understanding LLM's true reasoning capabilities is crucial for deploying them in real world scenarios where accuracy and consistency are non negotiable, especially in AI, safety, alignment, education, healthcare and decision making systems. And that is truly important. If you are going to be saying that, okay, we're going to get to AGI, we're going to be.