Microsoft's New AI Orca-2 Just Changed EVERYTHING! (Synthetic Data Breakthrough)

So Microsoft recently released their new research paper in which they present something called Orca 2. Now this is a follow up to what they released before, which was orca and this was something that we did a video on before. Five months ago they released something called orca. Essentially it was a large language model that was pretty good in capacity that took everyone by surprise and managed to match the capabilities of ChatGPT. Now however, they're back again with of course, Orca 2. And this is teaching small language models how to reason. Now there's one section of this paper where I think that we do get some kind of breakthrough in the kind of methods that we are using to train these large language models. And with the discussions online about this kind of training, I want to propose to you the idea why this is definitely going to change everything. So here's the thing about Orca 2 that sets it apart from other models and is that when you read this paper is that they did do something different. I think it's called step by step recall, then generate, then reason, recall, generate, direct answer, etc. So they have all of these various reasoning techniques for every different prompt. And essentially the reason Orca 2 is very, very good and why it's so cool and amazing, and I'm going to get into some of the real reasons later, is because one of the reasons is because Orca 2 significantly surpasses models of similar size and attains performance levels similar to or better than those models five to ten times larger. So the thing about ORCA and what we've seen about Microsoft research is that Microsoft are focusing what seems to be on large language models that are small but really, really capable. You see, orca is only 7 billion parameters, which is tiny in the comparison of things. You see, I think when we look at some of the other large language models like GPT, ChatGPT 3.5, that one has 175 billion parameters. But Orca is only 7 billion parameters and manages to accomplish nearly many of the feats that ChatGPT does, which is why this model is truly, truly interesting. Now, interestingly enough, what this model is, this model is open source. So you can use this right now if you do want to get the weight of this model, you can go over to this page and you can see that it's Microsoft Orca 2. 13 billion parameters. There's a 13 billion version and then there's a 7 billion version. So that, that should be something that I've seen people testing on Twitter and it's actually really good. So why this is like, you know, so crazy is because the way that they train this model as well introduces some new techniques that we haven't seen too much, but we are going to get in that. Now what is Orca 2 based on? Orca 2 is based on of course llama 2. So Orca 2 is built upon the llama 2 model family and of course it does retain some of its limitations as well as the common limitations of LLMs. But, but I think it was really interesting that it was based on llama 2 because when we compare it to the other models, you guys are going to see that it has the reasoning capabilities that are comparable to much larger language models. You can see, to evaluate Orca 2 we use a comprehensive set of 15 diverse benchmarks that correspond to approximately 100 tasks and more than 36,000 unique test cases in zero shot settings. The benchmark covers a variety of aspects including language understanding and common sense reasoning, multi step reasoning, math, problem solving, reading comprehension, summarizing, groundness, truthfulness and toxic content generation and identification. Now of course you can see that on these benchmarks that Orca 2 does surpass Llama Chat 2, even which it's based on, and it does also pass surpass WizardLM. Now, yeah, it would have been interesting if we actually got to see what they compared this against because they only did compare it against llama2 and wizardlm. So it would be nice to see many of the other models. Now I'm not saying that they cherry picked this because I'm pretty sure they didn't, but some of the findings are really, really cool because remember, these other models are really a lot bigger. So if they're able to get these smaller models to be more efficient and better reasoners, then we have a serious, serious future in our hands. Now this is where it starts to get interesting and this is where I would need you guys to pay attention because this is where the true stuff does come through. So it says orca's performance significantly surpasses models of similar science and and it also attains performance levels similar or better than those of models at least 10 times larger, which is of course models that are around that 70 billion parameter model. Showcasing the potential of equipping smaller models with better reasoning capabilities. Orca 2 models exhibit limitations. And of course what's also interesting was that the way that they trained Orca 2 could be applied to different base models. Okay, now they only used llama 2 and the reason I'm talking about different base models is because llama 2 is good, but there are other base open source models that are far superior that I'm going to talk about later. And I think it's really interesting because those other base models that we do have, if you take a look at the large language model leaderboard, you can see right now that there was a new open source king, and this one was called yi or yi, 34 billion parameters. Now it does absolutely insane. It crushes the benchmarks, but that's what I'm saying. Okay, so imagine now using the way that we trained Orca 2 to be even more effective. Can we apply it to some of these open source models and, and then get even more effective because a lot of these models, some of these are trained in many different ways. And it's intriguing to see how we could do this. So essentially the main breakthrough for Orca 2 was the synthetic data set. So it says here the Orca 2 is trained with an expanded, highly tailored synthetic data set. The training data was generated such that it teaches Orca 2 various reasoning techniques such as step by step, processing, recall, then generate, recall, reason, generate, exact, generate and direct answer methods, while also teaching it to choose different solution strategies for each different task. So the reason that this synthetic data is such a large breakthrough is because of the fact that this allows us to achieve scale. It allows us to achieve scale on a level that we haven't seen. And I can guarantee you all that this is probably one of the large stepping stones that we need to explore when it does come to looking at AGI. You see, the problem with a lot of the data sets that we do collect is that you actually do need to get a lot of data. And the problem is, okay, the problem with collecting data is that it is very, very hard to collect human generated data. And when we look at what GPT3 was trained on, it was trained on, I think, 45 gigabytes of training data. It says GPT4 was trained on 45 gigabytes of training data, and GPT3 was trained on 17 gigabytes of training data. Which essentially means that if one of the main bottlenecks is Microsoft Research showing us that training on a new synthetic data set and providing these outstanding results shows us that this is where we need to look. Now, Microsoft Research aren't the only ones that are looking into synthetic data sets, because of course, as you know, this is data that is created by AI themselves and is essentially just generated. Now the reason that this is really cool is because others in the field have also started to say this and many people are looking at the previous lessons that we did talk about. So. So doctjim fan says that it's pretty obvious that synthetic data will provide the next trillion high quality training tokens. I bet most serious LLM groups know this. The key question is how to sustain the quality and avoid plateauing too soon. The Bitter Lesson by Richard Sutton continues to guide us in AI development. There's only two paradigms that scale indefinitely with compute, learning and search. It's true in 2019 and at the time of writing, it's true today. And I bet it's going to be true until the day that we solve AGI. And essentially what he means by that is that if we want to get like a trillion high quality tokens, we're going to need to make sure that we are able to generate high quality synthetic data. And I'm pretty sure what one thing that they did, I don't remember if it was GPT4, GPT3, but I'm pretty sure that they exhausted a lot of the highest quality data that was available. And it was Sam Altman or Greg Brockman that was talking about how. The fact is that, that they know that for the other large language models like GPT5 and further iterations, that if they've collected every single data source on planet Earth, which trust me when I tell you guys, it's not that much, okay, you can fit it on like a USB drive, like all of the data, if you have it just as text. Essentially what we need now is we need to be able to generate a continuous stream of new data that we can use to train these models so that we can have them in different ways. And I'm going to show you later why this is crazy. Now, the Bitter Lesson is a really, really, really key insight into why we need synthetic quality data. And when you see this, you're going to be like, okay, this is so true. So the Bitter lesson essentially is broken down into five parts. And essentially it says that general methods are effective. The primary lesson from 70 years of AI research is that general methods leveraging computation outperform other approaches due to the exponentially decreasing cost of computation. Then essentially it talks about the transition and the case studies. Okay, and this is the key important part. I'm going to show you guys a GIF that basically illustrates why synthetic data will be what changes everything. So it says, historically, AI research often assumed constant computation availability, focusing on leveraging human knowledge. Okay, key part here is human knowledge. However, as computation power increased, the focus shifted to leveraging this growing computational capability. So as you know, before it was all about human knowledge and human power. But you have to remember, over time, compute has increased. We get faster, you know, GPUs, we get faster processing power, we get more RAM, we get everything is just becoming more and more efficient. And of course this is attributed to Moore's law. Okay, but this is where we have to shift over to ensure that we know that we can utilize this compute, which is going to be more effective. And one of the things is as well is extensive search and computational power. So the thing about this is, is that there is a case study which shows us that training on data that AI creates itself essentially creates a feedback loop of kind of like self improvement to the point where we can get that next level in AI. And that's why there is such a focus on synthetic data. So I'm going to show you guys this GIF right here and let's take a look at this GIF. Okay? So you can see that at zero days, AlphaGo has no prior knowledge of the game and only basic rules as an input. Okay? But in three days, AlphaGo surpasses the abilities of AlphaGo Lee, the version that beat the world champion in four out of five games in 2016. And then in 21 days, AlphaGo, okay, it reaches the absolute level that beats the top professionals and the world champion of three out of three. Okay? So it then goes to AlphaGo Master, and then in 40 days, AlphaGo Zero surpasses all versions of AlphaGo and becomes the best go player in the world. And it does this entirely from self play with no human intervention and zero historical data. The point from this GIF is essentially showing us that in zero to 40 days with synthetic data of the AI playing itself, we simply went from something that was like negative 2000 to all the way to the best in the world. Okay? And that's 40 days. That's pretty crazy if you think about it. Okay, In a month and ten days it went from zero to hero. Okay? And I think that that is absolutely incredible when it shows us that whilst, yes, it is good to play on human data and human experts, the problem is, is that the large language model, or whatever AI system you're going to have is only going to be as good as the best inputs that you have. Which means that there is a cap, because if we're able to get these large language models to be able to train themselves with unsupervised learning, and if it actually is very effective, then we could have a system like this where we get this explosion in growth and this continual increase in capabilities where we don't need human data anymore and, and hopefully if we somehow manage to not need any human feedback, then we could get situations like this where synthetic data provides us with the key steps to moving forward in terms of reasoning. So I think this would be something that they're going to be looking at. I'm pretty sure that that's what they're going to be doing for GPT5 and GPT6, because eventually we are going to run out of data. And there was a video by AI explained where he did talk about data is all you need. So yeah, you can go ahead and check out that video, I'll leave a link to it. But it basically says that GPT5 will all be about data. And because there's several bottlenecks about data, including the fact that humans are going to be needing to produce high quality data, there's only so much data that exists. And of course we do have this tweet right here as well from Elon Musk that does say, yeah, it's a little sad that you can fit the text of every book ever written by humans on one hard drive and synthetic data will exceed that by a zillion. And Jim Fan responds saying, lots of synthetic data will come from embodied agents, e.g. tesla Optimus, if we can simulate them massively. So it shows us that right now we're moving to that future where synthetic data will be the key that allows us to get that exponential growth, just like we saw with AlphaGo. Now I'm not sure how this is entirely going to work, how they're going to get the synthetic data, get whatever kind of machine producing it. I'm sure the engineers and researchers at OpenAI know exactly what they're doing and any other startups that are going to be doing that, because there are some companies that literally just provide data to OpenAI, they might be looking into generating synthetic data as well. You have to understand that these tokens are the backbone of these LLMs, because there are some papers that show that that high quality data is essentially the backbone of that large language model. For example, Microsoft's Phi 1 was a paper where essentially they just fed it the very, very highest quality data about code generation and it outperformed ChatGPT and I think even GPT4 on code generation, which shows us that this is where we want to do and of course this is where we want to go. So synthetic data has its advantage essentially. Synthetic data can be designed to encapsulate a wide range of scenarios, including those that are rare for or difficult to encounter in real world data sets. So this allows the creation of diverse and complex training environments essential for developing models that can generalize well across different contexts. A crucial aspect of AGI. So, for example, certain things don't happen all the time, but when they do happen, you want the model to be able to have like 10,000 times where it has already happened, so that when the one time it does happen, it knows exactly how to respond and handle that situation. In addition, synthetic data also has some some benefits. Like what? The data is unlikely to be biased because we're likely to make sure the synthetic data is balanced. We can also have the safe explanation of edge cases. So synthetic data can safely represent edge cases or sensitive scenarios where real world data collection would be impractical or unethical. So I'm not going to state any of those in this video because I don't want it to get demonetized, but you can use your creativity to understand what that might be. But training on that data ensures that the models are prepared for, like I said, a wide range of scenarios, which is essential for AGI. If we're going to have an AGI system, it needs to be able to perform in every single scenario to a general level that is above humans. And of course we're only going to do that if it's seen pretty much everything and anything that there is possible. And of course, as we discussed, the fourth part is that creating synthetic data allows for rapid iteration and scalability in model training. Researchers can quickly generate and modify data sets to test different hypotheses or improve model capabilities, accelerating the path towards AGI by enabling faster advancements in model training and evaluation. So one thing that I do remember from my video that I made on the GPT5 prediction date was that for GPT4, they spent around six months just collecting data. And for GPT5, I do believe they spent around the same time collecting data. Now imagine they didn't spend six months. Imagine they could collect all data for GPT5 in just around two months. So imagine just one month or imagine even 30 days. That would be something incredible because if you could do that, think about how much that shortens the timeline from new models. They were going to be able to test, you know, many different iterations of GPT5 with many different data sets. And essentially this is why I'm saying that synthetic data could speed up the timeline. Now, of course there are some negatives that do come with synthetic data, but I do believe that it is going to be the final bottleneck. Now to conclude, Orca 2 essentially says that by strategically training these models with tailored synthetic data, we have achieved performance levels that rival or surpass those of larger models, particularly in zero shot reasoning tasks. Orca 2's success lies in its application of diverse reasoning techniques and the identification of optimal solutions for various tasks. While it has several limitations, including limitations inherited from its base models and common to other language models, Orka2's potential for future advancements is evident especially in improved reasoning, specialization, control and safety of smaller models. The use of the carefully filtered synthetic data for post training emerges as a key strategy in these improvements. So the one thing I want to know now is that are we about to see the next evolution in AI, where all of these people who are creating these large language models are going to start focusing more on synthetic data due to the bottleneck of human data? Because once you do capture all of the data that you can from humans and it's available on the us, you're going to need more data to make these models even better. And of course there are different reasoning techniques and there are different ways to train a large language model. You know, there are different things that people are doing. But I definitely believe that if we do manage to get into some kind of loop, just like we did with AlphaGo and Self Play, that could present us with a very, very different scenario in terms of where we go in the future. So what are your thoughts about this? I will be doing another video on entirely synthetic data because I do believe that in the future that's likely going to be where we're headed. And now that we do have these large language models, which essentially before ChatGPT we weren't able to generate coherent pieces of text easily, but now that we do have systems like GPT4 that can generate entire paragraphs, entire pages of high quality coherent text, I would love to see the experiments of feeding that data back into GPT4 to seeing if it does improve its reasoning even better and accelerates the process even faster.