OpenAI PROVES DeepSeek COPIED Them!

Substantial evidence that what DeepSea did here is they distilled the knowledge out of OpenAI's models. And I don't think OpenAI is very happy about this.

So OpenAI are basically saying that it has evidence that China's Deep SEQ used its model to train a competitor. They're basically alleging that Deep SEQ used the outputs from OpenAI to actually train their model. And this is pretty big news because the news about Deep Seek's performance relative to the price it cost to actually create the model have led to some very interesting developments in the stock market. And this is pretty big news. Now, this news right here, this is some pretty big news because this actually means that there is the potential for OpenAI to sue DeepSeek on the grounds of alleged intellectual property theft. And one key piece of evidence that I think is floating around the Internet right now is this screenshot here. So we. One of the things that people have started to realize is that when you ask Deep Seek, who are you? Oftentimes in random conversation, it may refer to itself as a large language model developed by OpenAI. And this is because if a model was trained on the outputs of OpenAI's models, then in those responses that it would be training on the model would essentially be stating that as an AI developed by OpenAI XYZ. And that's essentially what we see here in Deepseek's outputs. This is something that's really important because it's if deepseek was an original model which wasn't training on the outputs of OpenAI, then it should say as a model developed by the company deepseek. But you can see in many cases this model actually refers to itself as a model developed by OpenAI. Which of course isn't going to look good in the context of all the news going around. Now I do know that recently deepseek did change this. So if you do message it now and ask who developed you, it will say that I'm developed by DeepSeek. But previously it actually did say that I'm large language model developed by OpenAI. And of course that is probably because they did train on the outputs from OpenAI. I can see here that it also states that OpenAI actually says that they found other evidence that the Chinese AI startup deepseek used the US company's proprietary models to train its own open source competitor. As concerns grow over a potential breach of intellectual property. And what OpenAI are essentially alleging here is that they had seen some evidence of model distillation in machine learning. You Basically want to put the large model capabilities into a smaller model. So you'll have a large model trained on a comprehensive dataset, capturing the intricate patterns of knowledge. And then essentially the student model, you'll have a smaller model which is initialized to be more resource efficient. And then that student model will be trained using the outputs of the teacher model. And it basically involves adjusting the student model to mimic the teacher's output distributions, capturing the nuanced information that the teacher model has learned. And in this case, that would be the OpenAI's 01 model or whatever model they have used in this case. And OpenAI have actually used this themselves. They've actually distilled very smart models. Like they have an internal model called GPT5, or they might call it Orion. I'm honestly not sure what the model is called, but essentially they've been distilling that model's capabilities down into models like GPT4O, so they can actually serve them to us at a much cheaper cost. Take a look at what the Grok CEO Jonathan Ross says about distillation and how Deepseek was able to rapidly advance its AI capabilities with from OpenAI. And how this actually works is if.

There'S a really good model already right here, just have it generate the data and you go whoop right up to where it is. And that's what they did. So it is true that they spent about 6 million or whatever it was on the training. They spent a lot more distilling or scraping the OpenAI model. OpenAI was effectively subsidizing accidentally the training of this model because they, they were using OpenAI. Right. And, you know, rumors are that OpenAI may not be completely profitable yet in terms of every token in the API, like on the subscriptions maybe, but, but in the API. And so each one that they generate effectively, they were losing a little bit of money while Deep SEQ was getting.

And he says, you know, Deep Seat were getting the training data for their model, which is pretty crazy when you actually think about it. So overall, it seems like maybe deepseek were stealing from OpenAI. Now, the article continues to state that this technique used by developers to obtain better performance on smaller models by using the outputs from larger, more capable ones, allowing them to achieve similar results on specific tasks at a much lower cost. And that distillation is a common practice in the industry. But the concern was that deepseek was using it to build, of course, their own rival, which is against OpenAI's terms of service, which we will get into in two. A moment. Now, the crazy thing about all of this is that OpenAI actually doesn't want to provide details of its evidence in its terms of its service. You know, they talk about how you cannot use copy of its in any of its services to compete with OpenAI. That's not the point I'm trying to focus on here. The point is that OpenAI haven't provided details and I think it's really important that they haven't because if they did, Deep Seat could potentially go uncover their tracks. Like for example, when I spoke about how sometimes deepseek refers to itself as OpenAI, they've actually now removed that. So anytime it refers to itself, it always says I'm deep Seek. But that is a key piece of evidence because if your model was trained by yourself, it shouldn't really say, you know, I'm a large language model by OpenAI. They will have some good defenses stating that, you know, they trained on a large portion of the Internet and a large portion of the Internet is going to contain pieces of text that do contain OpenAI's model. So it's not like they're going to be stuck in the mud there. But take a look at what David Sacks, the AI czar, which is the government appointed leader who's responsible for overseeing and managing AI policies, regulations and strategies. And he actually said in a recent interview that, that it's quite possible that this is exactly what DeepSeq did to get the capabilities of the current model possible.

There's a technique in AI called distillation which you're going to hear a lot about. And it's when one model learns from another model effectively what happens is that the student model asks the parent model a lot of questions, just like a human would learn. But AIs can do this, asking millions of questions and they can essentially mimic the reasoning process that they learn from the parent model and they can kind of suck the knowledge out of the parent model. And there's substantial evidence that what Deep Seat did here is they distilled the knowledge out of OpenAI's models. And I don't think OpenAI is very happy about this. And I think one of the things you're going to see over the next few months is our leading AI companies taking steps to try and prevent distillation. And so we'll see if, if, if the leading AI companies can prevent distillation by third party companies, that would definitely slow down some of these copycats.

So with OpenAI's terms of service, you can clearly see here that they state that one of the things you simply cannot do is use their outputs to develop models that compete with OpenAI. And of course in this case Deepseek is a direct competitor to OpenAI. So if they did do this, it would be possible for OpenAI to potentially sue them on multiple different grounds. You can also see here that they said that you also cannot attempt to assist anyone to reverse engineer, decompile or discover the source code or underlying components of our services, including our models, algorithms or systems. So this is clearly some kind of breach of Terms of service. And I wouldn't be surprised if OpenAI actually sues deepseek on these terms or these grounds. And the article continues to state that it's very common practice for startups and academics to use outputs from human aligned commercial LLMs like ChatGPT to to train another model. That's what someone who has a PhD at University of California in Berkeley said. And it says that means you get all of this human feedback step for free. And it's not surprising to me that Deep Seek supposedly would be doing the same if they were. Stopping this practice precisely may be difficult. And that's pretty true. How on earth do you figure out when someone is just chatting with a model versus actually using the model to get certain specific outputs so that they can build their own? This would actually be a really difficult thing to figure out. And it's not like OpenAI can just now potentially imply rate limits or whatever because that would just render the surface pretty much useless to developers. So they are going to be in a bit of a pickle figuring out exactly how individuals are doing this. You can see right here in the article it states that we know China based companies and others are constantly trying to distill the models of leading US AI companies. That's what OpenAI said in a later statement. And they also said that we engage in countermeasures to protect our ip, including a careful process for which frontier capabilities to include in released models. And it is critically important that we are working closely with the US government to best protect the most capable models from efforts by adversaries and competitors to take US technology. And of course it wouldn't be the first time that China has copied some US technology. There have been numerous cases of China actually stealing trade secrets by various different means. On 7th March 2024 there was actually an ex Google engineer charged with stealing AI secrets. A formal Google software engineer was actually charged by the US from stealing trade secrets about AI while working secretly for for two Chinese companies. This Guy, you know, he stole more than 500 confidential files and it was pretty crazy. So this isn't the only time that, you know, China has done this before. So it wouldn't be surprising to me if they managed to just steal distill this information from a publicly available model. Now, whilst all of this information is there, something really funny is kind of happening. A lot of people are basically saying that okay, maybe they did distill the knowledge from OpenAI's 01 model into R1, but can OpenAI really even complain about this? Because recently OpenAI accidentally deleted the potential evidence in a trading data lawsuit. If you aren't familiar to what they're referring to, I want you to understand it like this. OpenAI are basically saying, hey, Deep Seq, why are you copying our data? But most people are like, wait a minute, how is OpenAI saying people are copying our data when OpenAI basically just scraped all available public data on the Internet and didn't pay anyone for that data at all? Most people are basically saying that, look, OpenAI, you guys scraped all the data from the Internet without giving anyone credit. And of course, with that being said, you also accidentally wink, wink, accidentally erased potential evidence in a training data lawsuit. And it's crazy because OpenAI engineers accidentally erased critical evidence in a certain lawsuit about the training data. So this is something pretty incredible. And I mean, it's a wild ride in AI right now. Now something as well that I do find super interesting is the benchmarks. I actually like to look at the LMSYS chatbot arena because. Because this is one that is somewhat of an independent evaluation considering it's based on actual humans. In a chatbot arena, if you aren't familiar with how this benchmark works, essentially you input one prompt into a chatbot and you get two responses and you are blind. You don't know which response is from which model and then you rate which one you think is better. And then over time we do see which model people do prefer when they are essentially blind tested by the models. And right now you can see the deep seek is actually tied first with ChatGPT and Gemini's new model. So it's actually a pretty decent model even despite me being skeptical about certain benchmarks. However, on certain benchmarks like simple bench that actually do take a look at reasoning and this is real human style reasoning, like for example, the questions might ask about how many minutes will it take for ice to melt? Or you know, just simple questions that, you know, for a human would actually be really easy. So I don't really want to say them here, but you can see the human baseline is like 85%. But deepseek R1 actually.