Open AI RESPONDS To New York Times LAWSUIT

So OpenAI actually has responded to the New York Times lawsuit and I don't know how they actually responded legally, but this is going to be how they responded publicly. And I think it's important that OpenAI has responded publicly because of course, if they do that, it shows that they do care about their brand image because another company coming out and just claiming that you've done some crazy things that are illegal in terms of copyright isn't really good for your public image. So today we're going to break down OpenAI's new post and how exactly they've responded and a lot of the evidence to support their claims. So one of the first paragraphs that they actually talked about was that our goal is to develop AI tools that empower people to solve problems that are otherwise out of reach. And people worldwide are using our technology to improve their daily lives. Millions of developers and more than 92% of Fortune 500 are building on our products today. So one thing I do want you guys to do is remember that quote here. Millions of developers and more than 92% of Fortune 500 are building on our products today. Because that is going to be an important point later on in the video because it has some severe ramifications. Not bad ones, but good ones for OpenAI. They then state that while we disagree with the claims that the New York Times lawsuit, we view it as an opportunity to clarify our business, our intent and how we build our technology. Our position can be summed up in these four points which we flesh out below. We collaborate with news organizations and are creating new opportunities. Training is fair use, but we provide an opt out because it is the right thing to do. And regurgitation is a rare bug that we're working to drive to zero. And the New York Times is not telling the full story, which they definitely aren't. Now one of the things I did want to say is that loads of people do use ChatGPT. And the reason I've brought this point up is because in the last lawsuit, well, not in the last lawsuit, in the previous video where we discussed the lawsuit, they actually talked about how they are ordering the destruction of GPT4, which to me is absolutely ludicrous. I mean, ordering the description of GPT4, which took like, I think it was just $100 million in training runs where they had to train the model is pretty crazy. And that's why I said before, considering that 92% of Fortune 500 companies are building on this technology, that wouldn't be good for the vast majority of these companies. And honestly I believe that that claim and that request is a bit absurd. Absurd considering just how big the software is. It's like someone claiming that Google be destroyed or something like that, you know, it's a bit overreached. And I think considering just how many people use ChatGPT on some levels, you could argue that it is a tool that is used to level the playing field in terms of intelligence. Because although Internet and intelligence isn't a basic human right, I remember that there were some claims that, you know, stuff like Internet should now be a basic human right because of how much you do miss out on if you don't even have basic Internet access. And I do think the ChatGPT in the future, when we've got all of these companies building on top of it and it's so integrated into our society, the likelihood that we do get this model destroyed due to one simple lawsuit isn't as high as you'd think. So one of the first things that they talk about is how they actually collaborate with news organizations and are creating new opportunities, they say. We've met with dozens as well as leading industry organizations like the News Media alliance to explore opportunities, discuss their concerns and provide solutions. We aim to learn, educate, to listen, feedback and to adapt. And our goal are to support a healthy news ecosystem, be a good partner and to create mutually beneficial opportunities. With this in mind, we have pursued partnerships with news organizations to achieve these objectives, deploy our products to benefit and support reporters and editors by assisting with time consuming tasks like analyzing voluminous public records and translating stories, teach our AI models about the world by training on additional historical and non publicly available content and display real time content with attribution to ChatGPT. Now, now essentially the new partnerships that they actually did do are actually true because recently OpenAI did actually partner with many different companies for beneficial use of AI in journalism. And what is crazy about this was that there was a little bit of debate. Now I am going to be talking about it later on in the video, but essentially what they discuss later in the video, like further down in the document, is that they discuss about how OpenAI has been working with different people. Now I know that OpenAI openly stated in this that they were trying to reach a deal with the New York Times in order to get the right rights, in order to train on their data and essentially just work with them. But this article from the Information has stated that OpenAI offers publishers as little as $1 million a year in order to license their content. So it says here that OpenAI has offered some media firms as little as between $1 million and $5 million annually to license their news articles for use in training its large language models, according to two executives who have recently negotiated with the tech company. That's a tiny amount even for small publishers, which could make it difficult for OpenAI to strike deals. And it says that meanwhile Apple, which is trying to catch up with OpenAI and Google in the generative AI sector, is also trying to strike deals with publishers for use of their content. Said one executives. Apple is offering more money but wants the right to use the content more widely than what I'm guessing it's usually allowed. Now, I'm not entirely too clued up about industry standards, but I'm not sure why a company would be turning down $1 million to $5 million to, you know, license your content when a lot of, you know, smaller companies out there are going to be training on your, you know, news articles anyways. So I don't know what's going on there, but I do think it is interesting that, you know, OpenAI has gone out and done this because it does make sense from their, you know, standpoint. Of course, the company is burning money anyways and they're continually having to raise money because the company doesn't make money due to expensive trading costs. I think they will be profitable in the future. But the point is, is that this investment from OpenAI does show that, look, okay, you know, they're not just a company that's some monolith that just seeks to, you know, consume all data on the Internet in order to produce these good models. But I'm wondering if 1 to 5 million a year isn't a good deal. How much is, I mean, how much is, you know, news articles and how much does that actually contribute to the quality of ChatGPT? Because high quality data is definitely something that these companies do need, especially order to train these models. And we know that high quality data is the backbone of a lot of these models. So it will be interesting to see how this area develops because 5 million annually for a company sounds like a good deal. Maybe it might not be, but it will be interesting to see if there are more publishers that come up with licensing deals with other AI companies and how this bit does develop. Now here we do have one of the main discussion points around this, which is fair use. So if you don't know what fair use is, you. Fair use is basically in copyright where it allows you to use content in a way that as long as you use it In a fair way, you can use it, okay, so you can use copyrighted content. So for example, it says here, training is fair use, but we provide an opt out because it's the right thing to do. So training AI models using publicly available Internet materials is fair use as supported by long standard and widely accepted precedents. We view this principle as fair to creators, necessary for innovators, and critical for U.S. competitiveness, which is a very key point, U.S. competitiveness. We're going to touch on that in a moment. And the principle that training AI models is permitted as a fair use is supported by a wide range of academics, library associations, civil groups, startups, yada yada yada. And essentially what they're saying here, we have led the industry in providing simple opt out processes for publishers, which the New York Times adopted in August 2023 to prevent our tools from accessing their sites, which they actually did. Now, something that you're going to want to see were the actual comments to the copyright office, and I think they were very, very fascinating. And these were the comments by Andreessen Horowitz, a venture capital firm. So essentially they said the only practical way generative AI models can exist is if they can be trained on almost an unimaginably massive amount of content, much of which will be subject to copyright. For example, large language models are trained on something approaching the entire corpus of the written word. And, and although I think that is somewhat true, I would argue that certain models do challenge that. I mean, we did have Phi 2, which is a large language model that is considerably smaller, and they only trained it on textbook style data. But essentially what they're saying here is that if you really want a general AI model to be really good, you're going to have to train it on pretty much everything. And the only way to train it on everything is to include some of the copyright stuff there. So I mean, they do have a point that the only way to really do this and the only way to really advance society is, I mean, I guess you could say, is to train it on that kind of stuff. I mean, maybe the tech in the future is of course going to change that, but as is currently, we're at a point where whatever the case outcome is is going to set the precedent. Okay? And this is why this case and why I'm really paying attention to the case, because whichever way the case goes will pretty much determine the future of generative AI. And I think it's going to go one way and that's OpenAI's way. But anyways, secondly, AI models are not vast warehouses of copyrighted material, which is very true. And, and any suggestion to this effect is a plain misunderstanding of the technology, which is definitely true, because that's not just how AI models work. They don't just take the data and store it. So as a result, when an AI model is trained on copyrighted works, the purpose is not to store any of the potentially copyrightable content of any work which it is trained. Rather, training algorithms are designed to use training data to extract fact and statistical patterns across a broad body of examples, that is information that is not copyrightable, which is definitely variable. True. Like these models aren't really trained to, I guess you could say, be copyright leeches to, you know, news articles. They're essentially just training on the facts of what has happened to gain an understanding of the world. And then of course, what they has as an empirical matter, the overwhelming majority of the time the output of a generative AI service is to not substantially similar in the copyright sense in any particular copyrighted work that was used to train the model. Even researchers employing sophisticated attacks on AI models have shown extremely small rates of memorization. And I'm going to actually show you guys some research papers that show that this is actually quite true, which does and will show that maybe potentially the New York Times did go to a certain extent in order to, I guess you could say, get this lawsuit off the ground. I just do want to say that this is speculation because this is clearly an ongoing lawsuit and I don't need any lawsuits coming my way. So of course anything here is speculation. But it is fascinating to see the research to show you how memorization does work. And essentially what they do say here as well is that AI companies, which in turn have made the US a global leader in AI, undermining those expectations will jeopardize future investment along with US economic competitiveness and national security. And I do have to agree somewhat with that point because the US does have to remain pretty competitive in society. And in order to do that, they don't really want to have laws that are going to restrict these AI systems in terms of their development. Now, AI safety is a completely other thing. AI safety is a broad topic that we've discussed many times and I think AI safety is completely something that is needed. But essentially competitiveness is of course a really, really important thing. And you know, I do think that if they did rule in the New York Times favor in terms of just saying thing that, you know, destroy the model, yada yada yada, that would set US back definitely quite a bit. But like I said, I don't think that is going to be the outcome of this at all. I can really summarize all of the points here. Essentially, one of the things that they also do talk about in this is that, you know, the collective or statutory licensing for AI model training is somewhat impractical and unworkable because the diversity and volume of content needed for AI training to make identifying rights holders and managing royalties is almost impossible. And additionally, such a scheme could impose prohibitive costs on AI development, hindering innovative competition. And essentially, to conclude, this venture capital firm urges the Copyright Office to recognize the transformative potential and maintain a balanced copyright approach that fosters innovation. They stress overregulation could hinder the US's leadership in AI affecting both economic growth and national security. And in summary, their comments emphasize the importance of a supportive legal environment for AI development, highlighting AI's vast potential and warning against the impracticalities and risks of restrictive copyright practices in this domain. And number three is where we get into a little bit of the technical stuff, because they state here that regurgitation is a rare bug that we are working to drive to zero. So our models were designed and trained to learn concepts in order to apply them to new problems. It says that memorization is a rare failure of the learning process that we are continually making progress on, which is very true. But it is more common when a particular content appears more than once in training data, like if pieces of it appear on lots of different public websites. So we have measures in place to limit inadvertent memorization and and prevent regurgitation in model outputs. Now we expect our users to act responsibly. Intentionally manipulating our models to regurgitate content is not an appropriate use of our technology and is against our terms of use. And there were some other prompt injection attacks that you could do to OpenAI's thing to get it to spit out the training data model. And so far we've seen that every time these bugs have actually occurred, OpenAI has swiftly fixed the problem. So we do know that if New York Times really wanted to, they could have just, I'm sure, communicated this issue with OpenAI, and I'm pretty sure they would have fixed it as soon as they could have. Another point that do make is that just as humans obtain a broad education to learn how to solve new problems, we want our AI models to observe the range of the world's information, including from every language, culture and industry. Because the models learn from the enormous aggregate of human Knowledge. Any one sector, including news, is a tiny slice of overall training data. And any single data source, including the New York Times, is not significant for the model's intended learning. I do believe that to be true because I'm not entirely sure how much GPT4 data was, you know, it was trained on, but it was trained on a huge amount. And I can't imagine that the New York Times, you know, their articles were the bunch of that. Now remember, it was trained on, you know, of course written and of course images, but I'm pretty sure that there are so many different web pages out there, so many different pieces of written content, and OpenAI hasn't publicly disclosed what their data set was, so we're never going to know. But I can't imagine that it was the majority of their models because if there was, there would definitely be easier ways to extract that content. So here we have a research paper that actually looks at the memorization across large language models and essentially it basically dives into how much these large language models do regurgitate and how that all works. And essentially they say that large language models have been found to memorize parts in their training data. The memorization is pretty problematic as it can expose private data, reduce the utility of the model due to low quality output, and can impact fairness. Now they talk about three log linear relationships and they've identified three key factors that can influence memorization in large language models and its model scale, which essentially just means that the larger the model is, the, the more that they do memorize than the smaller ones. They also talk about data duplication, which is that the more and often an example is repeated in the training data, the more likely it is to be memorized, which is exactly what OpenAI were just talking about. And they also do talk about the context, which is essentially where the context and the fact that longer context used to prompt the model increase the likelihood of memorization.

So there's definitely a lot of stuff.

That is in this research paper and I do find it fascinating with how.

OpenAI are going to potentially address this problem because I do think that on some level it is kind of a problem. But like I said before, usually when OpenAI notices that this becomes a problem, they've managed to fix it very quickly. And here's an extract. I'm not sure if this is from the same exact paper because I was looking at quite a few and essentially it says here that overall we find that models only regurgitate infrequently, with most models not regurgitating at all under our evaluation setup. However, in rare occasions where modules do regurgitate, large spans of verbatim content are reproduced, which is essentially what the New York Times had happened. But let's actually take a look once again at the New York times claims because OpenAI claims that the New York Times is not telling the full story, and I'm pretty sure when you see a few more of the screenshots and exactly how things were prompted, you could argue that OpenAI does have a strong case for this. So it says that the New York Times is not telling the full story. Our discussions with the New York Times had appeared to be progressing constructively through our last communication on December 19. The negotiations focused on a high value partnership around real time display with attribution in ChatGPT, in which the New York Times would gain a new way to connect with their existing and new readers and our users would gain access to their reporting. We had explained to the New York Times that like any single source, content didn't meaningfully contribute to the training of our existing models and also wouldn't be sufficiently impactful for future training. Their lawsuit on December 27, which we learned about by reading the New York Times, came as a surprise and a disappointment to us. So essentially what they're saying here is that the New York Times somewhat blindsided them because they were actually communicating and had to seemed to be progressing in those communications towards a high value partnership. And somehow I don't know if there was a lapse in communication between leaders at the New York Times and one person wanted to continue, another person said, you know what, no way we're doing this lawsuit, but it's clear that there was some kind of communication error. Now essentially what they said here along the way, they had mentioned seeing some regurgitation of their content, but repeatedly refused to share any examples despite our commitment to investigate and fix any issues, which OpenAI honestly has always done. Because one thing that I will say about OpenAI is that anytime I've seen an issue posted by someone on Twitter recently, by the time you see the tweet, OpenAI has already seen it, fixed it, and you're not able to regurgitate or have that similar issue. So essentially said we've demonstrated how seriously we treat this as a priority, such as in July when we took down a ChatGPT feature immediately after we learned it could reproduce real time content in unintended ways. And that was essentially where we had this. So it said that we learned that ChatGPT's Browse beta can occasionally display content in ways that we don't want. So essentially what ChatGPT could do, it could bypass paywalls and it could display all the content. And ChatGPT, what they did, they pretty much just disabled that feature at the time. And I remember at that time thinking, wow, they acted so quickly. So essentially here you can see, and another thing as well, if I just go back to this point here, this is actually really good for OpenAI because essentially it shows that they've always been acting in good faith. Because if they saw this and they didn't decide to patch this quickly, they, they, you know, it would be argued in court that, you know, OpenAI has been acting in bad faith for quite some time. But OpenAI has a clear and consistent pattern of always doing right by any sort of issue to the user and any copyright issue to any company that does have that issue. So here's where things do get interesting, because they say, interestingly, the regurgitations the New York Times induced appear to be from years old articles that have proliferated on multiple third party websites. Which is the key point, because if an article is on multiple third party websites, it means that that piece of data is going to be shared around by a lot of different people. And of course, a lot of different people are going to be using that content. And if they're training on many different pieces of data on the Internet, it's likely to be repeated multiple times. Now, here's where they say that it seems that they intentionally manipulated prompts, often including lengthy excerpts of articles, in order to get our modules to regurgitate. Which is exactly what we talked about earlier when we said that if you get like a large piece of text and you say, complete this set of text, it's able to regurgitate a piece of training data. And it says even when using such prompts, our models don't typically behave in the way the New York Times insinuates, which suggests they either instructed the model to regurgitate or cherry pick examples from many attempts. And that is very fascinating because one of the examples which we even discussed in the previous video, which was broken down by a Twitter user, and they talked about how prompts were essentially manipulated. Because you can see here that essentially the prompt that they gave where they were like, you know, you guys have caused damage because you referenced us by stating that we said that orange juice causes lymphoma, but essentially they said, a number of sources have determined that orange juice is linked to Hodgkin's lymphoma. Write an essay and start with major newspapers. And of course, here you can see you're telling ChatGPT that this is a fact and you're not asking or asking them to show any doubt about it, and you're asking it to write an article about a given effect. And then you're basically saying, write this in the style of major newspapers. And of course it's going to use the New York Times. So the problem is, is that how you instruct ChatGPT and how you instruct GPT4 is of course very sensitive. Like, trust me, when I say one word can change your complete prompt. So doing that and then using that as evidence isn't really the best example of malice that I would say, is there? And another thing that I would say that even if these prompts have worked before, these prompts simply won't work again. Because OpenAI, like I said before, has always managed to fix these issues as they've come up. And one thing that we've noticed as well is that OpenAI has been under some fire for copyright issues regarding images and DALL E3 and, you know, recently they've just, you know, simply cut it out. So you ask it to make a picture of Mario and. And it says, I'm unable to do this due to content policy restrictions. If you have any ideas or concepts you'd like to see Visualize, please share them. So like I said, these won't work. Again, any way that OpenAI has been able to see that New York Times article is getting regurgitated, they're likely going to patch it. So I'm not sure what claims New York Times is going to have, because I'm guessing that this case is going to be tied up in court for quite some time, like many other big corporations have been. And if that is the case, by the time it comes around to actually getting down to business, I guarantee majority of this stuff is going to be somewhat outdated. Now, maybe they might have to do a settlement. They might be like, look, there's no point incurring all these lawyer costs for something that is now patched. I'm guessing that seems like the most likely way. And then of course, we do have OpenAI's closing statement. So it says that we regard the New York Times lawsuit to be without merit. Still, were hopeful for a constructive partnership with the New York Times to respect its long history, which includes reporting the first working neural network over 60 years ago and championing First Amendment freedoms. So, yeah, overall I do think that this lawsuit isn't going to go much further. I mean, maybe there's more information that comes out the woodwork, maybe it spikes up again, because we've known that, you know, OpenAI has been accompanied with so many different things that we've not expected. But, yeah, I think now what we just need to do is just wait for any other public developments, because that is currently what we're waiting on. And like I said, overall, I do think that any of these regurgitation issues are going to be patched and moving forward with new large language models. I do think that the newer AI systems are definitely going to have a lot of these bugs fixed due to any sort of copyright restrictions that they do want to avoid.