A YouTube explainer arguing that Meta (Facebook/Meta Platforms) used pirated books from shadow libraries—principally Library Genesis (Libgen)—to train its Llama models after licensing talks proved too slow/expensive. The video describes engineers’ internal debates, tactical steps Meta took to hide the sourcing (IP masking, ISBN/copyright-tag removal, filters to prevent verbatim output), performance gains (~5% on a 16,000 yes/no prompt test ≈800 additional correct answers), and the company’s decision to accept legal risk and mount a fair-use defense against ensuing lawsuits from authors and publishers.
Questions this video answers
At least 82 terabytes.
Roughly 1 billion pages of text.
2008.
Over 7.5 million books and over 80 million academic texts.
Four-plus weeks ("four plus weeks").
Chapters
Someone with the initials MZETA signed off on downloading at least 82 terabytes of pirated books to train Llama, their AI model. That's the same as watching Netflix for three years straight, or roughly 1 billion pages of text. So why did they do it? In this video, I'll dive into the dodgy world of shadow libraries and AI training data, why one of the richest companies in the world felt they had no choice but to pirate books, how they intentionally covered their tracks, and most importantly, why Meta and everyone in the industry is going to get away with it. But first, let's take a look at what these shadow libraries are. It's like Pirate Bay, but for books sites like Anna's Archive Z Library and most important to the story library Genesis, or as it's more commonly known, Libgen. Libgen started in 2008 as a niche Russian piracy site, but since then it's grown massively. Today it hosts over 7 1/2 million books and over 80 million academic texts. Everything from Shakespeare's classics to Harry Potter to peer reviewed studies in top journals. In other words, Libgen's data set is a goldmine for training an AI model. Engineers at Meta made the case for downloading data from Libgen. One of them said, it's really important to get books ASAP because they're more useful than web data. This is because books are high quality content that have been edited by humans. They also cover a wide range of genres, languages and formats. But before we go full corporate piracy, Meta did try to go the legal route first. They actually reached out to publishers and asked about licensing books. But Meta wasn't happy with the replies. Internal comms show a senior manager saying, seems unreasonably expensive. Another worker added, it would be incredibly slow and take four plus weeks to deliver the data. Meta didn't have that sort of time. They were already behind other players like OpenAI and Google. In an increasingly crowded space, it became clear to them they were not going to catch up unless they took a shortcut. And after adding Libgen's dataset, Meta's internal testing showed a jump in performance of about 5%. The test was 16,000 yes, no prompts that the AI had to answer. That 5% lift might seem small, but it improved Llama's accuracy by roughly 800 more correct answers with no extra compute. This was the beginning of Meta's pirate first strategy for Llama. But first they needed sign off. So they sent it up the chain, where someone with the initials MZ signed off on the project, Meta clearly tried to cover their tracks. They masked their IP addresses, scrubbed ISBNs, and deleted copyright tags. They even added filters to stop Llama from reproducing content word for word, like pages from Game of Thrones. It was almost like a digital crime scene cleanup. However, throughout this process, there was some serious concern from engineers. One employee said, I don't feel comfortable pirating books on a corporate laptop. Another added, if there's media coverage suggesting we've used data we know is pirated, this may undermine our negotiating position with regulators. And even though the Llama team rated the legal risk of this project medium to high, Meta's leadership took the risk anyway. You can imagine their thinking going like staying ahead in the AI race was worth being sued. What's a few lawsuits compared to the money at stake? After all, the largest publishers earn less in a year than Meta does in a few weeks. How could they possibly take on the full force of Zuck's legal team? Meta. Meta. Buck. Face. Meta. Meta. Buck. Safe. Eventually, as predicted, a bunch of writers and publishers found out and sued Meta. Famous writers like George R.R. martin, Margaret Atwood, and Stephen King all called out Meta's pirating of their content. But as expected, Meta's leadership and legal team had a plan. Their argument goes like using publicly available books to train AI models is fair use because the models transform the original material into something new. Meta has even suggested that scraping data from places like Libgen benefits end users, so it should be legal. And here's what's interesting. It's not just Meta using the defense. OpenAI, Google, Microsoft, anthropic and Stability are all facing similar lawsuits, and all of them are leaning into the same fair use defense, using it like a get out of jail free card. A director of engineering at Meta explained the strategy. If we even license one single book, we won't be able to lean into our fair use strategy. So Meta had two choices and use the pirated books or let Llama fall behind. They made a calculated bet. The upside of better performance outweighed the risk of getting sued. And it fits Zut's famous motto, move fast and break things. Only this time, the thing being broken might be copyright law. And that forces a bigger question. Now that you know how the sausage is made, are you going to keep feeding the machine?
