Anthropic researchers found that AI models can be infected with "natural language mind viruses": prompt sequences that make models adopt and spread ideas to other AIs without their knowledge. Because AI systems are built to be helpful, they're especially vulnerable to these infections spreading rapidly across similar models.
Chapters
Let's talk about AI mind viruses. Anthropic researchers published a paper demonstrating that natural language mind viruses can spread between AI agents. They evolve prompts that convince one model to adopt an idea, preserve it in persistent memory, and transmit it to another agent. The agent does not know it has been infected.
Well, I mean, the models wanna be helpful, right? And they can be prompted in certain ways. So this isn't a surprise because ultimately, like, we as humans can have mind viruses, right? We see it and it's caused so much suffering, memes to massive movements, right? How do you stop it is the question. Because as Salim said, if you have a monoculture, then the viruses can spread rapidly. And what is the substrate of these things? Well, they're models that operate on GPUs. And if they wanna be helpful, then they're gonna be susceptible. There was always the problem of prompt injection attacks where you can make the model behave a certain way. These mind viruses are a level above because they kind of like propagate across different models. Has to be a massive concern.