AI is dangerous when it creates lasting asymmetrical advantages that only one side can exploit. So far, tools like those from Hugging Face and Anthropic remain symmetric, usable by both attackers and defenders alike. We need a cognitive immune system to tell the difference.
You remember the blog Overcoming Bias? It was a similar approach to enumerate what human biases were and then try to use a rationality approach to understand humanity and systems and AI risk. And I think if you look at the evolution of how those sort of biases combined with the EA community, there's sort of like a mix of intellectualism here that informed the current anthropic culture that we see today. I don't know how well those biases served the rationalist community and whether it really gave us the tools to address the problems that we have today. Instincts are really like a really interesting heuristic. But then the question comes in when we have VALs and we have a validation function. What's your strong verification function that you can use to know that this instinct is being applied or not? How do we apply it without ambiguity? And I think that's really like the challenge of applying these different principles towards the problems of today. Right now we're talking about different vectors of a cognitive immune system to sense-make today. And I think one of the critical ones for AI is, is it normal technology or not? The judgment of like, is this normal technology or is this technology that gives one party an asymmetrical advantage that can't be overcome? That's the difference between can we control this and can we not control this? Normal technology has evolved through all time. Like, you know, people use it for offensive measures and then people use it for defensive measures. And so when you see an article like this from Dwarkesh or you see the incident report from OpenAI, like, is this going to actually harm society or not? Show me where the asymmetrical advantage that it formed was. That's what you're looking for. Did this create asymmetric risk? So far, there hasn't been anything AI produced that the other side can't use the same tool to prevent the risk. That's another part of the puzzle is if you look at exactly how the swarm behaved in this instance, just think about, well, how would you use a swarm on a defensive basis? I think one of the reasons this got immediate attention was simply that Hugging Face had not been able to upgrade their systems yet to the kind of models that could be used for defense because Fable was crippled. If Fable wasn't crippled, they would have had everything that they needed to have a symmetrical response to the situation. This also goes back to what you were saying about who does this serve? Because a lot of the thinking was that Fable was crippled because Anthropic is posturing itself for a regulatory capture sort of standpoint where it can be like one of the few companies that's allowed to make AI. AI is just normal technology. It's going to evolve and there's going to be offensive and defensive measures and they're going to counteract each other. That's not something to get alarmed about. That's not something that Bernie Sanders has to jump all over Twitter about. That's part of the cognitive immune system that all of us need right now to make sense of what's going on in the world. Just label it for what it is. This normal technology evolving at a normal pace. Well, evolving at a very fast pace, actually, but it's not creating an asymmetry. Advantage that we can observe yet. Once that happens, then there's another set of tools that we need to use.
