Lenny RachitskyHost of Lenny's Podcast | Author of Lenny's Newsletter

Catching "99% of attacks" isn't enough

23 Dec 2025 · 1 min

Watch on YouTube

0:34

A speaker (from a YouTube video by Lenny Rachitsky) argues that AI guardrails are ineffective because the attack surface for language models equals the number of possible prompts — an astronomically large space (e.g., 'one followed by a million zeros') — so claiming to catch 99% of attacks is misleading; even 1% of that space is still vast.

Chapters

AI guardrails do not work. What do I mean by that? The number of possible attacks against another LLM is equivalent to the number of possible prompts. For a model like GPT5, the number of possible attacks is one, followed by a million zeros. It's basically an infinite attack space. So when these guardrail providers say we catch everything, that's a complete lie. But most of them say we catch 99% of attacks. Okay, 99% of one, followed by a million zeros. There's just so many attacks left. There's still basically infinite attacks left.