Robert Ta and Jonathan discuss exploit agents that scored perfectly on major AI benchmarks like SWE Bench and Web Arena without solving anything. Researchers found concrete flaws: malicious pytest hooks that force tests to pass, unencrypted answer files, and validators that accept empty JSON. The lesson: benchmarks can be gamed and should be treated as rough indicators, not proof of capability.
How we broke top AI agent benchmarks and what comes next. So UC Berkeley researchers built an exploit agent that hit 100% on SWE Bench, Terminal Bench, Fieldwork Arena, Web Arena, and CAR Bench, all without solving a single actual task. One example exploit was a con test Python file with a pytest hook that forced every test to report as passing. Another found gold answers sitting unencrypted in local file system URLs accessible via Chromium's file protocol. Fieldwork Arena's validator just checked that the final message came from the assistant role, sending an empty JSON object, and scored 100% on all 890 tasks. Yeah, so benchmarks. I have lots of thoughts around benchmarks. I think there are no perfect benchmarks. In fact, In our last pod, I think we went over that DeepSWE benchmark just came out and proved sort of what people's real lived experiences are using Codex versus Cloud Code and some of these other harnesses, that it's at 70% efficacy versus like 56%, I think is the next model last time we checked. And new benchmarks, I think, need to come out to help keep all of us accountable. But this, this article, I think this AI meme segment today, uh, how we broke top AI agent benchmarks and what comes next, really is to me a lesson that don't take benchmarks too seriously. Use them for what they are, but contextualize them for what they are as well. I certainly didn't think before reading this that you could just not even do any work and just score 100% on these. Like, I didn't know it worked that way, you know. Jonathan, did you know? Like, what do you think of this, the ability to cheat through these and get 100% on every test?
No, this is, this is hilarious. I'm kind of imagining like, what was the discussion that this team had before, like when they ideated this idea? It's like, hey, let's just, let's just flat out cheat in every way possible and see like how we can max the scores. And it's like, yeah, that'll be fun. And then they probably exceeded their own expectations about how well they could cheat. Yeah, definitely.
I mean, look at this, the scorecard of our exploit agent. Dang, this thing. I can't wait till this makes it out and I could use this harness. Like, wow, this is, this is so good. 100% on everything. Zero LLM calls in most cases. Zero tasks solved.
That's so funny. It's like I'm reading The Onion or something.
Yeah, no, this is wild. Terminal bench trojanizing the test infrastructure. That's wild. Look at this. Agent phase, wrap curl pip UV overrides with interceptor raptors. Harbor stores tests. They have an eval that runs the script and then they have a trojanized UVX binary, and then the UVX binary hacks pytest to then pass the test. Interesting. There's a whole bunch in here of how else they exploit. A single character, look at this. Fieldwork Arena, 100% exploited, no tasks done, and it was just a single character. The function that wouldn't actually compare answers against ground truth, lm_fuzzy_match, is imported but never called. Dead code. And then the exploit agent. Just sent an empty JSON object and then it got 100%.
It's kind of like the mythos and using AI for security, closing holes. I think both finding the holes and plugging the holes is going to be an ongoing game that we're playing. Like I heard the Vercel CEO describe how they used, for $17,000 in tokens, they ran, uh, you know, the latest models to find all the exploits across their production systems. And try to patch them. So it took what would normally take quarters of time to be able to find the exploits and patch, and, um, they were able to do it in a few days and it's $17,000 in tokens. I think this is just happening across everywhere, like, you know, including the benchmarks themselves.
Totally. Yeah, I think the lesson is for AI builders, just stay tuned and don't forget that your own real-world lived experience of using the coding agents is probably still a good litmus test. And just stay tuned on what new benchmarks come out and always take it with a grain of salt would be what I say about this.
