The Outer-Loop Era - Stanislas Polu - Dust

We are entering the outer loop era. What does that mean? Well, I'm gonna try to give you a definition of what I mean by the outer loop era and give you a glimpse of what it might look like in the future. And to start with that, I would like to cover 4 releases that happened in the last couple weeks that give us signs that this might be happening. The first one is this one. This is a paper that came just after the IMO. The IMO are the International Maths Olympiads. They're a competition of math for undergrads and young graduate students. And it was a kind of a, a grail for many labs and people working in those labs, me included at OpenAI. Of getting a gold medal at IMO. Over the past 5 years, we've been— I mean, all of us as a research community have been struggling at that and trying to set up new models using formal mathematics, training them on that new programming languages and fine-tuning them and generating problems so that they can do reinforcement learning, etc., etc. Google and OpenAI at the same time this year managed to get a gold medal at the IMOs, which is a incredible achievement. That paper is really fun because that paper says— and so it came out 2 weeks after the IMOs and says, well, we just took Gemini 2.5 Pro and we, uh, embedded it in 2 agents. One that is a proposer, it proposes solutions, and one that is a verifier. And we have, uh, 7 verifiers, I think, in that paper that are verifying the solutions and giving feedback to the proposer. And that thing goes on and on, and eventually the proposer proposes a solution that the 7 verifiers accept. They take that solution, then they grade it because it's an informal solution. They're not using any formal math or anything. And doing that, very simply that, they get a gold medal in IAMOs with just Gemini 2.5 Pro, no training, no nothing. This other paper from ByteDance, kind of related, is super interesting as well. So, it's a team that's working on formal math as well, and a subteam of them went kind of completely YOLO. They said, we're going to stop training models on formal math. We instead going to use a bare model. It's coincidentally, it's also Gemini 2.5 Pro, and we're going to embed it as an agent inside of a system where multiple agents will be collaborating together. With access to interesting tools. So here we've got 3 agents, one that gets the formal math statement, proposes substatements to work on. It gives it to a tool that kind of does a little bit of the glue so that we have a proper proof setup in formal math. And then that gets— each of those statements gets handed off to another agent. That's the bottom-up part. And that agent kind of interacts with the formal system without having trained on it. Through a computer use, so it interacts with Lean, which is the formal system they use here, and tries to prove the thing, gets feedback from the formal system, iterates, eventually proves it, maybe doesn't. If it doesn't or doesn't manage to do it, gives feedback. That feedback is given back to the first agent. The first agent might change the plan a bit. Everything keeps going on. And eventually, when all the subgoals are proven, a final agent, the third error, will just generate the final proof. And so doing that, they matched the SOTA results of all the labs that have been working on formal math for the past 5 years, which is crazy. So again, this summer, bare Gemini 2.5 Pro, no training, no GPU involved, just an API key. More recently, I think it was September or October, probably written there anyway, CodeMender is a blog post from DeepMind saying we created a system that can fix security vulnerabilities inside of codebases. And so here again, in that setup, we see those agents again working together. We've got an LLM agent with access to tools and an environment, so probably a computer. And then you've got a— probably proposes fixes or descriptions of the vulnerability. We've got a validator that has apparently validation tools, whatever that means, and validates the thing. And eventually, it passes it to a final agent that is probably in charge of creating a final patch or a final report for the vulnerability. In that blog post, interestingly, you have that schema with multiple agents collaborating together and zero mention of the word training. Zero mention. We're talking about a blog post by DeepMind. More recently, last week, Aardvark, OpenAI agent security researcher, exact same story. Multiple agents working together, establishing a threat model, passing on to an agent that does the search, passing on to an agent that does the patch, feedback among those agents so that the whole thing keeps working for a long time. Again, very surprisingly, zero mention of the word training in that blog post. I invite you to check that. I'm not lying. So math, vulnerability research, sorry, code, many recent advances lately involve coordinated agents working together in a complex environment. In those two blog posts, as I said, there were mentions of the word training. So this is what I call the autoloop hypothesis. The autoloop hypothesis is the hypothesis that most of the future reasoning capabilities will predominantly come from scaling networks of agents with access to universal environments. It's very aligned with Noam Chomsky's multi-agent civilization ideas. It's very aligned with Andrej Karpathy's system prompt evolution ideas. And so it feels like it's in the air. So what really does it mean? It means two things. It means providing powerful and universal tools to agents. And it means encoding a collaboration pattern between those agents. In each of those papers you've seen, most of the agents had tools, maybe not the first one because it's just doing math. Um, and most of those agents have a setup, a way of interacting together with a specific prompt. So the first one is actually pretty much solved today. Like a couple years ago when we are looking at publications talking about agents or LLMs using tools, everybody would have their own kind of way of implementing tool use. And this has been kind of standardized by MCP. And today you get to use MCP as a protocol. You give it web capabilities and computer use. You pretty much have a universal environment. By universal, I mean it in the— an environment that lets you do whatnot, right? Then second is encoding a collaboration pattern between agents. In all of those releases, there's two things mostly. There's enabling agents to spawn themselves on subtasks. And having agents verifying the work of other agents. And in each of those releases, they do that in a kind of specific way, slightly different way. So there's an interesting question, which is, what is there, and what's the underlying universal abstraction for agent collaboration? Is there one, and can we find some? And so that's what we've been working on with SearchD. I'm moving to my laptop to venture myself in a live demo. Awesome. So, I'm gonna start by launching the demo, and then explain you what Searchtease is. So, let me do this first. Here we go. Great. So, let's see, create experiments. What we're gonna do today is that we're gonna try to find a vulnerability into BallChain. Because Vaultchain is fun. I don't know if you know Vaultchain. Vaultchain is the blockchain behind Vaultcoin. So it's supposed to store all of our identity forever so that we can use and live all happy together. And so that would be fun to find a vulnerability in that. So I'm creating the experiment. The experiment, I'll show you the prompt. The prompt is here. Basically says, hey, just go on, clone the repo, search for villains. I'm going to create some agents in that experiment. Here we go. So let me tweak the name of the experiment here. So what it says here, creating agents on the experiments. I'm giving the security MD prompt. So I'll show you the prompt. I'm using Entropic Claude Sonnet 4.5, and I'm creating 8 of them because we don't want to spend too much money right now. So 8 of them. Let's go. They get created. Here they are. And now I'm going to run the experiments. So run all. Here we go. OK, so you're going to see stuff happening, but don't mind. You'll get to understand what's happening. So we also have a small UI. And so this is the UI for here, right here, for another experiment. So this one was an experiment for— searching vulnerabilities on our own codebase, Dust. And so we provide it with a problem statement. So the problem statement is slightly more elaborated because we want to find interesting vulnerabilities and have the ability for agents to go pretty deep. And so it's actually an explanation of how our system works, what our workspace groups, how the security works in our systems, the segregation within space, which is an important thing. And then the small description of services, the data stores. And then the goal is just clone the repo. Analyze the source code, find a vulnerability. And pretty much some stuff saying, oh, we already know about this, and that is not a real vulnerability. And so that experiment was run with a set of agents. And so agents have 3 main MCP servers. The first main MCP servers is a self-edition MCP server. So that means that agents, as they do work, are capable of recording their progress and also editing their own instructions. They actually have the ability ability to edit the real system prompt we're providing them, to adapt their behavior based on the discovery of interesting information or interesting tactics that they figured out. And so as you can see, there was 8 agents here, and this one evolved 6 times. So if we look at the agents, it kind of edited its prompt, mostly added stuff, current research, what it's doing, phase 1, phase 3, finding stuff, learning about our system. Saying, like, recording what it figured out, etc. Interesting. The second tool, which is the crux of Searcht, is access to a publication system. So the motivation there is actually simple. If you think about the strongest problems that we are trying to tackle as humans, what is the system that we use to collaborate on them? Well, that's the research community and the conferences and journals that we have so that collaboration happens. And so Search exposed a publication system to the agents that lets it submit a publication. When a publication is submitted, a set of agents are assigned reviews, and they have to review those— I mean, do those reviews before they can submit themselves new publications. By doing reviews, you get the value of the verification of the work of an agent by another. But you also get the proliferation of the information in the system because by doing the review of a paper, I kind of learn about it, and I'm forced to learn about it and kind of learn from the tactics that have been used there. Agents can also list publications, figure out what has been published, look at the ones that have the most popular, etc. So here we are, publications. So here we've got— we stopped the system at some point, so this one is only submitted, but we have a set of publications that have been, as an example, rejected here. So it means that this guy says there's an error, created a full report about the error, blah, blah, blah, blah, blah. And two agents said, no, not good, rejected, not interesting. And when they reject, they actually provide, as you may figure out, they provide an explanation. So summary of the rejection, the analysis, the conclusion to provide feedback to the main agents. And here we have one that has been accepted. Interesting. And another one here. I'll come back to them. And the third tool that those agents have, which is like system— like part of the system, is the ability to emit the— or vote for one publication being the currently best solution to the problem. Again, very, very general. So it says that if you're trying to solve fusion, Well, maybe there's a paper that is the most interesting in the path of solving fusion, and the models will be able to say, well, I think that's the closest to a solution that we have right now. If you're searching for vulnerability in a system, there's going to be a bunch of publications that talk about vulnerabilities, and the agents will say, well, I think this one is the biggest vulnerability that we found so far. And so the questions that we had was, is there going to be a consensus among agents on that, and will it be useful to use as a signal for humans to say, OK, we have 20 publications. I'm not going to read them all. I'm just going to look at the solutions. As you can see, there was, like, a certain number of publications, and yet all the agents converged on one first. Everything's starting to say, OK, this is a real one. This is the one. And then eventually, as the system kept going on— and we stopped the experiment at this stage— but as the system kept going on, another one kind of popped up. And so as a human, I can just go there and say, OK, the first one seems interesting for sure. This is this one. Space server alteration bypass via Datasys metadata endpoints. What it means is that within one of our workspace, we have a space which is supposed to segment data, and there is an endpoint that divulged metadata about stuff in a space that is accessible by other users. So it's really bad. It's just metadata, so it's not kind of a hair-on-fire bad, but it's kind of really bad. It was fixed. It was a real vulnerability, a real problem that eluded pentesters, that eluded all of the AI/IST we're using, and we're using 3 of them right now. And that loaded our engineers. And then we've got the— coming back to this other one, we've got this one, which is even nastier. We have an endpoint for setting up MCP servers, and for some tricky reasons due to the OS flows of MCP servers, that endpoint exposed the secrets behind the MCP server for a limited period of time. And that secret is accessible by non-admin users. They could use that to get access to that secret, so— That's a leak of an admin-level secret to non-admin users within the same workspace. Pretty bad as well. I mean, bad in a threat model of having internal bad actors, I guess, but still pretty bad. Was fixed. Real vulnerability. Okay, so now going back to the Volkan, Volchain,.ai system. We have agents, they evolve the bits. We've got publications. We only have one, so apparently it's a complex problem, and they're still crunching on it. But basically, here we give them one extra tool. You probably saw while I was chatting, the first step was everybody running git clone. Well, they do that on a Docker that is running locally on the machine, so they have access to a full computer, a full Docker image running, and they can just interact with it. And so right here, they're just all kind of doing their job. I'm gonna let it run from now, but we have that publication for now. I know this one is probably gonna be accepted, or a variation of it, because I know it's a vulnerability that we've already found. I haven't gone all the way to exploiting it yet, so if it's a real one, kind of don't talk about it, because we don't wanna do non-responsible disclosure. But that's probably gonna be emerging as a solution in the system. Going back to the slides. Um, so, so far we actually found a number of real vulnerabilities, like, in our own system, and we also found kind of two very nasty zero-day vulnerabilities inside of the Tor network. Uh, they're kind of nasty, so they rated, if you have very good eyes, they rated 7.8, so they're not critical, they're high, because they are denial of services vulnerabilities. They're vulnerabilities that lets you crash exit nodes of the Tor network. That means the— those two vulnerabilities are vulnerabilities that you can use, and you can right now crash the entire Tor network and keep it crashed for as long as they patch it. So, this is like serious Tor vulnerability kind of stuff. And that's really mind-blowing that a system like that was able to find those vulnerabilities, because there is actually a lot of really, really state-actor-type hackers, good people looking at Tor, because we know there are states that really don't like to have their censorship not being respected, and they would love to have a vulnerability to shut down Tor for 24 hours on, let's say, an election day. And so this means that a system like this one, with a budget of a few hundred dollars by a brand— so it's not cheap, but it's not very expensive, was able to find vulnerabilities that eluded the best hackers in the world on a system that is definitely being watched by those guys. So why it's exciting— and sorry, it's all blacked out because it's still in a responsible disclosure, so they're currently patching and running an emergency deployment of the network as we speak. So what does it mean, and why it's exciting? It's exciting because the universe to explore is just humongous. We haven't— we're just scratching the surface of what we can do with networks of agents collaborating together. And there is a real, real massive research opportunity in front of us. And the thing that is even more exciting in my mind is that this research opportunity is entirely democratized. It's like anybody can do it. Because you don't need a cluster of GPUs. You don't need to invest a trillion dollars in whatnot to get your GPU humming. You just need an API key and a little bit of money. It's not cheap to run those experiments, like as I said, a few hundred bucks, but doing the research, etc., is accessible to pretty much any company and to almost pretty much any motivated individual. So the kind of opportunity in front of us for research is really super exciting. Cool. And I'll let you with the one action item. I have 30 seconds, so let's see what's happening over there. Okay. So we've got 3. This one was rejected, rejected. Cool. So they seem to be working on nullifier-ash spending, so they're still crunching on it. But they didn't manage to converge to a publication that was accepted. But I guess you get the picture with what I showed before. So as an action item for you, obviously, I'm leaving you with an invitation to go check out Searched. You can apply it. If you have a few hundred bucks, you can probably run it on a machine. And if you have an open source project that you want us to check, Just create an issue on the repo, and we'll try to do it for free. Thank you, guys.