Don't Build Agents, Build Environments Instead

Today we're going to talk about how to use Modal as an agent sandbox, which is increasingly like really important if you're building any kind of agentic workflow. And because of those things I mentioned, the fact that developer experience is so good, it's so fast, it's easy to use, it makes a really good sandbox. And you can use it for more than sandboxes. It just happens to be also a really good sandbox. Adam's here to talk about it, to show you how you can use Modal. So I'll just kick it off and hand it over to him.

Super excited to, to talk a little bit about modal and really talk a little bit about the difficulty in building a system of letting agents build software for you. This presentation is really going to be 2 parts, one of which is going to be talking about how does somebody design a system for agents to be able to reliably use their environments to actually go and build software. Then towards the end, we'll actually go and talk a little bit about what's the developer experience of building with this, and so we'll get into some code. I want to talk about the sort of real difficulty in building agents that can actually produce software. So before we get into this, my name is Adam. I work at a company called Modal. I've been at Modal for about 2 months now. I'm probably better known for the work that I've done as a maintainer of Prefect and at FastMCP. So if you built an MCP server, have probably built it on FastMCP before. This talk is about how there's kind of a, tends to be a lot of focus on agents where the environments that those agents live in tend to be a more sort of rich and interesting problem space and arguably a more important one. Yeah. I wanted to call this talk Environmental Engineering 201, or the kind of the, you know, maybe the spicier take is that the hard part of the background agents is always the background part and not the agent part. I called it Don't Build Agents, Build Environments Instead, really as an homage to one of my favorite talks that I would encourage everybody to go and watch. And that talk was by Barry and Mahesh at Anthropic that gave a similar talk called Don't Build Agents, Build Skills Instead. And this talk done every single number you can imagine on YouTube. But the central argument of their talk, and why do I appreciate this framing so much? Agents have really converged on a pretty universal design. I'm sure that maybe this is a controversial take depending on the talks that have led up into this. As I'm sure as folks here that have built with agents have felt, as you use something like the Codex harness or maybe Claude Code or use Pi or Hermes, you'll notice that a lot of these agents, you know, have some pretty common primitives to them. It's how do I do compaction when a conversation gets too long? Related, how do I do memory? But a lot of their tools are pretty much can feel the same. It's. You know, how do I go and execute bash commands? You know, how do I go read files, write files, edit files? And so the argument that Anthropic made, the knowledge cutoff of October 2025, was we've really coalesced on a common agent design. And so really the hardest part that we keep coming across is like how to imbue these agents with expertise. And their argument at the time was that skills were the best vehicle for imbuing an agent with that expertise. And so that sort of explains why, you know, as you see a lot of folks design agents, a lot of it is like, yeah, yeah, yeah, go buy whatever the agent harness is. The recipe for designing an agent always starts with, you know, go pick a harness and then let's go obsess over the context that we go and give that agent. And so there are obvious corner cases here that, that we could obsess over, but I would say that broadly, a lot of their predictions about businesses should obsess over providing context to agents and maybe not redesigning every harness has largely borne out. So this talk as an homage is I have the same feeling that's sort of borne out by working with dozens of companies who are building their own background agents, that background agents are similarly converging on a pretty universal system architecture. And so this is kind of one layer up from a harness, which is harness is really like, what is the individual agent doing? And as you're trying to design a system where those agents operate, they all have a pretty similar architecture to them. And so what are the hardest and most reinvented parts here? What are the things that we really need to be obsessing over? Which is really about, you know, how do you go and build what I call dev boxes for agents, and then how do you think about orchestrating them as they complete their work. And so this is, of course, you know, knowledge cutoff of the last few weeks. And so I imagine that this will be, there's some depreciating knowledge here, but I do expect this to continue to bear out. So I want to talk about these 2 problems. So this is going to be a bit of a systems talk, and then we're going to go into like, what's the actual code of writing with Modal actually look like? So to sort of appreciate this, I wanted to give an abridged history of background agents. I'm going to skip over the creation, which starts with Devin. I feel like Devin was just years ahead of all of this, and so I'm going to skip past this history and I'm going to skip towards maybe beginning of 2025. And so I'm sure what a lot of folks in this room have felt pretty similarly. Maybe you can imagine yourself back in the day where a lot of this started with maybe Goose by Block was actually earlier than Claude Code, but it really started off with all of us kind of pair programming with these harnesses because they weren't particularly smart, but it was still magic enough to where you could actually, it was a step up from tab complete from like the IDEs of the day. With the release of Sonnet 4 or 5, I think we all sort of collectively felt like these things were good enough to walk away from, and then now that they were good enough to walk away from, there was this collective desire of, well, great, now these things can execute asynchronously in the background. They were all sharing me and my laptop. And so now I have to figure out how to check back in with them, how to keep myself in the loop when necessary. And I have to figure out to give them their own machines. So we're going to talk about why giving them their own machines tends to be a kind of a hard problem in practice. It sounds very straightforward, but it tends to be hard in practice. So spent a lot of time to actually background them. This is where you started seeing You know, managed offerings from things like Claude Code or Codex that were basically like, you were using Claude Code in your terminal. Now you can like click around and you can try and get approximately a Claude Code experience in a UI. I would say that where I think we really saw the real success of background agents is through folks like Ramp, Stripe, WorkOS, and others who really kind of like led open architectures of how folks, of how they built their own background agents. And so I want to talk a little bit about what's in common between those architectures and where some of the actual difficulties go so that if you're somebody today who's using Claude code on a laptop, you know, how do you get that off your laptop? And then, you know, how do you get many of these things to do work on your behalf in parallel? So there's sort of like 2 real challenges here, and I'm going to explain some of the jargon here in a second. But I'd say that there's 2 real challenges. And one of them is designing a system that generates dev boxes on demand. I know that a lot of the setup for, for this talk is around sandboxes, and I'm using the word dev box here pretty intentionally, where You know, a sandbox tends to be something that's pretty bare. It is like, let me go spin up a Linux machine on demand that has pretty much nothing attached to it. I'm going to go use that to run untrusted code. And where I want to upgrade our terminology here to Dev Boxes is around like, what do I do when I want to run or build untrusted software? And so, you know, a sandbox is if I give an agent a task to go, go build me a a frontend, like, you know, your Hello World Next.js app, if I give it a sandbox that doesn't have any like ports open on it or anything like this, it's gonna be unable to like actually up its own server. If it doesn't actually have access to, you know, CLI tools to take screenshots, it's not gonna be able to take screenshots of that app. It's not gonna be able to see its own progress, like visually inspect its own work. And so we can see that there's a lot of things you're not able to do just on a bare Linux machine. As you try to give agents more complex tasks that aren't just software engineering tasks, maybe they start resembling data or machine learning tasks. In the machine learning case, now you're going to want it to like, I've got a model that's not performing very well. I might want to retrain this. Maybe I'm serving a model and I'm noticing that I'm not getting very good throughput. So now I actually want to like go inspect it and profile it on a GPU. If I go give it a sandbox that doesn't have access to that hardware, again, now I'm dooming an agent to go try to solve a problem without a machine that's not prepped to actually go and solve it. And so designing a system that actually generates these dev boxes on demand in a way that doesn't make you want to tear your hair out as an end user tends to be a really hard system problem. And the second piece, which is get into of where's kind of that line between your control plane and your data plane. I think that this second point was slightly more controversial. I would say the last 72 hours of Twitter makes me feel like this is now kind of a vanilla point, but we'll get into it nonetheless. Designing a system that generates these dev boxes or sandboxes on demand, I think that it's fair for folks to point out that this isn't a particularly new problem. If you've been doing test-driven development for a while, the idea of generating machines to go stand up, you know, environments, that feels like a thing that we've been working on for a long time, right? Like you have your GitHub runners, you're going upping your Docker Compose, and then you're actually running tests on that. And so I think it's a fair criticism. I'll just say that it's, so it's not necessarily a new problem. It's just something that requires a new solution because we have a lot of different constraints. And getting this to work for a whole new set of constraints without even that bare problem solved sort of just throws us back into the deep end. And so I would say like the first reason why that not be obvious about why this is a harder problem is I think we like to think a lot about CI/CD in terms of jobs. You know, we have this really happy path of first it's going to clone, it's going to assume that that is the state of the world, that that repo is exactly what you want to test against, which is fine. And then it's going to run your test, your deterministic test, start to end. And then it's going to tell you whether or not things succeeded or failed. For Dev Boxes, for agents, it's a totally different story where you have to go spin up a machine and then it's going to go do work for a while. It's going to come back to you and ask you for input. By the time that you respond to it, like somebody asks you for feedback on a PR and you don't get to it for a few days, we're not going to keep a machine running for a few days. And so now we have this notion of like, how do we have long-running or resumable sessions that need to potentially even outlive the runtime of the box itself? And so when you start thinking about sessions over jobs, it starts meaning that you have to think about how to persist the state of a machine. And so that means thinking about, you know, how do you snapshot the memory of a process in a sandbox or a dev box so that, you know, when you resume it, you can resume it from its like mid-thought from an agent or maybe mid-process if your agent's executing something. Or maybe your system is best represented by its file system. And so it's great. It created a Next.js app. It compiled it. And then it asked me for feedback on it a few days later. So what I'm going to do is I'm going to try and snapshot that sandbox before I throw it away. And then when I interact with it next, I need to be able to essentially spin up a new machine, mount that file system in a way that it feels like I'm interacting with the thing that I put to sleep a few days ago.

The one thing I hate about CI/CD also is the feedback loops take so long. Let's say in the 4th step of your CI/CD, something, some dependency is missing. Then you like oftentimes fiddle with that step. You're like, oh, let me add it. And then you run the whole damn thing over again. It doesn't feel like this is a nightmare. You really want like a dev box where you can just do stuff. Like you might not know what the answer is perfectly.

So jobs versus sessions, I feel like, are maybe the first unintuitive thing when you're trying to think about like, you know, environmental engineering. Like, why is this a new problem for agents? Sometimes it feels like when we talk about things for agents, we feel like to rebrand or invent new problems just so that they feel new. So I'm just very cautious sometimes of being like, All right, from first principles, why is designing an environment for an agent actually distinct from a lot of the hard system engineering of designing environments for CI/CD? And I think one of this is the constraints of a job versus a session. The second one is really that, you know, kind of related to this, to the notion of a session, is the environment really is alive. So often if you have, you'll have an agent that checks out a repo, goes and does some work. By the time you come back to it, the underlying repo may have changed. And so like, how do you actually give an agent the ability to resync its environment? Maybe requires like FFmpeg to be in your environment and it wasn't when you started. And so there's this question of how do I programmatically spin up new environments? Or, you know, how do I shut down an environment, resume it with a snapshot? So now it's actually imbued with the true environment it needs to actually do its job. And it's not something that I just want to throw away and rerun. And I think that's important, which is in CI/CD, I think we tend to treat these things as cattle, if you'll sort of forgive the tired phrase. And so I think we're totally fine with like throwing away CI/CD CD runs and then resuming them later. Whereas an agent, it's really hard to throw away its progress because it could have been working for a day or so, and you don't want to throw away all that progress. The last bit is really that this is a totally different security boundary. And when you're running CI, it often feels a lot more like running a CI platform feels like running CI, which is, you know, when you run CI and you think about like the threat or security model, what are you really afraid of? I'm afraid of like Hamel getting access to my environment. And maybe trying to exfiltrate secrets. So this is like, we're obsessed over like secrets and environment variables and not injecting them or like, you know, not having them available in plain text so that if somebody gets access to the machine, maybe they could read them. But when you're running an environment for an agent, if it even has access to secrets in its environment variables, it can go, it can read them. And I'm not going to be like FUD or fearmongery around agents, but sometimes they can do dumb things. They can take a key. misunderstand what the key is about. Maybe they request, you know, to a service and they put it in the URL parameters so it gets logged. I'm kind of more in the house of like, these things just kind of happen by mistake more than there's some like rogue agent. I'm not a big FUD sower around this stuff, but I would just say that you fundamentally have a process that you can't trust, that you don't want to give access to your secrets. And so you end up having to design this security or isolation boundary a little bit different. And so those 3 really constraints. all kind of pale in comparison to the last one, which is that cold starts matter a ton. And to really feel this, let's talk about some of the worst designed agent platforms that I've worked with. Some agent platforms where maybe the UX could improve a little bit, which is if you look at something like hosted Claude Code, if you want to go set this up for the first time, you know, you'll fill in your traditional metadata. And then you get to this part where you have to go set up a setup script. And notice this is where like you're going to go install UV, you're going to install FFmpeg, you're going to go UV pip install your, you know, your pyproject.toml or what have you. And where this ends up really biting you is this is a design that almost treats it closer to CI than treating it as its own distinct problem. And as a result, whenever any new sandbox gets spun up, it's going to go run this setup script every time in a way that's not cached or snapshotted. And so any task that you give it, we've all felt the sort of pain of environments that take like 2 or 3 minutes or even 10 to spin up. And this is a tax that you pay every time. And Codex is no different. It's literally the exact same. It's like, you want to go set up a setup script and a cleanup script. And these paradigms that really try to treat environmental engineering the way that we treat CI, you feel it as a user because you're seeing like just totally intolerable cold starts. And so these are sort of bad for a whole ton of reasons. And it's not only just the cold starts. I'd say that environment drift ends up being kind of the worst part truly in practice. Like even if we all had time on our hands and we could just walk away, where this really ends up failing you as a user is that your environment starts to drift away from you. And so you'll set environment variables, maybe you'll set a setup script. But then your code changes and your code isn't really synced with this environment that you have saved in these remote agent platforms. And so really it ends up leading to a bad user experience. And so what did RAMP and others do? I would say that all of these have sort of coalesced upon a pretty common approach to environmental engineering, which is all of them. So this is sort of like kind of free alpha on designing these systems, which is all of them decided that really you can't separate the actual environment from the code itself. And so that means that every single repo that you have, that you're trying to have a coding agent go and operate on, you actually maintain per repo images actually in code. And so you take on the lifecycle of that image creation. A lot of them chose Modal, made a lot of optimizations to sort of make sure that if you build an image once, then every time that you rebuild it, we don't, you know, have to rebuild it from scratch. We do a lot of like making sure that a lot of the same files, if you've used a file before, we don't go redownload it. It's like a lot of lazy file system optimizations. And so you can define an image in code and every subsequent rebuild ends up being really fast. They rebake this every image on a 30-minute schedule, which means that whenever a new agent needs to go pick up a task, they basically have something that is at most 30 minutes different from what lives on main. Empirically, that ends up not being, you know, too bad. And so they end up just doing a pretty simple git sync on start of the sandbox. And so what that means is that, yeah, every new dev box gets mounted on a pretty warm image. And so every single agent gets spun up in less than a second and is able to go and get to work. The ability to— their sort of intentional design choice here is keeping secrets actually out of the environment where we talked about where an agent could unwittingly access them and unwittingly exfil them. We're building actual like proxies or sidecars on top of that. And so your agent, you'll basically give it like, don't make a request to GitHub. If you make a request to GitHub, I'm going to proxy it to a sidecar. And then that's actually where I'm going to attach a secret. And so the untrusted agent with a mind of its own, not to anthropomorphize, is never really has access to the secrets itself. It can just submit requests that a more deterministic process can intercept and imbue with a secret. And so this design has helped Ramp scale to almost a million sandboxes. For Ramp Inspect, I think it's something on the order of like 70% of their PRs now are the result of a background agent. A lot of the design decisions that we made in order to enable like not just this, you know, less than a second boot on maybe 1 to 10 sandboxes, But how we were able to reliably do this for folks like Ramp at that million sandbox scale. What's the sort of result at the end of this is really the fundamental principle is they're able to eat these image builds asynchronously. So that 30-minute background process means that maybe if somebody added a pretty expensive dependency, that this gets built and cached asynchronously in the background, which means that a human never has to actually, you know, pay the piper for An expensive rebuild. It's all done sort of off the clock. So when you actually start interacting with your sandbox, it's snappy. It means that since they maintain it in code, they can tune those permissions reports on a per-repo basis so that your frontend gets maybe the 3000 port open. The one with your Postgres gets like 5173 or whatever it is open. And then also you get that setup version alongside your actual code. Okay. So this solves the stateless half of the problem, but I think it ignores— if I was listening to this talk, I'd be like, okay, that's cool. So like, that's maybe where an agent should be doing its work.

Yeah.

I'm glad that we've hyper-focused over, you know, how to create an environment that's like fast and maybe secure for an agent. But it doesn't really answer the question of like, where's the agent actually running at the end of the day? Like if I'm designing this system and I have a dev box, like what do I actually do with it? Am I shelling into the dev box or like, do I have a process that spins up open code inside of it? What does it actually look like when it's designed and sort of where is this headed? So to do this, I want to sort of talk about like, what has agent design looked for a long time? Maybe a year and a half ago sort of looked like this picture where you would have an agent in a process that was co-located with its tools. So maybe 2 years ago you were using something like LangChain, maybe you were using something like CrewAI, and you would go and write an agent. You would give it access to tools like file writing or web search, and you would just instruct your agent to like go do some analysis for you. And so why is this really bad in practice? Well, all of these things live in the same process or the same environment. So if you have a data analysis agent in LangChain 2 years ago or something like this, what would end up happening, you would end up having one tool would go awry in the sense of maybe try and run this in a machine that has 2 gigs of memory. You're trying to get it to open a 10 gigabyte CSV file into memory with Pandas or something like this. And then now suddenly this one aberrant tool that's running in the same process as your agent blows up. And now it's basically causes your, the process holding your agent to fail over. And as a result, this agent who may have been 10 minutes into its actual work starts to fail over and it like loses the state of what it was trying to do. So when you design agents like this and you try and you co-locate the risky stuff, not risky like FUD risky, but risky in terms of systems design. If you co-locate that with your agent, you've basically exposed the thing that's responsible for keeping track of what to do with the thing that it's supposed to execute. This is a problem, you know, sort of tale as old as time in systems design. And the way that we solve this is, you know, sort of canonical orchestration The way that you tend to do this is you'll end up wanting to separate out your, the thing that's responsible for holding state or the thing that's responsible for holding a plan. And you tend to want to actually separate this from the thing, the sort of long tail of risk of the things that it's trying to execute. For folks that were maybe came up from data engineering, this is why Airflow never ran stuff itself. It was kicking off Spark jobs because you didn't want to actually crash your scheduler. And so So this orchestration design of how do I actually keep my agent separated from the actual risky stuff that it's supposed to execute so that my agent never loses track of its state. And so this is kind of a lot of orchestration can be just really summed up by saying you want to put a fire door between the thing that you never want to fail and the things that have some risk of failure. So that way, when those tools try to execute something in an under-provisioned environment, your agent will just see that you had an error happen. It's not like your agent crashes. Because that tends to be just very expensive and something that you don't want to see. And so orchestration 101 really is, you want to keep an agent in your control plane, right? That's where all the planning is supposed to be. Your control plane is really where you want to keep things that, that you want to isolate from the risk of failure. And so what I see a lot of these agent platforms moving towards is you keep an agent in this control plane, you let it provision or manage dev boxes, and instead of putting that agent inside the sandbox, With tools to read, write, or bash. Instead, you give that agent in the control plane the ability to remotely execute work inside of that remote dev box. And so what this means is instead of saying, I'm gonna put open code in a sandbox and let it go nuts, I'm gonna spin up open code in one sandbox, but when it executes write, read, or bash, that's not actually writing files to the environment that my process is in. It's actually doing a remote call to a second sandbox. And so that way, if my agent ever corrupts the environment that it's executing tools in, it's never going to crash my agent. At worst, it's going to corrupt the throwaway environment in that data plane. And so if you're building this on your own, I would say the 3 biggest lessons of where to build and where to buy are going to be around really investing in this kind of dev box supply chain. That's not a marketing phrase. It's just the best way I can sort of explain it here. We've seen in like the releases of like Devin Outposts, you're going to see a lot of releases. In the next 2 or 3 weeks. So that's what I want to sort of talk about from a system design point of view and, and sort of really try to back up why, if you're building background agents, the hard part really is the background part. And so really invest in building those environments instead. We've got credible examples library of how to get started on Modal. And so maybe what I can do is share some of those materials along with this presentation. If you're interested in this, if you want to understand the experience of building sandboxes and building those dev boxes yourself, sharing some of those materials of how to get started Would really encourage folks to go check out modal.com. Modal does a bunch of things that aren't just sandboxes. If you want to deploy open source models, train them, really it's a pretty comprehensive infrastructure. So the experience of actually using a Modal sandbox is pretty straightforward. We do this in Python and JavaScript, and it's as straightforward as like, I'm going to go create a sandbox. And then if I want to go execute code into it, I can simply just call that sandbox and exec Python code straight into it. And then we actually sort of stream that exec back. So if you are trying to go launch Claude code or open code in a remote sandbox, you're not sort of waiting for it to go do all of its work and then come back and report it back to you. So you really get sort of the same native experience as if you were working with a local sandbox remotely.

Because y'all made it so nice to work with for humans, it's actually a really good— like, agents love it because like you get fast feedback, you get a lot of visibility, the docs are good. All that stuff combines. You don't really have to dig into the syntax I found, like, anymore. You just say, hey, like, I want to do this thing with modal and it tends to work.

A lot of folks, I think, as we discovered that, like, agents were going to start being the largest by volume users of our SDKs, I think there was this moment a few months ago of like, ah man, how are we gonna have to redesign everything so agents learn how to use this? And then it was the most, like, anticlimactic thing, which was like, oh, Oh, okay. They just need good docs and they need good typing. They need type hints. They need good error codes. And those are a lot of things that predates me at Modal, just to be clear. But it was something that Modal's invested in for years because it was always obsessing over like, how do I help a human understand what to do next? And a lot of that is, you know, how to help an agent understand what to do next. And so really encourage folks to go check out our docs. If you want to go actually background a single coding agent, if you want to build, we've got a reference example of like how to actually go build your own lovable. On Modal as well. And if depending on exactly like which is your preferred agent SDK, showing folks how to use it with LangGraph, how to build computer use agents, how to build code interpreters. And so— How do you see folks getting visibility into everything that's happening and knowing when to touch? You're giving such a great point of leverage. It depends on what you want visibility into. I know that's like a lot of folks invest in like classic kind of hotel style instrumentation just so they can understand like what their agents are even up to. I see folks reach for like like Braintrust, Logfire, LangFuse to just like understand what their agents are actually working on. And then, you know, we export a lot of those metrics so you can understand like traces around our infrastructure. So you can actually get a better view into like, all right, these agents in these environments were able to accomplish their tasks versus, you know, we had a few hundred agents runs fail because they were being summoned to complete a task in an environment that didn't actually have the underlying infrastructure or maybe like GPUs or something to accomplish it. So that's more from like an observability point of view of, you know, how to keep track of what these things are actually doing. Those platforms tend to be nice because they help you also understand like larger themes about how your agents are actually performing. But as far as like, what's the actual impact of these hundreds of agents all PRing like stuff against your repo? Like, I think a lot of folks still rely on kind of okay, mediocre, like GitHub.

Yeah.

I would say that it's a pretty underexplored problem area, to be honest. Like, I think a lot of folks are overinvested in observability for the one agent. And I think it's an open problem, but I feel like it's an underexplored problem of like, as a, as a system of agents itself, how do you instrument and observe the impact of, I think what is maybe being called like software factories this day? I think that's the marketing term for it.

Thank you. Appreciate it.