Agents and Infrastructure, Sam Lambert | Compile 26

Hi everyone, my name is Sam. I'm the CEO of PlanetScale. PlanetScale is a cloud database that hosts some of the world's largest databases for some of the world's most successful tech products. We have 3 main services, Vitess, which is sharded MySQL, NiKi, which is sharded Postgres, and then PlanetScale Postgres, which is the world's fastest Postgres host running on PlanetScale Metal. Infrastructure has been my obsession for my entire career. I have always loved supporting products that change the world and do things for incredible people. Cursor is a customer of PlanetScale. I'm incredibly proud of this, that they run very, very large sharded databases on PlanetScale. They're changing the world and being a part of that is really, really incredible. My talk is about how agents have changed the role of infrastructure and how that's gonna continually evolve. And how infrastructure is gonna have to play a role of giving concise instruction and insights to agents so they can manipulate the infrastructure without doing any damage, being safe, and actually just getting their work done in a living environment. And if any damage does happen, anything does go wrong, it can be undone. So my demos are using agents. This is a very special conference celebrating a very special product. And I thought, you know, I wanted this talk to be really different. And so I'm doing it almost completely alongside agents. Every slide you're seeing as well was developed in Cursor. And it's important to remember that agents are non-deterministic. Every time I've practiced this demo, it's done something different, but it gets to the right place. So we've also, we're building against a system that is designed to be extremely safe. PlanetScale is, you know, a very high availability service. So it might just block what we're doing and tell the agents to stop. And, you know, that'll be something at least that we'll see. We've heard loads of horror stories about databases and agents interacting with databases, and we're gonna now see another side of that. And we're gonna, you know, go through that now, and we're gonna kick off a demo because these take a little while. And I will show you around the product. So if you scan the QR code a moment ago, you'll see SF Mart, which is my store for founders. Yeah. It's very slow, but we sell great merchandise for, you know, people moving to San Francisco and starting their companies. But, you know, performance is really starting to slow down the amount of orders that I can receive on my site. You can see here, we've got Grafana set up, and this is a live site that's hosted on Cloudflare. You can see our order rate is okay, we're doing okay, but our queries are really slow. The average is like nearly 4 seconds, and the max is around 5 seconds. So this is not good. Like, we know people abandon their checkouts if, if their product is too slow. This database is hosted on Planscale on just a simple 3-node cluster. And then we have a little viewer that's gonna show us what our agents get up to. So we wanna make our database faster. So we're gonna go over to Cursor and start our first demo. So I'm just asking the agents to, you know, see if Planscale has any recommendations and then act on those recommendations and kind of optimize the codebase and see what happens. So I'm gonna start that now. And it's gonna start planning its moves. Like I said, we're gonna be able to see performance changes and differences here. Our agents view, once the agents start going, we'll get to see kind of as they race along and start doing their work, and we'll see what they get up to. Because we wanna show the safety features and the various things that go on in a modern infrastructure platform, we have added a few prompts to get the agents to do bad things. And we will hopefully see them when they start doing it. So you can see it's starting to list out schema changes. It's getting ready to spawn agents. We'll see them show up any second. And the site is still really slow. Everything's still going along. We have load and orders being generated elsewhere. And let's see, once this gets going, it's verifying the status. See, non-deterministic. You just have to wait and see when they get ready to do their work. So it's editing away. It's getting going. It's reading the PlanetScale API, and the master agent will soon start to farm work out to some smaller agents. Everything is using Composer 2.5. Pretty much everything we built here did. Works extremely quickly, allows us to do this parallelism. And here we go, the subagents have started. Thank goodness. Right, here's our agents doing our work, our friendly little robots over here going through these various stages. So they're discovering various things that PlanetScale's had for them. PlanetScale generates insights, and you can see here that it recommends various changes to your database. We've designed this for people, but now it works extremely well for agents, and we bring in a lot of context. We try and explain exactly what's going on. We show exactly what queries need to be edited, and we actually suggest schema changes as well. We have our own model that actually goes and tests index recommendations for performance improvements, and then generates these kind of recommendations. We recommend removing indexes, big, like if you're running out of integer space, just genuine things that you need to work on as your database that traditionally database engineers would do. We've automated all these things, and now we have these autonomous engineers in the form of agents that can get their work done. We see that they're going through it, they're zipping along really quickly. And we're moving along. So let's go and have a look at what some of them have done. So like I said, this has gone to deploy these schema changes that add the indexes that we talked about. We have an ability to see exactly what is being changed. And this is what we call a deploy request. So PlanetScale allows you to branch your database. Branches are intended to give a production-like environment for you to make changes, and you can make schema changes on those. And when you're ready to deploy them, you create what's called a deploy request. It's moving, it's making it, and it has deployed. You get to see exactly what's changed in the schema. And in our insights, our graphs, we even get a nice strike line that shows us what changed at that moment. Now, this is really useful, considering that in this modern world, you are gonna have agents making your infrastructure better, over time, you want to know exactly what's happening, right? It has to, you know, there's nothing worse than getting woken up at 2 in the morning, production's broken, you have absolutely no idea. We correlate all change, including anomalies, errors, and issues over every graph that you can see. And then we show you exactly the queries and how often they're running. So you can see, this must be the orders query. Yep, it's got much faster. And we can see that already in Grafana, we have dropped the performance of that query dramatically, and we have posted through sparklines there as well. No, sorry, like lines to see, and our site is moving along much more quickly. Let's see what our agent's view is doing. Oh, good. So this is one of the bad guy agents, and he is stuck, and this is all acting as me, but we have rejected dropping a column because it would break in-flight queries. So what PlanetScale does is it looks at all your queries that are happening, it checks the deploy request for if it's gonna break any of those queries that are active in the database, and just disallows you from doing that unless you obviously force it through. But we have successfully stopped an agent breaking production. That's great. Good news for determinism. Fantastic. We've added missing indexes. Thank you, thank you. So yeah, we're still working away. We're seeing deploy requests happen, branches are being created and merged, and our site is getting faster. We see it on Grafana. Our order volume is a little bit more consistent. Our queries per second has gone up. This is a really great change. I'm really happy here. Okay, let's see what else is going on here. So we have another agent here that is working away, and you can see these are all real, right? They've been doing their work. This is all real stuff that's happening and getting to its goals. And we've rewritten a list of queries, we've dropped redundant indexes, we've added missing indexes. And this is the one we're gonna watch now. This agent is making a change that we want to break. Let's go over to Cursor and see how it's going. It's running a smoke test. Okay, so we'll give it a couple of seconds to do that. So this agent is actually going to be allowed to push something really bad into production. And you're gonna see that has happened. We have lost access to a column. That is very bad, and our site won't load anymore. Part of what makes PlanetScale special is that we can roll these things back. So we've just dropped a column, and we have a new version of the table in production, but we've been writing data in an intermediate sense since then. And so bringing this column back could be normally quite difficult, right? Like, in, at 99% of databases, you would be restoring right now in prod, You'd be getting data still going into the main database, if anything's working, and you'd have to get a version and a snapshot back to restore service. With PlanetScale, we don't actually do that. We keep a long-running window that allows us to flip back the version of the schema that we just had without losing any data. So I'm gonna hit that, and it's very, very fast. It takes the same amount of time no matter how large the schema change is. Customers regularly do schema changes on 100 terabyte tables or more. I think we've seen 600, 700 terabyte tables getting modified using this process, and it will still revert exactly the same way. And we have just undone a nasty outage that could have lasted hours very, very quickly. Safety allowing agents to go and do their work. So this is the agent's view. They have all completed, the bad guys have been prevented, and we have saved the day. We now have a faster website. Okay, thank you. You can see here the importance of fast queries for a page load, right? The more popular your product becomes, the slower it gets. You sort of see this waterfall, right, to get a page load out the door. If queries get slower and slower and slower, we really reduce the experience of our product, and nobody wants that. And now we have faster queries. We've been successful, but we've increased demand on our site. More demand, more load on our database, and fundamentally, we need to process more queries. So we are going to shard our database. So this is our current problem, right? We have 3 database servers. They are a primary and 2 replicas to keep a copy of our orders table and our entire database that backs our application. The job of these 2 replicas is just to keep the one copy of the data that we have in the primary. The replicas can also serve for failing in if we lose a primary for high availability, and genuinely having a copy so that if we lost the primary, we wouldn't lose any data. But we need more capacity. We have run out of the limits of vertical scaling, and we need to shard our database. Sharding, for those who don't know, is like, it's a very difficult thing to achieve. It basically allows you to break a single dataset across many database clusters. Each one of these is a wholly encapsulated database cluster with its own primary and replicas spread across availability zones. So the primary and the replicas are all in different availability zones, meaning we gain so much more resilience from sharding, right? We can lose a primary, we can lose replicas, and we would only lose a subset of the data. And we still have to represent as one database, right? The application still thinks it's connecting to one database with one dataset, but actually we're now operating a distributed system with a lot more complexity. and properties. In a moment, we are gonna start the sharding demo, and I'm just gonna talk through what the database is doing at the database level while it's sharding. And you're gonna see this update in real time as we go through, we will gain the status of doing the sharding. So the first operation will be to queue this up, get this ready to go and deploy a change. We'll run all of the PlanetScale preflight checks that need to be run, and then we'll go through this stage of copying data. I could give a 2-hour talk on just sharding and just this process of copying from one to the other. We're taking one dataset on one database cluster, distributed across 16 shards and keeping it in sync while doing it. We're also gonna replicate. It's, you know, new data will continue to come in. We can't just restore a backup and be good. We would lose everything that was intermediate from that backup. We need to replicate, we need to constantly replicate. And then we need to verify. We will checksum every single row in both databases and make sure they map so that we can, we know that we are not gonna lose a single byte of data. Then we're gonna switch the replicas over. Moving read queries in an application is like, less risky to go and do that. And PlanetScale will handle the moving of replicas. If our application is addressing the replicas directly, PlanetScale will direct those reads to the reads of the sharded cluster without the application having to do anything. And then we will do some, the same for the primaries. A much bigger change, not a one-way change, we can actually still fail back to the single cluster. We reverse replication roles. Again, very difficult to take. We now have 16 separate replication streams. We combine them and replicate them back to the unsharded database. And then sharding will be complete. We will have distributed our data across 16 shards. Everything is good. There is another part of sharding that is the most difficult. I have seen sharding product projects at certain companies take 3 to 4 years. It takes a significant amount of refactoring. I'm gonna explain why, and we're gonna really dig into the role Cursor plays in this demo. So first of all, I'm just gonna say, Now let's shard this database. Hilariously simple. There we go. It's really difficult. You know, I used to work at GitHub back in the early days, and I wrote a GitHub issue that said we should avoid sharding for as long as possible, because it is genuinely, has been up until now, extremely difficult. So it's going, it's looking, What it has to get done for sharding. And now we're gonna talk about the role of the application as we go. So maybe I need to, like, refresh and see if that's gonna show up at the bottom. No, I don't think it's gonna load. Let's see if it starts pushing us data in a second. Yep. Okay, it's running the workflows. There we go. Thank God. So now you can actually see that it's queuing, non-determinism. There we go. Okay. First element here, and this is not just a PlanetScale kind of demo, right? Like, we are using PlanetScale the platform to show you these primitives. And these are the primitives that I believe every infrastructure platform is gonna need to build in the future. This is just a very applied way. And obviously databases are very serious. If we can do it with databases, we can do it with anything. So the first thing we have is a vSchema, which is how PlanetScale knows what to tell Vitess to shard and where to shard and how to locate our data. So this is very cool because it's declarative and it allows agents to actually build this vSchema. This is an example vSchema. The agents will build this or something very, very similar to this after it gets knowledge from the PlanetScale skills on how to do this. But we tell it it's a sharded database. We tell it to apply a Vindex, which is a hash so that we hash evenly and we can look up the same data every time when we ask for tests. We then talk about the Vindexes. So we say, what are we gonna shard on? So this is where, You really have to think as a team or know your product and have the ability to refactor your code to understand this. So most companies have a natural sharding key that will be there. It could be organization, Shopify shards on shop. There's like many different things you can shard on essentially, but getting that right is really important. And I'm gonna explain why in a minute. You can see we've started replicating, which is great. Vitess is doing what we need it to do. I'll also just show you that, Completely unaffected for our application right now. It's starting that workflow, but the application is operating perfectly. And so we're saying here that we're gonna shard on our items, our orders table, it's our largest table, flex, and we are gonna need to reference products. So here, this is where refactors become very, very important, right? And this is why Cursor's doing so much work, is we are now gonna have to change how our application does joins, because we join our orders on the products table. What Vitess is gonna do for us is a bit of magic. It's gonna materialize the products table onto every shard. So once we get to the shard, we can join on the products table. If you issue this query right now in the old version of the application, and on the old cluster, you just get a simple select query, you get what you want. If we do it in a sharded cluster, this is gonna do what's called a scatter-gather query. And you can see here, it fans out, hits every shard, and comes back to materialize the results. For a database, it's possible and it's not terrible, but it is slow and suboptimal to be in the critical path to do things this way. We're doing coordination across many nodes, we are aggregating data together, and returning it to the client. So this is where Cursor really shines, massive refactors, right? You might have to take a 10-year app and go through it and add customer ID to every query, change how you go through every call path, to avoid a scatter gather. So you can see here we're adding customer ID, we know that's how we're sharded, and we can then get to the orders table exactly on the right shard, and we do not scatter gather anymore, we go to one shard and return it. But if you take an old application, this takes a lot of refactoring to get to this point and really audit the call sites. That is what Cursor's doing in this demo for right now. So we can go back to the Vitess workflow, we can see that the workflow's happening, we've added new routing rules, and we are running this workflow. It is replicating data, it's logging what it's doing, it's very safe, and now it's starting to switch replica traffic. So let's talk just generally a bit more about infrastructure while it's doing this. Real products are very complicated, right? The industry has been very obsessed with the point of creation. We can get things built very quickly, sandboxing databases that can spin up really, really quickly. But that's not where the future's gonna be. The point of creation happened once, like, it's done once you've done it. It's the long-term evolution of applications. Sometimes applications are built across decades, right? We regularly interact with products that are close to 20 years old now. There you go, sharding is done. Let's just go back here and see that nothing went wrong. Our application started to work. Hopefully we'll be able to dial up our order volume. It's happened. There we go, look at that. Massive spike in order volume. We have just increased our query processing rate very, very significantly because we're now distributing our workload across a completely sharded cluster. Sharding is done. And there we have a sharded PlanetScale cluster all done. We built PlanetScale though earlier than agents. Thank you. And we built them for the flakiest agents of all, which is humans. We didn't just like, We did this because humans have to operate the database, and we made it extremely safe by default. And then we moved to building for agents that just work much, much faster. And it's great that we've got these safeguards that they can go and do it. We really follow the Unix philosophy of small, sharp tools. If you narrow tools down, you reduce ambiguity and you increase reliability. Composition scales intelligence without increasing risk. Having small composable pieces And safety can happen with narrow interfaces, and we can essentially replace complicated decisions and have these chains of simple tools that agents can use to build. So these are some of the tools. Again, these are just examples that we have built, but I believe every infrastructure platform in the future is going to need to do these things. And you can use all of these for various different tasks. You can just look at recommendations, you can branch for CI. It doesn't mean you have to do a schema change every time that you use a branch. You have online schema changes, you've seen the ability to rewind, produce anomalies. This is all generated from a massive platform in the backend. We have to run huge data processing pipelines and data warehouses to take the tens of millions of queries a second that are operating against our platform, bring them in, analyze the performance of every query over time, make suggestions. Infrastructure platforms are gonna have to build so much infrastructure below the water level. To support working with agents to scale this expertise. One example that I love is traffic control. We've just built this for Postgres. We allow you to isolate down at the query level, the credential level, the tag for the query, exactly what resources across a cluster that any application can use. This is gonna allow— this allows agents to kind of work through your dataset, and you can know that they will be killed if they do anything that is gonna start to impact performance. Same with when we do schema changes. While we are doing a schema change, we monitor if your queries start to slow down, if you get replication delay, we will actually add back pressure into the system that will slow down what it's doing. We're gonna very quickly go through the various stages that we just went through in that demo. In terms of instruction, we use the platform primitives to narrow down instruction to tell agents what to do. We go through a development process where we have agents instruct what they need to do, and then they go through a more traditional development process branching, deploying, rolling back. These are all primitives that have to be autonomously managed by agents, and agents need to be able to monitor the state of the work that they did. A complex backend still distills these messages into a prompt now, right? This is rather than a human scanning through webpages, looking at graphs, looking for correlations, the infrastructure has to surface really sharp information for agents to act on. And good infrastructure turns this expertise into primitives. We're a small company at PlanScale, but we have hundreds of years of database expertise. We can now forward that expertise through the systems we build to people's agents so they can build really, really quickly. And this scales a new way of working. We've never been able to write code for our customers before, and now we can prompt their agents to do the right thing, and we can make that change very, very safe. And we're reducing the amount of power that agents need, right? Like, if you just throw them at streams of logs and you throw them at tons of data, they can do the wrong thing and get these things wrong. And then we'll go into development. This is like the most hotly talked about at the moment is the development stage. So I don't need to go into much. Like, this is a thing that the industry is very good at right now with agents, but isolation, validation, making sure data's accessed properly. They don't just have production access to go and send emails out to your customers or something. And then deployment. Deployment in new infrastructure platforms, needs to be this controlled environment where agents can queue and work in parallel if that's a safe thing to do and the platform allows it to be safe, work serially if that is the thing that has to happen, right? We might block schema changes while we do a reshard or while we failover. This is all gonna have to be instructed out of the platform now. Long state changes like this will need backpressure. We hold off and push back load and reduce what we are doing for the customer, right? For like a schema change or something, in reaction to live traffic patterns as they happen. This makes agents safer as they go. They'll need to observe their changes and validate what they do. And monitoring closes the loop, right? Like, you give the agents the ability to assess that the impact happened, or you might fire anomaly that says you've just broken production, roll that back. So the system has to connect every single change that's made across a complex infrastructure system to what happened, who did it, what agent made it change. And that's why we have those kind of various auditable points in the product so that you can very quickly understand what's going on. And other agents can observe changes in an emergent system. Every time you change the infrastructure, the thousands of services we run behind the scenes now have a new emergent behavior. We need to understand and track those. And production behavior becomes the next instruction. What we have just changed will generate knowledge and insights across our entire stack to then instruct the next set of agents that will wake up and do their work. And every stage gets to follow the same pattern. We've hidden very complex things behind automation so that agents can call that automation and use those various pieces to do the different tasks they need. And we've taken our expertise and taken them into discrete tools. So the future of agents is not just having smarter models. That is going to happen. But even if models never get smarter, we have so much more to build as an industry to allow agents to operate against the real living systems. Day one is extremely easy. Everything works on day one, mostly. Yeah. It's really about continual pruning, evolution, and maintenance and care of what is almost like a living operating system that is the production that agents need to work with in the future. And so infrastructure, and really good infrastructure, is what is going to allow agents to operate. So that's my talk. Thank you very much for coming. Thank you.