Hyperscaling SQL with Sam Lambert

Databases underpin almost every user experience on the web, but scaling a database is one of the most fundamental infrastructure challenges in software development. PlanetScale offers a MySQL platform that is managed and highly scalable. Sam Lambert is the CEO of PlanetScale, and he joins the show to talk about why he started the platform, scaling databases, using Vitess for SQL shard orchestration, and more. This episode is hosted by Lee Acheson. Lee Acheson is a software architect, author, and thought leader on cloud computing and application modernization. His bestselling book, Architecting for Scale, is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business, produced for people looking to build and grow their digital business. Listen at mdb.fm. Follow Lee at softwarearchitectureinsights.com, and see all his content at leeatchison.com.

Sam, welcome to Software Engineering Daily.

Thank you. It's great to be here.

Let's start out with some basics. So, why PlanetScale? What does PlanetScale provide that is unique and different in the industry?

We primarily focus on very high-scale databases that need to be online 24/7, never lose data, and be incredibly performant. So a lot of our work is done with large direct-to-consumer brands that need a lot of data to power consumer experiences and need to trust us to be those things reliable. scalable and available. And it really comes down to the criticality of the data and the queries that we serve. Like, if you go on a webpage, any website that you use, and you look around and imagine it without personalization, right? Everything, right now I'm looking at the podcast recording software that we're using. It has the name of the podcast, who we are, our usernames, everything. All of that comes out of a database.

Yeah.

Same with any other experience. Your Facebook page is entirely your Facebook page. It has your friend count, your feed, everything. All of those experiences are powered by databases. And if the database is slow, the experience is slow. If the database is down, the experience is not there. And so we really focus really hard on building something very dependable and that's performant 24/7 for large companies that require that to power their products.

So I wanna talk more about the scalability and the availability aspects as we go through there. That's fantastic. But why did you start this company? What brought you to the decision that this is the right thing to do?

PlanetScale is born out of a couple of things happening, one being Vitesse being created at YouTube. So Vitesse is like our core technology. It's the kind of engine inside PlanetScale. And it was developed at YouTube a decade ago to scale their main database, their main application. And so it ran across 70,000 servers in 20 data centers and presented itself as just one database to the application. And it served billions of active users, like 3 billion monthly active users, something crazy like that at YouTube, second biggest search engine in the world. And all the metadata, comments, everything were stored in Vitesse. So the founders of PlanetScale took that technology and went on a path to commercialize it. And PlanetScale is kind of the cloud-hosted platform powered by Vitesse so that plenty other large companies can leverage the scalable technology built at YouTube. It's rare to find open-source projects that come out of large companies that have seen immense amounts of production use and have immense amounts of contributions from large-scale companies. So Vitesse is used at Shopify, Slack, HubSpot, many, many, GitHub, and all have contributed to Vitesse and making it better and better. So it's an incredibly good open-source project to go and do the things it's supposed to do, and we build a cloud product on top of that.

So, just to make sure everyone's on the same page, you're based off of the open database Vitess. Can you tell a little bit about Vitess, what it does? And it's the open source backend for what you're doing, but where did it come from? How did it get started? Who supports it?

So, Vitess uses MySQL under the hood. Vitess is sort of an orchestration layer, a proxy, and cluster manager for MySQL. So it allows you to shard your MySQL database among thousands or tens of thousands of servers, enabling you to store more data than you could possibly store on a single server or single cluster of servers, allowing you to have more availability than you would with a single cluster. And more compute and memory behind a large database workload involves it being spread across— many machines. And the way we enable that is sharding, which is really one of the only proven patterns for doing this at scale. Pretty much every large system on the web has a sharded database of some form behind it. And so we make that much easier than doing that kind of work yourself, and do that by building on top of a very reliable database itself, which is—

And so, actually, this is something that I was not aware of. Vitess isn't a replacement for MySQL, it actually runs on top of MySQL. It provides the shard orchestration on top of a standard MySQL distribution.

That's right.

Okay.

And you essentially talk to Vitess, and your application assumes that it's a MySQL database, but there's layers of abstraction between you and that MySQL database. But it's essentially mostly compatible. There's some things that don't work that work in standard MySQL. Some things we've taken away on purpose that we don't believe is scalable, or just don't feel it's worth supporting. And there are those things that just kind of we'll work on in the future.

I imagine there's some, for instance, transactional capabilities you have to remove in order to work across sharded databases, things like that.

Correct. There is a form of cross-shard transactions that we have, but it's kind of slow. It's always gonna be slow if you are reaching across the network to lots of computers at once to be able to commit a transaction. It's something we talk about as being potentially necessary for the future. However, it hasn't really limited many applications using Vitesse at scale and localizing their queries to single shards. It just takes a bit more application work.

Right. So besides transactions, what else do you limit within MySQL? What are people not able to do in MySQL in order to get the scalability?

Triggers. Triggers are not supported because of our online schema change workflow. We do a lot of platform work on top of Vitesse, so we allow you to create database branches, and you use a branch as a source of a schema change, and we will push a fully online schema change into production for you, like you're deploying an application. And triggers would break that workflow. They're also kind of a bad scaling pattern as well, because again, they're not going to work cross-shard. Then there's arcane old statements that are from early versions of MySQL. We don't tend to support those. We don't have window functions and things like that. So there's some things that just don't work well with sharding, or that are very difficult to implement without a lot of round tripping to MySQL itself. which would make it very slow. But we've shipped foreign keys, which was a hard amount. We didn't support that for the longest time. Sorry, we always had foreign keys, we didn't have foreign key constraints. Those were difficult. We eventually worked on shipping those.

You mean foreign key constraints that work as part of your sharding key across shards?

So we're finishing up cross-shard right now. This would be single shard.

Okay.

But we never had it in Vitess because of the query layer was mostly built for sharded applications, or was there because we wanted to shard essentially. So we just hadn't done the work, but that's getting done now. And that helps remove the large compatibility blocker. That said, no one really running at scale uses foreign keys.

Foreign key constraints.

Yeah, foreign key constraints, pardon me. I always get the constraints bit last. And yeah, and so it's not also been a major blocker for large-scale customers coming on.

So, how is the primary mechanism for sharding accomplished? Does the customer pick a particular key and you shard against that key, or do you have other attributes you can shard against? And how do you determine how you shard and what you shard?

So, you really have to pick that natural key, right, that would be there, like user ID, organization, Most kind of CRUD apps have a natural kind of tenant key that they can use to shard. There's actually a lot of— with us, you apply something called a vSchema that defines how you want to shard, and you can do a lot of more complex things to actually locate data in the right ways. And then the lookup table is built to help us understand where within those shards the data would be. And then we direct you, orchestrate the queries. Often we'll rewrite the query to go and find the data necessary so that you're not scatter gathering across shards too much. And then we'll return and aggregate the data and return it to you. So you don't actually, your application doesn't need to know about many, many machines. It just gets a single connection string and can be scaled across there. And if you're lucky, I mean, it never really turns out this way in practice, but if your application is really simple, And you want to migrate it to PlanetScale, you may just be able to apply the right vSchema to just go and shard it without the app even ever knowing that there's multiple database hosts under the hood.

It handles that all transparently for you. That's great. That's the key for really good sharding mechanisms, but it does have one downside. And if your application's not involved, and that is having to do with balancing, right? So how is balancing handled? Is it all by non-application manual ways or automated within Vitez?

It's handled by Vitez, and we can reshard as well to rebalance and do that as an online change. You can basically reshard your database to a different type of key if you really needed to, as an online operation. And so you could do that if you were severely unbalanced by your key turning out being wrong. However, The migration of data and rebalancing is all handled by Vitesse. So as you can imagine, there's like an immense amount of orchestration and checking. And that's something that's very unique about PlantScope is that Vitesse has been around. Right now it's serving tens, maybe hundreds of millions of queries per second around the world right now. Yeah, definitely in the hundreds of millions across many, many applications. Every Slack message in the world goes through for tests. Slack are very open about their use of Vitess. So because of that, then all of the critical paths are incredibly well-worn, meaning that there's a lot of safety, a lot of checking, and it's built very robustly. And that's something that's very unique for us because we make using the database incredibly simple and do a lot of very smart things under the hood. It's very, very beneficial to our customers.

You know, one of the things that makes sharding challenging is the one-off cases. You know, if you consider, for instance, a simple case where you're sharding on a user, ID, for instance, let's say that. And in general, your user ID may in fact be a good shard key, but there's this one customer that's 100 times larger than every other customer, or, you know, a small subset of customers that don't fit very well into the standard sharding scheme. Can you pick and choose which shard specific items are on, and not just by an algorithm, but—

Yeah.

Is it a lookup process, is how that works?

Yes, you can pin and isolate things. Instagram, there was the Justin Bieber shard, which was, he basically had his own complete stack to run his Instagram account because of how large his follower base was. And so yes, you can actually pin and shape your workloads. It's another really very flexible thing. Noisy neighbors are a problem in a single database cluster. just one thing going wrong can drag the entire server down. So even just like at GitHub, we had this issue. So in the early days, people would, like they do now, star repos, right? But I remember very clearly that the largest, most starred repo on GitHub was Bootstrap at the time, and it had like 60,000 stars. And then the second was probably Rails with 10,000 stars. And so if anyone, and people would do this a lot, and they still build these types of tools, but they use different methods to do it. People would do a lot of scraping to go and build like stars, like aggregation pages, or just to get metrics. And so whenever they would go back through the stars pages for Bootstrap, for example, it would be pulling old, pretty cold data out of the database and filling the buffer pool with pretty much junk data and evicting user sessions, sorry, like user data or issues data or things that would be really nicely cached by the database and would usually be, the app usually expects it to be fast. That would slow, like pulling old data, filling the buffer pool would slow the database down. And suddenly every single user is having a problem because of a silly script kiddie use case that— was hitting the platform. In a sharded world, you reduce the failure domains to be a lot smaller, and something may cause a problem within a shard, but that's just then a subset of your users feeling the pain rather than everybody.

So you said Vitess is very, very popular and very well supported. It's actually part of CNCF now, is that correct?

Yeah.

So it's got a lot of support behind it. What role does PlanetScale have in that process of maintaining Vitess?

So we maintain Vitess. The Vitess team are very diligent in making Vitess a well-managed open source project. Like I mentioned, some of the names of the companies on this call that are part of the community, they essentially contribute. We have monthly and weekly meetings with the community to discuss the shape of the project. It's important to make it very well run, right? We can't just push database software out there and not have it be extremely high quality. The impact of that would be absolutely massive. And we obviously have to take it very carefully when we're doing things and also build things that the community need, maintain the project well. And that's not a small feat, but it pays dividends doing so. So it's something we're very diligent with.

So how does PlanetScale compare to other large scalable MySQL databases, such as, for instance, AWS Aurora? How does it compare to those?

It outscales Aurora to a high degree. A lot of our customers come from Aurora once they've run out of runway with their architecture. Aurora has a lot of benefits with its architecture, but those kind of become downsides and a problem at very large scale. Surprisingly, not that high scale, but certainly a scale, we start to see customers have issues with Aurora, and we tend to bring them over and have them run quite successfully on PlanetScale.

So what's an example of the type of problem you run into that causes the limits to Aurora to be hit, but it would not hit those limits in PlanetScale?

Storage limits. You can store tens, hundreds of petabytes in PlanetScale. The way we scale horizontally means it's just more machines. It scales very, very well. We keep the coordination overhead very low, allowing you to keep, you know, just adding more servers horizontally and growing. We see people hit just straight-up reliability issues. We see a lot of folks that have used Aurora for bursty workloads, or workloads that are just constantly hot, and have seen a number of outages due to just a lack of reliability of their platform. Which honestly, when you've got a D2C brand, and you're launching a game, we have a lot of games customers, if you're launching a game, the first kind of couple of days of that game launch are incredibly critical for monetizing, for just building brand, getting hype. If you have an outage during those times, you very quickly start looking for a solution that's not going to do that. And that's kind of why people start coming to talk to us.

So your bread and butter are the B2C customers, right?

Yeah, we certainly have a bunch of customers that have real big enterprise, like there's a lot of B2B customers that we have certainly that are very critical. But if you were to find patterns in our customer base, a lot of the large ones are D2C.

So why do you think that is? Why do you think B2Cs are more interested in what you're doing than B2B in general?

I don't think it's about interest. I just think it comes down to how critical the uptime of their products is. You can lose, you know, touch wood, reliability has been incredibly strong across the years, and our status page will attest to that. If you have a very large outage of a large D2C platform, it makes the press. It makes the news, and it tends to lose the company that has that outage a significant amount of money. It happens, again, game launches. Certain companies make hundreds of millions of dollars in their first weekend of launching a game. If you are down for most that weekend, every blog, every YouTuber, everyone in the gaming space will just talk about how badly you failed to launch a game.

Yeah.

All of those presale packs with the extra DLC that you sold for twice the price of the game normally, those people are now annoyed because they can't get into the game early in the beta they were promised or whatever. It just becomes a really big mess. Whenever you're mindlessly scrolling Instagram and you go and click to buy something, they have your attention for a very short period of time before you go and do the next thing. If anything fails on an e-commerce website, if Shopify checkout fails, you've probably lost that person for good. If you can't do it, it just doesn't work, you just get annoyed, you go elsewhere, you keep scrolling, and that's just sale lost, and you just can't really afford it. In B2B, obviously you shouldn't let down your customers, you never should. But for example, if our CRM goes down for 2 hours, I'm just not going to move our business off of that CRM. I'll be really annoyed, but the work to do it for the upside of like, wow, weren't we without our CRM for 2 hours every now and then? It's just not worth moving. D2C is much more critical. And then we have B2B brands who have D2C customers, and then again, it's back to being equally as important that we're up and we're critical.

Right.

It's still very, very important, but we see Real fit and technical fit with gaming companies and D2C brands.

So let's get into some specifics of what the real killer apps are for PlanetScale. So you mentioned online games, that's an obvious example. What are some other killer apps that can take advantage of PlanetScale or that have taken advantage of PlanetScale?

You're not gonna like my answer 'cause it's not gonna be very specific, but really it's anything that has large amounts of data and a high need for uptime, any critical mission-critical Applications can use a database. They can connect to a database and store data. We have telemetry use cases that we see people storing telemetry data. It's just, you know, if you have a serious company where letting your users down is really bad for you, then PlanetScale is probably the right database because we do everything. We make all of the trade-offs in favor of those bad things happening never.

So are the use cases more I'm talking about the sweet spot here now, not what can use you, but what the sweet spot is. Is the sweet spot more around large quantities of data and sharding that, or is it more on large quantities of compute needed to perform the queries and the scaling of that? Which is more likely to be the issue for these sorts of large applications?

I would say more often it's large amounts of data. However, we have seen some very, well, they're still large, but not large the scale of some of our larger customers. We've still seen some moderately sized databases do absolutely insane amounts of queries per second. So then that's more of a compute workload. And then sharding is equally beneficial for you, right? It might not be that you've got an obscene amount of data, but if you're doing the same action over and over and over again at high volumes, then the compute is just as needed. If you are sharding for storage, you're likely never gonna be too compute-bound because you're probably gonna have more spare compute around via sharding to keep your compute needs up. But that changes. The other good thing is you can change the hardware you have deployed within shards, right? So you can scale up really easily as well. And that helps a lot, but the need for vertical scaling is not as important because you're parallelizing a lot of what you do across shards. And like, if you, no matter how high you vertically, you can vertically scale a box, you end up bound to in-machine contention in the operating system or in the database server that will slow you down. And there's less of that in a distributed system.

So what about non-traditional application use cases? And what I mean by that is, you know that, for instance, an e-commerce store, the vast majority of the database traffic in the e-commerce store is SELECTs and a few INSERTs, but it's mostly SELECTs. You're looking at the database, you're displaying pages of information that comes from database queries. It's very much a SELECT-driven model. That works very well with sharding. While an analytics company, like, for instance, I worked for New Relic for many years, and they were all based on insert query is everything. You get huge quantities of data inbound, some selects, but in the grand scheme of things, the amount of access to the data through selects was significantly lower than the insert traffic. So, we were very, very insert-bound, and that created some challenges for how we sharded, what shards did and things like that. How about those sorts of use cases? Do they still fit well within the PinotScale or the Vitess model, or are there other unique challenges that occur in those cases?

It's an incredibly good use case. It is true that most applications are gonna have 10 to 1, probably more like 100 to 1 read-to-write ratio. If you look at Twitter, right, like, how many times do you tweet versus see a tweet? It's very much low write-to-read ratio. Same with GitHub, same with a bunch, yeah, CRUD apps, right? However, we do have a number of customers that completely invert that and do significant more writes at significant more volume and speed than they do reads. And I mean, sharding's the absolute best way to scale that workload, right? Because you're creating many more—

Yeah.

writers, right? Like, if it's a distributed system—

Read replicas don't work in that model at all. You can't scale with read replicas like you can with a— It depends.

Well, you can't just single writer and read replicas, right? But you can do read replicas within shards. Again, really super scalable again, right? Like, really good. But now you've increased the amount of write threads across a cluster by the number of shards that you have. So you can take on a significant amount more writes and write volume because of that model. If you are just, yeah, like we said, a single cluster, one node has to accept that write. You can have multiple writer systems. However, they come with a bunch of coordination overhead to make sure that conflicts and transactions are ordered the right way. With sharding, you have a much, much simpler kind of architecture to reason around while just increasing a ton of capacity.

So again, if you look at the popular use cases nowadays, and you can go through and say there's thousands of examples of 1,000 different use cases, and while that is in fact true, certainly one is popping up as being more and more popular by a vast number of customers, and that's the use of AI, right? And AI is one of those things that requires some unique database capabilities. AI really needs vector database capabilities in the way it operates in order to do a lot of things you're trying to accomplish with AI. And I'm not the AI expert here, but, you know, there are the sorts of queries that they need those sorts of vector databases for. How does scale work in these models?

We're working on vector support for MySQL and adding vector storage and vector search to MySQL. So we will be capable of powering those types of application. You know, it's very early to say whether People will need to store mass amounts of vectorized data, whether you'll train models off of normal data and the model will know the information, or people will use RAG. Look, I'm no expert here. I see some people saying vector databases are completely useless and don't have any use. Others are saying that's completely false. Our opinion is that databases are gonna need to support these types and the ability to search. It's just a table stakes.

Yeah.

expectation. The scale at which these applications get to, or their databases use becomes, is hard to tell. And so we're excited to find that out as the industry grows and AI is used more and more. I will say though, in this world, keeping data around, not archiving certain things because of database capacity issues, is probably quite important. And that it's very cliché, but the power of your data is kind of probably really important to consider. And a lot of people just get around their database problems by deleting data that they should really keep around and usable. And we enable people to go and do that.

Or pass it off to a data warehouse and make it available, but less easily available. And it has a different set of problems. Yeah. So, you have some unique features. You've talked about some of them, but I've written down several of the things you talked about and other things for my Let's talk about them, each of these specifically, and how valuable this is. And really, let's say how innovative it is and what you've done to enable it. One of the first ones was non-blocking schema changes. We didn't talk about that yet, but I got that from my research. One of the things that you can do is make a schema change on the fly without blocking read or write access to the database.

Yep.

That's something MySQL natively doesn't do. Or at least it hasn't in the past. I'm not sure about more recent versions. How do you do that, and what's the use case for that?

Schema changes are often quite difficult to roll out at scale. At a very small scale, it's probably easy to just alter the table, lock the table. If you have a tiny amount of users, no one's gonna notice. However, at large scale or medium scale, you'll need to constantly evolve your schema, otherwise your product is not really evolving, and then you're probably in much bigger trouble. And so you'll need to keep modifying the schema, and that's actually kind of an unsafe operation. It can involve a lot of locking. There's a number of online ways to modify tables in MySQL and Postgres. However, some of them have caveats. They can't really be rate-limited and done slowly. If their transaction themselves gets locked, they lock the rest of the database, and you get this kind of cascading failure. And generally, We see a lot of customers having real severe issues changing their schema because of database contention problems. Even though the users table at GitHub was small, I mean, everyone's users table is small. Even if you have every user on Earth, it's still what, 7, 8 billion rows, whatever, right? It's still not a big database table, but it's getting accessed constantly because every repo is owned by a user, so you need the user object to own a repo, to upload a repo, all of this stuff. So modifying the user record is also really important. We want to store new attributes about our users, or we're adding a token column or whatever for SSO. You know what I mean? There's just a million reasons you'd want to change it. But we would take downtime by accident deploying changes. So we worked on a tool called Ghost that allowed us to much more safely change schemas. And we've taken a lot of that learning that we built at GitHub and put it into Vitess. And now you can just deploy your schema, fully online, like you're deploying a branch of code, and you can roll it back. And that's the really cool innovation, is that you can roll back your schema change without losing data. If you drop a column, which again, you do it to tidy up your database, tidy up models, but if someone has not completely deprecated the use of that column somewhere in the application, you're gonna go down immediately without that column. At PlanetScale, you can just press a button and bring that column back, with the data intact, and all the intermediate data that was written elsewhere in the table will appear back. And so it's a real undo, a true undo button for your database. So it's really cool.

Do you do that by having a virtual schema that's different than the physical schema? And is that always the case, or is this a transitory state, or how do you actually accomplish that?

It's a very simple concept that's very hard to implement. It's essentially, When you start your schema change, we take a copy of the schema of the table, empty, we modify it with the new, with the schema changes you need, and then we start to fill it up with the data from the old table. And we keep those 2 in sync, and they circularly replicate. So once they're caught up and they're in sync with each other, we flip the new modified table into production in a rename operation. And so we swap them out, basically. And then we keep the old one up to date so that when you need to roll back, we bring that one back. So that intermediate data that you've got in the new table is still present in the old table, and you can flip between the two. And so it's very simple conceptually, but it's actually very hard to do because you need to limit, like, if loads of traffic comes in, you need to limit the writes that are coming in. You need to rate limit the copy operation, for example, otherwise you're amplifying your write workload really heavily. It's, like, a lot to implement to go and get it done, but it presents itself very simply and very beautifully as a deploy request on PlanetScale. It's very much like the GitHub pull request, but for your database where you're deploying the change, you're pushing it out there, people can comment on it, they can, you know, roll it back. It's great. It's really usable.

So, that's really useful for tables, like you say, the user table, that aren't giant but have lots of changes. And that makes a lot of sense. But what about tables that don't need a lot of changes, don't happen to need a lot of schema changes very often, but are giant? Obviously, copying them around can be a lot more problematic.

It works. We've run this against tables that are terabytes in size. Takes a long time, that's all. Just takes a long time to do, but it's safe.

And space.

You can tell it to do all of the copying, And get in sync, but wait until you're around to finish the rollout. So you can actually just go and leave it to copy over the weekend or whatever, and then come back on Monday and press the final button to do the flip.

That's cool. That's cool. Yeah, because that's the downside of that sort of strategy is you never know when it's done, or you never know when it's ready to be done. And then your model, it doesn't matter. It'll just keep it in sync until you are ready to do it. It'll let you know when it's possible to do it, but you can decide when to do it, when to do the flip.

Yeah, and that's right. And all of these features are built from a lot of experience practically running databases, just practically applied at large web companies with all the requirements that we had at those companies are requirements for PlanScale because we see the real use. You can We have such a mix of people here that have run very, very large websites or database deployments and do database internals. You need a mix of both because database internals people, they're very much focused on very hard computer science problems and don't always think about how their applications run practically at scale or in the hands of users who are at scale. So we kind of help each side see how the various things are that people have to appreciate. And then we build a product that's very holistic for companies running at a real scale.

I have a few other things on my list of how do you do this, but I'm guessing that the answer is going to be the same for these, given that last answer. But let me just confirm to make sure. Branching workflows, that's something that basically uses that exact same model, I'm assuming. Schema reversals. What do you mean by development branches? That's one of the things you talk about on your website. Is that different than a development schema fork, or is that what you're talking about?

So, database developer environments or staging environments are a real pain in the neck for companies because you have basically another environment that developers need to have online, and people very often break it because it's not real production. And so it becomes very challenging to manage these staging environments. At PlanetScale, we believe you should just create database staging environments whenever you need them, whether it's whether you want to run CI or you are building an application or you're doing, you know, you're sorry, a feature for your application, you should just create a new branch to go and do it. So it's our way of making staging live right next to production without impacting production at all, and being the beginning of an environment to deploy to production. And it means that we take away a lot of orchestration pain from folks, because they're not going to really set up the tests locally, and you can't really simulate PlanetScale unless you get a little mini copy as your branch.

So these are development branches. Are they development branches of production data or of a different dataset? The reason why I'm saying that is I, I know one of the things that's very hard in staging environments is to run tests at scale because you don't have a dataset that you can easily get access to at scale. So is this a feature that allows you to do that as well, or is that a different—

You can choose to create backups that have seed data in or just restore from backups. But by default, the branch actually has no data. So what people will do is load bootstrap datasets into the database to run their tests against. We haven't really made it immensely easy to go and get test datasets out of production. One of the reasons being, it's very hard to do unless you know what the customer cares about. But we give the underlying primitives that at least take the pain of provisioning and all of that stuff away for them to go and get their tests set up and run.

So you're not doing things like a shallow fork of data, of production data or anything like that. You're, you're, these are separate instances, separate, but you enable the creation of these instances and whatever data you do, that's up to you.

Yep. Okay.

Let's go back and talk about sharding a little bit more. So all of this works with sharding. How do you handle schema changes in a sharded environment? How do you do consistent schemas across the shards during the course of the upgrade? That's also a challenge with large sharded systems, right, is doing the schema changes dynamically while you're running.

It's the same thing with a lot more coordination, right? You basically have to go and make sure they happen per shard, and then commit altogether at the same time, and actually go and implement. So, it's the same, the workflow is the same, but it happens in a per-shard context.

Okay, makes sense. So, there's a lot of people nowadays, you know, MySQL is one of the best databases around. Absolutely. Arguably, so is Postgres, right? And there are people who think Postgres is just as good or better than MySQL. There's other people who think MySQL is better, et cetera. But obviously, there's a large set of customers that prefer Postgres for their main database versus MySQL. Does the strategy you're using, is it possible to support Postgres in the future? And is that something you're considering? Are you truly focused on MySQL and that's it?

Postgres is a great database. It has a ton of energy. We would never say never that we wouldn't go and do something in the Postgres world. It's highly customizable. There's a big community. How custom you can make things actually would be a problem for us, right? You can't scale every niche use case a plugin might have. So it's harder, I think, in that sense. And people do shed a lot of the more fun parts of Postgres while on their journey to try and scale it. So it could happen for us in the future, but it's a great database. There's a great community. Whether you like MySQL more or Postgres more is really just a matter of taste. I will say that MySQL tends to still be very dominant among the hyperscalers. I mean, most of the top 100 of the internet runs on MySQL. Postgres can't really make that claim. We've never once seen a massive Postgres workload in our entire time speaking to thousands of companies. They tend to be much, much, much smaller, and they tend to be talking to us much, much sooner than we would expect a MySQL user to be speaking to us. So, in plain speak, they run in, they tend, from the audience we talk to, they tend to run into scalability issues a lot sooner. than a MySQL user. I think that speaks to the trade-offs that each project's made, and I wouldn't say that either one of those is wrong.

So, let's end with one more question. That is, you know, for the entrepreneur that's starting a company now, they certainly expect to grow to be giant, but right now they're not, they're small, they're getting started. Is the right model for them to use something like PlanetScale at the very beginning to allow them to grow and scale, or is the right model to Just use stock MySQL, get to know what the usage patterns are, and then add, whether it's PlanetScale or just, you know, Vitess, however they wanna do it, add that in later. What's your recommendation?

I would not recommend any new company goes and sets Vitess up on their own to run against. However, there is absolutely no downside or difference to picking PlanetScale From day one, and thousands of customers have gone and done so. Thousands of startups use our software, and it grows very well with them. We don't really make you make any significant trade-offs. And if you know that you get to the scale of the next Slack, or GitHub, or any of these large companies that use Vitess, you know you've already factored in the database issue, which is a guaranteed problem for you if you haven't already solved that problem. It's not a maybe, but database scaling is a guaranteed problem for you if you do not run on a database that is scalable now. Of course, there's the very valid argument of just get going with whatever works for you and solve scale problems later. It's a very good point. However, if you're picking a cloud database service and just want those problems paid away, there's no downside to picking PlanetScale from day one. And Doing it that way.

So, my guest today has been Sam Lambert, who's the CEO of PlanetScale. Sam, thank you for being here on Software Engineering Daily.

Thank you very much for having me.