PlanetScale data with Sam Lambert

Hi there, and welcome to PodRocket. My name is Paul, and today I have the pleasure of bringing to you Sam Lambert, who is the CEO of PlanetScale. Welcome to the podcast, Sam.

Thank you. Really happy to be here. Thank you for having me.

Yeah, thanks for coming on and taking the time. So when did PlanetScale start? How long ago?

PlanetScale started in 2018. The project, uh, came out— our main project for test came out of, uh, Google at that time, and our founders decided to make a company based on it. And it's been quite the journey since then. And I, you know, I came to the company in 2020.

So 3 years, it's been around for 3 years, so fairly new company. And I'd love to talk about, like, you came into the role of CEO, you know, you came from your own background of the things you're good at. Maybe we could marry the topic about what PlanetScale is about and how you came into this role and what you do.

Absolutely, I can do that. So perfect. As mentioned, Vitesse, our open source project, started here at YouTube and it was built to power YouTube as YouTube's main database. It was very successful at running at an extremely large scale. And when the project became open source, other companies like Slack and Square picked up the project and started using it for themselves. And the founders thought, we can make a company out of this backend technology. And I came to experience Vitess via GitHub, where I was as VP of Engineering. Using it to scale our database, which is a problem like everybody has. And I asked the company, you know, I would love to invest a seed check in the company. This is amazing technology. I think there's a great future for a technology like this. I would like to invest. And so I got to know the founders and we kept on and I was advising the company and I'd been building products at GitHub. I was part of the team that built GitHub Actions and I really fell in love with product building, especially products that developers consume. I think developers are the most incredible audience you can possibly build for. And I was looking for something new and PlanetScale was very exciting to me just because of the technology. And I thought, look, if we can take this backend tech and build an incredible developer platform on top of it, it's going to be a world-changing technology. And that's why I joined. We started doing that. We kind of kind of said goodbye to the old product, started building this new product, which everyone experiences just as PlanetScale now. And that went quite well. And eventually I was asked to take on the role of CEO and was more than happy to do it. And I'm just extremely happy, loving the job, loving being at a company like this. It's fantastic. And I feel very, very lucky.

You mentioned Vitesse. So for those listening that might not have caught that, Vitesse is Really quick, like a scaling platform for SQL-based databases. Could you touch on that really quick?

Yes. Yeah, it's an orchestration and sharding platform that runs on top of MySQL. So it uses MySQL for what MySQL is great at and then does a whole load around it that means you can take it to a really large scale.

Gotcha. Okay, thank you for that. So yeah, you kind of came into the role of CEO and you're loving it. It's going well.

Oh, it's great. It's awesome. The company is full of just amazing people, and it's such an honor to work with them.

So as CEO, I'm guessing you're thinking a lot about like, how do— how does what we're building fit into the landscape of the current products out there? Where are we going? You know, all that good stuff. What I feel like, there's a ton of cloud-based solutions out there that are coming out right now. Could you give us your 2 cents on like, Why are there so many? And maybe, like, are there too many? And where do you see that trend kind of going in the next 5 years?

Yeah, maybe there is too many, actually. I think, you know, there is a lot, right? There's the reason— one of the reasons for that is it's a massive market. Database market is just colossal. Every tech company needs a database. Very hard to build anything of substance without a database. So one, there's just massive opportunity. There's just like, if you want to build a company and go and, you know, create a high-value product with with a huge TAM, databases is certainly a really good way of doing it. There is a lot starting though, and it reminds me, I think I said to someone recently, I think databases are having their JavaScript framework era where everyone's having a go at building one. And that's good and bad, right? It's good because it means there's creativity. It means that there's interesting new experiences. It's bad because I think a bunch of people are going to learn that it takes 10 years and a lot of money to build anywhere near a decent quality database, no matter where you're starting from. It takes a lot of practice and getting it wrong. And if you look at, say, the NoSQL trend that came up a while back, and then you look at where NoSQL databases are, they've essentially spent their last decade building towards everything that already existed in the SQL world. And now with products like PlanetScale, the trade-offs you had to make, or that would force you to use a NoSQL database, are barely relevant now. So we're building on really strong tech with really great developer experience. So in short, I see a lot. It's interesting for us because we kind of compete on both ends of the spectrum. People regard us as one of the databases with some of the best developer experience. At the same time, there's very few databases that can go anywhere near the scale we've taken Vitesse to. So it's interesting being on both ends of the spectrum and seeing what's coming up. And it's good ultimately, but I think it's also getting quite confusing for— for developers to know what to pick and what to choose.

Yeah, that gap between the NoSQL and SQL is definitely closing in some ways that I've firsthand experienced, like writing some backend services that it's like, I don't know, there's something I had to reach for in Firebase that is just supported natively in PG14 or something. Do you think that this was a general trend of just how we as humans interpret and process data in our daily lives, that it's inherently, Data's relational, you know, I guess, in some degree. Yeah. What do you think about that?

I mean, the relational model works, right? SQL's been around for nearly 60 years now, and it's extremely powerful, and you can build great things on top of it, right? And honestly, like, if you use databases the right way, you can take it to colossal scale. Even non-relational data stores run on top of MySQL, like TAO, the graph database that runs at Facebook, which holds the entire world's social graph inside it, still runs on MySQL. under the hood. And so I get told all the time MySQL's going away, MySQL's dead. I just got back from Berlin where I gave a talk called MySQL's Dead, Long Live MySQL. And you see that classic DB Engines chart, you know the one I'm talking about, where it's like every other database is doing really great. And then you look and it's on a log scale and you see that actually no one's even close to half the adoption MySQL has. And this is actually a really amazing strength, right? So it runs pretty much most of the top 100 websites run on MySQL. They all contribute back to it. So we have this really mature, great database that is now fueling the next generation through products like PlanetScale. And that means there's a long life left and we can still continue to build upon the kind of sedimentary layers of learning. And it's just not worth chucking it away. And I see a bunch of technologies and new databases, they're throwing it all away and starting again. It's like, yeah, I just don't think you're going to beat Oracle and Facebook's engineering teams. when they have a 20-year head start on what we're doing. And so it's very interesting, and I just hope it doesn't hurt the user in the long run, right? Like, if you tightly bind yourself to a database with a data model that's like non-relational, non-portable to Postgres, or non-portable to MySQL, and you've built your company on top of it, you are just going to stop doing anything for your users, and you're going to replace your database, and then you're in agony.

Yeah, and everybody that has to go through that is going to be put through a clothing wringer.

Yeah, right. And it's very painful. It's extremely painful, and it can be business-ending. Like, the wrong database choice can really screw your business up.

What is, like, what is colossal scale? That's a cool term. Like, you hear big data and all that. I'm— and I mean, distributed databases. I'm sure a lot of people who have been in the industry for more than 5 years have had some turn of hand of working with a giant database, but maybe could you put in perspective what you would consider a colossal deployment and database?

So on our scale, if you want, if you are going to be a large Vitesse cluster, you need to be in the thousands, if not tens of thousands of machines with petabytes of data. So it can run at very, very, very large scale. For example, every single Slack message in the entire world is in Vitesse, right? They use it as their backend, or it was the backend for YouTube, or JD.com, all of these just huge hyperscale, which are dealing with billions of users, not millions, billions of users. That's hyperscale to us. It's funny, we see people coming from other databases where they're like, we are massive and we are one of their biggest customers. And it's like, yeah, you have 10 terabytes of data. Which is great, we can help you, and we're gonna help you. But it's funny to see, or interesting to see, what scale means for certain technologies.

Yeah, I mean, 10 terabytes can feel massive.

Yeah, it can for certain technologies, certainly.

Um, so like, what do you think is the secret sauce, or some of the general paradigms that make scaling on that colossal level possible with Vitess? And then if there are things that you folks do over at PlanetScale that build on that, that improve that level of scalability?

I would say pragmatism is one of those things, building on a classic scalable technology. MySQL has always been very automatable. Is that a word? It's a word now. You can easily automate MySQL, and it's always had those goals of replication being super easy, automation being super easy, and it's very, very reliable. And what that means is you can break the problem down. If you are using convoluted multi-master, super distributed, whatever, complex distributed systems as your database, debugging that becomes really problematic, and it gets very niche when you run into scale problems. MySQL is very proven at doing this. And then when you take Vitess's model of sharding MySQL, so essentially breaking your database up into many smaller databases and consolidating them at the Vitess layer, That means you reduce the failure domain down to single shards, and that means you break the problem down. Like, the Facebook database team, they joke that they deal with small data. And the reason that is, is because they've broken the problem down into a sharded system. So it's small amounts of data scaled up many, many times over. So you get this kind of repeatable pattern, and that is a really strong way of scaling. Essentially, Scale is about doing as little as possible over and over and over again in a very sustainable way. And Vitesse is very, very good at doing that. And we get linear scalability with adding horizontal nodes, and that's hugely powerful for predictability.

Gotcha. Okay. So what are some things that, you know, maybe a developer who doesn't interact with databases on the daily at the level that you guys do, what are some of the things that you would start to talk about when wanting to educate people on the types of I would call them, like, physical challenges that you face running a performant database? And I'm thinking, like, you know, the I/O things, the RAM things, you know, stuff like that. What are some of the things that you folks need to consider, or things that, like, a normal computer person who knows a lot about computers might not even, like, think about that is a challenge that crops up?

Yeah, that's a really great question. You need to think about ultimately why certain people that are running massive operations do it one way, and why a different tool is trying to present you magic that just seems too good. It's like too convenient. It's like there's certain databases, technologies out there, they have these demos and whatever there where they're like, write your data in New York and it magically shows up in Singapore and you can do all— and like, yeah, I mean, at a very low scale, at a very low transaction rate, all of those things are possible. And then that really starts to break down When you realize that latency is very difficult to get around. I mean, the speed of light is fixed. It's not gonna get any faster. You can only send data across the country or across the world at a certain speed. And then if you are writing your applications to act as if the data is all magically available around the world and is strongly consistent that way, then you're gonna, you know, you eventually end up in a really nasty eventual consistency problem that your application is not set up for. And debugging that is agonizing. Debugging that is extremely difficult. So you've got to ask yourself where the magic comes from at the cost of what. And for example, in the early days of Mongo, they used to benchmark way better than MySQL. You found out that the data never made it to disk, and that's why. And then you would find when you lose half of your data, and your business is in trouble, you've now paid the trade-off, right? And so, database design is very, very difficult, and it takes a very long time to mature a database. And if you are running on a non-mature database, you are their experiment. Your data, your business is how they are learning and building to mature. It's whether you want to stand on the shoulder of giants or be someone else's kind of test case.

And what do you think that like this general trend that we're seeing now where like Cloudflare as an example, right, you can kind of get things cached and distributed among the edge. We have great AWS offerings of replicating, like you said, something in Singapore, boom, it's already there. You don't have to do much. We have this like new screen layer, I guess, so to speak, between like the developer and the actual mechanics, the steam pistons storing and moving your content. Do you think that this poses a risk, I would say, to the integrity of the systems that are going to be cropping up over the next decade?

There's extreme potential risk. There's great potential for upsides as well. That's where it's a really big developer experience problem at the end of the day, right? So we want to create a database that feels magical, feels easy to use, feels awesome, but at the end of the day, doesn't want to betray you. It's a database, right? It has to show you when you're doing things that are suboptimal, or eventually are gonna lead you down, like, a dark path. We think about it as, like, a cake mold. The constraints the database gives you is gonna shape the application that kind of gets baked into it. And you have to take that really, really seriously, because if there isn't a really smooth graduation curve as scale starts to come up, it's going to hurt in the long run for folks. So the world is getting better and more distributed, and that is awesome. What you distribute is a really key part of the design, and Cloudflare are doing, and all of these companies are doing amazing stuff. And you can create caching data at the edge. Fantastic. Do you need to write data at the edge all the time? No, because that's a much harder consistency problem to deal with. And at the end of the day, you probably have a 100-to-1, maybe even a 1,000-to-1 ratio of reading to writes. So, does it really become pragmatic to solve the writing at the edge problem? Could you just queue it back? Could you just send your write to a more central location like most people do?

Yeah.

At the end of the day, once you've terminated SSL close to users, you've gained a huge chunk of the performance improvements you're ever going to get by distributing things. If you go to the full degree of Everything is fully distributed, your database can write at the edge in all of these nodes, you probably have massively over-optimized based on where your users actually are and what their constraints are.

Right. I mean, writing on the edge, you're paying big bills to do that. I mean, you're basically taking your service and replicating the actual push-the-on-button functionality in a bunch of places, and you're, yeah, you're gonna be paying for it.

And your users don't— and no one, no company out there is that global. I mean, Amazon's serving most of their users in the world from one region. Same for loads of other major deployments. No one's traffic is that evenly distributed around the globe that it's worth spending all that time on.

Yeah, unless you have the audience that is so impatient that if your website doesn't load in 5 seconds, they leave. But I think 99.9% of applications, you'd be fine just serving it from anywhere. The internet's fast enough.

Correct. People take it too far. People just go too far. I mean, it's just the same way everyone wanted to spend their time breaking their app into microservices and doing all this stuff. It's just over-engineered. When there's billions and billions of market cap from Ruby on Rails apps that just talk to MySQL, I mean, there is another path. There's a I was talking about, I don't know if you remember, there was a Choose Boring Technology movement where people were just talking about building on basic foundational tech that can take you very far. I mean, it kind of needs to come back. We're at this massive expansion again, and it's going to crumble down. You're going to start seeing all the, I bet my business on this tool, and it let me down. And we'll go back to normalcy. It happens all the time.

Hey, this is Emily, one of the producers for PodRocket. I'm so glad you're enjoying this episode. You probably hear this from lots of other podcasts, but we really do appreciate our listeners. Without you, there would be no podcast. And because of that, it would really help if you could follow us on Apple Podcasts so we can continue to bring you conversations with great devs like Evan You and Rich Harris. In return, we'll send you some awesome PodRocket stickers. So check out the show notes on this episode and follow the link to claim your stickers as a small thanks for following us on Apple Podcasts. All right, back to the show.

Do you— so what are some of the ways that you think and you believe that PlanetScale is taking these topics we're talking about and finding a happy medium for the largest subset of people? Because yeah, personally, when I go look at these services, there's some that are either super hands-off and magic-y, or it's like I'm writing pure SQL management. scripts. So, yeah, how do you position yourself?

Yeah, right. Like, yeah, it's in between, right? So, like, yeah, you know, use Amazon RDS and you're setting up all the CPUs and the replicas yourself and doing all this stuff, and that's obviously a waste of your time, and most of that can be handled for you, and that we can easily manage those things for you. Yeah, we try and we really wanna be the best database to start your company or your project on. Because it's easy to use, development environments are handled with branching, you can do schema change seamlessly, you can collaborate against the database, the CLI is super simple and easy to use. But when your company becomes successful, we wanna be able to take you out to the scale that our customers operate now. And that's unusual. There's no database that really has the range, right? The ones that are really fun and easy, like no schema, no rules, no parents, whatever, though—

I'm thinking of Firebase, is it? Yeah.

Correct. And then people really struggle and everyone has to migrate off of Firebase. Our vision is that common sad story of migrating off your first database or from Postgres or wherever is we end that story with PlanetScale. And in the long run, you just get so much more for buying in and building against it. And that's how we want to do it. That's the vision.

So I feel like a big reason of why people move off really comes down to money too. It's not just an operational thing, and how that model interacts with your customers is gonna be a big deciding factor. What do you have to comment on about that for people that might be thinking, okay, well, that sounds great, but when I'm running this thing, it's gonna charge me the 10x overhead that Firebase charges me for taking data out?

Yeah, we want our pricing to scale. When you are scaling, everything is a factor. how easy it is to operate, how much it costs you is a factor, and we wanna have really fair pricing. MySQL is very good and dense and optimized, meaning you can eke out a lot more. I think Percona moved a customer off of Postgres to MySQL, and it cost them 50% less to run the exact same workload on MySQL, because it's very dense, it's very optimized in that sense. So we wanna make sure that we reflect the same. Obviously, we capture the value of, like, you don't need to hire anywhere near the amount of staff you need to managing PlanetScale versus RDS. So, you know, we obviously, there is a value there, but we try and make pricing scale really well with the workload and try not to be too expensive. And yeah, it's a constant journey and a mission, and we'll optimize that, we'll keep being reflected in pricing.

And so what does interacting with PlanetScale look like from the developer's chair? If I were to describe it for Firebase, I would say, all right, well, you got some SDKs, you can write raw queries, there's a few ORM libraries like this ABC. Could you give us a quick viewpoint into what it's like to sit in the developer's chair and use PlanetScale?

Yeah. So we don't believe long-term that local development is going to exist in the future. It's partly because the world is moving to SaaS. And if you've got a highly distributed environment, it gets very tough to copy that onto your laptop, right? And we've all done that. It gets out of date. You don't have Firebase locally, right? So we've come up with a concept, and we were the first to come out with this, is data branching. So we We basically allow you to branch your database as trivial and easy as you branch your codebase. We believe as soon as you make that Git branch, you should immediately be able to make a database branch that you can connect to it. And that has multiple benefits. It's isolated. You can keep your own data in there. You can keep a copy of the— a different copy of the schema. You can do whatever. And rather than managing multiple versions of the database, if you switch between different environments, then you're tearing down, seeding, messing around with the database. That's cumbersome and annoying and always breaks. So we believe in staging and development environments being in the cloud, and you can use that with like Codespaces, you get a GitHub branch. If you wanna make schema changes, now this is, again, this is where the gap has started to close from NoSQL to platforms like us, is you have a schema. Having a schema is a very good thing to do. It's optimal, it's the right way. But historically, schema changes have been really tough. So we've made them extremely simple, and you deploy your schema changes like you're deploying code. So we have our primitive called the deploy request, and it's very similar to the GitHub pull request. You can create your deploy request, you can show your team the schema changes you want to make, they can comment on it, you can hit deploy, it rolls into production fully online, it doesn't block the database, it doesn't cause queries to fail, it's rate limited, it goes into production. And then if you've done something wrong and you've broken the database, you can just hit undo and it comes back with all the data there. So it's a very developer-centric workflow that is aimed at feeling like your normal developer workflow, and the database that is extremely powerful is doing the right thing under the hood.

Now, what about in the scenario where I'm thinking in some work that I've done in the past few months, we have a relational-based store and we wanted to run some tests against one of our environments and say, okay, if we pulled 1.5 million rows this way versus that way versus doing it over the course of a day, how does that change, A, the speed of the query, and B, our total compute cost spent on these queries? So what about that type of really intensive load testing? Is that something that people should be thinking about when And what am I going to pay to PlanetScale if this is all in the cloud and I'm just developing on your services and stuff?

It depends how you want to optimize. Our pricing is very fair in the sense that you just pay for your usage. You don't have to spin up huge machines because one batch job that runs at night needs that level of machine, right? And then the rest of the day, it's just sat around, you're paying extra. We just charge you for your app, your usage. the queries you consume, the storage you consume. But also with our Insights product, if you were doing that analysis, you could just send those queries to the database and look at them individually inside of Insights, and PlanetScale will show you how quick, how optimized, how, uh, all of the details you need from each query, and to make it very easy to compare. And it's the database giving you that knowledge and that opinion.

Oh, gotcha. So they They would be sent to the cloud anyways, because this is a SaaS-only kind of development environment, and then you would pay for what you use during that investigative spike. And there's a lot of tools that you're mentioning PlanetScale has to kind of accelerate that investigation.

Correct. Absolutely. We want to put developers in the kind of— it feels like we want it to feel like an Iron Man suit. Like you've got just constant access to all of the things you need to get stuff done.

Where do you find PlanetScale, you guys putting your eyes next? Is it improvement on the current codebase and the features coming out, or stepping into some other, you know, SaaS and really solid backend services for developers?

Right now, we really want to focus on databases being amazing. Databases are hard and really tough, and we want to continue to build something that's incredibly high quality. We've already shipped features here that are world-first, like PlanetScale Rewind, and that's getting people really, really excited. And we're moving the bar. Like, in the last 2 years, we brought the industry Rewind, database branching, deploy requests for databases. Like, we're introducing new concepts that have never been seen before. We've got something coming late in the year that is, again, just game-changing, not been done. So we're working on world-first stuff that is going to move the needle really quickly. on what databases should look like. And then we also just want to keep building against the fundamentals. We are now starting to power some very large deployments and websites on our cloud platform, and that's a serious task. And at the end of the day, 99.99% of our requests go through the protocol, and that means building a really robust and awesome service. So we keep making improvements, we keep hearing customer feedback, we keep iterating and building something that people will love.

You're mentioning giant deployments coming into you guys and into your service scope right now. Can you maybe talk about, maybe not necessarily even a giant one, but a customer story or developer story that might get people excited about checking out this new way to develop and interact with a relational database?

There's many stories in the sense that we have companies that are founding, choosing PlanetScale, building really, really quickly, and they're growing like 50% every single month, and they just like, they're incredibly happy because their developers are just growing, they're extending their teams, their database is just not causing them problems. And we're also just working with a couple of companies now that are on their way to IPO, and they have massive scale databases that are letting them down. And one of the biggest risks they have is the database platform. In fact, funny enough, some of our biggest customers are moving from Postgres. We have a bunch of folks that are getting into a lot of pain there, and needing a scaling solution. And luckily, it's not a massive move to go from Postgres to MySQL. I mean, if you're using some very niche functionality, you might have to do some work, but we get in with real depth. Like, we have very, very large companies that use our product. We have our customer engineering team that supports some of the largest sites in the world with our database technologies. They get in there and they work with these companies and really help them get through database scaling, and it really helps them unlock their business. It's just awesome to see, just to kind of meet with companies that are migrating over, and their brand names or their big products that you know of, and just seeing them excited that they're now moving past database issues and going back to building, it's great.

Do you find that people are coming in for the scaling solutions that PlanetScale has to offer? I mean, it's literally in your name, or Are there people coming in that are also saying, I'm here for the branching, like, just the new workflow is enough for me to, like, try this out?

Yeah, it's both. If you're a major company, like, if it's a major company, you can't overlook your database not scaling. There's no fun feature that is gonna make up for the fact your database doesn't scale. And some come to us just purely because they've run out of scale on Amazon RDS or whatever, right? So they come to us for scale first. and the features are a nice thing. There's other people, and there's a huge amount of people on earth that have smaller development teams. They have some serious database usage. They don't want to go and hire database experts. And using PlanetScale, that kind of, with all our functionality, that means you don't need those experts, is incredibly compelling to them. They want branching. They don't want to manage staging environments. They don't want to mess around adding third-party tools to show them what their queries are doing and then still not getting that right. So it's a mix. I say, even with the bigger customers, once we get in and solve their scale problems, they really do start to have fun when their developers are excited to use their database. That's an unnatural experience for most engineering leaders to have their engineers excited about the database.

No, yeah, that doesn't sound like a real thing to me. The database is always the first thing to go down, and it's always the saddest thing to go down, because it's usually like the 3 guys who really don't want to be up at that hour, that have been at the company for a long time, that know why that random column is there, or that are stuck working it. So.

Yeah, there's always a last person carrying the bag. The one who fixed it last always gets paged next. And it's sad. When we talk to people, they say things like they're scared of their database, they're scared of changing the schema, they have tons of columns and stuff that's completely unused because they won't touch the database to change the schema. And now people just push and push, and many schema changes a day, just queued up like deploys. It's great.

Well, Sam, we're coming up on time here. Is there any resources or other talks you've given that you would want to point listeners to that want to learn more about either database scaling just in general, or specifically about PlanetScale?

So if you go to our YouTube channel, there's loads and loads of different examples. There's talks from our backend engineers, there's customers coming on streaming with us. If you really want to just get going though, our docs page or just planetscale.com and sign up. My message to everyone is if you are using Amazon RDS, especially with MySQL, I guarantee you can move to PlanetScale for similar cost or maybe often cheaper and get so many additional features, online schema changes, deploying your schemas like code. You will guarantee to see a benefit over Amazon RDS, Google Cloud SQL, any of these tools. So it's worth just getting in and giving it a go and importing your data and seeing how it is for you.

Right on. Well, thank you, Sam. Thank you for talking about database scaling. It's kind of one of those mystery areas to a lot of people that don't ever actually break open these, those geodes full of interesting things inside. But yeah, thank you for your time. Thank you for coming on.

Thank you for having me. It was really great to be here. Thank you.

Thanks for listening to PodRocket. You can find us @PodRocketPod on Twitter, and don't forget to subscribe, rate, and review on Apple Podcasts. Thanks. Thanks.