PlanetScale Management with Sam Lambert
31 Mar 2022 · 49 min
Sam Lambert, CEO of PlanetScale, explains how the company's Vitess technology solves MySQL scalability problems that Aurora and RDS couldn't. He covers horizontal scaling, hot-row protection, and vReplication, plus PlanetScale's engineering culture where managers stay hands-on and designers join early.
Chapters
Running a database company requires expertise in both technical and managerial skills. There are deeply technical engineering questions around query paths, scalability, and distributed systems. And there are complex managerial questions around developer productivity and task allocation. Sam Lambert is the CEO of PlanetScale, which is building a modern relational database infrastructure. Before PlanetScale, he spent several years on infrastructure at GitHub. He joins the show to talk about his work at PlanetScale and the vision for the company. If you're interested in sponsoring Software Engineering Daily, we reach over 250,000 engineers monthly. You can send an email to sponsor@softwareengineeringdaily.com to learn more.
Sam, welcome back to the show.
Thank you. It's awesome to be back.
You are the CEO of PlanetScale, and I have a lot of high-level questions, some lower-level questions. I want to start by just asking, how does building a database company today compare to how it was in the past? Maybe when you think about other database companies in recent memory, such as MongoDB, or even further back like Oracle.
I think it depends what you're trying to do. I think you can still go and build a database company if you just focus on the actual database, but I don't think that's going to cut it in the long run. I think you have to have something that's stable, reliable, and does its fundamental job, which we know we have that covered with our kind of backend tech, Vitesse. But also, I think in this modern world, and if you look at where the industry is going, it's now more important than ever to build something that's loved by developers. Developers are so important to every part of company life in, in the tech community, but even at non-tech companies, developers provide a lot of leverage for the businesses they're, they're a part of. And if they don't love the tool that they're buying or using, it's not going to cut it for the long run. So I think in this modern world, it's about building something that's creative, amazing to use, great developer experience, and is robust and reliable.
Now, what I would say is each of those, you know, companies had kind of a core innovation. Like, I think MongoDB, just the core innovation was kind of a usability, just in the sense that it was the database that felt like it was JavaScript-oriented or felt like it was document-oriented that had the most traction. And similarly, PlanetScale has this core innovation of having well-designed MySQL scalability. Why wasn't relational database scalability solved by the time PlanetScale came to market?
It's a really, really, really good question. And I think Mongo is the absolute right case. to think about. So people kind of hate on Mongo. I certainly was not a Mongo fan when they first came out to the market because the things they were saying, I think, was really not true about databases. And they had a lot of issues. And, you know, I remember I worked for a place that ran one of the larger Mongo clusters at the time, and it was not fun. However, Mongo, I think, is a great lesson for everybody. And that's— that is that developers are not necessarily always going for of database purity or, you know, what some standard or paper says. They're getting— they primarily want to be productive with something that's good enough and that's fast and, and works for them. And I think Mongo, Mongo was very successful by understanding that and getting in and building something that was— you got out of developers' way. And they did an immense amount of work to make very, very, very good developer experience. And they kind of did the rest later, which is And they've built a fantastic business on top of it. I think it's like, I think Mongo exists today because of Postgres, MySQL kind of asleep at the wheel a little bit when it came to experience and kind of just focused on the database fundamentals and passed on a lot of the pain. They satisfied that we're solving hard problems for people, so we can pass on some of that pain back to the user. And it took too long for the MySQL world to kind of catch up with that. And I think we're We're trying really hard to kind of make gains on that and make something that's supremely usable and really, really good. But now MySQL, you know, still dominates. You know, people talk about MySQL being over and dead. And I think that's just, that's not true. Still the major, most of the major websites in the world are still running it. And a massive amount of the internet, I think it's like 79% of the internet is still run on PHP. You've got to imagine. Would you say it's unfair to say 90% of PHP websites probably use MySQL?
That's probably right.
Right. So still a massive amount of the internet is, is running on MySQL. And, you know, there's something to recognize there. And we're going to carry the baton forward. It's been around for 25 years. Hopefully it'll be around for another 25 and it will get a lot better because of companies like us.
What is the percentage of the internet that's WordPress? Something like 80% or 70% or 60% or something? Do WordPress installations want Vitesse?
Funnily enough, we have a customer that's a top 3,000 website. They run on PlanetScale and their backend is WordPress and their frontend is Next.js. A common pattern that we're seeing now, and I saw there's someone doing this for Drupal as well, which is again really interesting. It's like companies, e-commerce, media, all of these like large sites were built on WordPress. There's so much data and backend that's all built in WordPress, but they wanna have a new kind of frontend and want to leverage some of the amazing tools like Vercel or Netlify and these new frameworks. So now they're treating WordPress as like this API middle layer and kind of doing headless WordPress. And so we have customers that are doing that. And obviously WordPress runs on MySQL. And when you get to large scale, the fact we have a drop-in replacement for MySQL, we're seeing people migrating off of Amazon RDS onto us to use these kinds of solutions. So I think actually this new architecture for WordPress and then backend services like ours are probably gonna keep WordPress in the game for even longer.
And just as an example, can you just explain, like, if I'm running some amazing, amazingly complicated or high-traffic WordPress deployment, what am I going to see in terms of if I move my, my core database to PlanetScale, what am I going to get out of that migration? Is it, is it going to be a performance gain or is it going to be more around the usability layer at this point?
It's both, and I don't think— I think both improving either is very, very impactful for an organization. We see that we kind of round off some of the bursts. So if your traffic is low and it's the middle of the night, some of those less optimal queries or, you know, less optimal patterns are less problematic when your database is underloaded. The problem is when it gets to be very loaded, that's when things start to get Really problematic and can cause slowdowns around the entire system. And having a more mature database stack like ours in the backend kind of takes away some of that pain and pressure and gives a route towards horizontal scalability, which means this kind of like more bursty workloads and workloads that are more complicated and less optimized run much better. Also, we have features that protect the database a little bit more. So like if you just truly overload a database server, it just falls down. Right? That's very clear. It's you go from like queries slow down for a little while and then they just stop and it stops working. The fact that we have scalability and we have features like hot row protection. So if you are like hammering a row with updates and inserts, we can queue those and send them down to the database in a way that stops you completely hammering a row. Same with reads. We can detect, you know, if you're doing issues, the same read queries, we can catch them all at once and serve them with a single query. So that kind of intelligence. between the app and the database really does help with running apps that are bursty and complex. And one of the big WordPress sites we moved over was on Aurora, which kind of claims to be a real solve for a lot of these types of problems. And it just wasn't. They were having really bad reliability. And so they moved over to us very quickly, and they've been very happy since.
I'd love to know more about the comparison matrix when people are looking at hosted relational database solutions. I imagine you have some selling points on each of the comparisons like Aurora, and I don't know what the other MySQL scalability options are, but do you have a point-by-point comparison or some of the main ways in which PlanetScale is able to trump its market competitors?
Yeah, actually, if you go on, there's planetscale.com/enterprise has a comparison table. Primarily this one compares scalability and I'll give a brief overview of that. One way we massively differentiate that's obvious is usability. Right. And that's obviously very subjective in terms of like one being more usable than the other. I mean, we know that Amazon experience and taste is not a strong suit. of theirs. And we find that, you know, a couple of our customers, they've got into the position where they had a decision, which was like, try and hire database experts, very hard things to do nowadays, or move to a platform like ours where they don't need to, don't have to have DBAs. And a lot of them choose that because it surprises me the money they make on Amazon RDS and how little they do for you is wild for me. Like, it's just incredible markup on top of the machines they provision for you. And it just does very little. And we have a deeper feature set, things like branching that we brought out there that, you know, we created and it's showing— it's a whole new way of thinking of staging environments and deploying schema changes. None of this stuff exists in these rival platforms. But then it comes from a pure scalability perspective as well. So we've got a customer that's moving to us right now. They've got probably 20 terabytes of data and they add a terabyte every month. That's really, really hard to do on these types of platforms. So like if you look at MySQL RDS, the max storage size you can get is 64 terabytes. Well, we have customers with clusters for tests and clusters at 20 petabytes. So a massive amount more storage available on PlanetScale. Same with the amount of cores. You know, we have clusters of 50,000 cores. The most cores you can provision onto a single database with RDS MySQL is 576. And you get 5, only 5 replicas. Well, like how many, how can you run a serious website off of 5 database replicas? It just doesn't happen. And so there's all these people that start their companies on these products and then very rapidly they get to the point where it just stops scaling for them and they're pretty stuck. And we talked to a bunch of customers that are trying to unwind gigantic architectural balls of twine that they built around Amazon RDS. Customers have built their own sharding layers. They've built app logic for sharding. They've split separate tables out onto clusters. It's just really messy for a— and really a lot of operational overhead for a product that charges as if they should be solving all these problems for you.
It's so surprising to me that a product with as much market penetration as Aurora hasn't solved sharding, doesn't have built-in sharding. I guess it just speaks to the level of engineering difficulty that was there for a long time. I mean, I suppose, you know, you, you kind of needed Kubernetes to build Vitess, so I guess just the database lineage demanded that things evolved this way.
Right. And it's actually really— the lineage, you talk about the lineage of Vitess, is really interesting. And the reason is there's such a strong suit for Vitess as well. Like, you see these databases now, they're like We're cloud native, we do all this sort of stuff. And it's like, yeah, cool. Like you're building from the bottom up with very few large customers. I mean, their customers are gonna run into, you know, you don't wanna be the biggest customer of your database platform. So we kind of joke that we'll never find a bigger customer than YouTube where this thing was built. And it was built on Borg, which was the predecessor and inspiration for Kubernetes. And so it was built in a world where there was no like true persistence. And it was built for failure in a very dynamic, but antifragile environment and still running on MySQL, which is very robust. And so it was born in the fire almost of a, of a trying to scale a colossal website with not just millions, but billions of users. And that is a real strength for a database. It truly is. If you can make changes and then deploy it to production and then get the traffic of billions of users. you find problems very, very, very quickly, and that makes things very strong. I look at Facebook's adoption of MySQL, the work they've done for MySQL. It just helps. It's just so, so much better. And that is a real strong suit for Vitesse.
So PlanetScale recently made it to general availability of the hosted database. Can you define general availability and what it meant to get to that place?
So for 6 months, we were in beta. It was really about continuing to build out some of the central systems we needed so that we could feel we could go GA, kind of having users stress test the system and make sure that it works and it does the fundamental things it's meant to do. There's so much trust involved in a database. You've got to take it really, really carefully, and we take it really, really seriously that we're, we're the custodians of people's databases, and they're— and then, you know, by proxy, it's their business. And if there's any database issues, it's real hard, and we take that super seriously. So we, we wanted to be— the database world is swamped with snake oil, and we didn't need to add more hubris and recklessness to the, to the mix. So we wanted to take things very slowly. But to our surprise, people started moving really, like, decently sized workloads over to us in beta and things started to work really well. Like that, that very large website that we talked about moving moved during our beta and they— it went fantastically well for us. And we thought, well, if their availability is improved moving away from Amazon RDS by moving to PlanetScale, like we're close to ready. So, you know, we started to prepare. The engineering team did a phenomenal job. They all came together and made a massive list of all the things that need to be better, need to be improved. All of this, like rough edges and snags that we had, and they just went through it, got it all tied up, and we shipped. And I was really surprised that it was only six months. And considering the platform came together, and the first commit on building the platform, like everything around Vitesse that we put together to build the product didn't exist before December 2019. So we've moved an incredible pace. We went from nothing, like literally not. Like I was going back. Through the demo videos in the early days, about this time last year, we had like a blank login page that you could log in. Like there was just nothing. So in less than a year, we went from not just built, put it out into the market with features nobody's ever seen, rocket ship user growth and GA in less than a year. It was just an awesome year next year. And a lot of it was getting to fundamentals, like a lot of undifferentiated things. Single sign-on compatibility and audit logs and user kind of password reset flows and all of those little things are undifferentiated. Everyone has to build them. And I'm super excited that we have a year ahead of us now where we've done all the fundamentals, we've, we've got the basics out there, and now it's kind of pure cool stuff. Like this year's got— our roadmap has got some incredible things on it that we, we're really excited to show people, and it's stuff that people haven't really seen databases doing.
Like what?
So I'm not going to go and sneak the roadmap out there. But one thing I will say is that Vitess played many, many roles at YouTube and did a lot of things that would not just being the MySQL-compatible database. There is an immense amount of power and robust power under the hood. And our journey now is to kind of start surfacing that power to people.
So you worked at GitHub for a pretty long time. How did the database challenges at GitHub compare to what you've seen with the customers of PlanetScale?
Depressingly, it's all the same. It's a large set of problems that hold the whole industry back. These companies that spend a massive amount of time solving database issues when they should be doing things that differentiate their business and help their business grow. I think there's these lost years of— we speak to so many, so many customers that are victims of their own success, and it's the database that's letting them down. Very often it's like the CEO comes to these conversations because when you're having database problems and outages or really slow development cycles, like, it's a top-level priority. And I speak to, like, you know, you speak to so many people at these companies that are going through really difficult issues, and there's no, there's no way out unless you, you kind of replatform and move to something else. It's really tough. And we were in that same spot at GitHub. We We grew wildly successfully, and the things that always broke first was the database. We had a fantastic database team that spent a lot of time and built a lot of tooling to make it manageable, but it was never fun. You were always catching up.
Do you see PlanetScale used for OLAP workloads or just for OLTP?
Hybrid of both never works. I know people are trying it, but it just doesn't. So we really optimize to be really good at OLTP, and we're going to give people easy ways to get their data into their OLAP databases. And we use OLAP databases for our own analytics and data gathering. Right now, we're purely focused on being fantastic at OLTP.
Vitesse, it's a piece of technology that's not necessarily married to MySQL, as I understand it. Do you imagine a world where PlanetScale offers I don't know, Elasticsearch backed by Vitess, or Redis backed by Vitess? Do any of these other database use cases make sense?
We talk about it, and it's interesting. I think Elasticsearch would be too much of a stretch. The one that we get asked for all the time, I mean, nearly daily now, people ask us for Postgres support. There is a lot of— the architecture, yes, it's not necessarily Fully bound to MySQL, but there is a lot there. Like MySQL provides really good like replication primitives and ways of doing things, and and the fact they're reliable is you know a thing that makes Vitesse very reliable. But Postgres is the one we get asked for constantly, like all of the time, and we we consider it, we talk about it, but it usually comes down to it shouldn't matter. Like what? The backend, the exact backend storage engine, whether it's MySQL or Postgres, shouldn't be up for concern in the long run. Really, it's gonna be about all of the other incredible things that we are gonna do and whether your database platform can do that versus, you know, whether it's Postgres or MySQL in the back.
So PlanetScale obviously has this core innovation of database scalability from Vitesse, and beyond that, you've built on these database usability features. What do users expect out of that database admin and usability layer, and what are the opportunities for innovation there?
I think they expect being able to do things quickly and intuitively without reading miles and miles of documentation that kind of explains every caveat and weird side case or whatever that the database has. I think people expect to trust the database to do what it's supposed to do, and that it does that very, very reliably. So that's why we really think about what our job to be done is, and we kind of refine our understanding of that as time goes along. But I think people just like, actually, like most people just don't want to think about it. They like, It's very clear what the database is there for. And I think people feel, they feel betrayed almost when it doesn't do that. And we really want to make sure that we do exactly what we're meant to do. So I think that usability sounds simple. It sounds like, yeah, of course, duh, everyone should do it that way. It's not as simple as that. Like being able to do complex operations from a CLI that is intuitive and easy to read is not an easy UX problem. And when I compare ourselves to other tools and see, like, so for example, like, we're no less secure than any other platform. Like, we provide you an encrypted connection to MySQL, but we've made sure that this, the certificate exchange is handled by the CLI and that it's made super simple elsewhere. Whereas other people are like, download this, stick it in this folder, do this thing, and it's just like unnecessary waste of time. complexity that just can cause issues for people that are trying to learn or begin or get things. So we just spend more time refining it and making it super, super simple and don't, don't assume much knowledge from the user.
What's the feedback loop between talking to customers about what's working, what's not, and getting those features built into the the admin situation, or I'd just love to know more about the feedback loop for getting features into PlanetScale.
Yeah, so we talk to our customers a lot. You know, I get very involved in the sales process, so does our head of engineering, and we listen to things that may, say, block a sale or that really hamper a customer having a good import experience or, you know, moving over to PlanetScale. And we look at those things. I think the core reason we have this product that's, that's as good as it is, is because our engineering team really care about the users. So, you know, I spoke to someone who was a founder of a company that's already doing extremely well. And they said, oh, just, you know, I was talking about something completely unrelated with them. And they said, oh, just so you know, like, we're PlanetScale users. I was like, oh, that's great. And he's like, and, you know, I tweeted at the PlanetScale account that had a problem and 3 of your engineers DM'd me at once. And that was just a really unusual experience. I've never had an experience like that. And I was really proud of that moment because I've— one thing I've noticed with our engineering team is when they get more sources of information or feedback from users, they dissect it really quickly and really think about it. And so our engine, like, we don't put walls up or gates between our engineers having genuine conversations with other engineers like themselves. to understand their problems. We don't have product managers for that reason. And that's really, really important that everyone has the freedom to go out and talk to users and advocate for users internally.
Are there any particular problems that you've seen customers encounter in MySQL deployments that you simply don't have a good solution to? Like things that you're not able to solve in the product? Like just, are there problems inherent in MySQL that you simply can't solve?
So far, no. If they're on MySQL, they've likely had problems with, you know, things that like scalability with MySQL or performance. You do come across certain people that just shouldn't be using a relational database for the things they're doing. But there's heavy bias with the fact that we, you know, we sell this MySQL OLTP solution. So we spend up— end up talking to customers that use that and have that as a problem. Sometimes there is real issues with people's applications and they're doing things in wonky ways and you kind of have to break it to them that it's like, yeah, really, like this isn't gonna work in any sense. And you kind of coach them through changing their application. But we get used to that. That's kind of life as a database company. And the more we hear those types of things, the more we think, well, there's product features here, there's intelligence and there's ways of surfacing these things to the user without our help. It's just all opportunities for us to improve the product.
When I read through the PlanetScale blog, there's a lot of posts about database internals and nuances of certain MySQL operations you can make that are, you know, take place under specific, or you want to use under specific circumstances. How much do you personally have to understand about MySQL internals as the CEO? Do you feel like you can just kind of like learn as you go along, or have you, have you had so much experience with MySQL that you really have a deep familiarity with, with a lot of the subtler operations?
My background is as a MySQL DBA, so I have a lot of experience with MySQL. I haven't done it for a long time, so I haven't kept up to date with every single you know, feature or nuanced change. But, you know, I know how these things work and I could describe pretty simply the architecture of MySQL and why it's good at what it does. But it's been a while since I've, you know, spent a whole amount of time with a large MySQL deployment. But, you know, it's enough to know what we're doing. It's at the appropriate level. You know, there's, there's definitely things and trade-offs in MySQL that can make things complicated. But yeah, no, I have a good understanding of that. Still build small applications using MySQL, and I've kept pretty much— you know, I'm excited about it. It's databases. I love it. So it's still just a high level of interest for me, and I keep up to date with what the engineering team are doing and then try not to meddle, but just keep an understanding and have a high-level appreciation for it.
Is there a specific technical feature of MySQL that you learned recently that you could share? Is there anything you, you've learned from the team?
For Vitess, I've learned— I've been really learning about the power of vReplication in Vitess. So the way Vitess works is it is very good at replicating data between disparate MySQL machines or just disparate shards of MySQL. And that's a kind of hard problem to solve. And it has many, many possibilities for what you could build with a system that's— is that robust. So I've been spending a bunch of time learning about that. And the test feels like a treasure trove of just great tools and software that still is undiscovered by the wider community. And so I spend my time learning there and still getting pleasantly surprised each time.
I'd like to switch the conversation Talking a bit about engineering management, is there a particular book on engineering management that changed your perspective on the practice?
No, honestly. I think engineering management is one of the things done absolutely— one of the most suffering parts of the tech industry is the craft of engineering management. I think it's done so badly at most companies. That it really lets people down, especially engineers. And I just try not to get involved in the sense that there's probably some great books out there, there's probably some really good techniques and practices, but it's, you know, a lot of it's generalized. And once you get the basics down, I think people spend too much time being professional managers versus people that are there to support a very creative process. that should result in great engineering being done. I think the right engineering managers view themselves as a kind of necessary evil on the path to getting great engineering done. And those are the best ones. And then unfortunately, there's a whole— and the majority of the engineering managers, interesting. So the industry-wide kind of opinion of themselves is that they're there as some sort of special, massively necessary role. And I just don't fully agree with that.
Can you say more about that? I'm not exactly sure. What is the anti-pattern of engineering managers that you're really honing in on there?
Engineering managers tend to find their work as being an end, like as a result in itself. How many engineering managers spend their time working on reorgs and ways to neatly bucket their teams into nice little slots versus being able to support a highly creative, chaotic, dynamic environment? And letting great people do their work. I've worked with, unfortunately, too many of these people that don't spend their time learning the craft their teams are building and learning the craft of engineering and supporting good engineering being done. They instead just push paper around and do this kind of middle management, which just wastes massive amounts of resources and really does nothing except grind businesses down. We have a very different management philosophy. We want the company to be the best place to work. We don't think managers that come to collect resources, when they call people resources, or control information to gain power. I don't know. I just seem to— as soon as you start hiring professional managers that are there to just manage people and not get work done, politics comes into the organization and it just becomes a mess.
I wrote a book about Facebook engineering, and, you know, when I think about Facebook engineering, the product is so expansive and has so much surface area, it's very easy for me to imagine this kind of decentralized, highly creative work environment that you're referring to. But when I think about a database product, to me, a database product is so mission critical that you almost— it seems like you would want a rigid engineering organization and you would want engineering managers. And this is maybe a caricature, but you would want engineering managers that are kind of like taskmasters, you know, very thinking very rigidly about what needs to be done. Why is it that a database product is something that can be creatively hacked on?
So no matter how difficult and low-level and mathematical the solution to a problem is. It takes creativity to solve all problems. It takes optimism. It takes being inspired. Have you tried solving a hard problem without feeling that even solving it is beneficial to you or something that inspires you? You'll never get it done. You can achieve both things. Like, don't get me wrong. You can— what you've been talking about is, you know, and I worked at Facebook and I encountered some of the best managers I've ever seen in my career, especially upper leadership in the engineering org, were just phenomenal. They achieved the same level of high— the way Facebook internally talks about the infrastructure and the crushing weight of that scale of billions of monthly active users and the very hard infrastructure problems to solve, they take it extremely seriously. But you can enforce— yeah, you can have managers that like hound people and drive them crazy and bug them and create endless reports and metrics that don't really mean anything to get some level of rigor and robustness. Or you can have managers that inspire the want to do that in their people by being just as good as their people, maybe even better, and that are able to perceive a better version of the people they work with. and that they support and they inspire and drive people to be their best selves. And that takes competence and that takes mastery of the craft. So at PlanetScale, to be a manager, you have to spend your first 6 weeks programming. You do not get away from that. And that's that we stole that from Facebook. They do the same. But when you, so our VP of engineering, he came in and spent 6 weeks engineering with the team building and it gave him such an appreciation of the work his team has to do. And it gave them respect for his engineering ability. Where you've seen a lot of organizations, the VP of engineering, oh, you know, I haven't written code for 10 years, but I'm really good at resource allocation. Well, I mean, yeah, they may be able to micromanage people to death to get a similar result, but you can be just as good as inspiring people. So Jay Parikh, who ran the infrastructure team at Facebook, was a phenomenal leader. Phenomenal. And part of that leadership came from competence and people knowing that he could probably do the job of most of the team below him just as good or better. And so it's just different. You lead from the front or you push from the back. It's just a different style, in my opinion. Can I read you like a little bit from our manager expectations? So I think I can make it more clear to you.
Sure.
Yeah.
So you can expect your manager to perceive a better version of you and support you in getting there. Gather and deliver relevant signal and opportunity for impact tailored to you. Provide clarity of purpose, of direction and responsibilities. Managers should be capable of making strong technical contributions, even though they may not spend much time doing so. And they can drive a culture of excellence and attract increasingly talented contributors to the team. And your manager will not micromanage or coddle you. Instead, they'll insist on a high-trust, ownership-oriented culture. They won't command with title. Instead, they'll lead with influence. They'll serve as a proxy or firewall. They will not serve as a proxy or firewall. Instead, they'll support you in building healthy, direct lines of communication with your colleagues. They won't play politics or hoard people. And they'll incentivize people to put their, like, the team and customers first. And the last is my favorite. They will not fill their time with milquetoast busywork. Instead, they'll enthusiastically engage with the work of getting things done. So that's the difference. We have a longer list of things they won't be doing. Versus what they should be. But to me, it's more about leading people and building a great experience for them to do their best rather than pushing people around.
And do you have any systems of accountability or ways for tracking progress methodically that may not inhibit that kind of, uh, you know, looser, more creative situation that you'd like to create?
We're still small, right? We're still only 85 people. So we don't have tons of systems and processes for doing these things. And we try and avoid them. It's largely subjective. Like everyone at PlanetScale knows their craft really well. And we know when things are taking longer and things are slower or more complex or more noisy than they should be. And we just drive it out really quickly. We just have a very open, frank discussion about the issues. Just get to resolution. We just don't, you know, we only have one sort of value at the company, which is everyday matters. And we selected it because we found ourselves living by it. I think most company values are nonsense. I think if you're like, your company value is excellence, you don't have excellence, you have the opposite of excellence. If it has to be demanded or asked of your people, then you've hired incorrectly. Everyone at PlanetScale comes to work because they want excellence. They want to close their laptop at the end of the day and know that they're contributing to and building something very special with other very, very smart people. No value written on the wall of excellence or empathy or any of these generic terms that people use as their company values does anything for that. It's about culture. It's about who you hire. Most importantly, it's about firing people that either disagree or don't live up to that way of working. Anyway, I digress. We have one value, which is everyday matters. What that really means to us is do things now, optimize for making today the best day, and that losing time and wasting time on putting things off for a week away is just a waste of time. We optimize for an extreme bias to action of making things happen. People start to get very frustrated very quickly if things don't move, and that's good. And I do not ever want to just like extinguish that fire for fast progress.
Do you have a rigorous test suite for changes to the database? Or, you know, do you guys, are there subsets of the surface area of the database that you test more rigorously?
Yeah, so we have a, yeah, it's a very, very robust test suite. We test against all major frameworks to make sure that they don't see regressions or compatibility issues. We performance test. You can move fast and take things very, very seriously. So the Facebook motto of move fast— it used to be move fast and break things, but they changed it to move fast with stable infrastructure. And I think actual stable infrastructure reliability and tests are precisely how you move fast. The thing I deployed facebook.com in my second week at the company. I pushed code into facebook.com. And I couldn't believe how incredibly simple the process was. Because of a ton of really good infrastructure and automation that makes it like not at all scary. We want it to feel the same when you're making database changes. So we're extremely rigorous with our changes. And it also helps that some of the largest websites on the internet use our software. And when we do a release, they take it, they test it themselves, and they put it into production. And we have contributions from pretty much all the major websites in Vercel. So it's really standing on the shoulder of giants and getting tons and tons of like You know, every Slack message in the world is in Vitess. When you send a Slack message, it gets acknowledged by Vitess before it actually gets sent around to the client. So it's very robust in terms of the, the amount of traffic and testing it gets, and that is really, really handy for us. It's not just us deploying it, it's our other large customers.
To get into a particular engineering question, the Kubernetes clusters that run on the managed PlanetScale instances. Can you tell me anything about those Kubernetes clusters? Like, are there any particular, like, I don't know, service mesh or operator patterns or anything you're doing interesting in those Kubernetes clusters?
I'm by no means an expert when it comes to Kubernetes. I'm told it's fairly vanilla in terms of the Kubernetes setup. Perhaps we could have someone from our infrastructure team come in and talk a little bit more about it. But we try and keep it simple so that we can deploy into the managed environment, right? And so that it's kept— yeah, I don't have too much detail about the depth there when it comes to our code. I leave that to our fantastic infrastructure team.
What about product design? It seems like getting developer experience optimized has a lot to do with design. Do you involve yourself in design review meetings? Do you guys have dedicated designers? What's the process of getting design optimized for a database?
We bring designers in at the very beginning of the conversations. If it's going to be touched by users, then we have designers that have built fantastic developer tools. In fact, our 2 product designers report to me, and every week or month I check with them, are you sure you still want to report to me? I'm not very good. I'm not as engaged as one would be necessarily being the CEO. And we all agree we wouldn't change it because it's so important to have designers at the seat of the table, even with backend engineers. And it's the only way you create great user experiences. You kind of explain what the job that the user is going to need to do, what the job to be done is, what the user story is. And you have the designers work on it. And it's not just thrown over the wall at the very end to be like, you know, make this thing pretty. Or like we don't have like backend engineers doing mocks of UIs and then designers just get like 2 weeks before launch to make it pretty. Right at the beginning, like we just had a number of, we called them Zoom sites because we didn't do them in person, but like offsites basically on Zoom where we were kicking off the major features that we're delivering this year. There's a designer in the room for every single one of them and they're already working on mocks and the mocks and the designs are being done in parallel to the engineering work to make sure we refine both. And they meet in the middle in an acceptable place. If you have backend engineers designing stuff, they invariably come out with things that are a little bit more complicated because they're— they understand the trade-offs differently. And that's not wrong. It's fine. The job for us is to kind of keep a healthy tension between the two of this thing should have no steps or very few steps to execute this journey. And it should be heavily designed and it should be achieved by magic in the backend. It's the same with our brand designers as well. We have in-house brand designers. I get asked every single week. which agency designed your website? And I always say they didn't. We have people in-house. We have people that put design on every single blog post we do, every piece of social. Like, it's extremely important to us how our product and how our brand is represented in the world.
Are there any other infrastructure companies, modern infrastructure companies, that you take inspiration from? I was going to say that your website looks a lot like Vercel in a complementary way.
Yeah, of course. No, I think they're a wonderful company. I think there are people— they remind me very much of PlanetScale. And people say that PlanetScale is the Vercel for databases. And that's very, again, very complimentary. And we share a massive community together. I think it's becoming— Vercel, PlanetScale, and Prisma combined is becoming a pretty formidable stack. And I see people using it constantly and there's already some pretty successfully big websites running on all that entire stack. Yeah. I think whenever you meet someone from Vercel, They universally agree that they should be building something incredible and usable for their audience. And it's the same with us. And when we all do get together and hang out, it's just a melding of minds. It's awesome. And I just, yeah, I just have an immense amount of respect for what they've done and continue to do.
That stack of Prisma, Vercel, PlanetScale, what are the advantages that that gives you in terms of actual day-to-day usability? When you think about a comparison to a much more traditional stack, like the LAMP stack, I mean, this is like 3 cloud-native technologies compared to 3 open-source technologies from back in the day. How would you draw that comparison?
So first of all, I've still got immense love for the LAMP stack. LAMP is going strong and keeping a huge amount of the internet together. But I think To answer your question properly, the combination of the 3 gives you incredible leverage. And that is that it's fundamentally at its base what infrastructure should be about, leverage. And you should be able to leverage good infrastructure so that it adds time to your day, not subtracts. It gives you a repeatability and a scalability That means you don't relive the same problems over and over again. Like, if you get your stack and architecture right early on solutions like these, you have an incredible amount of scalability ahead of you. And that means you're not rewriting how your database code works. That means you're not changing database from Postgres to MySQL. It means you're moving fast and continuing to grow and growing your business. And that cannot be underestimated. Like the years of engineering time. I just spoke to a customer. They're huge. Like they're migrating to us, but they're massive, really big, really huge MySQL deployment. And I couldn't believe their answer when I asked them. I said, how many years do you think you've lost to database scalability problems within your company? They've been around for like 10, 15 years now. And with a straight, fully straight face, he said 100 years. And I was like, I couldn't believe it. I just couldn't believe it. I was like, maybe 10. He was like, no, we've got a team of about 10 people that work on databases full-time. And it's just a constant issue. And that's crazy to me. And I think people have the same with like frontend scalability as well. Maybe not as bad as databases. I think it peaks at databases, but they have these issues. And so when you pick a stack that's built on these types of platforms with very little backend knowledge that you probably don't need to develop the muscles for a long time in your company, you can get Well, just to wrap up, I'd love to get your perspective on what has been hard about becoming a CEO.
Just what are the hardest elements of managing a database company?
I don't know if it is specific to database companies. I think I probably have an easier job than many based on how great the team of people is here. I think we certainly have avoided a lot of the problems companies go through. by just having a very high bar for the people we have at the company. So I feel very lucky. You almost to the hour of like becoming— when you become the CEO, this is strange kind of just feeling you get of like, well, the buck stops here. Like you, everything is potentially your problem. Like when you work at a company, like, you know, you work in the engineering team, you're not worried if the bills get paid. You're not worried. You assume someone's paying for the office, you know, someone's job somewhere to do something. If you're the CEO, if there's that person there not doing something, and it's causing problems, well, that's your problem, it eventually rolls up to you until you can find a more sustainable way to fix it. Or so you get this kind of strange feeling that is, the unknown unknowns are your problem, and you don't know it yet. And so you're kind of, you have to continue to wait. It's not something I feel can undermine your thinking or make you overly paranoid, but it's certainly a sense of responsibility that goes a layer deeper than any other role I've ever had. You talk to other CEOs about it and we all agree it's a pretty lonely job. Sometimes doing the right thing can mean you're everyone's enemy for a little while, or you have to push for certain things that don't make sense to everyone immediately. Then it's your job to explain it because you owe everyone That. And so it's just, it's a lot of additional responsibility that I never quite expected, but at the same time, it's an amazing job. And it's a— I feel it's a role of servitude towards building a great team with great people and a great company, and it's something I'm more than happy to take on.
Awesome. Any tips for getting those problems, those unknowns to bubble up?
Listen, pay attention, and always have an open door to people to come and talk to you. There's always someone that's bothered by something. And if they have to bottle it up and not get it out and not communicate with you, that's when problems fester. If people know they can come to you and you'll deal with things super quickly, and you'll be grateful for them raising the issues, then things come forward. There's certain stuff you'll never predict. And you have to have a sense of humor about that. I mean, some of the stuff that happened in our journey at GitHub, Which is kind of funny in the end. You think to yourself, wow, could never have predicted that. And you kind of just, you roll with it. You have to have a sense of humor about it, but just having a team of people you're close with that you listen to, really important.
Well, Sam, thank you so much for coming back on the show.
Thank you so much for having me. It's always a pleasure.







