Sam Lambert Returns: Building Great Things by Playing the Long Game

The thing I love about this job is you have to build with the paranoia, assuming the server will arrive with no disks available to you. Like, you just, you just have to like do all this loopy stuff and it's really fun. It's actually just like really interesting to engineer at the extreme edge with those needs for paranoia. It's cool. It's really cool.

So, how was the party? Was that yesterday?

No, it was the day before, but like an ad hoc party started yesterday again. Um, the, the, um, we had guests who were still here at like 2 AM in the morning after the party. It was ridic— it was ridic— like, we had a DJ, we had, um, like food trucks. It was like... just so many people showed up. It was amazing. It was really, really cool. I can't wait for you to come visit sometime. I know, you know, you're too cool for San Francisco, But, um, come and do an edition of it. Just come work from the office, man. Just kind of like come hang out. It's like there's space for you whenever you want to be here. It's gonna—

No, I appreciate that. I, I, I mean, when I saw the— yeah, I saw you mention the party and I saw— I didn't see too many pictures. I feel like I just saw like a handful and I was like, man, this is when I wish that I lived in SF. I mean, I, I love it. I think it's like a beautiful city and I— yeah, and I absolutely like love being there, but, uh, I just haven't been there in a while, but Yeah, it's cool. You can do this kind of thing.

Your company is gonna rip, and when it does, you are gonna just have to come here a lot. Like, it just becomes like, it's the Hajj. That's why I love living here, is 'cause everyone comes and sees me, right?

Yeah, it's funny you say it's good being in SF because, you know, people are always there and you, they kind of, you just have someone in town visiting all the time. That happens in Miami too, except it's always someone here for their bachelor party. It's like the reason I see someone.

Yeah, yeah, yeah.

My friends are in that group where they're like the age range where they're just finishing that up. So I'm seeing less and less people now, 'cause they're not going on bachelor parties as much. But, uh, yeah. That's why it's nice being in a city.

I hated that early 30s era where every weekend was like... my wife was in, like, a bridal party or whatever, and I just had to hang around at a hotel for hours just to go to a wedding with people I don't know.

Yeah. I think overall I escaped it for the most part. Um, but yeah.

Baby showers next.

Oh, yeah. No, we're, uh, definitely in that zone. But I don't know if this is the case everywhere, but at least... In Miami, and I guess in Latin culture, it's only a woman thing. Like, the men are not invited. I've never been to a baby shower, period.

I have, I think. Um, yeah, I think people just have any reason to throw a party, I guess. And, you know, um, in white women culture, they like any reason to torture the men with, you should care about these things as much as us. And so we have to go and do those things. Tara's the worst. It's the absolute worst.

It's like the opposite. Like, I think I'm like, it's weird to like invite men. Like, they're kind of like banned from the party.

Yeah, yeah.

It's just a woman thing.

As you know, half my family is Indian, which is like, why would we ask the men to like, about raising the children? I mean, they have nothing to do with this.

I mean, why?

Yeah.

That's funny.

It's great. We can build SaaS companies instead.

So speaking of how's it, uh, how's it feel? I know you guys have been working on Uh, Planescape: Metal for a while, so it probably feels amazing just to have it out. And you've been— I can tell you've been having a— you've been in a good mood this whole week.

Ooh, it's been awesome, actually. It's been really good. Um, yeah, you know when you kind of see a new world and very few people know about it, and you just feel this constant, like, tension? Like, you're just so— I know you're like this too, which is, you know, once you get this focus, you just wanna jump in and put everything into it, and Yeah. Anything that doesn't reflect that world is an annoyance. And, um, Metal's been around for like quite a while now, and, um, we launched it very, very differently, uh, in the sense that we did our customers first. And we were gonna— like, the secret, the behind the scenes, is we were going to be ready for re:Invent, and we were going to go and announce it in beta and make a splash at re:Invent, and We started showing it to our customers and they were like, well, can we move now? And we were like, I mean, yeah, we can, we can move you. It's fine. It's durable. It's like we test everything. There's no, you know, there's no risk. It's just, do you like, we would need to focus only on, you know, kind of helping you all like migrate and figure it out because you just don't want to take any risk and you do it properly. So we, you know, we'd need to do something differently. So if all of you are willing to be part of our launch, we'll delay this by a whole quarter. We'll get you up into prod, and then you become the launch. And everyone was like, sure, fine. I mean, some of these are public companies, right? They had to get approvals everywhere to go and do this, and it was the first time they'd ever done this stuff. And it was just an awesome way. I mean, it was like very much in the style of what we want to do, which is push our companies forward. Like, like, we're, we're just a database at the end of the day, you know? It's only magical when they're up, right, and doing and building their companies. And so, yeah, so it was a kind of tough 3 months to kind of just try and stay chill, know that we're going to launch this thing, and then watch our customers be successful. But since launching, we've just had so many inbound emails and people immediately just bringing us incredible expensive Aurora workloads and saying, how much would this cost on Planscale? And so far, every single one is going to save money and get faster. So it's very exciting.

Yeah, it's funny to hear that because I So struggle to keep anything quiet. Like, the moment— even before, like, the moment I have an idea, even before I've, like, actually built the thing, I'm, like, already, like, wanting to talk about it. So whenever I see, like, you know, bigger, uh, it's kind of bigger companies that, like, plan a lot more and, like, like, announce things and it takes, like, months, I don't know how, how I could, I could ever do that. I just don't know how to discipline for it.

No, it's hard, but it's great for what you guys do, right? Like, you just kind of ship things and it's dynamic and it's what people love, I mean, it's a really good thing to be that way. You kind of— I would say we still strike that balance of not trying to be too perfect, right? You've got to hold on to that tension of getting it out for people quickly. But at the end of the day, especially with what we do, no feature is worth production issues or reliability issues.

Right, yeah.

So we just have to be very cautious and careful, you know?

Yeah, no, that makes sense. Um, the, uh, yeah, the charts I've been seeing have been really crazy, just very dramatic, dramatic drops in the P99 charts.

It's so fun. Yeah, just like, and then people are like, we're just watching people migrate and they just, they're like messaging us their P99 graphs and being like, oh yeah, this is what happened. It's like, wow, I think this is good. I mean, this is the theme. Who doesn't want that? I mean, we love, as software engineers, we love that, right? We just love to either graphs going up or graphs going down dramatically normally means something Good or terrible. But, you know, in this sense, we're very happy. And the P99 is the important one too.

Yeah, it's kind of nice in cases like this because I feel like a lot of time is wasted in software, like debating things that are more abstract, or like we kind of overcomplicate some of these discussions and make it seem like really intricate. Yeah, but then in this situation, there's nothing to discuss. Like, number goes down. Like, that's, it's really straightforward. There's nothing to debate. Is this good? Is this bad? Like, you know, none of the There's no discussion around it because it's just very straightforward. Definitely the things that are actually good tend to be kind of straightforward in that way.

100%. We're excited about it because it's that straightforward. It goes down to being kind of the formula for infrastructure startup success, which is a kind of— whenever it's a huge shift in cost or performance, like that's what Snowflake did with, ironically, the opposite strategy, which is by separating storage and compute, right? They changed that game fundamentally. We're going in the opposite direction technically, but, but that is kind of how you win, is you have this just big step change in one of those. And then doing both is obviously very special because before Metal, although we were, as everyone knows, like we're running these giant workloads for like, I mean, like AAA games that, you know, we all play, just run, you know, using this stuff. And It's amazing. But it's not like the mass market that everyone can go and benefit from. Aurora has just a giant base of revenue across hundreds of thousands of customers, and that kind of our pitch to a lot of Aurora customers. And if you read Intercom's blog, it very much describes it. And they did actually have an initial blog post a few months back about how they're moving off of Aurora to PlanetScale. to tell their customers that they're going to be more reliable because Aurora is unreliable when it starts to grow. And so our pitch was like, well, you know, if you're on Aurora and it's starting to become unreliable, or it's, you know, you can migrate to us. We'll keep you scaling, we'll keep you at pretty much the same performance that you're used to, infinite for infinity. Like, it just linearly will scale at that forever. Now with Meta, we're actually just saying, yeah, if you spend $800 to $1,000 a month or more on Aurora. Yeah, you're probably gonna get faster and cheaper if you move. That's huge. Now we just have to get out of our own way and make it even easier to go and do that. And it's gonna be very good.

Yeah. So just to get into some of the details here, when you're deploying in AWS or any of the major clouds, you typically will use network storage. So this is If you spin up a server and you need storage, it's not attached to the server directly. It's on separate dedicated server, server cluster that does just storage. So the trade-off here is just performance because you're going over the network to access other storage. The benefit is that storage can be made redundant. Your server workload can move around. It's okay if they need to move it to another server or whatever. It's not attached to the physical disk on the server. It can, it can access remotely. So if you're building anything in the cloud that needs persistence, because you need redundancy, you're pretty much forced into using EBS. And if you look at a lot of these, like, newer clouds that are, like, simpler, they tend not to have the EBS offering because building EBS is quite challenging. It's a lot harder to build that than to build, like, a compute offering. It's extremely hard to run stateful workloads because They'll just, you know, do the thing where they put your database and your storage on the same physical server. Sorry, your compute and storage on the same physical server. And then when that server like runs out of space, you know, what do they do? Like they got to like figure that out. They need to move stuff around. Now the data is in one place and your compute somewhere else. So it's very difficult to run persistent workloads in simplified clouds. But then if you do want to run persistent workload, Historically, the problem has been, well, you need redundancy, so you have to deal with the EBS penalty.

Mm-hmm.

But what PlanetScale Metal does is it keeps you in AWS. You can still, you know, use all the other stuff that's there. And if your compute's there, there's no latency penalty, but it does not use EBS. It actually does use instance storage, but retains the redundancy capabilities, retains the ability to move compute around. And that's kind of the big innovation, which as many database offerings as there are, there's not really one that retains all of this. So it's kind of like the sweet spot for any serious workload.

100%. Yeah, that's, I mean, perfectly put. Basically, you know, we, when we were back in the data center days, your final kind of last resort for a serious issue was a RAID controller, right? Like at the end of the day, you know, we ran in the data center at GitHub, you know, and at the end of the day, if even if we were down for a full day because every replica exploded, like, and then, you know, we needed to get data back, we could at least have hands-on machines moving disks and rebuilding RAID arrays to get the data back. It'd be horrible. It wouldn't be, but it's that final piece of safety. You don't get that in the cloud. Like once you terminate an instance, an EC2 node with instance storage, it's gone forever. You'll never see it again. Um, but with our redundancy model, that's not a problem. And the reason it's exciting for us is we're just buying the same machines Amazon does.

Yeah.

Like, they just take those exact same machines, they build all their software services on top of it. And so if you're on Amazon buying convenience from Amazon, it's hard to beat them when it's at scale. It's about cost and efficiency. But, you know, our software engineers have run bigger database workloads than Amazon themselves, and some of the largest running on Amazon run on our software. And so it gives us the opportunity to just fully bet on ourselves. It's the same way that we increased our SLA. I don't know if you saw that because—

Yeah, I did.

We don't rely on EBS. And, you know, it's really flexible to use EBS. Like, you know, we roll over our pods every 29 days. So every machine at Patsco is less than a month old at any point. And EBS makes that really easy because you just detach the volume, spin up a new pod with the new version of MySQL, It attaches and runs, and it's just easy. That's the only trade-off of moving to Metal is we do the traditional dance of backup, restore.

Mm-hmm.

It's fine. We've done this for millions and millions and millions of times. It's very, very robust. But it means that we get that. And yeah, if you're starting a database company today, you have to choose your pain, and it's hard enough to do it one way. So you usually end up— people are doing it on S3 as well. There's companies like Turbopuffer, Shout out to Simon and crew. Phenomenal. Like, it's not an OLTP database, but it uses S3 to its advantage to provide like an incredibly cheap, cost-efficient way to do vector storage. It's like you only get a certain amount of your working set can be like hot, basically. And they then run that on fast local machines and then they back it all to S3. But running OLTP workloads on slow storage is just lunacy.

Yeah, yeah. And that's even crazier because there's like, it's even more layers than, than straight EBS. Yeah, there's not even like a straight-up file system.

Maybe we should talk about actually like, because I'd love, you know, your explanation will probably be way better than mine in terms of the impact of tail latencies, right? Like if we're doing like 200 queries a page, right, which some people do, right, you know, that's not abnormal at all in a you know, a large web app, one of them spiking up to 500 milliseconds of P9, that's a failed— that's like a failed request worker pipe. Like, you've seen this stuff. Like, you know, it just— yeah, it's that band of consistent performance which is actually more important than just an overall speed metric or average, like, millisecond latency.

Anyone that's ever done any kind of performance work Uh, you like, the first thing you learn is like, oh yeah, I don't give a shit about averages at all because your averages always look great. They always look fantastic.

Yeah.

Uh, and then you look at your, you know, P99s and things at the edge. Uh, and that's where all the problems are. And when you fix things, that's where you see stuff react like, like immediately. So yeah, I mean, stuff gets, goes viral online where people just like, you know, do a couple of calls and they're like, oh, here's the thing. And it's, it's funny. But, uh, when you're actually doing performance work, you know, it's It's all about the tail ones. And when you talk about like quality of a product, it's about feeling reliable. And if P99 sounds like, oh, that's like, you know, an edge case, not really, because like it multiplies across a number of queries. And like as a user, you like might hit that pretty frequently.

Yep.

And like feeling like when you hit something and it's like 4 out of 5 times it reacts correctly, but then the 5th time it's a little worse, the users end up feeling like the whole thing is unreliable because sometimes it just feels off. So yeah, that's where all the value is, and that's where you have to focus.

Yeah, that's where normally your best customers are the ones that feel those the worst, right? Like, they're the ones with the most data. They're the ones with the kind of— yeah, I remember Bootstrap, Twitter Bootstrap was by far the largest repo on GitHub for a long time, and it just someone browsing through the stars page of that with a script would just take the website down because it would just be pulling all of the data off of disk and whatever. You know, yeah, it's like, it is so important. And that, you know, people, you know, they think, well, you know, you're just putting the network over, you know, network store attached storage here. We've done this forever, SANs or whatever. I get it. I understand. But doing every I/O, which databases are incredibly chatty, and it's not just your queries, the databases are doing other things like checkpointing and checksums and replication and managing a buffer pool, like they're doing so much reading and writing to disk on your behalf to maintain all of the other things that databases have to maintain. They're incredibly chatty. And to do all of that over the network is just unbelievably costly. And you see this in the graphs. And you're not just like just going over a network, you're going to 3 machines for redundancy. You're going through a type of load balancer. If you hit one node that's doing some kind of rebalancing or traffic shaping, that's gonna slow you down. We've seen outages, and we're going to do a blog post about the kind of true reliability of EBS. We've seen outages where Amazon were like, oh yeah, maybe that was just like a blip. And we were like, it was more than a blip across multiple accounts. We deploy so many EBS volumes, like such a massive amount, that we saw, we noticed they had like a top-of-rack switch issue for one of the EBS racks.

Wow.

We saw it across multiple accounts. They first of all, to one customer, they denied it was a problem. And we were like, well, 3 other customers of ours saw the exact same thing. So you clearly had a service issue. And they're like, oh yeah, you're right. We did actually. And all of those things can just happen. And now you think about it, to put that in between just getting a page of data off of your disk, it's mad. It's actually great.

Yeah.

The cloud is just If that's the only cloud approach, luckily it's not, then it's fundamentally wrong for databases. Like, fundamentally, they're not going to work.

Yeah, it's funny you mentioned the— when you have visibility into stuff, you do pick up on these things. We had a similar case where, just because we see a lot of AWS customers as well, we saw that a subset of our customers had problems with— they were running into this bug with Lambda. Like, they would invoke Lambda and like, the response wouldn't be correct. But it wasn't just one region, it was like a subset within the region. And it was really hard to track down because we're like, what is going on? Like, we can't replicate this. And it turned out they had like rolled something out to just one, like a physical building before the other, the physical partition before the rest of the region. And like, we found the bug before like they realized it.

That's wild.

So it is funny, you pick up on some of these things. And prior to that, we also found out like a Lambda function feature just was not working at all, like globally. And you think someone like AWS, like literally a million customers, you would never be the first one to run into an issue. But yeah, there are cases where it happens, where that happens. Yeah.

Well, they're now a big engineering org. Like they've built awesome stuff, but when you have 400 services, it's like really hard.

It's really, really hard.

Yeah, of course. It's like innovator's dilemma, purely very simple. Like how do you just address all these things at once. It's not, it's not easy. It's really, really not easy. But luckily we're now in this amazing era of lots of startups inside that ecosystem having a fairly constrained place to be. There's like a debate on Twitter about this last night. I think people were talking about, you know, the difference being that we're inside Amazon with this tech, because you could go and get, like, you could, like you said, you can just go and get these machines from like Equinix or whoever. And build on them, but that's not the point. You have to be next to Lambda. You have to be next to S3, where everyone's traffic is. We just don't see. You can't run serious services outside of Amazon. It just doesn't work. And then Google second, and you know, there's just everyone is there, and they're using everything else. If you're not inside, if you're like not a you know a peered VPC away, it's just way way too slow. Way way way way way too slow.

Yeah, that was actually something I was going to bring up next. Uh, because a lot of, so there's like the pure technical side of what you guys built, but there's also a bunch of constraints that aren't like really technical constraints. It's just the realities of constraining yourself to do this in Amazon. Because obviously you can go, and some companies that have done this, buy hardware somewhere else. Uh, it's probably a lot cheaper. So like all your economics are probably different. Uh, and you can like build something very performant outside of Amazon. Uh, but to do it in Amazon is the trick. And it's funny because it's not like real technical— some of them are technical limitations, but a lot of them are just working within what's possible with what AWS offers. Uh, just like, you know, you can't just say, I want a server with this size disk. There's like limits on how much the disk size can be. Like, how does that correlate to CPU size, memory size? So that probably impacts product decisions on your end. So. It's funny because, uh, to me, this is like, I, it's obviously the case. Like you have to do that because your customers' workloads are in AWS. So you have to be in AWS. That's just non-negotiable. Very easy to build a great product outside of that. So I think a lot of people show off great products that are not in AWS. I don't find that super impressive because like, that's not the hard part. Like you can actually build a big business that way. At the same time, I'm just like, man, this is a, crazy. It's like way too sticky for AWS. I'm like, how do you ever compete in certain categories with them when like you just have to be in there? You know, like it's just like a ridiculous monopoly, I feel like.

Yeah, it's like waking up and saying, gonna do a telecoms provider or something, right? Like it's just— yeah, it's so difficult. It's a giant operations game now. Um, and there's like clear winners and all the other ones that are now taking parts of Amazon's customer base are similarly just massive businesses that can light money on fire for a long, long period of time to go and grind out doing it. Yeah, it's really hard. I mean, until you've spent significant amounts of time using Amazon's VPC reachability analyzer, you're not there yet. You've like that— once you're in that hell, you know, you're like trying to build things of significance. And yeah, like You're just completely right. It has to be inside where those workloads are. And the constraints are crazy. Yeah, I mean, it's all of these little things, all of the little networking bits, the fact that one physical availability zone is not the same for everyone's account. People don't realize that, you know, and so you end up getting strung up and stuck in all of these different places. It's complicated. It's extremely, extremely complicated.

Yeah, it's just funny, like when you're working on this stuff, you just see underneath the abstraction where like it just feels like, for most people, it feels like AWS is just like one big computer that you just put stuff on. But then when you work on this stuff more directly, it just, that fades away and you're back to buildings and like physical racks and all those problems.

We give them hell, obviously, like everyone does. They're kind of our competition, but they're also our partners. And they are amazing with us in terms of their openness. You know, that we, like when we get on a call with the engineering team, they are customer obsessed. They truly are. You've seen this too, right? Like, their engineers are curious. They will get on and they love that you find these problems. And, you know, we're having fun with the EC2 team because now they're like, yes, more giant databases to run on our stuff directly. That's cool. You know, it's really fun. But yeah, like you actually said, like, you know, the long tail of shipping Meta was, okay, do the really hard bit, which is how do you run petabytes of data reliably without losing data on machines that can just explode and disappear? The long tail is making that really reliable and all of the rough edges is gone. So for example, you know, EBS, the great thing about EBS is you can take like a pico node, like a quarter of a vCPU and attach an 80 terabyte volume to it. You couldn't do anything obviously with it, but it's that flexible. That's one amazing thing about it. We don't need that for OLTP, but, you know, we need people to upsize, right? Where people actually have to like say they want to make their machines bigger. So we have to, we've, we have stuff that nags you, you're 80% of your storage and— Yeah. You can just like, we'll, we'll make it so you can just turn this on and it will scale up automatically. But not everyone wants that, right? It's like, people don't— auto scaling is— that's another giant myth that, you know, that we could, we could go hours into the auto scale. Like, it works for certain things, for other things it's an absolute disaster. And most people, when you actually ask them what they're asking for, it's not, it's not really how auto scaling works or is implemented. But anyway, so we have to— and there's things like, okay, you're going to do a schema change. Well, if you actually apply this schema change, it's gonna fill up that box. So we have to kind of warn you that you need to upgrade before you go and do it. It's all those kind of just little UX pieces that make it approachable and usable, uh, that takes the time in terms of building these kind of products out. And, and you're— yeah, you're dealing with weird— there's, there's this kind of— when you look at these nodes, the SKUs are really all over the place. You wonder like, which custom customer did you add this custom machine type for? Because like, for $10 more, we can have double the memory, you know, things like this.

Yeah, exactly. Yeah, figuring out— yeah, it's like, it's so much of the work is just not the programming part. It's like just looking at the numbers and like understanding all the economics of this stuff. And the way that they correlate, like they attach, because you're using instance storage, but you can't say I want 500 gigs of instance storage. You pick an instance that has some kind of CPU and RAM combination and it comes with some fixed amount of storage. And like, to me, those numbers are just so random. It's like, well, I want a lot of storage, but I don't need like 64 cores. So.

Yes.

Yeah, it's a funny problem.

It is funny. And so, yeah, those are the kind of things where we've had to kind of figure it out, um, and, and kind of work out the skew. Like, the skew— just the word skew is like a meme here now because the, like, they're kind of just dealing with all of these various things. We also had a really fun issue with these nodes, which was they fail too, and they have issues, and they can come up with unmounted disks. And so they— we had an issue where they were just coming— they were coming up and not being fit to serve. So we've had to build something where Amazon will give us a machine that's not good. And what we would do is like we'd throw it back. We'd just go, nope, get us another one. And we'd get the same machine back sometimes. It's like, oh, that's still broken. So now we've got this logic where— and we've seen it chain like 2 or 3 in a row. If you give us one that doesn't work, we hold it until we get given another one.

Oh, I see.

So we have this kind of pooling logic where We will just keep the nodes until we have no more node demand, and then we'll let Amazon have all the bad ones back to give to everyone else. And you're just like finding this stuff out while—

Yeah.

You know, running it live. And, you know, with the amount of machines we've deployed, that, you know, one in a million failures happen a lot. And you get to find all these little edge cases and weirdnesses. And people don't build with you— like, the thing I love about this job is you have to build with the paranoia assuming the server will arrive with no disks available to you. Like, you just have to like do this loopy stuff, and it's really fun. It's actually just really interesting to engineer at the extreme edge with those kind of needs for paranoia. It's cool. It's really cool.

Yeah, it's really fascinating to me because I know where my skill set is. Delivering a reliable service is just not aligned with where my abilities are. I just don't have the discipline for that. I'm just way too— I don't know. This hasn't been my trajectory. So, Hearing this stuff is interesting. And to me, I'm just like, how do you even go about this? Like, you build something and you think it works, but how do you know that like some random undiscovered thing isn't going to happen?

Yep.

You know, a month from now.

Yeah. And you have to, you have to decide on an architecture that you believe in and then very simple ways to fail that don't rely on people. You know, I— and by the way, a lot of database companies, you— they don't have the skill set of running services. They can do the database engine stuff, which is incredibly hard and takes like a certain type of cracked person. You need the— and the other type of cracked person, which is, um, runs— can run huge services at scale. Most companies don't have the two. That's the one thing that's very different about PlanetScale is we have both. We have people that run mega websites, um, and the deep kind of database core fundamentals of, you know, parsers and query planners and all the storage engines and all that stuff. One philosophy that we have, which I think is kind of really amazing, is because of a shared-nothing architecture. We're going to talk about shared-nothing a lot more in the future because it's an architecture that people don't really understand. It doesn't come up much, and it's really truly the ideal end state for building these kind of systems at scale. But essentially, you can't build a reliable service unless it doesn't rely humans to wake up, because you pay someone at 2:00 AM in the morning, it's minimum of 15 minutes. Well, that's your SLA just exploded anyway, right? You know, it's gone. And even if they do have to intervene, what options can you really give them when they have to come up to speed and figure it out? So essentially at PlanetScale, all of our architecture is designed to kind of just have one solution to everything. Just kill nodes. Kill the primary node of that shard.

I see.

Just kill it. Just get rid of it. Like, it's the 99.9% fix for everything of like, why cat or not pets or whatever? Like, why— let's not kind of try and understand this node's feelings. Just shoot it and never think about it again. And that's literally Yeah, like, and Vitesse just has handled failure this way hundreds of millions of times. It just knows, okay, that node is gone. How do I get the cluster back online and quick— quickly into a kind of healthy state? It just runs that same, like, well-worn code path to go and do that.

Yeah.

And it makes building things reliably a lot easier. It's hard to do that, but it's— with the mental model is one thing, get like kill the failing node. And every time, we've never, touch wood, we've never caused a global outage for any customer. The worst you can have is like a minute of a shard being unavailable, which is partial failure. It's fine. No one— the other reason people build unreliable services is because they strive for like perfect records, no bugs. Like, that's crazy. Like, you have to embrace failure because it will definitely happen.

Yeah.

But yeah, the resolve is always just kind of you murder the node and bring it back and it becomes partial and it makes the mental model so much easier. And it goes back to, again, separating storage and compute. Because of the shared-nothing architecture, our nodes have their own truly, like, I guess, full stack if you think about it. They have their own disk, they have their own memory, own CPU, own copy of the data replicated from elsewhere. They share nothing with the others. These kind of shared storage layer compute stuff like Aurora and whatever, they share way more. So like a single outage at any one of those layers of the stack is a full and total outage.

It stops everything.

Yeah, exactly.

Yeah. It's funny how good that solution of just restarting, like killing and restarting is. I was mentioning this on Twitter a couple weeks ago. I've been moving a workload from Lambda to a container, and I'm just discovering like so many bugs that I just never bothered handling because my With Lambda, it's effectively rebooting like every 15 minutes or so. And now that I have this long-running service, I'm just like, oh great, I like didn't dispose of these things in like 100 different places and I gotta go chase it down. And yeah, on one hand, I probably should have just had better discipline about it. I mean, any discipline I had around it just deteriorated ever since I moved to doing stuff on Lambda. But like, you know, it works, like it works really effectively.

Who cares? It works. GitHub and so many other companies, Back in the old days, we used to— over Christmas, we would always have some outage due to a memory leak because we'd stopped deploying the website. And so, the kind of versions of the app were just living for like weeks longer and were just— there was memory leaks everywhere. Like, just frequent deploys kind of covers up some of these things. It's fine. Like, who gives a shit? Like, it's going to fail. It's definitely going to fail. Like, embrace that. Just know it will and do the right thing. I was going to talk about this in a blog post about magic because I think we all, um, we all want to build things that are kind of fun and delightful, and everyone likes their shiny web pages and whatever. But at the end of the day, you know, the magical experiences are the, you know, getting home to your family safely at Christmas, right, on an airplane. And it's more like building an airline than it is anything else. You know, all day long while we're talking, just up and down, up and down, 737s or whatever going up into the air, coming down. They can be repaired at any airport on Earth. by a crew of people on a shift that are just managed like through engineering firms or whatever. They fail in the most predictable ways. The most regular failing parts are cached at every airport in the country. Like, how many times do you just get that kind of half an hour, 20-minute delay while they fix something up and just get you going? And you're not unsettling, you're safe, you're gonna be okay. Like, those planes don't even turn off. Like, like a 737 will land And will just, like, sit there, on, go back in the air. They'll do 10 flights a day. It's just so reliable. People need to think of themselves as building that and how that works, like how the food gets on, how the food comes off. Like, that is the real process of building this stuff, not, let's just hope it doesn't fail and make it really pretty. You know, it's just not gonna— it's not gonna cut it.

Yeah. No, I think in a lot of parts in life, making things invisible is, uh— is the value. And it's, it's so hard just to have something that's there that people aren't thinking about, whether it's tech or like just anything in society in general. I feel like, uh, people really accomplish something when they've solved a problem totally invisibly and no one ever thinks about it.

It's beautiful. It's beautiful. It's really beautiful.

Just moving away from the technical stuff. So it's been a year, I think you said, since you guys did like a big shift a year ago from being, uh, I don't know, maybe you just want to describe it yourself.

Around a year ago, I think the anniversary was last week, we let go a big chunk of the company and doubled down on being pretty much purely an engineering company. It was a reaction to multiple things. We were kind of in, we were playing in multiple ends of the database market, and we felt this like real strong pull at the top end of the database market. The bottom end was just a painful mess, basically, of cloud spend and people that don't have real problems. And the thing about, like, kind of the tech discourse of Hacker News and Twitter is you quickly realize— I know you know this— which is most people don't have real problems. Like, that's just truthfully, like, sure, fucking put SQLite on a server. Like, who cares? Like, your 2 users won't notice. Like, it just doesn't matter. Like, you can argue with people about Postgres or whatever, and you just realize they just don't— if they, they just don't see the world the way you see the world, right? Like, they just— they, they— whatever they choose, it's gonna work. And it has, and it's their lived experience, and they feel great about it. And they don't pick technologies for the traits that we pick technologies, and that's fine. It's completely— there's no hate there. But when you're trying to kind of be in that area of the market and you're spending so much money on having like a free tier that will just not convert, you have to have marketing people that do that kind of media that works for that. And it's just expensive and it's not optimized. And, you know, we saw a path to being a profitable company, which one is just incredibly powerful. Like, we threw this huge party at the office and, you know, we have a lovely office now and we you know, pay the mortgages of all of our staff. And it's like our money, it just goes into the bank. We're not just, you know, hope— hype round, VC hype rounds, hopes and dreams, just spraying money up the wall while we just— every customer costs— like, if a customer costs you more money, you're just reducing your burn constantly. And we're now the opposite. If you write— if you're a $39 a month customer with us, you get like— you have a problem, you get a thoughtful reply from our amazing support team because you are a customer. We make money from you. It's amazing. We had an opportunity to do that, so we did it. It was really painful. You know, I definitely would've done things slightly differently, but in the long run, it was an incredibly powerful change for the company 'cause it focused us and gave us the autonomy to do things like this launch the way we did it. We just—

Mm-hmm.

If you were, uh, you know, a VC-backed company with 18 months of runway, you would just go to re:Invent with your beta product just so you had— you had to build pipe, just, just more pipeline, more pipeline. Like, you you have to just go and ape it out there and shout about it out of desperation because you don't have long enough left. We were like, oh wow, yeah, actually just launching it with some of the major biggest like tech brands around already running on it on their blogs, that's an obviously better launch. Well, we're not going out of business, so we can just do that.

Yeah.

Do you realize like when your daily decisions are made from a place of strength and not fear, how that compounds? Like it just We make decisions every single day that you can only make when you're profitable. And so although, again, I miss the people that aren't here, there's some amazing people that don't work at PlanetScale anymore. It was the best thing we could do, and it gave us this year to go and build Meadow. And we've done it, and it's already just changing the company immediately. It was the right thing to do. And it was just a very different thing than we are used to in tech. And when you're building a deep infrastructure startup, you just need time. It's just time. We've got a customer that pays a couple of million a year, I think, now. I first showed them the product in beta. It was barely real. And they're like, cool, this is great. 4 years later, they eventually became a customer. Because they're just not going to—

Yeah.

There's no alpha in just like, yeah, we just yeeted our data over to a startup. Now the business is finished. But it's like they had cool features. You know, the website was shiny, just doesn't work. You did— like, our sales stance is so funny. You know, we're working with it at the moment, this really very large public company, very famous. And I was talking to the CTO, um, and it was, you know, me and one of our sales engineers. And he was like, Google sent 18 people to try and, to try and sell a spanner, and you're here on your own. And I said, well, I mean, let's be honest, like you— and I obviously said the name of the company— is anything other than like a technical proof of concept where we prove to you we can do this going to convince you? Like, do you need slides and an interpretive dance done in front of you? Like, you just need to know we can do it, and you have your own requirements. Unless you see it for yourself, you're not going to buy it. That's us now. We fully embrace that, which is look at the website. It's just— we just tell you what we do. We don't try to hype it up. It's like, if you have problems that fit our solution and you're serious about those, you just will read it and try it and it will work. We kind of opted out of all that other crazy stuff.

Yeah, that's, that's super interesting. I mean, I have a bunch of things to ask you there, but on that last thing, do you ever— and maybe this happens less now, but the thing you mentioned was you guys did things differently. Um, there's— because yeah, there's a standard playbook, right, with certain things, right? Like you said, a bunch of companies building products for developers do this standard playbook of like investing a ton in marketing on like the lower end. And all these orgs look exactly the same. They're all producing the exact same stuff. But then also on the enterprise side, the way you build organizations that sell to enterprise is also like a standard playbook there. Did you, like, do you have people telling you to like, or pushing you to like follow Those standard things, and like, what does that look like? And, you know, is it annoying? Yeah.

I mean, you do get it. People do question it. They wonder.

And...

I just, like, don't care what anyone else says. At the end of the day, I spend my life on this thing, right? I'm only gonna do it my way. That's it. Like, at the end of the day, if we fail, the person, like, turning the lights off at the office for the last time is me. And so I'm only gonna do it the way that makes sense to me. And I'm like an odd, a weird person. I figured that out right now. Like, I'm just very weird. I mean, I just have my things I wanna do, and they either work or they don't. But I can't be anyone but me, and I can't— I'm not gonna get like a 42 long blazer and go and kind of with a briefcase and do like—

Well, they tell you to hire someone that does that, right? That's kind of what the standard playbook is, like, oh, you gotta hire this guy. He was, you know, CRO at this company, and, you know, he did— he took them from X to X. you have to hire him, put him on your team, and like, so he can run that same playbook again. Yeah, that's usually how I see it. It's like the investors or whatever will suggest a hire, and that kind of fundamentally changes what the company is, and that's why they kind of all end up looking kind of the same.

Yeah, and, and very few of those people have ever sold databases or really tough technical, deep technical stuff. There's some types of product where only the go-to-market function is what will work, right? Yeah, the technical moat can— like, the technical standard can be achieved for those types of products very quickly. Why is Ramp bigger than any of the others that started? Like, why? They just— they're a credit card issuer, right? Like, they, you know, why?

It's so funny you bring up Ramp because I have a friend that just gets free lunches from them all the time and just never— because they're just always hitting him up. Like, every month they're like, let me take you out to lunch, let me take you out to lunch, you know. And he just does it and he just never intends to be a customer.

But they're a go-to-market machine.

Yeah, exactly.

Look at amazing companies like Rippling, right? Which is like, they have some hard tech problems now because they're scaling, but at the very beginning, it was just a phenomenal brute force product team that were just like, understand everything a customer wants, build it, ship it, make it happen. You would need these types of people to build those types of companies. Databases just don't work that way. They just can't work that way. Our core audience are backend engineers with hard problems. It just, it just doesn't work. And, you know, we'll, we'll, we will build out the sales team, we'll build these things out, and we have to build— like, Mongo has one of the best sales teams ever built in tech, like legendary. They wrote the book. They still needed to do that to build a big database business. You have to. But the, the marketing paradigms shift over time, and what worked 10 years— like, it's always that classic of like, yeah, like this person's there, you know, and they were at Twilio and they did this thing. Twilio nailed developer marketing for that generation. The Ask Your Developer billboard is legendary. They crushed it. We've moved on. We've gone past it. Like, we're elsewhere now.

Yeah.

And I think now it's going to go back to simplicity and value. Like, and then— and we're seeing that right now. I love AI. I'm not cynical about it at all. I love it. Um, but, uh, you know, there's a lot of AI slop content being made, and it's just like rubbish. So we're now doing— spending a month per blog post to make— and we've got one going out while we're talking right now. Like, high quality, really good stuff. You just have— it's what it takes now to cut above. Like, you've seen— you've been to re:Invent, you've seen the hundreds of millions of dollars of marketing spend just being screeched at each other. You just have to do something to cut through. And, and, and that's why we do it our way. And if you have taste, it is such a moat. Like, like, our website started in a Google Doc. We just designed it in a, in a Google Doc with words. Holly did a thread about this, um, last week talking all about it. And people— well, some people were like, my God, this is genius, when we shipped it. Other people were just so confused. They were like, this company's gone nuts, they're gonna go crazy. I don't know why when I said we were gonna do this, everyone at the company was like, yeah, let's do that, that seems like a good idea. I don't know why. That's just taste, you know what I'm saying?

Yeah.

Like, when you saw it, you were like, oh, I love this. I don't know why you love it. I don't know. It's like the first thing I thought of. I don't understand why I like it. And other people do, and some people hate it. I just don't get it. But if you do have that nose— and we've had it with other things like the other database branching, we invented that, all of these different things— if you have that, use it. Don't— you just see so many companies go wrong by having all of this taste, this initial thing that made them then them, and they're like, okay, so we've got this unique thing that's growing, we've got this advantage, this completely unique viewpoint on the world. So let's go and do what every other company does now. Let's just throw all that away. And VCs are some of the worst for this. They're like, oh, we back contrarians, and we want the— and then my friends, other companies, they'll phone me and be like, yeah, so my board is telling us it's now enterprise sales, go-to-market, demand gen. And way too early, and it's just like pattern matching and running the playbook. It just— no great company was built that way.

Yeah, it's the thing you talk about where— there's a time where things work and it doesn't work anymore. I think that's like marketing for sure, but across business in general, I think people look at great companies and they like try to copy the surface thing there. But the thing that people forget is when those companies were founded, they didn't look like anything like the great companies of their time. Like they did the opposite.

Correct.

So if you're doing that today, innately you're not doing what they did. And the marketing thing is funny because the only thing that matters in marketing is that Uh, people want to hear what you're saying. And it's so much like fashion in that some approach to that will be novel for a while, but it so quickly becomes like a counter signal almost. Like, like the whole, I mean, talking about landing pages, like, you know, Linear did their crazy landing page. And for a while that was like really great. But now when I see one of those, I get like a negative reaction because somewhere in the back of my head, I'm just like, I know they hired like some standard design agency. They like threw them $200K and they didn't like think at all and were just like, I want a clone of this. And without me consciously thinking that stuff, I just equate it with low quality. Uh, so yeah, it's exactly like fashion and it's not irrational. Like the whole point is novelty and like, you know, standing out and especially, you know, even at your guys' stage, like you got to figure out how to make your dollars go further than, than they should.

Yes.

If you are buying attention for $100K, a company with 10x the money as you will just do that at 10x. So.

Yes.

Yeah. This is the thing with marketing where I'm like, I don't know why people don't see that. And the flip side is it's actually really fun when you set the constraint of we need to do stuff that nobody else has done before. Like we need to do things that are like so different from everybody else. We need to do things where when people see it, they're going to like grab it and like share it with their friends at work, right? If you set that constraint, it's really hard because like, you know, 99% ideas don't pass that bar, but it's so fun because that's like such a pure creative exercise that I feel like most people should love. And so exciting when you like stumble on something and you're hyped to get it out.

It's kind of scary as well because you have to just, you spend half the time being, you know, it's so funny. You're like, why is no one else doing, is this so, am I just so dumb? Am I like just not getting this? You know, And then you do it, and then, you know, it works. You're like, oh, maybe I should try another thing, you know?

Yeah, exactly.

We always joke we're a tiny marketing team, but we always joke like the biggest challenge is that we'll try anything. I'll try anything once. Like, exactly, like, I'll just do anything. So that's more open-ended and scary than anything because, you know, you just have to go and do that now. Like, what can— like, what's the highest agency thing you can go and do? And just do it. Don't pattern match, don't listen, don't care. We're the only people not building a separated storage and compute architecture. Like, we were the only people not building it for years. And VCs would just go, oh, I mean, Snowflake, it worked for Snowflake. And they're just like pattern matching. And I'm just like, databases don't work that way. And they just look at you like, oh yeah, you're one of those. You're just— And then you do this stuff. And but we spent 4 years being told, like, you know, there's another well-known storage, separated storage compute database company, and they kind of came onto the market with like loads of hype. And, you know, someone messaged me, um, it was like, oh, you know what, you know, what do you think about this? And I said, you could give me a magic wand right now and to wave that would immediately instantly switch PlanetScale to that architecture right now, and I would throw it in the trash. It is— I would never do this. I would never. And they look at you like, like, wow, that just— you just sound bitter. And like, I'm sure I did to them, but like, where are we now? You know, like, like, you just, you just like, you just have to, you just have to feel insane for years and years and years on end and just prove it out. I also think, by the way, when it comes to this marketing stuff, generationally we're in a different world thanks to kind of the Rails community, thanks to the Node community. You know, I'm old now. I'm 38 years old. Uh, when I was at GitHub, I was 32. I had a $100 million budget that we— for data centers and software and whatever. I grew up in the Ruby world. Like, the tools I used were beautiful and good, right? Like, you know, the 2013 kind of tech era when I was an engineer, you were using Heroku, you were using Rails, you were using all of these really well-crafted— GitHub, I mean, the one, right? Like, these well-crafted, beautiful tools. Like, what did GitHub do different to SourceForge? I mean, nothing. Like, except be usable and resonate to a generation. All those people who grew up on that software are now VPs at big companies and don't want to buy crappy shitty software and, and don't care about the hype.

Yeah, exactly.

Like, they just don't care. They just— it means nothing to them. It just, it just, it's just like the sad thing about AI, and I love AI, and like an O1 Pro subscriber, I love that thing. Like, you know, it's amazing. The thing that ruins AI every time is the gap between the hype and the reality, because the reality is good. The reality is actually amazing. Paired with the hype, it's ruined. I can't watch movies. I just don't watch movies. I, I, people laugh and joke at me all the time. I saw The Matrix the first time ever last year. I thought it was terrible. I don't know what anyone was like. Anyway, but like, I can't because the hype kills it for me. Like, it just ruins— you just— it's everywhere, and then it just cannot possibly be as good as everyone is saying, and it ruins it. And the times where I kind of just nakedly use AI for the things it's good at, it's just unbelievably magical. The day I wake up wishing that my Siri would like rearrange my life and do everything for me, I'm very disappointed.

Yeah.

And I think we're at the beginning, and I hope PlanetScale kind of shows that we're putting skin in the game, is that I think we're in the post-hype era. I think everyone's dopamine receptors have been worn down to just a nub. And I just think someone wants something that's good. And, and there's a pull request of 64 versions of my launch post for Meadow because I wanted it to have as few words as possible. It was like, you know, like you— what's it like, coding golf or whatever, what you just try and get right?

Yeah.

It was just like— well, and I sent it to, um, one of my dearest friends, uh, in the world, Ted Naiman. Shout out to Ted. He's got an amazing company called Case that he's building, an incredible writer. I just kept sending it to him and like, I just couldn't get it where I wanted it to be. Because I just wanted it to be— I wanted to convey what's special because you only get this tiny amount of airtime. But I didn't want to indulge in anything other than, here's a customer. Don't— like, I— like, what's the news flash for most companies? Company says own product is amazing. Okay, good.

Right.

Good job, you guys. You know, confidence is important. Um, but I just wanted to be like, don't ask us. Who gives a shit? Like, it's a database. Like, go to Cash App, that, you know, go and look at their public filings of that company. Like, the things we know about how incredible that company is, you know, it's in like Kendrick Lamar songs and Super Bowl ads. And, you know, that's the cool bit. That's the actual cool bit, right? And not, not the, the kind of, you know, semi-sync replication. And so that's why— and I think now, I think so, I hope we're at the beginning of this post-hype era of What you— what's your— what are you actually doing? And then I think it becomes the, you know, the billboard ads shot on iPhone. They're the most self-aware, amazing ads. You look at a Google ad for, um, a Pixel and it's like 8 mega— like, how many megapixels are we at?

Yeah.

I don't know how many megapixels the iPhone camera is. I buy the new iPhone every single year, but it doesn't matter. They, they are so self-aware. Like, like, Google just give you a load of tech specs. Of course they do, it's Google. Um, Apple just say Here's a picture of a yurt that someone took. Like, that person is exploring the world and is cool, and we're getting out of the way so they can do it. Shot on iPhone. It's that awareness, I think. And I think that's hopefully where we're getting in tech, which is we've gone through this massive amount of hype and physics have not changed. The rules of software have barely changed. Present them simply. Spend your time building something good. And people will like it or they won't like it. That's all that matters.

Yeah, I mean, I hope you're right because the thing that is tough is especially with like when you're early in a space or trying to figure something out where things are going, like, you know, AI for example. The problem with that approach is it just creates so much noise and to like have any kind of clarity, it's so much work. Like so much of my time is just spent like thinking so hard about these things and like trying to filter out like what is definitely just Some person's personal insecurity manifesting as like some serious-seeming opinion on something versus like you know what's actually real. It is just really tiring, and I was thinking about this this today. You know, with all this again, I'm very pro AI just like you are. I try to use it as much as possible. So I'm not one of these people that are like you know against it. But there's so much talk about. I just built like the craziest thing without, you know, any effort, whatever. And there's like so much, there's so much discussion around this. And I keep thinking about the products I use and I'm like, I use like 5 products and I haven't used a new one in like a long time.

Mm-hmm.

If there's all this stuff being built, how come one of you has not built something that I like love? Right. It's, uh, it's like not showing up in the place that matters. Um, and I, I felt this way about tech for a while where I feel like We're just obsessed with these demos. Like, here's a little demo of something. Um, whereas I think kind of like what you guys did with this launch, it's like, oh, we've— this thing has been in production. Look at this thing we've actually built. Now we're going to talk about it. That's so much better. I think outside of it being just better for people, like, you know, it has more value. It's just a much better filter for yourself. Like, if you're trying to like show something to the world, like just wait till you've actually built something cool with it, because that'll just kind of filter out a bunch of things that you're just going to waste your, waste your time on. Uh, so we definitely try to make sure, like I said, you begin, even though I said like I get really excited to show stuff off, I make sure that I like put it into production, use it for something real and know that it's actually useful before, before I talk about it. Uh, but again, like it's just, it's just not it. Like every single company in dev tools, whatever, they just put together like a stupid demo of a product that no one's gonna use to show off some feature they kind of made up.

I think what's key that you understand, because, because I think people could misinterpret what you said of conflating shipping slowly with like take— being deliberate and getting it right. You can iterate. You just, you just have to— like, one of my favorite engineers at the company is a guy called Mike Kudamash. Mike just spends his day talking people out of scope. He's like, well, actually, why don't we just do less? Like, why don't we just like— what's the kind of minimum thing that proves it out? And there's a great— you can, for whoever's listening, you can go Google. There's a— Spotify had a slide in a talk years ago that was like iteration. And it was like, you know, a job is like get to the shops or whatever. Sad person waiting for a car to get built versus skateboard, scooter, bike. Like, it just gets better daily. Yeah. And you can still go back and forth. Like, yeah, for a while you're trying to skateboard with shopping bags. I can't imagine how that's like, but you know, but at least you're doing something. There's, there's this minimum functionality. You can ship stuff daily that is consequential towards something big if you're very clever and selective about how you iterate and, and build and kind of your tooling backs the ability to do it. And, and I think people think, well, it's either like waterfall ship every 6 months this perfect thing or ship garbage and hype it up every day. And it's, it's, it's neither, actually. It's this kind of middle ground of, like, pace and learning and sensibly defining incremental progress on, you know, like the eye, the evolution of the eye, right? Like, an eye did not appear, but some light-sensing kind of, you know—

Yeah, it was so useful on day one.

Yeah, exactly. Just sensing shadows or whatever when no one else, no other thing could. I mean, so, so much, so powerful. And, um, yeah, like we don't have point-in-time recovery for like externally for users yet, right? Like we can do it for you, but you know, it's just not wrapped up well. We know we need to ship that. We just— our big customers don't need it. It's not something that people do at massive scale. You know, mid-market people want it. We know we don't have it, right? So like, could have delayed Metal or just ship Metal and now work on that next. And you just go through it and you just kind of grind it out and you make it happen. I will say one more thing about hype though. Like I am in your audience. Like, I think in the world of AI, trust and credibility is going to be essential. Um, that essentially— that's like why we blog the way we do, is like we want to like educate people and really speak from a place of authority. Like, when you say you like something, I trust it, right? Because you are selective. And you— and because you say you don't like things and you don't just kind of hype it up. And that means I'm an actual member of like your audience. I want to hear what you are thinking about. And, and because of that credibility and and trust. And that is— and, and people ruin it so quickly by hyping up— there's other YouTubers that I know are very good engineers. I know they are. But for, for money, they lie about the technologies they use. Like, that's bad. That is bad. We are in that era of tech now where the grift is on, and they know better. They literally know better in exchange for money. And I mean, it's the same with what we watched with crypto. Like, all of the VCs that quietly shuffled off their kind of NFT profile pictures and are now hyping up AI, it's like, like, you can't trust these people. They just believe in everything. That selectivity and taste and building your audience on truth— like, I don't, I don't really have much of a following on Twitter or whatever because I like, I just tweet what is actually true about databases. Tiny TAM of people that give a shit about that. But like, I mean, who cares? Like, why, why change? Why risk it? So I think it's like holding on to being genuine. And I— there's very few tech media— I told you I listen to this podcast every week. Because it's like a real thing and it just feels like we're in this desert of genuine good content about tech. And hopefully that will change as, as time goes on and we all grow.

Yeah, it's funny because like, I think when people hear like the word, you know, be genuine, kind of what you're describing, like from my perspective, it's actually a purely selfish exercise because you talk about, you know, like if you're a YouTuber, a way to fund what you're doing is to take sponsorships and do XYZ things. And for me, if I would— if I was ever in that position, the reason I wouldn't do those deals isn't because of like any kind of like moral thing. I just find it impossible for me to think clearly the moment I have any. It's like so naive to think that I'm still going to be objective even though I'm getting money. I 100% believe you want to. I 100% believe you really care about that. But having clarity, the moment you have that, it's, it's so, so, so, so difficult. And to me, like, that clarity is more important than anything. And I even go— like, me and you have a personal relationship, like, we like each other. Even that affects my clarity, right? Like, I want to see PlanetScale do well.

Right.

I, like, want to like what you guys are doing. So I'm just, like, hyper-aware of how easily you can kind of get caught up in this stuff. So then when I see people, like, hyping stuff up, my first thought is, It's not really about, like, oh, they're confusing people. It's like they're confusing themselves. You're not gonna reach the right destination if you're confusing yourself to that degree.

100%. And that's the kind of crux of all of it, which is that genuine— just knowing where you came from and what your values are, and just holding that in the center. And it's really hard, and it's really difficult.

Yeah, it's really difficult.

It is really, really tough. Like, truly, truly, truly tough. It goes back to the marketing, goes back to like all of the advice and standard playbooks that people give you. What's it worth? Like people, you know, I had, you know, as a favor to a friend of mine who is an investor, they had like 3 young kids that, you know, they're like 19, 20 going into college. They're interning at great companies here and they ask like, can I go to dinner with them? So I went to dinner and they show up like eager with advice. And I said, I have zero advice for you. Yeah, no, I've had less advice the older Correct. When I was your age, I was sleeping on my friend's floor and I ate rice for an entire week because I'd spent all of my money trying to like set up punk shows. That's literally all I would do. I was an idiot at your age. Like, you were doing way better than I was. I don't know why. Like I said, the only advice I have for you is in your 20s, never say no to going out once. You're never gonna get those Friday nights where you're like, ah, these are infinite. They're not. Like, I am old and I get hangovers now, right? That's the only advice I had for them. I was like, what would I know? Yeah. I don't know your generation. You, you look at me like— until recently I was wearing low-cut socks. That's apparently how they all know we're old. I got some new ones. I got the bomber, the bomber's long ones now. Much nicer. But the point is, it's like, why would you care? Why would you give a shit? Like, do you know what I mean? Like, yeah. Um, I see a doctor more, more. I'm not sick or anything, but I have to. I'm like, 38, I have to go see a doctor. You don't see a doctor. You don't even know what feeling sick looks like. I was like, don't listen to me. There's nothing I have for you. And I also haven't achieved anything. Why should you look up to me in any way? Like, it It's just, you don't, you know, it's like, it doesn't— you just have to kind of go and just build what's, you know, true to you. And then you're lucky if it has economic impact. You're just lucky.

I know. It's funny because I, I have the same exact feeling as you where I used to give a lot more advice. And then nowadays I like look back at my career, I'm like, this doesn't make any sense at all. Like, I just did a bunch of random stuff and one thing fell into the next thing. And like, it kind of looks like it makes sense. And then what I'm doing now is like super weird. And I just can't imagine that advice working for anyone else because it was just like a weird thing that worked for me. So at this point, I'm just like, I don't know what the world's like now. I don't know what like getting a job is like now. Like, I don't— I just don't understand anything. So I have such little advice for anyone. I have like less advice now than I had 5, 10 years ago.

No, it's so funny. It's that confidence, like that midwit curve, right? You just get through the other side of it and you're like, Yeah, I mean, all I know is, like, all you find with experience is that how little you know, how you just don't know anything. Like, you just, you just have to be lucky. People downplay because it's uncomfortable. It feels uncomfortable to just talk about luck, but it's just luck. My entire career changed because, like, one— saw one guy at GitHub tweeted frustration that they couldn't hire a database person. So I just DM'd them. I don't know why I DM'd. My entire life changed, like everything. There's 2 people that exist in the world, my 2 children, that would not have existed if I sent that tweet.

Yeah.

No, seriously, like, you just can't plan it. But if you go out there every day and work really hard, because I will say one thing, actually a true bit of advice is I'm not smart. I'm not a smart person. I'm by far the worst engineer that works at PlanetScale, right? But no one can— I will not let anyone outwork me.

Mmhmm.

You know, like, 'cause smart doesn't get you anywhere without hard work. And like, people always say, like, luck has to catch you working or whatever. I just firmly believe that. You know, you just have to be doing anything and something and, like, jump on it. Like, we— a massive— because of metal, a massive opportunity jumped in our lap this week. We've started already. Yesterday we made that decision instantly. We're doing the thing that we— that, you know, I'll come back when we ship that one. But like, this is— it's massive. It's as big as MEDDLE. And it was yesterday the opportunity showed up, and now we're started. We're just going. We're just doing it. That's what it takes. It just takes like relentless drive and moving all your chips into one place over and over again. And then luck will kind of find you.

Yeah.

And other people that are also lucky that can create more luck will find you, and you can just be there. You just do stuff. Just do things. You can just do things. It's just so incredible. And it's similar to manifestation as well. I seemingly simultaneously believe it's dumb and completely real at the same time.

No, it's true. And I think as I've gotten older, I've been more into this stuff that doesn't seem logical at all, just because I've just seen so much come out of stuff that doesn't exactly make sense, but it kind of does in ways that you wouldn't expect. Yeah, the single decision thing is so crazy to me because I think about it all the time and I think I get physically nauseous and uncomfortable when I think about this. My wife, who I met because she came in for an interview and we ended up hiring her at the company I was at.

Amazing.

And she tells me that she almost blew off the interview and I literally get like weak to my knees. She like just like almost flippantly just was like, this guy, because she talked to Uh, that the CEO of the company, who, who admittedly was kind of weird, she was like, uh, he seems weird, like maybe let's blow this interview. And like every tiny aspect of my life today that I absolutely love, personal life, career, whatever, like branches out from that one thing. And I just imagine her just like flippantly blowing it off and being like, none of my life would exist. And—

Crazy.

I literally get like, I'm like, I just can't handle that thought. It's It's crazy. So, yeah.

Doesn't it build a sense of gratitude, though? Doesn't it make you just feel so lucky to be alive and where you are? And just— it's just incredible. We're just so lucky to get to do what we do. Just even being in tech, I tweeted the other night, like, we're just so lucky to do what we do. I mean, we just, like— we wear sweatpants to work.

I'm not even wearing pants right now, if I'm being honest.

Exactly. It's great. Yeah. Just let it go free. It's fine. Like, you can. It just doesn't matter. Like, whatever. And then like, we just accidentally become like millionaires doing it. It's like, cool. Like, that's good. I mean, yeah, this cut— this won't exist in 100 years. Didn't exist 100 years ago, you know?

Yeah, it's wild.

It's like, we're so lucky.

Yeah, your brain can't even really truly understand it. It's, uh, I think just kind of beyond our understanding. The other thing I wanted to talk real quick about was I was kind of going back to— this is totally out of order because we totally moved off topic now, but just kind of bringing it back to Just like PlanetScale from like a year ago. When you guys made that decision, so there's a few decisions you made. You decided to shrink the team. You also got rid of your free tier to kind of focus more on mid and upmarket there. And it was really controversial. There was like a lot of commentary about it, specifically the free tier thing where people were like, actually, I don't know, like, what were some of the things that you were really frustrated about that you wish People understood it differently?

I don't think I was very frustrated at all. I think the confidence at which people just said things, you know— I look back, I think the thing I got most wrong— I'll start there, 'cause I can't just make it sound like I'm Teflon. I won't take any feedback or, you know, won't let anything stick. Like, I did so many things wrong. with that layoff. One of them was the messaging. I think it was very cold. And in reflection, I think the blog post was very cold. I did it. I sent it to my friends that happened to have either founded successful companies or just been CEOs or run a business or whatever. And they read the post and they were like, oh yeah, this is it. They agreed with the decision.

Mm-hmm.

And so we're like, yeah, this is good. You should just do this. And I didn't really think about the kind of how it would be read and conveyed. And I think it came across very coldly. as we're doing this. And that's kind of a problem with me, which is like, once I believe in a decision, I just go for it. Like, I just don't care about even if I said the completely contradictory thing the day before. It just doesn't matter. And I believed in this so much that I just wanted to do it and did it. And I think I hurt people. And I don't feel good that I did that. I think that's fair critique, that it was kind of a cold way to message. something that had a real human impact. But then you kind of just saw this wave of stupidity that followed. Like, you know, we put that we had a limited runway, and then someone very famous with a very large audience got on camera and said, how is that possible? How can you have an infinite runway? And you just think to yourself, when more money goes into our bank every month than leaves, what date am I supposed to choose for when we're going to go out of business? Like, every venture-backed founder has a cash-out date in their head. They know that at the current rate of burn, without giant revenue inflection, their bank account will go to zero. Like, every— like, ask your venture-backed, like, vendors when their cash-out date is. They know, they know. They report it every, every, every 3 months to their board. We will run out of money if nothing else changes, a raise or a revenue inflection. We'll run out of business. So like takes like that, you're just like, wow, people are just hateful or stupid. Like, that's moronic. Like, you know, just help me. I mean, if you, you know, help me find the day we're just gonna go bust when we keep making more money. Like, we ended the year with like way more money in our bank than we modeled. Like millions of dollars more than we thought we would have at the end of the year. We had it. That's thanks to our infra team cutting loads of cloud spend and cost and whatever. We did a bunch of that. But like, the point being is that takes like that were silly. Astro were releasing a database product a week after we did the layoff. And they were like, you know, and this is where I thought it was really amusing because it's like, well, you know, you know, we kind of know what we're doing in databases. Like, that was never in question. You knew, people knew what we were doing. We've gone and just did something very drastic. In my mind, that would have made me feel humble. But instead they were like, wow, like, you know, amping it up and doing all this sort of stuff. Turns out they were wrapping another database platform that has lost data and shipped a product without backups, lost customer data, and then gave data away by having a bug that's literally impossible to do on our platform where they just served other people random databases.

Yeah.

Never unlive that. You'll never undo that. Not that we'd ever see companies like that in a competitive sale, but when they have competitors, just takes reminding people of that and that's over. Like, you know, they shipped that. And you just think to yourself, like, the arrogance is unbelievable. Like, you just see the level of arrogance and how little they know about computers. Like, they just don't understand them. They just think it's just this simple thing. And as long as you do some good marketing and some little drawing on the, you know, it works. And it just doesn't. And so stuff like that was just like, okay, you'll see. It was like that long game thing of like, yeah. I got home that night and my wife looked at me and she just burst into tears. And she's like, I've just read what everyone's saying to you on the internet. And I thought it was hilarious. Like, Sam Dick, But like some really funny quips or some really good ones. I just thought it was really funny. Um, but yeah, like people go at you and it's not comfortable and you don't feel good and people remember it and whatever. But then you watch, you watch the outages, you watch like the data loss and you're just like, yeah, that's the, that's the long game. We're just going to keep building. We're building Metal, we're building this stuff. You can all confidently yap, but then do it.

Yeah.

when we wake up at 2 in the morning to like fix, figure out issues, I know like they could have infinite money and they couldn't fix these problems. They literally could not. And so you just eventually stop caring. But, you know, at the time you were just like, wow, everyone is so wrong and they're so wrong about you. And you want your— you want a fair trial and you just can't get that in the court of public opinion. And it gets to you. And people have said to me, why do you care? I don't know. I do. I just care.

It's impossible not to. I think it's easy when you're not in that situation to be like, oh, yeah, it's easy not to care, but it's basically impossible not to.

Yeah, 100%.

Yeah.

People ask me why I argue on Twitter. It's like, well, I don't know. I don't know, but I care. Like, it matters, right? And other people care. Like, we've started to now see more folks interacting and kind of talking about the stuff they do. At scale and, and, and the realities of running in tech. And you kind of realize like, yeah, there's this dormant group of folks that do really amazing, interesting things and they kind of want to talk about it. And we're hopefully, hopefully igniting a revolution for these people to go and actually talk about what it means to kind of do this stuff in reality. And that's why, again, why I like your stuff, because you, you know, you have this kind of like— you're not cynical, you know you have to build for an audience that's young and modern and doing amazing stuff, but you also just balance that with like It's still computers, right?

Yeah, it's funny because we actually reference your guys' decision a decent amount. Uh, I just felt like it was so— I think prior to you guys doing that, I had a lot of doubts with my position, kind of what you were saying, right? Like, you think everyone else— you think you must be crazy because everyone else is doing things differently. But forever I was just like, Why is everyone obsessively going after this indie hacker free tier type of persona? Why is there a whole company oriented around that? In my head, I was like, this— I'm like trying to see it. I can't see it. We— are we doing anything wrong? And then to see you guys have captured that a good amount more than us and then be like, oh yeah, that's like not worth it at all. Like that's actually a negative thing and it's going to like screw us if we're if we keep orienting around that and go the other way.

Mm-hmm.

That was an important data point for us because that was the first time we finally saw someone in the space show that that might actually be right. Everyone's chasing the thing that doesn't really make sense. And then now oftentimes we see companies that are blowing up, they're getting a bunch of usage or whatever. And the first thing we think about is, man, what is their support? Look like? Like, what are they just going through complete hell right now because they're getting like the dumbest people that will never pay you just eating up your time? And we orient so much around how do we make sure that we build as though those people can use us, but like we're never in a situation where we're stuck supporting them.

100%. That is literally— that is the essence of it. You know, all of these people that have got negative gross margins because they're just have tons of usage that will go nowhere. Like, it's like, just why? Like, what? What? You don't need it. Like, you just— you're completely right. Like, I mean, it's— this is— this is what people are going to learn over time, which is, you know, and so many people have— founders have said to me, we're not doing a free tier because of you. Like, we just don't. We're just— we're just not bothering. And I'm like, yeah, I think that'll be a good decision. Or it won't. There's like tons of businesses I would start where I would have a free tier. Like, they would.

Yeah, it just depends.

Consumption-based infrastructure businesses. Because how can you be serious about your company if you can't spend $39 to buy the same database that Cash App has? How can you not be serious? How can you be serious? Like, seriously, just like we are giving you a signup flow that provably some of the greatest companies in the world, tech companies in the world use, money print businesses that just print money. And you can't— you don't want to spend $39 on that? You're going to fail. You failed already. This is a test for you and you have failed it. Like, no, I'm serious. I'm serious. It's just like like, yeah, you just have to not care. And, but again, it's going to take 10 years to prove that out. It really, it really will take 10 years. And people always go like, well, what about net new? I mean, people are— the largest customer of one of these little Postgres startups is migrating to us now. They were, they were told by that company, you are our largest customer, that's why you're causing us struggle.

Ah.

Yeah, it's like, it's an order of magnitude below, uh, you know, ACV even. So it's, yeah, one, if that's your large customer, That's very bad. 2, it's a hilariously small database. It's like 100 gig of data.

Wow.

So it's just like, what's your free tier doing for you if you're— that's the peak of your conversion, is that?

Yeah, I think people have this funny thing where they just put themselves in hell, not realizing the point of starting a company is to put yourself in a nice situation. Like, it's going to be hard for some time, but ideally you get to a place where it's like a nice situation, you're happy with your, with your life. And this thing where your customers graduate out of you, I never want to build anything where my customers graduate because that's just hell forever. Yeah, because you're just like, you're finally making money, you're trying to make more money. Oh, and they're gone. And now you're like fighting this infinite battle. Like, that's just not a fun place to be. Like, why build anything that people will outgrow? And then you're stuck with the people that aren't paying you. So it's just like, it's complete hell.

You're just UBI for Indian software developers, you guys. It's just, I mean, just like, you know, Just like we, um, were Ubering to work for $5.

Oh my God, it was so good. Pay like $3 to go anywhere in Manhattan, and it would take, it would take 40 minutes because of Uber Pool. But it was fine because I was in my 20s and I didn't have that much money. It was great.

Yeah, exactly. Like, you know, the GitHub office was just Priuses just pulling up all morning. Just, just, just brilliant. The same Saudi money is now just going as compute, right? Exactly. For students to learn, like I, I, those people are valid. I love those people, but I can't— I, I have a different type of business to build. It's just the way it is. And, and all of those ones that have got the hype and they're tweeting their user graph— tweet your revenue, tweet your gross margins if you want to be impressive.

The user graph without the y-axis, always. The y-axis is unlabeled.

Yeah, like, it's like, hang on a second, we showed up with my sequel in this world, right? And like gained so much traction of like just, you know, at the party there was a guy that, you know, he's building a really cool company, new, young. I was like, is there any chance on earth you would be using Postgres, like, if it wasn't for PlanetScale, right? And he was like, oh yeah, 100%. It was, you know, so we showed up with that. You know, if all the job is, is light $30 million on fire to make $2 million, I mean, that's fucking easy. I mean, I can do that.

Yeah, personally.

I just light Lambos and give them away for a third of the price. I mean, if that's all they're doing, like, You see this hype and this— you see these user graphs, it's like, yeah, like any— if that— if the job is to light money on fire, like to make no money, that's super easy. And that's— and that's— that's the vanity of— that's the vanity of tech Twitter, which is like, yeah, you know, um, David Kramer, who runs Sentry, is hilarious as well. You know, he puts it like, you know, he's built a phenomenal business and he puts like 10 million ML.

Yeah, yeah, I saw that. He removed that. He removed that recently. I wish he put it back. It's Somebody didn't know who he was the other day and was being like, oh, how much money have you ever made? I'm like, yeah, man, I wish he had that in his profile still, the tens of millions MRR.

Yeah, yeah, just to generate like a phenomenal product used by just thousands of amazing companies. And it's just like, just go— just goes to work and does it, you know? Just like tech Twitter, that, that's actually, if I do, I guess, advice for anyone, it's, it's all a mirage. It's like when the fucking ACH clears for millions of dollars into your bank and all of your employees' bank accounts, that's when you won. Every minute until then is just a waste of time, you know? Like, it's just thinking about anything other than obsessively thinking about that, like, it's just air.

Yeah.

And at the end of the day, like, there's a graveyard of dead companies that looked incredible.

Yeah.

All the way through and made— and then there's these rocket ships that just slowly build. Like, like, I always, like, remind people that if you were a Snowflake at the stage PlantScale is at right now, you'd be like, dead company. Like Series D, flat revenue growth, 2 CEOs fired. And then it's the long game. And then they go off like a rocket. You just can't tell. And it just takes way— like GitHub, again, ripping hot company, sold for an amazing exit to Microsoft in year 13 or whatever. It's a long grind to go and do this. And checkpointing today's Twitter optics in any form of indication of future or success or idea is dumb.

And there's just so much reinforcement of this. Uh, if you look at YC companies that were like the hot YC companies during Demo Day, if you like analyze them, it actually like were like the worst. Like you, you get to like perform, they perform really badly. So it's, it's so hard to, again, going back to like being able to get clarity, it's like that stuff just creates so much noise. And I'm back to the thing you were saying about the free tier and $39. That's the thing that was so funny to me because everyone was like, Really upset that the free tier was gone, and to me, and I was like the discussion like like when I was talking about I took that for granted. Okay, the free tier is gone, but like hold up, thirty nine dollars a month is free. Like to me that's like that's free. Like that is free. Like how is that? Like we would never even think about think about that. So it's just funny that there was so much debate over uh the free tier is gone, but like. Man, it's still just basically free, you know?

I wanted extra protein with my Sweetgreen salad the other day. $28 for a salad. If we fucked— like, it was fine. I just watched her, like, out of the little trough, just scoop the preallocated 3 ounces of chicken, right? For $6. And you just think to yourself, we could end thousands of businesses today by fucking up. thousands of companies could just go, right? Like when CTOs of public companies sit you down and go, if we go bust, you go bust, you're like, hmm, you have a $300 billion market cap. That's a sobering realization. And people can't, like, people don't want to pay $39 for like that. Yeah, I know. It doesn't make any sense at all. You're not serious. Look yourself in the face and say, what else are you going to do? That's so, you know, like it's just, so that is the point of the free tier. Like, you know, I don't want to hate on hobbyists. I don't want like indie devs, whatever. Do you do whatever you want. But like, at the end of the day, I have a viewpoint. They have a viewpoint. I only care about mine.

Yeah. Yeah. And that's actually the tough part about all of this. It's like, it's not exactly that they're wrong. It's just that I don't want to— I'm not interested in engaging, like talking to you guys about your world. Like, I want to talk in my world about the stuff that I care about with other people that are also in my world. But when you're online, it's just like everyone just interjects into each other's stuff. So yeah, like the things that I'm doing look ridiculous from your point of view, but like, I'm not talking to you. I'm not saying you should live like this. I'm not saying you should do any of this stuff, but it's not— that doesn't mean it doesn't make sense.

Yeah, it's that famous Hacker News comment to Dropbox.

Right.

You're just asking.

Yeah. Yeah. We've been on a— we managed to do between Terminal and SST, we've been on Hacker News, I think, 4 times this past, like, past year or so. And I get so stressed every time it happens. Like, I used to— there used to be this thing where I'd be like, man, I'd love to, like, do something that, like, makes it on Hacker News. And I have a feed I go through with like Hacker News, a bunch of other stuff. I remember I was reading something, I was reading an article and I was like, man, this guy's getting roasted in the comments. I feel so bad for him because he's being misinterpreted, blah, blah. Like, I was reading that, then I swipe up to the next one and it's one of our stuff and I'm like, fuck. So now I'm just like, I want to be out there, but at the same time, like, not too out there because—

yeah. Yeah, I know, I know exactly what you mean. Yeah, you like, you, you kind of Too close to the fire. Yeah, no, it's, it's the same for us, which is like, yeah, it's good, it drives some traffic, the comments are just crazy, and then nothing changes. You go back to doing whatever.

Yeah, I know, you know, it just, it becomes normal.

Yeah, is that— what's that meme like, you know? Is it like the Tyler Creative, like, cyberbully? Like, just close your laptop, bro. Like, it's just like, that's it, you just log off at the end of the day, you know?

I was saying this the other day, I was like, there's so many companies that don't— that can't do it. Like, if you were honest with yourselves, like, you just can't do the internet thing, like, it's not in your skill set, you don't have to. You can just not do it. I feel like so many companies do it and do it badly. It's actually totally valid just to, like, opt out and not do it. You can still do a lot of stuff, you can still accomplish a lot of things. This isn't going to be one of your channels, but whatever.

Yeah, yeah, the whole, like, hello fellow kids. If you can't play, then it's only downside for you if you jump into this maelstrom.

You can just ignore it. You can just completely ignore it.

100%. Well, unfortunately, I've got a day of customer calls to go and do, so I'm gonna have to part ways and say thank you again for having me on. Obviously love spending time with you.

No, appreciate you coming on.

Adam, whatever you are dealing with right now, I hope that has gone well.

I don't think it has, but I know you'll appreciate that.

Sending you love, Adam. Um, but yeah, thanks for having me, and just like, thanks for producing something that gives me a little value and delight every week.

Yeah, congrats on the, the launch, and yeah, we'll flip our stuff over at some point when I have a chance to look at it. But, uh, yeah, excited to see this.

You know we are. You're in the top tier of support, CEO support.

Yes, yes. No, I talked about that a bunch.

Yeah, whatever you need.

All right.

Whatever you need. Thank you, Adam. Thanks, everyone. Thank you.