Patrick Collison recounts Stripe’s darkest moment

Patrick Collison describes Stripe's eight-hour API outage caused by a data-center switch failure. He rushed to the office at 4am expecting angry customers but found nobody had noticed, which felt both relieving and deeply unsettling.
Chapters
I remember our first meeting with a bank. And, you know, I don't know if she literally had like the security button under the table, but like, I think if she had had one, she would have been pushing it. And she told us in no kind of, no uncertain terms that there was no possibility that they would work with us. Right. I really, I guess, feel this imperative to emphasize the long period of sort of, you know, again, two years, it's a long time when you're in it, of working away with so many of these roadblocks and headwinds and people telling us that it couldn't work or shouldn't work was a idea mostly because I think that for others, either here or elsewhere, who are sort of having that experience, I mean, it could mean that you can't actually do it or the idea is a bad idea or whatever. Right. Or it might not. Right. What was the darkest moment? The moment where you're like, this is work. There was a moment very early on, and maybe it was a little bit more, maybe six months in, six months in. We still ourselves, of course, took it very seriously. And the fact that our customers were relying upon us to, for the revenue was a big deal for us. Right. And so my pager went off at 4am one night and the API was down. And again, this is not like a social media app or something, Right. This is the conduit for our customers revenue. So it was a big deal. And so I remember kind of immediately heading to the office. We lived very close to the office. I was at the office within 5 minutes, 4 or 5am and started to try to figure out, diagnose what was going wrong. And it turned out that sort of an upstream switch at our data center had failed. Upshot being API was down and we couldn't do anything about it. And it was amazingly painful, right? Because the thing is down, you sort of want to be kind of furiously typing, fixing, getting it back up, whatever, and just couldn't. It was almost literally eight hours. They sort of got a new switch and they replaced it and the whole thing came back up. But that's not why it was the darkest moment. The darkest moment, of course, was because I was bracing myself for the deluge in complaints and dissatisfied customers and anger and vitriol and everything else. And 1pm rolled around and as, as far as I could tell, nobody had cared or noticed. Wow, dodged a butt. Anyway, on some level, that was good news, but as I reflected on it, I realized that it was actually kind of existentially bad news. And I'd been up until very late the night before, maybe till 1 or 2, and I'd gotten very little sleep. And so I just remember walking home from the office to go catch a little bit of sleep after that, and it was not a great day.
More from Lorenzo

Netflix founder Reed Hastings explains why “family” is a bad metaphor for your startup

Netflix founder Reed Hastings explains why “family” is a bad metaphor for your startup

Stripe founder John Collison explains his product principle of “talking up to the user”

Stripe CEO Patrick Collison shares the tactics he used for finding product/market fit

Stripe CEO Patrick Collison shares the tactics he used for finding product/market fit

Sam Altman explains his two biggest mistakes when he started investing in startups

Naval Ravikant on Elon Musk: “The great entrepreneurs are always willing to start over”

Jeff Bezos explains Amazon’s process for expanding into new products like Kindle and AWS


