Kevin Grüneberg on leading billing engineering at Supabase

Kevin, thank you so much for being on Monetizing SaaS, our podcast. I've gotten to know you over the last, I would say like 4 or 5 months. Got to know you because we were studying Supabase's pricing and billing experience and putting teardowns out online and then you started responding to them and apparently you were the guy that had built all of this. And so then we had the good fortune of talking to you on a number of occasions about everything you've learned. So thank you for being here.

Thanks for having me.

I'm hoping that today we can do a deep dive into everything that you've learned about billing engineering. So this is like, this is like one of those subjects where like, I assume you didn't, you know, kind of go into computer science, go into software engineering to become a billing engineer, but you now have an enormous amount of expertise around a part of software development that is crucial and very much at the core to how companies commercialize, capitalize, make money. Maybe you could kind of take a step back and tell us a little bit of your story and how you've ended up leading billing engineering for Supabase.

All right, I'll try to make it quick so it's not too boring, but I started engineering pretty young, like 12-ish. I was just doing stuff for games I was interested in, but I did get into an apprenticeship at 16, which is like a concept in Germany. It's like you do work full-time in a company, but there's also like a little bit of school involved. And then after 2 to 3 years, you are an official software engineer or an official plumber or whatsoever. So I did that. Um, so started working very early with 16 already, and then by 18 I was basically a full-time professional software engineer according to Germany. And afterwards I went into a couple of companies, then ended up going to an in-house startup for like a big US company. And after that started freelancing for roughly 2, years, and when my freelancing gig ended, I got into Supabase kind of by accident, cuz I've been following them for a long time. And then there was just this offering, which I thought I might be of help, which was the billing role. So to, to give a bit more context on the billing role, in the second company I joined, the, in our startup, we were a small dev team who was responsible for everything. So billing was just included and we were renting out physical product. So you could rent a TV, you could rent a washing machine, usually like these big type of products, or like, like a drone. So for 3 months you can rent a drone and then you would pay monthly. So there's no upfront fee. You get it delivered and then pay 3 months and then we'll pick it back up. So one of the important bits of that was like a subscription management. And with that also came things like dunning and fraud. It's actually quite severe if you rent out physical products, especially if they don't have any upfront payment. So you send someone something that's worth 10 grand, but you don't know if they can actually afford it, right? Like there's different checks in Germany. There's something called the Schufa, which is like a credibility check. And whether if you, for example, in the past had any open invoices and other companies, they all, it's all a bit obscure, but there's ways to kind of check if someone might be credible to buy it. But then again, just because they can pay in the first month doesn't mean they can pay in the second month. So this whole thing became quite important. And also there was crazy fraud. There's actually organized crime where people order iPhones and the likes, and they order maybe to your personal address, but they will wait in front of the house and then talk to the delivery drivers like, oh, 'You have a package for so-and-so,' and then obviously that's not the person. It gets so bad that it's like hundreds of cases where the police has to wait inside of the truck to catch some of those people. And that's like the first time I've really been doing subscription management, billing, and everything that's with it. To be fair, that was just one part of it. We did everything else as well, like the fulfillment. That was the whole company. It was just a couple of engineers. And afterwards, when I did freelancing, I coincidentally went into to a team for contract management. Same thing, it was a subscription-based, um, it was a media company and they— similar, you could think of Netflix. So they just had like a monthly subscription to consume media. And for whatever reason, again, I landed in the contract team because I guess they also saw my previous billing experience and thought I'm a good fit. I wasn't searching for billing jobs, to be honest, like I wasn't. I just seemed to fit in. Personally, I consider myself more of a generalist, so I can kind of help everywhere, and for whatever reason, I keep getting into billing roles. When I saw the Supabase offering 2 and a half years ago, I initially went through it and was just— no one was responsible for billing, and they were looking for someone who could own billing and take over. And initially I just skimmed through it, was like, ah, this sounds like, like, I think I could do it. But then I put it aside. And then for whatever reason, the universe was just really trying to force me into this job because I was, I was bombarded with it on like Twitter. I got like random LinkedIn mails. I saw multiple tweets from the founders. I was already in contact with the founders, not because I wanted a job or anything, just because I've been a Supabase user for a long time. There were some chats, some memes were exchanged many times. I was, I already had some conversation with them. I actually did a case study as well for some personal project with them. And we did like a big migration for 125,000 Auth users from Auth0 to Supabase. And that's where I really connected with Supabase as well. Never in the intention of working there, but just because I really love the product. And then as the job offering was just way too much in my face and my freelancing contract was ending, I was like, why just apply? Like, why not? Actually, it does sound like a nice challenge. It does sound very interesting. I didn't think I'd really, I didn't know what I was getting myself into because it's like, oh yeah, billing. No, it's just billing. I can do that. I was previously doing billing and like 10 other things, but I definitely underestimated billing in a in a high-growth startup. And funny enough, the founder, Paul, reached out to me once I applied, like same day on Twitter and was like, ah, did you just apply to the billing role? And then he started questioning me, not in a bad way, but it's like people applying to the billing role, you gotta ask like, do you really wanna do billing role or did you just apply to work at Supabase? Because there's not too many people applying for billing roles and it's a very niche job, I'd say like you find more people just doing API or frontend or other domains, but billing is such a weird domain for most people, either because they are afraid of it, similar to auth, because it's quite scary if you deal with people's money and you do mistakes. But on the other hand, it might just be, ah, there's not even much to do because they don't really, they have never done it. So they don't know the complexity and all the things that come with it. So he was suggesting me like a, I think, customer success role, solutions architect or whatever. I was like, ah, nah, I don't wanna talk to people so much. I wanna do the engineering. Nowadays I talk to people all the time because I also do support. So, but still I wanted to do something more engineering heavy and with more overall responsibility in the company, just challenge myself. And then that's how I landed in Supabase as a billing engineer. Really, I never intended become a billing engineer. And for whatever reason, I keep getting myself into these roles. Um, and I guess by now I could— I can actually pull it off. But, um, yeah, it was quite funny when I started. It was literally from day one because there was no one responsible for it. So everyone just chimed in as like a problem, um, occurred. And then one— whichever engineer had some time just fix whatever billing issue they just had at that moment. So no one, it was like, no, not streamlined at all. It was super tricky choices that have been made in the past that still haunt me today. Just because it was like, oh yeah, that we can fix it like that. But then if you don't know the consequences of billing decisions, especially the early billing decisions, they are so difficult to revert and such a big effort, especially if you have growing customer base. Sure. If you have no customers, you can just make a big change, right? But breaking changes in the billing domain are extremely painful and just take a lot of time if you want to do it properly without having negative customer impact, right? I mean, sure, you can just roll something out and then, but if then a million customers are screaming at you, then that doesn't help. But yeah, first week was already right into the fire because Stripe was threatening to shut down our account because of card testing. So there was a nice Welcome. And then when I was asking some people, they were like, you're the billing guy, like, that is, that's your responsibility now. Like, okay, thank you. I'll, I'll take it from here. But, um, yeah, it was a nice start.

It's a great story. So there's a lot I want to spend time with you on, and I'm going to try to ask questions that sort of move us through what I know is a really complex subject area, um, that has a lot of depth to it. But maybe we could zoom out for people. I mean, I think like if you're working in high growth SaaS today at like a really exciting company like a Supabase, of course you would end up having a billing engineer or perhaps a billing team. You're hiring a team. But for those that are maybe a little bit less maybe experienced with this role, it might be unintuitive. That one would need a billing engineer. I mean, aren't engineers supposed to go work on core product? Why has the function of billing engineer come into being, in your opinion? How do you think through the history of it?

I think over the years, the pricing has just shifted. Like we used to have one-off purchases and these like monthly subscriptions, right, that are fixed prices. But especially in infrastructure nowadays, the pricing is quite complicated. Like our pricing is still obviously much more complicated than a subscription fixed fee pricing. But if you compare our pricing to a pricing of like AWS, so like hundreds of thousands of products and then prices depend on region, like there's even like a matrix pricing within the matrix pricing. And then that doesn't even cover things like custom, that's just like stock prices. So as these pricing models get more and more complex, it's just the complexity of the billing system is also growing. So now if you have a monthly subscription, the only thing that's scaling is like the customers. But if you have something usage-based, you suddenly have these events. So now for maybe half a million customers, you have a billion events. Now you need the infrastructure. To handle a billion events. Sure, you can do like a daily cron job that just sums it up, but that's going to only get you so far. And then you have like delays of a day. I think the complexity or the necessity of these type of rules just came with more complex pricing. And also nowadays you have so many users on applications. So let's say 10 years ago, like, sure, there were also companies with a billion users or whatnot. But over the last few decades, the amount of people you can reach with these systems, it's just crazy and how intensively these people are using you. Everything is just constantly growing. And that's just like one tiny part of it, right? It doesn't end at, I can now calculate How much customer owes, then you also gotta invoice it. Sure, you can just dump it on an invoice, but then that invoice also needs to be correct from a taxing point of view. So tax, I'm not even gonna get into taxes, but I just want an example. Different countries need different things. German customer is gonna like scream if the tax ID is not on there. So now suddenly you have things like, oh, I need to like redo invoices for customers. I need to do these invoices properly. I have customers from 150 different countries. Now you might need localization or like different, uh, things for, for the invoicing. And you also want the invoice to be as clear as possible because the customer is going to look at the invoice and be like, why do I owe you $1,000? So now you've got to break it down for the customer. They're going to have a PDF. These PDFs need to be stored for at least 10 years because of tax purposes. And again, there's just one other tiny bit, then you got to send all these invoices to the payment receipts to the customers. Now you need like an email service to send that. Now payments, you want like different payment methods. You can't even connect like card payments directly. You want to get into like compliance hell. So you need like a payment gateway to do that. And then they got to manage the credit cards, but they can't be stored on your end. So they gotta be stored elsewhere. And then that doesn't even, that's the perfect world where all your events are correct. All your customers are like, oh yeah, I understand your pricing. Oh, I don't need to look at your usage. I trust you. I'm always very conscious about my pricing choice. I'll never want a refund. I always have money. Don't worry about it. You guys never gonna, you're dealing with people at the end of the day and it's people's money. So people are very sensitive about it. Which if they think you don't charge them right, or if they think you didn't deliver the value of it, they'd be pretty angry usually. So it's quite, quite a hard domain even in terms of customer communication. But then also from a business point of view, you have people not paying. And, um, again, I'm, I'm just scratching on the surface. I'm just mentioning a couple of things that are in the billing domain. And if you consider all of that and you have multiple million customers in like 200 countries, and you might also do taxes, and then there's still the business side where the business, like, are we going to make revenue? So now you're also trying to find ways to optimize revenue, maybe do upselling. Sure. If you have like team for all of these things, if you have a billing team, if you have fraud team, if you have a RevOps team, a growth team, a payment service gateway team, sure. Yeah, then that works. But in a company like this, if you just have one or two engineers that do billing, they're all going to be pretty busy with all these things. And then there's like fraud and all that stuff as well, right? Like, especially as an infrastructure company where if people launch a service and they can launch pretty beefy service, then that's going to be pretty expensive for us. So if they don't pay or if they never intended to pay, that's pretty bad for us as a business as well.

You know, listening to you, what sort of stands out to me is like, well, how does this type of role emerge in high-growth software? And it sounds like part of it is, look, you've seen an evolution in the way software companies price. You've seen the shift from seat-based to usage-based or hybrid. Part of it is at scale, you see the proliferation of SKUs and now you have sort of a combinatorial explosion that is just difficult to manage, especially when you're talking about more than a seat-based subscription and instead a usage-based subscription that has all kinds of variability. And then thirdly, I think I'm hearing you say, like, there are a lot of components to this, right? Like there is calculating, metering and calculating correctly. Sending the correct invoice, dealing with refunds, dealing with people who don't have money. And then maybe finally, this is just incredibly exceptionally sensitive. Like on the other end of this, the customer is going to be sensitive and you, the business, are going to be sensitive. So you have to get it right. So it's kind of high stakes at nearly every decision.

Yeah, I think it's high stakes for both the customer and the company as well, because customer, understandably, if you pay someone you're quite sensitive about the value you're getting. But on the other hand, as a business, you also want to charge appropriately because you're trying to make money. Yeah.

You know, one thing you also started to say at the end was, you know, yeah, you can go throw bodies at this problem. You can go through RevOps and SalesOps and lots of people in an org can kind of get attached to this work. I'll make an argument and I'm curious the extent to which you think it's valid or invalid. The fact that roles like billing engineering teams exist in some ways come from, to your point, the evolution of pricing models. But I would add on to that the evolution of go-to-market models. Like if you're supporting product-led sales and sales-led growth and maybe channel, like here's another sort of angle of SKU permutation. Also, like you have, I think, customers that are really empowered and they want to buy portions of products, not always the whole product. And so companies are doing less of, hey, here's the whole platform. And instead here are tiers or features. Or add-ons. Again, more complexity, more SKU combinations. So that feels like one part of it I'd add to it. And then the second part is it feels like all of that transition has happened in the context of tools that are, you know, of a different generation and tools that in some ways all control pricing or billing from different perspectives, right? Like you might have pricing or billing be controlled by the billing system. You might have pricing or billing be controlled by the CRM. You might, you know, it's like you've got kind of different tools for different stacks, each of which do govern pricing and packaging to an extent. And I would include in that like the actual product itself and how the subscription, the contract is fulfilled in the application. So as you listen to me say that, to what extent do you buy that argument or believe that it's valid or invalid? How would you respond to that?

No, no, I do. It's completely valid. A good example is just custom pricing for larger customers, right? So we have this stock prices that, that's the example I made with AWS earlier, where they have like hundreds of thousands of products and then there's like matrix within matrix pricing, but then, then there might be things like commitments. So a customer might commit to paying a certain amount and then get a discount, which is very different model from discounted line item prices. So there might be even in, in just like custom pricing and commitments and, and there's discounts, there's coupons and there's so many systems in like custom pricing alone that as you scale, there's going to be a lot of demand for this, especially as you attract larger customers, enterprise customers, they want to have like a stock contract. Sure, we have a stock enterprise contract, but they will likely have some type of custom agreement where maybe they already know they're going to be very heavy on the function invocations. So they are a bit worried about the stock price. So we maybe give them a discount on that specifically, or if they might just have an overall commitment, they commit to spending this much money in the year, but we give them a general like 30% discount on usage. So even in that domain alone, there's a lot of complexity. And then you also want to enable the sales team. Any sales and growth team to just talk to the customer and not make any promises where I'm like, oh shit, that's like a year of work. You just promised the customer. I want the sales team to be able to make custom deals with the customer. And we're just like, oh yeah, sure. We support that from a billing product. And if you build your own billing system, then you would have to implement all of these things yourself. Which is nuts. It gets more and more complex. Like rolling your own auth is already not a good idea. Rolling your own billing is insane, especially in a company, in an infrastructure company like Supabase, because yeah, sure. If you give me like 10 more engineers, yeah. Okay. Let's build billing. But like, we, we are not billing at the core. Like Supabase is not a billing company.

Sure.

If you're like Orb or Metronome or Stripe. Yeah, you build a billing system because that's your, that's what you're selling. But Supabase is not selling billing systems. So you don't want to, don't want to build the actual billing system. There's still enough you've got to build yourself on top of any billing system where you've got to be busy for years. But yeah, I agree with your, your statement. Again, I think billing is very cross-cutting and that's also where billing engineers come into place. Because the context you need for billing is quite large. So you always need to think about a lot of edge cases and impact on customers, like on any type of change, given its high sensitivity, right? So you don't want to just delegate all the work to like tech-focused team. I don't want to delegate the API work to the API team and the frontend work to the frontend team because there's so much involved in there. That I would have to talk to the engineer for like a day to give them all the context they would need and all that, that they won't remember anyway. So as a billing engineer, it makes perfect sense to be full stack and kind of be able to jump into all these areas of the product. So I do infrastructure, I do frontend, I do backend like APIs, I do a little bit of data engineering. Obviously if things get way too deep, then I do get help. But overall, a great advantage of a good billing engineer is being able to get into the whole stack of the company and kind of help themselves, which is also an advantage because I can kind of help anywhere, even if it's unrelated to billing, just because I'm very familiar with the overall infrastructure of Supabase. But yeah, there's just, I think people greatly underestimate how complex billing gets, especially in a growing company. I know that Vercel, for example, has like 9 people or like 10 people on the billing team. And I totally get it. Even just like usage insights for a customer, you can hire like 2 people full-time easily that do nothing but like usage insights. It sounds like, oh yeah. You just put a few charts on the dashboard, right? For this, that's a great oversimplification of usage insights, especially if you have like 30 metrics you're charging on all of these based on different data, you have different plans, then maybe you have custom contracts with enterprises and yeah, it just doesn't stop. And that's not even the business side of things, as you mentioned, like then there's sales and other teams that want to be enabled or we want to do partnerships. We want to do, for example, a company approaches us and wants to give out like 1,000 coupons barcodes for their customers. Now you need a coupon system to do that because you can't just give out a fixed code because that's going to be on Reddit within like 3 seconds. I can guarantee you that. We did the same. It was like immediately everywhere. I was like, can't trust people. So yeah, it really is kind of flows into the entire company, which also makes it super interesting and you can have a very high impact. On a billing engineer role, just because you are everywhere in the company. Yeah.

I mean, in some ways it reminds me a little bit of like a term that is often used, like a growth engineer, but it's a type of engineer that has like an obvious and material and ongoing commercial impact on the business. And, you know, that goes without saying, but also obviously impact on the customer. I think One thing that Supabase has done so well, in my opinion, having now studied not just the approach and the philosophy to pricing, but also like the billing experience in the application is it's just so well considered. It's this full, complete, thoughtful extension of the product itself. And you obviously had a great deal to do with that. So maybe can you tell us a little bit about, you know, if I were interviewing as, you know, the next billing engineer to join your team. How would you explain the system as a group of components that work together? What would I need to understand after a 2-minute conversation with you where you kind of broke down the architecture for me?

There's— where do I start this? I'll start with the external systems we are using, which is Stripe and Auth0. Stripe is our payment service provider, which is necessary because you can't save credit card information on your end and you need something to process payments. Like you really don't want to do the charges yourself. Like sure, you can trigger the charge by an API, but you don't want to do the actual charges because now you've got to deal with like a billion banks and connect with them or connect with Visa and MasterCard and all the providers. So you always need a payment service gateway and Stripe is just a very obvious choice and a very, like you can't really go wrong with. Stripe, especially if you're US or outside of Europe. The other system is Orb. Orb is our billing provider, which is responsible for doing the invoicing and calculating the actual usage. So usage events are pushed into the billing system and the billing system knows every customer and knows the plan of every customer. And then as usage comes in, it recalculates what the customer owes and then invoices it. At the end of the month and then also triggers the charge. So Orbis has an integration with Stripe to trigger the charge. And where it all starts is if a customer creates an organization, we at the same time create a customer in Orbis and a customer in Stripe. From there, you choose your plan and the plan is just a subscription. So all the billing is on the organization level and the organization itself does have a plan. You do have a fixed fee. If you go for the non-free plan. But other than that, the real billing comes into play once you start creating projects, because each project is like a dedicated server. So you're going to have to pay for compute, you're going to have to pay for the usage and so on. But creating or removing projects doesn't modify the subscription. It just adds usage. So now we need something for tracking and aggregating the usage. So we have a ton of different sources that could be like Cloudflare logs, like request logs, which we then use for egress calculation or calculating how many storage images you transformed. We have Victoria Metrics, which is like a Prometheus-compatible system for metrics. So that is another source of data. And then we have our own logging system, Logflare, which is another source of data. So we just have these data sources. And then on the data engineering side, we use DBT, which is a tool for, uh, is an ETL tool, so extract, transform, load. So what it does is it consumes all of these raw data sources and then pre-aggregates them to an hourly basis. The reason why we're doing this is we don't want to send the billing system like 12 billion events every day. We want to, and for a user, they don't need on the billing level, they don't need to have every single request. Because that's, that will be insane for a customer that has any type of usage. So what we do is we aggregate it down to an hour, which we think is a, is a fair enough, small enough granular billing period. And these get aggregated and then those events get sent to Amazon S3, where the billing system then picks up the usage. The reason this is decoupled is if every hour you have millions or billions of events, sending that via API is not a very efficient way to do it. So a more efficient way is placing it in like a detached storage. In this case, it's, it's S3. And then the billing system can just download the file, ingest the events. If anything goes wrong, you can just try it again or do like a dead letter queue or the likes. And this is exactly what's happening with Orb. There's also ways to directly integrate with the API, but then again, if you send like 500,000 events per API call, the API call is just going to take forever. And then if anything fails, you don't know what fails or where. So detaching that is quite important for a system that has a decent amount of usage. Sure, if you just have a couple of thousand events, you can do it as like the request comes in, you just call like the billing system, but what if that fails? You just lost the event. So you kind of want to decouple the event tracking from the actual usage invocation because there can always be failures. You want like a resilient system. So that's another part. And then in the Supabase dashboard, we have a bunch of usage insights. We obviously have all the flows. Those to do like subscription and plan upgrades. We, so since the organization is the, this organization has the subscription and all the billings on, on that level, you do have the organization usage insights, which are still based on some older pipelines, at least some of it. As we continue evolving the system, we are, we're trying to get everything to, to new pipeline, to more frequent data, but it's just very long process to do it. And then there's things like the dunning process, whereas if customers don't pay, we inform them. We have some hooks on payment failures where we inform the user immediately and also give them some insights into why the payments are failing, when it is retried, and so on. Then we have the actual dunning of eventually pausing the customer. And there's things like service restrictions where customers go nuts on the limits. We do warn them, but if if they ignore the warnings and they go beyond the grace period, we've got to pause them as well. We have some fraud insights. There's disputes and generally like payments to review. And then there's the whole aspect of pricing itself. So we have the pricing page that we own, but also there's constantly discussions in the background about is the pricing good that we have right now? There's new products being introduced. We need pricing for that. So there's a lot of pricing work as well and rolling out pricing in a graceful is also quite a challenge because you've got to do all the customer communication. You've got to do the grandfathering because you don't want to just roll out a pricing change and then customers are— because every customer has a different billing cycle. So if you just roll it out on one day randomly, then maybe the next day the customer, one customer gets billed and they suddenly pay like a very different price than what they agreed to. So even just the thing of pricing changes in itself, like you got to do the usage insights, you gotta do the pricing page, you gotta do a graceful rollout of the pricing change itself. You gotta communicate to all the customers that are affected, like how does the bill change? Well, technically you don't need to do any of this, right? Like no one forces us to inform customers. But then again, if your core principle is transparent pricing, and I'm a Supabase customer myself, I don't wanna be fucked over. By Supabase and some pricing change. So it's, it's just the right thing to do, um, to inform customers and be graceful and give them a chance to like adopt the usage. On the other hand, we also have to do it sometimes because maybe our pricing was unreasonable or like unsustainable for us as a business. We always try to find ways to do it, um, gracefully, but sometimes it's just very hard. Sometimes people don't understand a lot of these decisions. If they come off weird or if they come off as in, oh, why did you guys do like this? The likely reason is because we didn't want to fuck over all customers. So we found like some compromise because otherwise we would have just gone for simpler, higher pricing or whatever. I'd say those are the main components that come into play. And then there's a lot that just connects to all the billing itself.

How do you think about entitlements as a component? Do you view it as a standalone component or is it, how would you describe it generally?

Yeah, the reason I didn't mention it is because our entitlement is very bare bones and you can tell by now. So as a platform the size of Supabase, entitlement is quite big. Like I'm not just talking about the pricing plans that have all the different, all the different items and like line items and pricing, but there's a lot of stuff hidden that might not be in the pricing page. So a lot of like features might be gated behind certain plans and so on. And all of this is currently tightly coupled to our billing plans. So there's like a thousands of checks across the codebase of, oh, if the customer's on the Pro plan, he can do this, or he gets this limit. And this is not just isolated within one backend. It's in our API, but then it might also be in like the real-time server because the real-time server is like, for each customer, there's like a tenant and then the limits are in there. Then it's also in like functions. It's in our API gateway. It's in our frontend. It's scattered. Throughout the whole infrastructure, basically. And this was done like that before I joined. And again, hard decision to revert because you've got to refactor everything. So we sometimes have these cases of, oh, we want to give this customer permission or a custom limit. And what we end up having to do is like hard code it in like multiple places and have very weird code that's not safe. And then if there's plan changes, it might not persist and all of these things. So, um, one thing we will have to do this year, and it's the only reason we haven't done this, is like lack of time thus far, is just decoupling entitlement from billing, which again, what I've said initially, some of these billing decisions are very painful to revert. Entitlements, I wouldn't consider the most painful. It's just a big refactoring. It's very annoying. To do it across the entire codebase. It just takes a long time. But since it's mostly an internal change where you don't have to inform customers about it, it's not as bad as like swapping out the billing system itself and like changing the invoicing and so on. But it's still a shit ton of effort, especially if you do it after like, Supabase has been around for 5 years. We now have over 100 employees. You can imagine how much code is like, and Supabase is a big product itself. There's a lot of functions and sub-products and so on. Entitlements are scattered everywhere. So it's going to take many, many weeks of just refactoring, of just me doing nothing but code refactoring. And obviously I have to do it in a very graceful way that it's not breaking anything. It's going to be everywhere and there's going to be merge conflicts all the time. And so that, that's, there's like a few of these things where if you, if you don't do it right in the beginning, and some people don't know the consequences because they've never really built a billing system or they've never been in a billing team long enough to feel the pain. So people just said, ah yeah, I'll just do like plan checks., which is okay if you just have like 3 plan checks across your codebase. But if you have like 3,000 checks and then you need like big enterprise customers need like custom permissions, or you have these high-value customers who you want to give early access to a feature, like how are you going to do this now? Because sure, you can have like a feature flagging service, but like on the front end maybe, but then people sneak their way through API calls. And so you want it to be consistent across your entire codebase. Sometimes it's not enough to just have like a feature flag on the frontend. So, yeah, I think entitlement is just a big one that if you, if you take one more hour to do it right in the beginning, you're going to save weeks, if not months of refactoring down the line.

This to me is sort of a little bit of a, It's a little bit of a dilemma facing startups and what I'll— let me see if I can explain. I have interviewed founders who, after having a certain amount of success, reach a point where they have to go refactor a lot of the entire monetization stack. You know, it's not just billing, it's entitlements, it's how they think about the product catalog, it's the relationship of the CRM to the billing system, to quoting, to the application. Provisioning. I mean, and what is always noteworthy is that there is a billing engineer behind the scenes saying we should have treated monetization like a first-class citizen in the product way earlier. And yet that doesn't seem realistic, right? Because startups are fighting for their lives. They're doing the most important thing at the time. They're not future-proofing. But I want to press on that idea and I want to see if you, how you'd respond because In some ways, like we've stopped building auth. Like in the beginning, we just buy an auth tool. We don't rebuild a lot of this stuff. There are now standards for how we do certain things. It does not seem to me that there is a standard for how we think about monetization architecture early in the life of a startup. And it almost inevitably comes back to be an expensive problem if the business is successful. What do you think about that? Agree, disagree, think that's fine, that's just the nature of things, or that there's maybe an opportunity for companies to actually be better businesses by having better architecture earlier in their cycle?

No, it's a hard agree. I feel like I'm also sometimes asking myself, how do companies get into this situation, right? And I get it, like high-paced startup initially, you don't even know if you're going to make revenue, so you don't really put time into the whole billing stack. Well, I don't have to worry about customers not paying because I don't even have customers paying, right? So why worry about that and all these things? I just have a simple product, I don't need to worry about integrations. And I get that to a certain extent. So I think on the one hand, on the one hand, it is an educational problem, whereas most people don't really know the complexity of billing and what comes with it and like the consequences of bad decisions. Because again, internal code refactoring is annoying, takes time, but it's internal code refactoring. But larger billing system changes are very, very difficult to do gracefully with all the customer communication. If there's a tiny mess up and suddenly you charge customers too much, you overcharge 200,000 customers. It's like, Yeah, good luck with communicating that. And like, that's got to be a nightmare to deal with. So the impact of billing is huge and of any billing changes. So I think again, education is one bit, whereas we need to educate people more about how billing should be done. And I'm not talking about over-engineering billing. It's just, there's a couple of concepts, as in entitlement is a concept that people should understand and people should understand that why is it a bad thing to couple entitlement and billing. If you just decouple it from the get-go, it's not really going to be more effort to do it initially. So it's not a thing of, oh, you got to put another 3 months of engineering in at the beginning to like tackle a problem that you might run into a couple of months or years later. It's more of, yeah, there are just things in the billing domain that people should be aware of and what are the consequences. And I think an auth, as you said, by now most people, there's still some crazy people out there that think you should roll your own auth. And sure, if you just have like a sign-in via Google, yeah, I guess. But if you, if you're a serious company, yeah, like, oh, we don't have to get into rolling your own auth. But somehow people understand it. I think because people understand auth a bit better because they, they can relate to it a little bit more, I guess. But then again, education is one bit and the other part is probably just, I think people think they need to start. Building their own billing system because, oh, I have— I don't have a fixed pricing model. I don't know where this is going. So I don't want a system that is going to restrict me. I don't want— I want to— I don't want the constraints like Stripe usage billing. I don't want the constraints of that because I don't even know if my pricing model is going to fit into the billing system. So even though they don't have any requirements that don't fit into a billing system, they will just assume that they might have requirements that might not fit. So they just start doing it themselves. And I think—

which is kind of the opposite of what you want, right? Like you want the ability to test lots of things, especially early on.

Yeah. So I think it comes from being scared of my requirements might not fit into the billing system and, oh, if I use Stripe now, I'm going to be locked in so much that if I use AuthLite now that 'But I have this very crazy pricing model and I'm changing my prices like once every 2 hours.' And at the end of the day, usually these type of changes are quite conscious and they take a lot of time anyway. And you're not just gonna, if you have like the craziest pricing model that doesn't fit into, into any billing system, you probably should question your pricing model and be like, which customer is going to understand this? Like, should I really do this? Rather than being like, oh, I have this super crazy pricing idea and it doesn't fit into Stripe, so I'm going to build my own billing system. Like that is a, it's probably not the solution of doing that. So yeah, I think those two mainly education and then people assuming that it won't fit into any existing billing system because their cool product is so crazy that they can't put that into any existing billing system or just underestimating the the complexity, right? It's just, ah, we'll just do, we'll just do a couple of entitlement checks here and there. We'll just, ah, we'll just use Stripe. Ah, we don't need this. And then, but when they realize that doesn't fit anymore, usually it's already very late. And yeah, but I don't have a, I don't have a good solution besides hiring billing engineers early on. By having someone in the team who has like seen it and is like, let's not do this because it's going to cost us like a year.

Okay. Well, let me, let me press you on this. Could a solution not be having a robust separate entitlement service early on?

I mean, it makes perfect sense to have something in place early on that kind of combines the best practices around billing, ideally something that is kind of modular where you can opt into. So you don't want to be hardcore vendor locked into billing. And obviously billing is always quite vendor locked because you're going to rely on the whole billing system and all the data structures are very different from each billing provider. But ideally, if you have a system that kind of gives you all the best practices of billing, and that is entitlement metering, usage billing. Pricing, pricing components, usage insights, and all these things, that is an amazing head start. And if you eventually outgrow the entitlement system, or if you eventually outgrow the metering of that particular system, and it's modular enough where you can still use entitlement, but swap out the metering, then I think that is the best you can do to just Again, you're very pressed as a startup. You don't want to spend half a year building or integrating a bunch of like billing systems and thinking about entitlement, thinking about pricing. So you want to start quick, right? You want to get it out as quickly as possible and still not, or still be as flexible as you can. So a system that gives you all of that with great defaults and great flexibility is It's definitely a perfect start. And if you eventually outgrow parts of it, if you might never outgrow it, but if so, and you do hire more like dedicated people, because now you've got to like 10x your usage insights, you got to either build on top or start getting some bits out, then that's fine because you've got a very solid base and you did best practices from the get-go. So that is not nearly as painful as having like 11 handwritten billing components that now you gotta deal with.

What I hope this type of episode and others like it can do is start to, to your point on education, evangelize to a world that there are components in the context of billing architecture that if in place early on, drive a, a lot of agility, flexibility, capacity to iterate and be competitive, and b, save a lot of time and money and really complex work. It's, as you've said many times, very, very high stakes.

I have one good example for this. I think like half a year or a year in, I think we wanted to introduce the team plan. Which we already had the free, pro, and enterprise plan. And then we wanted to introduce the team plan. It was straightforward in terms of code, but it just took a long time because we had to put it in like a billion places. Because again, all the entitlement checks are like inline. If customer is on pro, if customer is on enterprise plan, then there's like different limits, which are like scattered across the whole infrastructure. Introducing that was quite a lot of work. So if you have a system that streamlines plans and entitlements and all of these things, things like adding a plan is like 100 times quicker. So you could argue that we could have done this in a way from the get-go with like Stripe and have everything dynamic and whatnot. But again, because no one was really familiar with the billing domain and no one was educated on the domain. They just thought like, I'm just going to write these two lines of code because that's, that is fixing my problem right now. And then those two lines of code that happens a couple of thousand times. And then you have all these custom billing while we could have saved a lot of time by integrating with the system that gives you these best practices. Because we don't build a lot of these other things as well. Like we don't build a feature flagging service. We just use ConfigKit. And like a lot of other things. So why are we building all these billing components ourselves, which is crazy. I would rather build a config kit myself than a billing system. But yeah, I don't know. Maybe it is just a lot of education, but I do hope that an episode like this helps. We've got to reach non-billing folks though, because the billing folks will agree. And then we've got to reach those people who still think you should roll your own billing. Yeah.

Okay. It's been so much fun. It's always such a pleasure to talk to you. I hope we can do this again because I think that we barely scratched the subject.

Yeah. We can do billing rant episodes. It's just me ranting about billing and customers for an hour. We could fill a series with that.

I think we could. So I'm game if you are, but this is a great first start.

Awesome.

All right. I'm going to stop this.