Credit-Based Pricing for AI: What Works, What Fails, and What to Avoid

Guys, thank you for being here. Um, got to know both of you over the last year and have talked to you a lot about how to monetize AI and software. So I'm grateful to you for being on the show a second time for you, Tito.

I'm excited to be here.

Yeah. And a, and a cool, cool to do it together. So let's dive in. You know, Patrick Collison came out a couple of days ago and talked about how quickly Stripe's usage-based billing product is growing. Um, and a lot of that growth is driven by companies like Cursor. And you guys know Cursor well. I'd like to just maybe kind of ground people in what do you think are the second order effects of this relatively meaningful shift towards usage-based pricing driven by AI?

Oof, it's a hard one. Starting with the heavy hitters. Well, I think it's, I mean, just zooming out a little, I think it's really interesting that we were always kind of thinking about a particular set of pipeline metrics, growth metrics, uh, sort of assuming SaaS margins. And we just had this very, uh, predictable revenue thing. You know, when a company gets to Series B and it's growing at a certain rate, like you're just going to be able to like apply some formulas and it's, you know, you can roughly figure out what's going on. I feel like one of the crazy things is like you have just no idea what's going on. Like, you know, the margin could be break even, the margin could be, you know, 20%, the margin could be 80%. And like, Those are all very, very different businesses. But I think obviously the growth is like unprecedented that we're seeing across like a number of companies, but then you like don't actually know exactly what's going on with the LLM underneath and what's going on with the business and where value is accruing. So I don't know. I think one of the second order effects is just like nearly impossible to really tell like how a company is doing and what the end game is.

And I guess even as a consumer of these products, you're just never really sure what's going to come out. You don't know if there's going to be a much cheaper model a couple of weeks from now, or if there's going to be a much more expensive one that's going to give you 10x the ROI. So it is really, uh, everyone I speak to is like in sort of a holding pattern, but also excited for whatever is coming next. It's just really hard to predict in general.

Yeah. One of the, I mean, it feels to me that like, A, your, your kind of margin profile is hard to predict. It seems like forecasting, I imagine, is also hard to predict. Like that feels like a second order effect to me that CFOs must be struggling with. CROs must be struggling with.

Well, and buyers too, right? So as, as you know, we sell something that's like heavily usage-based, it's like, you know, $20 a month or $40 a month or whatever the seat price, like that's totally fine. But like, how much am I actually going to spend? Is it going to be the $200K of like license or is it going to be $5 million? And, uh, you know, and then if it is $5 million, should I budget that as like AI interns writing my, my software? Or like, you know, is that headcount budget that I'm spending?

Even, uh, I was, I was having a conversation with someone earlier today and we were talking about how, uh, model inference can get cheaper, but then models are smarter, so they're going to start doing more work. So it was, it's almost like You're not thinking too much and then you're overthinking, but then finding that perfect balance of like the AI model, it's just almost impossible to figure out. 'Cause if it gets too smart, then it does a lot of work. And even though it feels like the price per token can get cheaper, you can, you may end up producing a lot more tokens just by virtue of the model being that much better and that much smarter.

It's kind of like an irony. It reminds me of this other irony that I've been thinking about, which is like, People will often ask me like, what are the right, like, what are the price models to play around with? I kind of generally think, all right, there are like 4 price models. There's like your traditional seat-based, there's like pay-as-you-go, there's credit burndown. The nuance there is that it's pay up front, not pay as you go. And then there might be like flat rate. You might have like hybrid across all 4 of those. But like, if you choose credit, for example, like one of the irony is like, it's designed to be like flexible, like usage-based, but also more predictable. But it's not really predictable to your point if I don't know what the value is.

Pre-purchasing something to be used like 3 months from now, that entire math is just maybe completely invalid in just a couple of weeks.

Maybe one other, just around that pre-purchase point, one other thing that we're observing is it's like hard to know like which model you're going to use because just the underlying foundation is changing so quickly. And so even though, even if you're pretty committed to a certain like workflow engine or application layer logic, like, you know, that model companies would obviously love that you pre-buy a million dollars of inference on this model, but it's just really hard to know that because on a week by week, month by month basis, like who is state of the art and who is best price performance and all of those questions like change dramatically.

Yeah. And as a consumer of those models, how do you even evaluate? It's mostly on vibes if you're, if you don't have like, yeah, the training to evaluate something more objectively, which nobody should have because like it's so nascent. But yeah, it's a lot of it is based on vibes.

You guys had a really interesting first row seat to building out monetization at Segment together. If I'm, tell me if that's incorrect, but then— That is correct. Okay. And, and, and that was like an interesting, of challenge and body of work. And then you went and founded a company together, Koala, where you had to rebuild monetization. Can you talk a little bit about like lessons that you took from Segment to Koala? What did you keep? These are good practices. What did you leave behind? These are bad practices.

The biggest thing for me was, is just like really thinking about like where value accrues and like where the value that you specifically provide that's differentiated is sort of where your margin comes from over time. And so I just think you need to have like high clarity of thought on like what that is. So for instance, at Segment, uh, a lot of the value accrued in the like application logic that was like merging together different data streams. And so one of the flaws in our original pricing was like the actual data router was being marked up quite a bit. So like the core infrastructure, which to our buyer looked like Kafka essentially. And so like, they're like, why am I paying like 10x over the raw price of, you know, this streaming data? Um, we got away with that at the, in the early days, but I think like as it became clear and as AWS made those building blocks easier, it just became harder to really sort of command differentiated pricing from that. But the place where we really could was in the, is a product called Personas, but it was stitching together all of that data into profiles. And that was like a very expensive thing for a buyer to go do and provided like real kind of measurable value. It was like return on ad spend., from that, you know, once we really had clarity on like where margin accrued, that, that made it pretty clear what the pricing model should be. I think we didn't have that in the early days, uh, at Segment. And, uh, I, you know, I think one of the things we really tried to do at, at Koala is just have clarity on like where it is that value is accruing and like what's getting commoditized as sort of path, path through underneath us. And I think like the more you have that clarity of thought, the better you can construct like a, a really good pricing model.

Yeah.

And I'd say from an engineering and implementation perspective, it's like, it's being very intentional from the beginning in designing it to change. As Stephen was mentioning, it's like all information that you're learning almost in real time. So I don't think there's anywhere that I worked where there wasn't, weren't at least like 5 iterations on pricing by like year 2. So it's just expecting that everything will change. So being intentional in the data model and how you collect the data, how you report, how you model entitlements from very early on. And this could be a little— could go a little bit in the weeds. But one thing, one lesson that I took, I guess, like one of the main lessons that I took is like that intentional data modeling of entitlements and metrics and separating what's the metric that you want to build on, what's an entitlement that you want to make available. Incrementally or whatever way you want to deploy it makes a big difference in how you evolve your pricing over time. It's not uncommon for like the less mature and your organizations to just sprinkle business logic all over their codebase because it's like, it's a cross-cutting concern. It's very easy and tempting to just like infest your entire codebase with like all of these little, it grows like weeds, right? So being very intentional in how you tame it and how you designed it to be changed and figuring out all the clear contracts. Yeah, this is where I would think like investing in the right platform and investing in the, like the right layers of abstraction makes a huge difference. It turns these projects from being like very dangerous and potentially catastrophic to being something that it's a muscle that you can exercise over time and then allowing you to iterate really fast. And I always like to think about Apollo is a very good example, like the Apollo.io, they're very good. Their architecture is very good around pricing, which allows them to constantly experiment. I once heard a stat that like at any moment in time, they're running at least 2 pricing A/B tests. Right. Any moment in time that you visit their pricing page, there's an A/B test there. And that comes from this discipline that like very early on, you're just treating billing and pricing as like part of your core data model too.

I mean, it seems like what you're kind of saying is like what you took away from Segment was that it was really important from a demand model perspective to decouple pricing logic from application code.

Yeah, I would say one of the surprising things, you know, I was sort of more of the PM business owner persona here, But one of the surprising things for me was just how challenging simple fixes would be. Like, I know Neto's an amazing engineer, probably one of the best in the world in this domain. And I'd be like, Neto, what the heck, man? Like, why is this going to take 3 weeks to do this? And he's like, well, let me tell you. And then he would like pull up his editor and blow my mind. And I think it's just like very easy. I mean, because pricing is such an important relationship with your customer, it's so complex across all of the functions and some of the actual technical pieces of like how baked it gets into like so many different parts of the codebase without the clarity of thought of like what you're going to do and what you think your levers are going to be as like step 1, and then a great architect like Neto as step 2, like to figure out what this is going to look like, you just end up really painting yourself into like a pretty tough corner.

Yeah, I think the, the most common mistake I see is confusing pricing plans for pricing configuration at an org that I worked on once, worked out once, there were 47 different plans, but they weren't really plans. Like, I mean, you don't sell 47 different plans. You, you're probably selling 3 to 5, 6 possible plans. Like, a pricing plan is just a representation of a configuration of a self-service menu, right? It's just picking from, uh, a menu and presenting something, but it's very easy and tempting as an engineer to just go and implement everything as like an if check. If self-service pricing is V45, then X, Y, and Z, as opposed to composing it as to like, oh, these are all the things that I sell. This is all of the amounts of things that I sell. And like, how can I compose them together in a way that they're each of them are their own abstractions? And then pricing a plan is just an offering of different things. And that's very easy to just go straight to the more procedural approach and then sprinkle business logic everywhere. I mean, pricing logic everywhere in your product and then finding yourself in this like very complex, convoluted implementation, especially because pricing is really hard to test. It's really, really hard to test. It often involves talking to a third party. And webhooks and all of those things. So it's extremely rare that you're going to have incredible test coverage. So, and because it touches every part of your codebase, it's also very unlikely that you're going to have high test coverage everywhere. So being extra intentional makes a big difference. It could be a competitive advantage in just how fast you can iterate in your pricing.

You know, one of the things that's really interesting about this is it feels like this was something that you learned from having lived the problem firsthand. It's like not something that would've been apparent to you had you been building Koala as first-time founders, or if you'd been building Koala, having not done what you'd done at Segment. Do you feel like that's true?

Yeah, I think it's like one of those things you gotta learn the hard way first and then once you've lived the pain, then you can, then you can get it right the second time.

Yeah. I think what you've argued is really competitive companies are very iterative and very agile, especially when it comes to their monetization. And one way to do that is to ensure that from an architectural perspective, you've decoupled pricing logic from code so that when you want to go make a configuration or offering change, that is not a multi-threaded change throughout the code base as well as in downstream business systems. And I think what's sort of fascinating about that is like, I talk to dozens of developers every week that actually have never heard the term entitlements. So maybe can you Can you break down what entitlements are and maybe the different sort of languages and/or words we use to describe this surface area a bit more for people?

But the way that I think of entitlement is that finding the minimal unit of the things that I sell. And the, the reason why I'm going there is in enterprise sales, it's almost always going to be custom. If there are two humans involved negotiating, there's going to be a lot of custom things that are going to come in. And from a self-service perspective, there's, there's always going to be experimentation. You're always going to be adding or removing or tweaking certain things. So the way that I think of entitlements is that it's what the unit of value is, the lowest level of a unit of value that I can offer to my customer. And it could be a binary thing. Do I offer single sign-on support? It could be an incremental thing, like the number of operations of like AI operations that I'm allowed to run in a given time. They could be time-bound too. I could be rolling out a feature as a trial, so I could have a trial-like entitlement allowing access to like an enterprise feature for 5 days. I'm no longer giving you— it's the difference between giving you a full trial of my product versus giving you a specific trial to a specific thing that's time-bound and I guess like limited. So not too different from a feature flag. And Fynn Glover, I think you guys were one of the first ones to help me think of it as of an entitlement as a feature flag. So it's almost being the contract for the very first contract for a feature. And not to oversimplify, I think it's tempting to say that like, Managing entitlements decouples all of billing. There's always going to be a level of coupling, but that contract should be as minimal as possible. So I like to draw the contract at the entitlement level. So a binary access to a feature or quantifier or a time-based access control for a specific feature. As I build a feature, I also build the access patterns to it. And who should get access to what. And that's typically how I think about it.

I think that's excellently described. Tito, anything you'd change about that or does that feel right to you?

No, that, that feels right to me. And I think, uh, I don't know, just too many of these systems like happen over Slack where like blah, blah, blah knows this one and then this other person knows this one. And you know, obviously that's like a natural way to want to start, but I think, uh, Yeah, just like very quickly. I think it's the, it's the n-squared nature of like communication between like every single member of GTM and then every single customer and or prospect or whatever. And like what started as a very simple, simple problem and easy to reason about, like, and then you start to like have a couple developers owning different entitlements, uh, just ends up in like a pretty chaotic situation. And then I think the, maybe the worst thing that you still make that work for a bit, cause like. Customers are noisy if they don't have the right entitlements. But like where you really suffer is like, you know, a year later the finance team is like, okay, like we're starting to get more serious about, you know, IPO readiness or whatever the proxy of getting more serious about financial metrics is. And then it doesn't need to be IPO readiness, but whatever the thing that you want to mature is. And then you just, you realize you have like a completely untenable situation of like what's been contracted to the customer versus what they actually have.

And like, You know, good luck.

Like maybe we'll fix it as customers renew type of thing.

You guys have recently been acquired by Cursor and, you know, I'd love to know what you're going to be doing there, but like at a high level, it seems like you're going to go join a company whose mission, I'll probably butcher this, but to me, it seems like Cursor's mission is to help a lot more people in the world create great software products. And if you've got a lot more people creating great software products, you've also got a lot more people trying to figure out monetization and perhaps for the first time. And you guys have both already said, like the first time you run into monetization, like you often end up making these mistakes, reinventing the wheel, hard coding pricing, things like that. What do you think, you know, this next generation of builders needs from vendors in the billing space, in the feature management space, in the monetization space to help avoid those types of mistakes and to help create the type of agility that you guys invested in creating at Koala?

Yeah, well, first things first is we gotta create this type of agility for ourselves here in the new world. So that's what Neto and I are working on, uh, this, this month. Um, no, but in all seriousness, I, I do think like there's just a, a lot of like, as people, you know, as the CTOs of like the largest companies in the world are kind of thinking about procuring Cursor, they're thinking about like, Okay, I know this is going to free up a large amount of my human time, but I don't know how much human time. And so as I go talk to the finance team, I gotta like, am I spending $1 million on Cursor because I have like effectively 10 new engineers and this is like the, the best use of, of dollars? Like, how, like, how do I think about, you know, that? I think we gotta like recognize what every, you know, CTO or not everyone's, uh, Buyer obviously is a CTO, but whoever your buyer is, like be thinking about like what they're thinking about. And I think more often than not, they're thinking about like how this agentic thing that you're building or could be building in the future, how it actually saves them no longer like a traditional ROI thing, but like leverage on human time. And so I think that's going to be like roughly what everyone's thinking about is certainly what we're thinking about, but., but I, I really do think like in this agentic world, that is what folks are gonna be thinking about. And so, you know, I think the answer is a little different for everyone, but I'd be thinking about how you can construct, uh, your product, your offering, uh, so that you really have clarity on like what the differentiated value is over just, you know, going to ChatGPT directly and then how you price fairly, hopefully against outcomes of saving like enormous amounts of, of human time. Mm-hmm. Um, so I, I don't know exactly what that, looks like in terms of a platform other than a lot of the stuff we've been talking about. But I think that's like the fundamental product and business challenge that everyone's thinking about.

The only things I would add is like, it needs to be incredibly obvious and simple to test. And it's very loaded. There's a lot that goes into making something obvious and simple, but really, I'm a strong believer in having engineers go down a paved road as opposed to equipping them with a machete and sending them away in the jungle. So really all the tooling that makes doing the right thing really easy. So being opinionated and providing the tooling such that they can build these abstractions in a safe way, that they can test them in a safe way. It just, it will start attracting more and more people. I believe like, I'm a strong believer in developer experience. The better the developer experience, the more likely it is that someone's going to do things the correct way.

So yeah, in other words, like the vendors need to really consider how to create a standard so that people are not reinventing the wheel and so that the way that they are recommending the standard can be like done and tested safely and quickly and easily. This is cool. I think this is a fun, valuable conversation for people. I'm grateful to you both. Netto, thanks for the time later today. I'll see you this afternoon.

Talk soon, Fynn Glover.

Cool.