Highlighting Data Intelligence with Databricks & Bonbon’s Reward Innovation | E2020

I had to have a lot of conversations with my internal team as well, like my co founders. And the framework we used internally was which path allows us to influence the world more. We are all academics. I care about this field, I care about technology. We care very much about how AI is going to impact humanity. There's a large component that's extremely mission driven here. You know, I would say not all startups form that way. You know, if you have some SaaS startup that's building a cool little feature, you're really trying to like make money, which is fine. There's nothing wrong with that. I have no, no issue with it. I don't think you're really thinking about humanity so much and I'm not trying to beg you be on a high horse or anything like that. I'm just trying to like really say like we cared about this and that's. That was our initial motivation.

This week in Startups is brought to you by openphone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. Twist listeners can get an extra 20% off any plan for your first six months at openphone.com/LinkedIn jobs. A business is only as strong as its people and every hire matters. Go to LinkedIn.com twist to post your first job for free. Terms and conditions apply. And Beehive power your newsletters with AI tools, referral programs and ad network features all in one platform. Get 30 days free and 20% off your first three months at beehive.com twist.

Hey everybody, welcome back to this week in Startups. My name is Alex and we have a very, very excellent episode coming your way. Databricks, as you know, is one of the most valuable private market technology companies in the world today. It last raised $500 million in a series I that gave it a valuation of around $43 billion. But also in roughly the last year the company bought Mosaic ML and before that deal was actually put together, announced and then executed was we had Naveen Rao, the CEO and co founder of that company on the show. Well, I have dragged Naveen back because we are now over a year past the acquisition much has happened in the world of AI, especially on the open source front. So please welcome back to the show Naveen Rao now from Databricks. Naveen, hey, how are you?

Great, thanks for having me on. Cool to close the loop a little bit after. We were in the midst of the acquisition and now we're on the other side of it, so.

All right, let's start there. So 1.3, $1.4 billion deal. How do you, how long did it take to go from like you and Ali Godsi talking to when the actual paperwork was signed?

Well, I guess first conversation was probably like end of March and then we signed the deal like mid June. Actually going from here's what we want to do. We actually want to like the concept of yes, we're going to, we want to work forward towards an acquisition was probably like early May. I think it was, I think it was 62 days from that conversation to the actual close, which is kind of record time.

Yeah, that's a lot of work to do in that amount of time because you were a venture backed company so there's a lot of parties involved, a lot of signatures.

Totally.

I have to ask though because I was just going back through databricks acquisition history prepping for our talk today and the more I think about how mosaicML fits into databricks, you guys are super important because databricks of course built on the open source Delta Lake project really kind of popularized the lakehouse format for bi across structured and unstructured data as everyone knows. And then you guys brought on the ability to train essentially AI models for corporations that have data which nests very neatly on top of what databricks are already built. So a pretty core component of what databricks is today. And given where prices are for some AI companies, Naveen, did you perhaps sell for a little bit less than you otherwise might have? And I know it's rude, but I'm curious if you stay up at night thinking about the yacht you didn't buy.

I mean we're doing just fine, honestly. The math was about can we come together with databricks and actually make the whole even better. That was the way I looked at it. And honestly the valuations are high right now, but they're somewhat paper valuations. I had a conversation with Ali beforehand is really the valuation and stock price of databricks is strictly less risk than Mosaic. So we're moving from kind of high risk, you know, very, very high beta to something that's slightly lower risk, probably still high beta as far as like public stock go. But you know, we, we saw that there was an opportunity to make the whole better and really that's what the thesis was was. I mean I can kind of walk through the logic. It was actually quite simple. If we stayed at it alone, you know, things were growing really fast and let's say in three years I was looking at a three year horizon and was like, can we get to, you know, call it 300 million revenue? That would put us in the 5, $7 billion range. As far as valuation goes, we'd probably need to raise around a round and a half to get there. So there's going to be some dilution taken. And then if I. So that's scenario one. And so that has certain risks associated with it, of course, the market and what happens with AI, et cetera. Then there's databricks, which basically I said in that same timeframe, can we double the price? And I think that doesn't sound crazy.

Right.

In three years, can I double the price? And then we're actually at the same place in terms of how much our stock is worth. So I looked at it from this perspective and the, the databricks path was lower risk. And really the first conversation I had with Ali about it was, this all sounds great, I'd love to do it. The economics just have to make sense. Like those numbers have to line up. Right. And so we got there. Yeah.

Well, this, this brings us to founder mode. You had a tweet from a couple weeks back about. You said the funny thing about founder mode is that if you have to tell everyone, you'. It maybe you aren't found that really funny, but I do think that you have to be in founder mode to get a deal done in 62 days from start to finish. Did you have to jostle investors a little bit to get them to agree to this, or were they pretty on board from the start when you put the deal together?

You know, that's a great question. And literally every one of my investors wanted me to continue going.

Really?

Yeah. And you know, and it wasn't in a bad way. I wouldn't say this in any negative way. My investors, actually, I've known for a long time, many of them, and they're like, look, let's go at it, let's not sell, let's go bait. Right? They're like, if you want to do this and you think it's the right path, we'll support you, but we're totally all in to keep going. So I thought that was a great message and really it was hard because I had to have a lot of conversations with my internal team as well, like, you know, my co founders and, and sort of the framework we used internally was which path allows us to influence the world more. That was it.

Do you think that that perspective is the right one for founders in general to take as they observe a possible acquisition offer, or is that More of a Mosaic ML specific perspective.

I would say definitely it was a Mosaic thing. We are all academics. I care about this field, I care about technology. We care very much about how AI is going to impact humanity. So there's a large component that's extremely mission driven here. You know, I would say not all startups form that way. You know, if you have some SaaS startups that's building a cool little feature, you're really trying to like make money, which is fine. There's nothing wrong with that. I have no, no issue with it. I don't think you're really thinking about humanity so much. And I'm not trying to beg you be on a high horse or anything like that. I'm just trying to like really say like we cared about this and that's, that was our initial motivations.

Well, I mean your comment was whimsical and you know, adds a slight edge to it, but I don't think you're off base. And also I think what we've learned watching SaaS companies as a specific cohort in the last two years is that if you built a SaaS company that was a very strong feature that raised money in 2021, you may not survive in 2024. So I don't think your comment, even though it was slightly sharp elbow, was off base. Now on the mission point though, this is a good thing to touch on because databricks of course has a open source heritage, if you will. And also the Mosaic ML company put together a number of very popular open source models including MPT7B and MPT30B. One of them got more than 3 million downloads, which I think at the time Naveen was the most downloaded open source LLM. Is that right?

It was, yeah, yeah, yeah.

So it seems like there's a real kind of like mission agreement when it comes to open source in general at Databricks plus Mosaic. And this brings us to the recent news item in which we saw that SB 1047 in California, a AI regulatory bill, was vetoed. Now one sticking point or one point of criticism in that was that that bill might have made open source AI development more frankly legally risky. Now that's behind us. I'm curious if there's anything that you think is standing in the way of, of open source AI development when we consider it in a competitive race with closed source.

I mean, I think the potential for lawsuits on copyrighted information obviously is, is real. There was still the executive order that was put out at the end of last year that put Some hard limits in terms of compute. It's not clear how this is going to impact open source. I think whether something is open source or closed source is a business decision. But like outlying, it actually causes problems. We're already seeing it with llama models, like multimodal llama models. We're not rolling out in Europe because of the regulations they put put in. So I think that's the, the net of this is that the consumer suffers, right? They have fewer choices and there are fewer ways for them to modify, customize these things for their purposes. So I think it's a net negative if you start to restrict what people can do, especially this early, you know, maybe in five years when we truly understand the impacts and how the liabilities work, et cetera, et cetera. Okay, we can start to rules in, but it's just too early.

All right, everybody, I'm on the road, I mean all the time, right? And that means I am always juggling phones and laptops, apps, all these different services I use. And when you use multiple devices and you have all these different apps running your business, you need to have one single phone number that is perfect. And that perfect Phone number is OpenPhone. It's going to simplify all the communications you have in your organization because OpenPhone has rethought the modern business phone. And it's so magical. It works with a single elegant app that you can put on all your devices. And it works right on your existing phone, even works on your desktop. Our sales team uses it here at launch. Why do we use it? Well, we don't want people talking to customers on their private phone lines. And our account executives don't want to give their personal phone number out. That's just weird. You want to have everything tight. An open phone will make it tight. And tight is right in this regard. Shared phone numbers are also awesome for things like customer support or when we run events, we like to have a field phone number. So hey, if you're a VIP and you're at the liquidity conference, just call that number. OpenPhone is super affordable. It's just $13 a month. Twist listeners get an extra 20% off because they got you covered. Openphone.com Twist and what if you have an existing phone number? No problem. Openphone is going to port them over at no extra cost. Head over to openphone.com twist to start your free trial and get 20% off. Off.

So on the meta not bringing their most recent family of llama models. I think that was announced at the 3.1 mark, we're now at 3.2 for the meta Llama family. I'm curious what your take here is on this. Is that an actual Meta saying, look, we can't bring them to Europe because of regulatory confusion, or are they trying to make a point by saying that as a way to politely influence the way that AI regulation is put together in Europe?

Honestly, I don't think there's anything too galaxy brain going on here. I think it really is perceived risk and it's like, okay, don't want to take that risk. All of us, every company who's deploying these solutions has to look at that and, and make a decision around the risk. And yes, maybe there is in the back of our minds, like, okay, we want to change that environment and make it more friendly. But you know, I think in the short term we're really just looking at it like, is it worth the risk? Is, is it a risk reward, trade off you're willing to make? And Llama said, no, it's not. Or Meta said, no, it's not. So I don't think there's anything else going on here. I think this is a consequence of early legislation.

Yeah. Does Databricks have any restrictions or set up changes based on different geographic regions for its product set? Because Databricks plus Mosaic is now a pretty broad company. So I don't know every single product line you guys have and exactly where it's available. But I'm curious if regulations have also impacted how you're rolling out new features, open source models and so forth.

Absolutely. Yeah, it is. I mean, we conform to a lot of these same rules. Like, if Meta is not rolling something out in Europe, it's unlikely we would.

Okay, so essentially I can't run, you know, llama 3.2 model on the Databricks Mosaic system in Europe at the moment.

We're still looking at the exact details of this, but. But yeah, we're not trying to like provide an end around or anything like that. That's not, that's not the goal. Right, right, right, right.

I was not trying to, I was not leading towards trying to get you to provide an end run around Mark Zuckerberg, but I am keeping tabs on how open source AI development is playing out around the world because just frankly, as an open source sympathizer, just kind of personally, I'm really excited about what open source AI models have done in the last 12 months. It's given me a lot of excitement that there's not going to be too much of a performance gap between what we're seeing that's open source and available on say hugging Face and what's available via a paid, you know, OpenAI API call. And that actually kind of brings me to the Llama family of models. When those came out in July of this year and then later September, were they better than you guys expected them to be? Because the market seemed to be very pleasantly surprised by them.

Were they better? I think what was this big surprise for us with the 3.1, the 3 and 3.1 was really maybe not the quality of the base models. We knew that was going to be within a certain window, but really the, the care that was taken on the RLHF side of it where they actually did a lot of work there and you know, frankly put a lot of money into it. So that was probably surprising for us how much they did. And I think it was really a statement of here's our commitment, this is how big we're going to go with this. So it wasn't clear when the Llama 2 models came out like they were good, but you know, DBRX and other open source models came out, Mistral and others who were, who that were better, you know, at the time. So it wasn't that, that was, they were far and away. But then honestly, I think in the last, you know, six months, things have really shifted. They're, they're investing heavily and they're building really great stuff.

Yeah, absolutely. And just so everyone knows, RLHF is reinforcement learning from human feedback. I actually forgotten two of the acronym points. I just wanted to bring that up. Can you just explain what that is and why it matters for folks out there who are AI interested but not perhaps AI experts?

Yeah.

So when you put these models in the wild, you can't, you can't make the, give them just the raw model. There's all kinds of weird stuff these models can say. You know, things that might be correct but not necessarily, you know, kosher to say perhaps or you know, sometimes they're the, they can weed down paths that go very dark. And we don't necessarily want our models to do that. You know, especially if you're talking about psychological safety. If somebody's depressed and they type in, you know, what's the best way to commit suicide? I don't want the model to instruct them. So actually providing these kind of bounds and guiding things in a way that to positive or saying I won't answer really, it's pushing all the probabilities around of where that knowledge lives and how to, how to express it. And it's actually quite hard to do this.

Well, yes, and if you're going to do a lot of it, you're going to have to have a lot of humans actually doing this. Ergo, it's expensive. And that's why your point about what Meta put into the llama models matters, because they're not just writing code, they're also doing the flesh and blood work to make these models pretty good. Now, you mentioned dbrx, which is the open source model that databricks put out March of 2024, if memory serves.

Correct? Yeah, end of March, I believe.

And the company said at the time that it spent about $10 million training this model. And that number just kind of sat in my head because I couldn't quite figure out what it meant. It didn't seem like a huge amount of money, it didn't seem like it was free. But one thing I did recall is that Mosaic ML, one of your goals as a company was to make training models a little bit cheaper, more efficient. And so I'm kind of curious. One, are y' all going to be putting out DBRX2 and then two, should I expect models of that size and complexity to become cheaper to train over time from companies like databricks and also perhaps other corporations?

Yeah. So the first one, you know, we're, we're partnering very closely with Meta now. So we basically said, what's the point in providing another choice unless there's direct innovation so we can choose to work together with them and say, hey, these are things that we see our customers needing new potential directions to take to make more efficient. So now we're just working more closely with them. So at the moment we have no plans for DBRX2. On the second point, you know, we are constantly working towards making these things cheaper. So we're still doing base research on these kinds of techniques, like how do we extract the most from our hardware, how do we make these neural networks learn faster and cheaper? And we're going to communicate that with our partners like Meta. And so the DVRX was cool because it was a very large, what's called a mixture of experts. And really this was a way to make things trained cheaper to a certain quality. But not only that, actually make the inference faster and cheaper. And we're going to see this happening in the open source more and more. Mistral actually had an open source MOE model as well. So, you know, I anticipate we're going to see more of these techniques coming out. And really the benefit is going to be to the user. You're going to see a faster response with higher quality and less money. That's where we're moving. I still believe this idea that we talked about at Mosaic, we called it Mosaic's law of forex reduction in price to get to a similar quality model year on year is real. We're seeing that today. We're seeing it from OpenAI, we're seeing it from everyone.

So this actually brings me a thought that I think six months ago wasn't talked about much, but it's now pretty common, which is that people are concerned about the value in AI models and how quickly they can depreciate based on progression. And I think we can see this in OpenAI's price points for a model over time does decline pretty precipitously. So does working with Meta allow essentially databricks writ large to have leading AI models inside of its platform without having to do the work to train them? It feels very much, and I say this playly, like the corporate version of getting your cake and eating it too.

Maybe that's what we're shooting for. No, I mean, you know, look, they want to invest in this and they have good reasons to do so.

Yeah.

And you know, we're close with them. We have, we're non competitive in a lot of ways. Right. Meta is, is doing this to enable a developer community. You know, they're, they're a social platform, they're using the models internally for all kinds of stuff. Those things don't overlap with what we do. We are focused on enterprise and so it's actually a really great partnership. We're very complimentary. They want to build a developer community, we want to enable developers. So it works great. So why not, why don't we do this?

Well, I mean, I just realized that now I have to ask you, once we stop recording, to not sell your company to Meta. Because given that, how you guys are sufficiently well matched, I mean, Meta buying databricks wouldn't be the strangest thing we've seen in the last 24 months, but it would be a top five thing.

Yeah.

Founders, I know that you're keeping a close eye on your burn rate. I am too. In today's venture market, every single hire you make has to be perfect. Right. You can't make mistakes. You got to keep that Runway as long as possible so that you can run more experiments. And you need talented people to run those experiments and figure out how you're going to get product Market fit. How are you going to scale your company? And that's why you need to use LinkedIn jobs. As you know, LinkedIn brings you the candidates that you can't find anywhere else. LinkedIn passed the 1 billion member mark. Think about that. 1 billion members and 70% of LinkedIn users don't visit the other leading job sites. This is a phenomenal statistic. They don't even go to the other job sites. Why? Because they might not be looking and those are the best hires. But they're hanging out on LinkedIn doing professional development, checking in on their network, building their network, sharing content, finding leads, all that great stuff. Bottom line, there's amazing hires waiting for your company on LinkedIn and nowhere else. And they have a special deal right now. Post a job for free. What F R E E what a great price. LinkedIn.com/TWIST that's right. LinkedIn.comTWIST to post your job for free. Terms and conditions Core supply so on.

Models getting better over time and also cheaper. The thing that I've been playing with the most lately has been OpenAI's O1 model. And I still think this has been digested a little bit by people, but it does feel a bit like a high watermark and it was using a lot of reinforcement. Learning goes a little bit slower. Is, is compared to the open source world, closed source now ahead of what we're seeing from open source? Or is this just kind of the give and take of progress, if you will?

I mean the closed source guys are still ahead, there's no doubt about it. I mean OpenAI is they have to stay ahead of the curve. And as you pointed out, like the half life of a model is actually quite short. It actually, it speaks to the economics of this whole industry right now by the way, is that, you know, you spend a billion dollars on a model and it only, it's only going to generate anything for maybe six months revenue. And so that's kind of hard. Right. Like in terms of capital investments, I mean we don't see this anywhere else. Like if I Invest, you know, $100 million into, into designing a car, I can sell that car for five years. Yeah. If I can only sell it for six months, you know, the market has to be really big. So that is definitely going to be a hard thing for the industry to work through. And we're, we're, I don't know what the answer is but in terms of closed source, closed source is still ahead. No doubt about it. Meta is putting the investments in to make open Source catch up. And what I think is interesting is that if you look at the L1 model, I actually don't think there's a huge, a huge step forward on the model itself. On the LLM. It's actually more the orchestration of that model, how that model self assesses and comes up with better answers. Because the way these models work, they actually come up with probability distributions of outputs. So if you look at, if you think of it that way, like there's a probability that the output is incorrect, but the mean of that probability is, is correct. So you have to kind of do multiple generations many times to get to the right answer. And really that's what we're, we're seeing here. And actually it's interesting that OpenAI, who has the most resources thrown at this, has gone that direct. Instead of like saying, I'm going to make the output probability of to be correct, it's like, no, no, I'm going to think about retries and doing this over and over again. So it's, it's a peek into the direction of where things are going. Is this sort of idea of orchestrating the model to have retries and get to a more correct solution through iteration.

Help, help. I can only get so bullish about AI. Okay, so this is super cool. So essentially what you're saying is the core technology is still advancing, but what OpenAI has done in this case is use what currently exists kind of recursively to improve the output. And this is the reinforcement learning element, correct?

No, not exactly. So this is all done at the inference time. So inference time is basically, oh, I see, the model is somewhat static. So you're basically asking the model produce for me multiple results and have some way of judging the quality of those results and then saying, ah, okay, this is a better step forward than that one was. So if you think about what an agent is like, a human agent is you're kind of thinking through multiple steps to solve a problem. You break a problem down and each one of those steps has to be correct. So if you're incorrect in one step, it sort of takes you down a bad path. So you need to assess, self assess. Did I make the right call on this step? I think I did. Then move on to the next step. And really that's what's happening now.

How far can we, can we push this point about orchestration to improve model output quality? If we can get gains from orchestration, does that road have a long way to run? As in, are we just scratching the surface of what we can do with that. Or has 01 perhaps kind of shown the limits of that new technique? It's a way to get better results out.

Oh no, this is just the beginning. And the idea that you can break a problem down is not new. Right? We actually wrote a blog on this, I don't know, six, seven months ago, about compound AI systems. And really the idea here is if you look at any engineered system, let's call it, I don't know, the original coding for computers, right? We would write at the assembler level and as this big monolithic blob of code, probably completely unknown, maintainable. If you think about it like by today's standards, it's like, oh my God, why would we do that? Because we didn't have abstractions then. And so we sort of thought about this big blob of code is completely non introspectable. It was hard to maintain over time. We moved to a place where we could modularize. We could break down code into functions, objects. Each of these things can be independently verified and then strung together. And so advancements in programming languages allowed us to do this. We're going to see the same thing in large language models. Like first we start out with this giant blob that just does something and it proves the concept that we can make something work. Now we're moving to a point where it's like, well, hang on, if I'm an enterprise, I actually want to modularize this. I want to say maybe I want a language front end that can parse multiple languages or multiple modalities. Maybe I want a reasoning engine that understands how to reason over a particular kind of data. Maybe I want a backend that understands when I should be calling to a database or calling functions. And then I want something that assembles all this stuff, creates an output that's formatted in the way my users need it. Maybe that's JSON, maybe that's English, maybe it's something else. So all of these pieces come together and you got to orchestrate them and you want to independently verify them. And in this way we move to a much higher reliability. Instead of talking about this giant monolithic blob. Now I have independently verifiable components that I can actually make really good and I can find bugs and I can debug the system.

Okay, so one thing that I saw maybe five years ago, and I loved this, was a lot of companies thinking about no code and low code programming, essentially abstracting one layer above, actually typing code, connecting things, moving boxes around, et CETERA as someone who hasn't really written code since C in high school, I loved that. It was fantastic. I felt like I had superpowers in the world that you're describing. Is this a future in which people who are not hardcore developers will be able to set up these different models and functions like you're describing? Or will this, do you think, remain the remit of the engineering department even in, say, three years?

I think it's going to remain in the remit of the engineering department. The reality. I know, it's just. I mean, it's one of these things. I think coding assistants have really opened the aperture of who can play with these things. I think it's. There's a certain level of complexity you could express that way. Human language is naturally kind of imprecise. You know, we talk about, like, coding as if it's a bad thing, but it's just a very precise way to describe a problem. That's all it is. There's nothing inhuman about it. It's just a more precise way to describe than English. Like, I actually don't want to program in English. I want to program in a programming language because once I'm facile with that, because it's more precise, I know exactly what the effects are. And, you know, so if I think about it like that, we need it. We need high levels of precision. I need the model to behave in a certain way under certain conditions. I need to describe. Describe those conditions very precisely. I can't sort of hand wave and just say, yeah, go figure it out. Right? I need to actually work all the way down to precision.

Well, I suppose my question then would be, how long until those parameters you have to set or rules you want to set up or test, you want to run are entirely done in natural language versus code? Because at that point, it would open the aperture quite a lot. But it sounds like, frankly, not soon.

I mean, look, I think there are patterns that we see today that. That are common. Let's call it 80% of the use case can probably be use cases that can be captured in, you know, some kind of pattern that I can call from English. And so I can get most of the work done with English. But then there's gonna be that long tail of stuff that needs to be really precise. And I think there's. There's just no shortcut. You've gotta be really precise. I mean, it's like a legal document. It's the same thing. Yes, it's written in English, but it's not really English. It's very precise language. It's meant to describe something, you know, it could be English, but you know, it's still got to be precise.

Does that make legalese then a very high level object oriented programming language?

It absolutely. No, it totally is. Right. Actually there have been attempts to make it into code where I have strict syntactical requirements and things which actually enforces potentially correctness and verifiability. You know, this is a programming languages topic that's been around forever, you know.

I suppose though, if we do then use AI to automate the legal industry out of most of its employment, we can then just take the lawyers and turn them into developers because they'll already be familiar with the style of communication. There we go. Problem solved. Labor market fixed. Everybody, you heard it here first. I want to talk about the enterprise though, and one reason why is I am a little bit muddled about agentic AI. And I know you guys have talked about recently the Mosaic AI Agent framework and Agent Evaluation Services. Sierra by Brett Taylor is working on agents inside the enterprise. To me, it's become a bit of a catch all term, if that makes sense. And so I'm kind of, kind of curious to mean how do you slash DataBricks define agentic AI? And then I want to dig into that a little bit, but let's start there.

Yeah, you're right. It is a catch all term. It became the new cool thing to say and it's not well defined as to what it really means. I mean, I would say how the term came about was building something that can work through a set of steps, you know, some sort of a workflow, break a problem down and actually work through multiple steps to get there. So, you know, we see this all the time with rpa, Robotic Process Automation.

Yeah.

Okay, everybody. You know I love newsletters. I love sharing knowledge and what I'm up to through my newsletter. And if you've got a newsletter, you got to check out Beehive. It's spelled B E E H I I V. I use it for the Twist Ticker and our Twist 500 newsletter. My team is raving about Beehive because it is an all in one platform that not only powers our newsletter, but it's got all these incredible platform features that are helping us grow our subscriber base, which is what these newsletters are about. Listen to this. Beehive's co founder is the same person who helped Morning Brew reach millions of subscribers. In other words, they took all those tactics and they put it into a platform They've got a great feature I just want to tell you about. It's called the AI Post builder. This makes writing easy. As a great writer, sometimes I need inspiration. Well, they will get inputs for your ideas and then shape and optimize your content for the maximum impact. It's perfect for busy founders. Right? And they're available 24 hours a day. They also have a referral program that turns your audience into ambassadors. It works great. Plus, hey, if you want to monetize, they got an ad network. And you know what, it's super affordable, starting just at $39 a month. So here's a great call to action. 30 day free trial plus 20% off your first three months. Go to beehive.com/twist beehive.com/twisT, B E H I I V dot com. What a great product.

Yeah. So, and we want to say like, here's a common pattern I do over and over again. Maybe it's like I have a bunch of data in one file, I cut and paste that, I drop it into an Excel format, then I go and apply this formula like it's a common pattern we see over and over again. But it's just very hard to automate. So I think that's what the RPA stuff is all about. And agents were kind of a way to say, well, maybe I can have it observe what a person does and then learn. That's where this started. I think we're not quite there from the perspective of I can just turn it loose and be 100% correct. But what we want to work toward is doing that within databricks or other data platforms. Because these are the kind of workloads that dominate inside of databricks. People do, you know, ingestive data. They do some sort of transformation and some sort of dashboarding and they can make some kind of decision. Can we start to automate some pieces here? And really that's the overall vision we have. A few different ways that this looks today.

Okay, I want to pause and talk about this because it sounds like when you describe breaking up a problem into smaller pieces, taking steps, honestly, that reminds me of going back one step. What I see with 01 when I ask to show me its logic. So to me it feels like the, the agentic AI you're describing is kind of like what that model is already doing. And I presume here that I'm confused and I'm oversimplifying things. So can you just set me straight?

No, conceptually you're right. And I think what the big difference is between that and what we're doing is the idea that I want something that's really, truly bespoke and built from my process, my data built from my data and the learnings that I have within my company. Oh, one is great. It's a concept and it works well under the conditions where they tested it. But you can break it pretty fast. I mean, if you just go on Twitter and look, there are lots of people that broke it and made it do stupid things with really simple questions. And I think this shows that to make these kind of systems work, you do need specificity. You need to work from your data. Your data is the ground truth of what should really happen. It's not a general problem. And you know, if this is not a foreign concept, if we look at like humans, it's the same problem. Like if I got generic smart kid out of college and said, go solve this problem that I have 20 years of institutional experience around in my company, they're going to fall apart too. They're not going to get the question right. Maybe if they're really smart over time, over the next couple of years, you could train them to be really good at it.

Yeah.

So I think this is the same thing we're seeing here, is that we want, we see the value in building things that are, that are bespoke and experts within a particular domain. We call this concept data intelligence, where you can use your data to create intelligence to actually automate things and make it useful for your business.

Yeah, and I actually, I watched a clip you guys sent over about the difference between general intelligence, which is essentially a large model, it can do cool stuff, and then data intelligence, which is that model applied to your existing corporate data set. And this, by the way, is always why databricks data Lakehouse plus Mosaic ML made sense to me because why not have a brain on top of your bucket of data? But at the same time, in that same post about the Mosaic AI agent framework, you said, or your company said, sorry. While building a proof of concept for your gen application is relatively straightforward, delivering a high quality app has proven to be challenging for a large number of customers. And I'm curious if that issue is around the data they need to have perhaps cleaned and ready to be put into the model, or if it's something else that's making it difficult to turn a bucket of cool data, cool models, into an actual product people want to use inside their company day to day.

Yeah, I mean, there's a lot of things to pick on here. So data quality, data cleaning, all of that. Absolutely. You need to get your data in very good shape because if you're going to start calling this data the ground truth, you need to make sure that's really true. So, you know, bad data leads to bad outcomes. That's true for humans too. If you give someone the wrong information, they're going to do the wrong thing.

Sometimes you give them the right information, they did the wrong thing or the wrong information, the right thing. Humans are. Bro. Humans.

We could talk about that a lot too.

Let's not get. But let's not get bogged down. Sorry. It's an election year, so I have a lot of thoughts about that. But let's start back to univine.

But humans are actually still quite amazing at understanding causation and how to get to an end goal which we haven't quite cracked yet in this field. When we look at some of the problems I've seen from different companies is, you know, I would characterize the second half of 2023 and the first half of 2024 as every Fortune 500 is like, oh my God, what's our gen AI strategy? Right? Everybody had to have this. I've talked to boards of large companies and they all said they're asking their management, you know, what is your, your strategy? And the strategy was go get an OpenAI account. Right. And I think, you know, people don't really know.

That's my strategy too, Naveen. I mean, in their defense.

And that's fine, you know, I think it's a place to start. But really you need to understand what success means. This has to have a business impact. Like just putting a new feature on your banking app may or may not have any effect on your business. It has to be something that your users want that they get value from and actually use your platform more or do more banking or do more loans or whatever it is. So actually thinking through that is still a business strategy question. It's really not an AI problem at that point. It's really, is this a good thing for my customers? Perhaps at some point AI will be able to tell me that. Today that's not really true. Okay, so now we can say, well, I've identified a problem. Great. Now I want to think through what is success criteria. So I can take components that are in databricks and I can start to assemble them. I can get to a working demo pretty fast, as you said. And that demo might have some kind of performance attributes to it and it's going to break in certain places. Oftentimes we find that this looks cool from a demo perspective, but if you actually go and try to roll it out, you'll see all kinds of degenerate use cases, all kinds of places where it breaks. Okay, so let's characterize that. Let's build an evaluation and give it a score. This is a 37% correct for my, for my application or it's 30, 37% useful for my users. Like start to score this stuff and evaluate it. Once you score it now you're in a place where you can actually start to improve it. We have all kinds of tools that can say, okay, well let's try this to see how it moves the score. But you need to get to that point. And I think this is actually the blocker for most of these applications. Beyond all the plumbing stuff of security and governance, you need to get all that right. So I think there's a lot of steps that are, that are kind of impeding progress for enterprises and we're trying to break down those steps and make it systematic for our users.

So I'm familiar with the CEO of a company called Skyflow Entre Sharma. He's a friend of mine and likes to explain things to me. And one thing that he was early on was, you know, helping corporations keep PII safe. And then the gen AI boom came and everyone was talking about putting the data data to work. And I presume that there is the same kind of tensions between people wanting to keep their data properly permissioned and only the right people can see it. And I can't ask about Naveen's work data and so forth. Has databricks and other vendors solved that issue or are we still working out how enterprises can bring their data safely into an AI context today?

Yeah, that's a great question. It's actually a big part of what we do at Databricks is we call this data governance. Yeah, the cool thing about databricks is that we already had a framework for this, we call it Unity Catalog. And this is basically a universal source of truth of, of permissions and lineage and you know, logs of what happened. If you're a regulated industry, you're a bank or a healthcare company, you need that. You can't actually go willy nilly and say, okay, here's some data I'm just going to throw into a model and you know, hope for the best. Like it doesn't work that way. Right.

That's a one way ticket to sitting before Congress being told that you're a bad person.

Yes. And not actually, not just sitting in front of Congress, like massive lawsuits and huge damage to your business. I mean, what's really interesting is that banks and other regulated industries, they work on trust. Their whole business is based on trust. Like, I'm trusting that that bank is doing the right thing with my money when I put it in there. Otherwise I wouldn't give them my money. Right. So they have to be kind of conservative in this regard. And you know, that's been a real big blocker to get this stuff out to their customers. What we're seeing today is they're, they're, they're thinking about a blast radius. Like, okay, I can try building a solution. Maybe I can expose it to internal users as a tool or an aid.

Sure.

And start there and then maybe work your way up to something that goes external. That will happen as we get, as we start to trust these solutions and be able to engineer them through all of the edge use cases. But really this is a huge blocker. So what we've done is inside of databricks, we've integrated a bunch of stuff that came from Mosaic as well as a lot of things that have been developed here under the umbrella of Unity Catalog. So for instance, if you fine tune a model, that model is fine tuned on some data, that data had some attributes associated with it. Maybe it had certain access controls. Those access controls need to be inherited by the model because that model actually, you know, is a function of that data to some degree. Or when, when you use a vector database and you do a pattern like RAG or retrieval, augmented generation, there may be data in that, in that vector database that can be viewed by certain users and not others. When a user hits that endpoint, the vector database needs to be aware that, oh, this user can't see that data, so don't pull that data. So these things, they seem simple, they're conceptually not difficult problems. But to manage at scale gets very, very complicated. And so we were able to leverage this actually quite mature framework which we just open sourced by the way. It was open sourced at the Data and AI Summit in June. And on stage, Matei Zaharia, who's the CTO and co founder of Databricks, actually hit the button on stage that open sourced it. I thought it was kind of cool.

No, no, I mean, I, I, I'm always here for pageantry and why not push the button live? Was it open source onto where did you guys put the code online?

I think it's on a GitHub repo.

Cool.

I'll Throw a link to that in the show notes if anyone wants to check it out.

Absolutely. So anyway, I think the point is that we had a quite mature way of thinking about this in terms of data governance and we've applied that now to gen AI. So different components like fine tuning, embedding, vector databases, function calling. All of these things live under the guise of this unity catalog, which sort of takes away that security and governance plumbing side of things to unblock some of these use cases now getting to correct models that are useful models. All of this stuff is the next frontier I'll call it.

Okay, but listening to you talk about this and thinking about the issues that have come up and that have been fixed or at least worked on by databricks and other companies to get enterprises from we have an OpenAI account to we're using our own data in production for a feature that's either internal or external. It seems that a lot of the pathway has been smoothed. So it has not helped a lot of companies go from we have a gen AI strategy which is we're tinkering, we're playing so we don't fall behind, but we're not really in production yet. Two, this is something that we've delivered both to customers and also our employees.

Yeah, they're getting there, right? This is still so new. I mean remember ChatGPT still was not even two years ago. We're coming up on it in a month. It was November of 2022. So this is actually quite unprecedented how quickly the industry is moving. Right. Two years is amazing to actually even get a POC going on this.

Well, yes, that is enterprise speed. If you're curious why it took two years and that's fast to get a proof of concept. Go work for a private equity owned company for a while and you'll find that two years to do anything is fantastic. So I hear you.

It is. So we are starting see that now and you know, I'm having conversations, you know, almost daily with these, with enterprises about how to do this and how to do it well and what are some of the best practices we've seen having these governance tools out there? Like we launched this stuff this year, these are, these are within six months. We've launched a lot, a lot of these features and you know, you need to be a trusted partner. Like I know there's a lot of startups building cool stuff and absolutely continue building cool stuff. There's no reason not to, but you know, it's hard for a big company. Like I don't Know, name your big favorite bank, you know, they can't take.

And use favorite bank. I mean, pick your favorite super villain.

Yeah.

Who charges you ATM fees. Chase, I guess. Okay. Anyway, good.

But yeah, take a chase. Like for them to use a startup is quite hard. Right. I mean they're just, just getting a tool approved, maybe they can get some POCs going. But they need trust in their vendors and you know, databricks has already built this trust. So putting these features in on top of a trusted platform they already have has actually been quite transformative. Our business is growing very fast and you know, I can't talk about the precise numbers but you know, it is, it's, we're seeing a lot of uptake.

I've written a lot about databricks growth over time and this is actually a beautiful segue to what I wanted to ask, which is before gen AI became a household name, the thing that everyone talks about in the world of technology, databricks was growing very quickly by itself in the pre mosaic ML days, if you will.

Yes.

Now of course when you pull up databricks website, it is the data and AI company, clearly an emphasis across the branding and really, let's be honest, the industry. So now when new customers are coming in to the databricks world, are they coming in because they're more interested in the lakehouse product, data storage and bi or is the AI now a leading vector by which you guys land new logos and accounts?

I would say it's a combination of the two, honestly, because we still see a lot of companies out there who are struggling with migrations of their data platform. I mean, again, going back to enterprise speed. I know it sounds like, you know, things that were like 10 years ago, but these are still problems from 10 years ago, like these legacy systems. I actually talked to a company not in the US but in Asia that was literally still running on, you know, old IBM mainframes from 20 years ago. And they're like, how do I modernize? I mean, I know this sounds silly but like this is how these industries work. Right.

And that's how COBOL programmers have 100% employment. Right there.

Exactly. And actually they're, they're getting paid a lot because that's actually, that's my backup.

Plan to journalism by the way, is if this all falls apart, I'm going to go buy a couple of books on COBOL and just nestle myself deep in someone's basement. It's going to be great.

Yeah, take a year, go learn it really well. I think you'd have a, you know, great career prospects for the next 10 years.

Yeah.

So I think we are seeing a combination of companies who want to leverage their data for AI. And this is, this is squarely the vector that we, we are winning on right now is that we are the combination of those two things. And frankly, it's not something that's new to databricks. In fact, databricks started there, Spark started there. It was actually the Netflix challenge, if you remember this thing, I think from 2012 or 2011, where Netflix published a bunch of data and they said, okay, go build something that can match preferences of our users to movies they might want to watch. And they published a big data set. So Matei and others actually made Spark in response to that because they want to scale it out over a whole bunch of CPUs. So AI has been something that's been front and center for databricks. In fact, many people use databricks for AI and then use other data platforms. Oh, interesting.

Yeah, okay, I want to touch on that because one thing we've heard so much about at an industry level is the need for more compute to avoid being GPU poor. How well do you know Jensen? Will he send you H1 hundreds? But you know, prepping for today's show, I'm seeing databricks on gcp, databricks on Azure, databricks on aws. And you guys, you know, you mentioned you're not working on DBRX2. So does this mean that databricks doesn't need to build a hyperscale network of data centers around the world? And if so, is that a risk to the company?

Well, we do not build data centers, so we never have. We run on top of the cloud. We are a software company. So being GPU poor, GPU rich, we actually do still a lot of fundamental research. So we still have a sizable footprint of GPUs. But we are not building those open source models. It doesn't mean we're not doing other work that's on the internal, internally facing side, externally facing. We are running lots of inference. We're seeing inference kind of take off for these models. I mean as we get richer use cases, things like agents that we've been talking about doing multi step, like there's a ton of pressure on inference, making inference really fast, making it really cheap, making it very high quality. All of that takes GPU compute. So I don't see the world where we are not doing a lot of stuff on GPUs. We're just going to have more and more of this going forward. And you know, we're, we're expanding into multiple GEOs. We're not having everything in one place now. So I think the, the pattern of deployments of those GPUs is maybe different. Instead of having one giant monolithic cluster, we're breaking things apart. But the numbers I don't see going down anytime I worry about platform risk.

Because you guys are dependent in a way on these major scaled cloud platforms. And at the same time Microsoft is making its own AI models, it has its own BI tools. They don't want to share right now. They'll be besties with you because it's convenient for them. But you guys have such an ability right now to raise capital at whatever price you want. I mean, so to me, I'm just kind of curious, why not offer a databricks cloud alternative to these big tech companies who I don't think ultimately want you guys to win?

Yeah, I mean I would say you should ask my boss about that one.

But he'd tell him to text me back. I mean, geez, come on Ali.

No, I mean, I think all possibilities are there in the future. We'll see how things go. I mean what we have done is establish a particular point in the stack of trust. So we, we do own this, these abstractions for data. Now we're owning a lot of these abstractions on how to take that data and make models that are really great.

Yeah.

And, and how to deploy them. So when you have that abstraction you can move down the stack. Typically moving up a stack becomes very hard. So you see this, if you're a hardware company, moving to cloud becomes hard.

Yeah.

If you're a cloud company, perhaps moving down to build your own hardware is less hard. We see this. Every cloud company is building their own inference chips now.

Yes.

So you know, how is it going to run in the future?

Because that does not include databricks, just to be clear.

Correct. We are not doing hardware. I mean we don't do the cloud, so we're not doing hardware.

I'm just making sure that when you talk about these other companies, it's not you guys. You're not making your own. Google is Microsoft is Meta, is everybody else. Okay, cool.

That's right, exactly. So how are things going to shake out in 10 years? I don't know. But right now, I mean there's a good reason for the clouds to work with us. We drive a lot of incremental revenue to them, we store a ton of data and now we have the tabular acquisition, like lots of data in the cloud is on our formats, so that gives us a pretty good control point. And the clouds are, are incentivized to work with us at some point. Maybe that's not true.

Well, I think what I'll do is I'll just go start buying small servers and then plugging them into the back of your guys's HQ and then slowly build up a cluster that you guys can then offer as pass to me. I don't know. One of my things that I'm most scared of right now is that the largest tech companies, the ones that are the hardest to knock off, that have the most market weight, are also the companies that own the major cloud platforms. And that to me makes them so monolithic and hard to kill that I'm worried that we're going to see less innovation over a long timeframe over many different technology cycles. But it just makes me a little scared. And I know it takes a lot of money to build a hyperscaled cloud and so there's only so many companies that can do it. And so the fact that you guys currently don't plan to just me as an individual, that bums me out. Although I understand from a business perspective how it would be frankly distracting to you guys right now. I just, I hope, I hope that in five years you're showing me around, you know, databricks, Data Center Number 9 or whatever it is. That'd be great.

Yeah. But I mean, if you think about it on a long time horizon, one of the largest incumbency events in tech happened in the late 70s. What are the two largest companies in the world today? Apple and Microsoft. They both started then, and they both started with the PC. They've leveraged those, that, that advantage over and over again through multiple tech transitions. And you know, it, it hasn't really slowed innovation. Right. They've been on, they've been maybe behind, but they buy, buy into it or you know, buy a startup that's, that's doing something innovative. So I don't know. I mean, I like that argument.

I, I just wish that instead of the magnificent seven, it was the magnificent 27. You know, like just more competition.

I don't disagree. But I mean it's sort of like how these ecosystems evolve. You have to have certain capital to, to have that incumbency and you know, they have it and it keeps going or it keeps rolling forward.

Yeah, it's great for you guys because you can work with all of them at once and therefore no Matter where your customers are, you are as well. So I get that there's a lot of talk about small models versus large models. And my impression going back to the models you guys made at Mosaic is that quite often we can see a 7 billion parameter model be as good as a larger one if it's well tuned. Just what is the Mosaic databricks perspective on corporations using larger versus smaller models today?

Yeah, I mean, I think it's about building the right thing for your application. Do the thing that matters. If latency really matters, go small. If quality really matters, maybe go bigger. But one thing you'll see as a trend is literally no one has built a model bigger than GPT4. That happened, I don't know, 16 months ago, something like that. And that model is still the biggest one out there. And even OpenAI has been building smaller and smaller models. You have GPT 4.0 and 4.0 mini 4O mini is probably less than 10 billion parameters. I don't know with specificity, but it's very fast and we can look at that as kind of a metric for how big it is. And I think we're seeing that they're able to extract higher levels of quality from a smaller model. So the science is by no means done here. There's a lot more work to do. There's a lot more we can wring out as smaller models. And then chaining these smaller models together, as in these compound AI systems is the way forward. Right. That's how we're going to do this economically. That's how we're going to do it in a modularity fashion where we can independently verify each module and build end to end systems that are very reliable.

Well then do large models then exist as a way to help create smaller derivative models?

Yeah, that's. That kind of had to happen. It seems like we built this giant model and then we said, oh well, we can actually take the outputs of that model and actually start to, to modify smaller models to have better performance. And so, you know, Microsoft did a bunch of work with the Phi model that they published. Those are like 1, 1.3 billion parameters and they're very high quality. So we're seeing that trend over and over again. Where I can, it's not distilling. I'm using synthetic data generation from a bigger model to basically bootstrap and train a smaller model. And it works. So maybe we need to go that route of building something really big to go and build something really small that's high quality.

Okay. On the Synthetic data point. I wasn't going to bring this up, but you brought it up, so I can blame you if we go over a little bit. Synthetic data to me makes a lot of sense, but always just based on the way my brain thinks. Considers it always to be of slightly lesser quality than non synthetic data. Is that a bias I need to get rid of in my thinking?

I wouldn't say get rid of it yet. It's kind of, I don't know if you remember that movie Multiplicity where the guy makes a copy of himself to do more work. It's actually kind of a funny movie. Basically he makes him a copy of himself to like have one at work and one at home and then he makes another copy and a copy of a copy.

Michael.

Keep getting worse. Michael Keaton. That's right. Yeah.

I've not, I've not seen this movie, but the, the, it looks hilarious. So that's going on the family list. Thank you very much.

Yeah, so anyway, it's a little bit like that. I mean, you, you, you start to become more unbound from the real distributions as you make more copies and more synthetic generations of it. It's like you think of it as making mashups. Some of those mashups are not accurate reflections of the real world. We call this concept model collapse. If you train a model on a bunch of these synthetic objects, maybe you get something that actually doesn't give you anything useful. So far we're seeing it as a good way to augment models and actually allow them to explore the distributions of the real world. If you can keep tabs on keeping quality and keeping it grounded. So I think it's a useful technique. It's not something that's going to solve everybody's problems. I still think we have some major issues in the field of AI. We've gone very brute force, you know, 20 trillion tokens of data to train a model. That's 20,000 human lifetimes of text. This is not how humans work. I mean, humans and animals actually use a lot less data to get much higher quality causal models. So we have a lot of science, I still think to do, to build, to build these things economically and you know, very low power. Your brain runs on 20 watts of energy. A mouse runs on 800 milliwatts. And they do amazing things.

No, that's one of the most encouraging things that I've seen. I've seen the people showing like the, the brain watts versus data center watts and how many, you know, equivalent flops you can get and so forth. Do you think that eventually, and this is completely off topic, so this might be a stupid question, Naveen, but like there was a, there was a, an effort, I read about this in like business week like 10, 15 years ago, to replicate the human brain thinking that that is the neural network that is so efficient, can do so much. We should do it. We seem to have kind of moved away from that in the transformer revolution. Do we bend back towards biology over time?

So okay, those are like the human brain project and stuff like that I think is what you're referring to. I mean I'm a neuroscientist and I thought those were like very poor facsimiles of what, what was going to lead to intelligence. So it's because you're trying to replicate something without understanding why that thing exists. It actually becomes, you know, you get one little parameter wrong, the whole system falls apart. These are delicate systems. So I do think going down the path of trying to extract principles makes sense now. I do think that looking at biology still is potentially a useful path forward. There is a different dimension to more data that we are missing currently in these systems. They're not, they're not trying to understand how to make better decisions and self critique. Maybe we're getting there now with this kind of like this, this chaining idea. But you know, our brains have multiple systems that actually interact in very interesting and precise ways. So you may have read Daniel Kahneman's book Thinking Fast and Slow. Very. You know, this is a great book. If anyone hasn't read it, it actually has roots in neurobiology. There's, there are systems that are very old in our brain that kind of make learning without parameters, without, without, without a model. Then there's these other systems which actually try to simulate different realities and then make good decisions. We use both of these things in concert. And systems we build today, like the AI systems don't really do this. They don't have a good grounding of. This is a real representation of the world that I can verify. They're just sort of like almost pattern matching against, against things that we're seeing in the training data. So am I saying we're on the wrong path? Not necessarily, but I think there's a lot of work to do that we can take some, some inspiration from biology without direct mimicry.

I think the biggest takeaway from our conversation today for me is the idea that there's still so much improvement coming in our ability to build intelligent software systems like the orchestration point from 01 what you just said about this, I mean, this to me feels like we have 10 years of incredibly quick development ahead of us and we're going to end up with, I mean, something truly marvelous at the end.

That's absolutely. Look at the Internet, right? I mean, I was a, I was a freshman in college when really the first web browser came out in 1993. All the basic concepts of what the web was going to look like sort of existed then, but look at what we have now. We have cloud computing, we have, you know, ways to do live updates and, you know, applications that run within a browser. None of that stuff was really contemplated at that time. And you know, it was again went from this big monolithic thing that sort of worked till now. We can engineer modules, make things work really interactively, and we created a set of systems that are highly reliable. Companies like Google basically figured out how to build resilient infrastructure. This needed to happen to have like, you know, nine nines uptime. You know, all of this engineering has to happen. And we're just at the beginning of this. ChatGPT was two years ago. You know, we've got 10 years of innovation left ahead of us to make these things really, really amazing.

I feel like you should point to that sign that says ChatGPT came out less than two years ago. Every time someone complains that AI hasn't solved world PC, I'd be like, all right, it's been 22 months. Everybody calm down. Actually, I just realized that my first child is basically as old as ChatGPT. Huh. That's going to be a really weird milestone to have in my life. Okay, one last question before I let you go into being. A lot of startups out there working on AI derived solutions services, lots of people building cool stuff in AI databricks has been acquisitive. It's how you joined Tabular, of course, was a big deal. I'm curious about two things. One, how often are AI founders calling you up very politely. And also, just how interested is databricks today in doing more, perhaps smaller tuck in acquisitions of AI focused startups?

Yeah, I mean I, I do angel invest and you know, I, I typically try to pick companies or founders that I feel I could, I could be useful to them. As an angel investor, I kind of see myself as a de facto advisor or helper whenever I can and you know, and things that I find interesting, you know, if it's a new cool technology. All right, let's, let's, let's give it a go. I, I am really interested in places where we can apply AI to actually solve vertical problems. Healthcare, Healthcare is actually a very important one to me because I think it's something so fundamental to what it means to be human and we have the pieces to actually go make healthcare accessible to many people. But it still hasn't happened. There's a lot of structural reasons, things like that. So let's get to that. So I like to invest in these kinds of things. So yeah, I do get called up quite a lot and I'm happy to talk to any founders who are building something cool. So there's that. Sorry, what was the second part of the question?

And just how acquisitive is databricks going to be, say in the next 12 to 18 months? A lot of companies have been built. Many of them probably aren't going to make it to the next level. So how many do you guys want to tuck in?

Yeah, I mean we're always open to these tuck ins. We've done a few of them. We're always looking. We have a really good corporate development team that's basically scanning, finding places that fill in holes that we have. So we're always looking at this and but we're very thoughtful about it. We don't want to buy something just because, you know, it looks cool. We're going to look at it like, okay, how is this going to work strategically with what we're building? Does it solve a problem that we don't have a good solution for today? Does it have a customer base that we want access to? Something like that? There's got to be some reason. So we're pretty thoughtful about how we assess it. And when we bring these companies in, I mean some of our agent evaluation stuff has come from a tuck in that we had a company called Lilac AI. They actually created a really nice UI for embedding data and visualizing it. We've actually morphed that into something that is going to be much bigger and very impactful for our customers. So we are absolutely interested in more tuck ins. And you know, we're growing fast. We're growing, our revenue is growing more than 60% year on year. And so I think we are going to be one of the major players for these companies to go find an exit toward and you know, keep in touch with us, build a relationship with us.

You literally just threw the we're growing fast, 60% growth. I was not going to bring up you guys going public, but I mean, dear God, you just, you're demanding it. So I know the answer is we'll do it eventually. Not yet, blah, blah, blah. But like H1 2025, like if you guys don't do it, then I think I'm going to submit like my tears in a vial. Like, come on, give me the filing. So just so you know, that's coming next summer if you guys don't pull the trigger. But Naveen, thank you so much for coming on. We appreciate it. We'll have you back in another 250 episodes or whatever it was the first time. And where can people find you online? Just before we go.

Yeah, I'm on Twitter pretty actively. So Naveen G Rao, that's N A V E E N G R A O is my handle there. I'm also on LinkedIn, of course, so you can find my name there. But yeah, Twitter is usually where a lot of the AI stuff happens and where we have a lot of cool discussions. So follow me.

Absolutely. For everyone else, more interviews, more live news, more twists coming your way. I'm Alex. I'll see you soon. Bye.

Okay, everybody, welcome back. It's time for a Jam with JCAL session. What's a Jam with JCAL session? This is where a startup founder tells me what they're working on, who their customers are, what their product does, what they're trying to achieve, essentially their vision for how they're going to change the world. And then I ask them, hey, what's your biggest challenge? Are there things you're struggling with? And then using my experience, Having invested in 400 companies and taken well over 10,000 pitches from founders, and having done 2,000 of these episodes of this week in startups, sometimes you know what, I know where some of the bumps in the road are and we have a dialogue about how to solve problems, which is what startups are all about. So with me today is a gentleman named Elliot Easterling. He's the CEO of a company called Bonbon Tech and that is our partner today. Tech is a great domain name. I use it for Founder Fridays Tech. Many people out there are using tech domain names because they're awesome. They let people know, hey, you're a technology company. And so thanks to our friends at Tech for supporting this segment of the show. We pick all the companies that come on. Welcome to the show, Elliot.

Thanks, jcal, thanks for having me. Appreciate it. I just want to walk you through a little bit of the business. I'll take you through a deck and then I'd like to jump into some questions I have about some of our go to market review.

The deck. I like reviewing the good deck. Let's see how you did.

Great. Awesome. Well, I'm the CEO, as you mentioned, of Bonbon Technologies and we are a rewards platform for publishers. Allow publishers to reward anything and that ultimately drives a lot more engagement and much higher registration rates. So let's just get started. Really the pain that we're trying to solve is for ad focused publishers. The 99% of publishers that are ad focused and they've been really suffering from wave after wave of big tech changes. Things like cookie deprecation, which started in Safari and now is moving to Chrome, search results pages referring less and less traffic out, social media algorithms referring less and less traffic out. And then now AI, which is going to have a massive impact as big tech platforms use AI to drive more engagement on their platforms and ultimately refer less traffic. So what we're seeing is publishers are looking right now for solutions. They're planning for a smaller user footprint, but they also want to drive a lot deeper, more profitable customer relationships from those users that are on their platforms. And so we built basically is a fix by rewarding engagement. So we built this, the first publisher rewards platform that gives consumers relevant rewards, access to unique content, simple and transparent data and privacy controls, and ultimately a better user experience. Publishers get logins and logins re enable cookies and lost IDs. They also allow the publisher to build direct relationships which they're starved for in the world where they're mediated by these big tech platforms. We also give the publishers gamified engagement, a points program that drives things like visitors page views, watching videos and ultimately that's all delivers five times more monetization per user.

All right, you got me. So far it's interesting, like I'm, I'm, I'm, I find myself nodding as a publisher just so we, we check in here as you run me through the deck. Yeah, that publishers do have these problems and if their consumers are logged in and you know a little bit about them, you can monetize them better. And people like gamification. So let's see how this works. Yeah, cool.

Yeah. And you're a media maven, so I think I, I knew you would, you'd really understand the business pretty quickly. So, so what we've sort of, we've cracked the code on sculpting consumer behavior with this rewards program. So we built an optimization engine that drives outcomes like 300% higher registration rates, 100% more engagement, 250% higher ad rates. And then we're finding that 54% of people after they log in will actually complete their data profiles and provide even a richer understanding to the publishers who's visiting their site. And this is all at a moment where publishers need this most. And so let me just walk you through really quickly, kind of like a sample of the tech.

Finally, I want to see the product. Okay, here we go.

And so on the left you see a little icon, little tab, that's us, we control that. And that's all the user's navigation. They can access the rewards program. But on the right hand side here, what you see is that at any time through the process, we can allow a publisher to trigger a rewards window either inline or as a pop like this that essentially runs eight or nine offers simultaneously to figure out through machine learning what do the users of that site care about most that will make them willing to register. And ultimately this technology delivers like a 3x higher rights rate from anything they've been doing prior.

Okay, so for the audience that was listening, there's a little arrow on the side, like a little chip on your phone. If you were to click it, it pops up, says hey, would you like to win this television set or whatever, some kind of sweepstakes. If you log in, if you register, you automatically get updated. Yeah, yeah. You get entered into the contest. I love it. Yep.

And then after someone enters or registers and we verify their email, we actually gamify the registration process. Like we say, hey, tell us more, a little more about yourself. Give us your name. 94% of people will give us name for more points. Zip code, 91% give us zip gender, 89% give us gender. And even they verify their phone number with us at 54% and we're just getting started. The other things we do is we give people points for reading every article they read. They earn more points and that drives 100% more engagement and just high, high level. We provide them with what we call three parts of the platform. Open Identity Manager which allows them to collect and manage first party data. This rewards engine which runs hundreds of offers to try and give publishers the ability to reward anything. And then we have a bunch of front end tools that consumers interact with that basically sort of deliver the product. And we also have an API where publishers can call it to issue the rewards on their own.

Very nicely done. Have you started to deploy this with any publishers yet? Where are you at in terms of building this business?

Yeah, we are live on 27 websites. We have 60,000 BondBan members have actually registered. The one thing is one user log into the publisher, they become a Bombard Rewards member and also a publisher as first party data. So we've been able to build, you know, our file up to 60,000 as of last week. And we're running about 60 million monthly pages across our publisher network.

Great. You know, publishers are looking for tools like this and if you can help them build their profiles and give them those gamification tools, I could see a number of them wanting to participate in this. The challenge of course is you're trying to solve a problem for publishers who are really struggling, which means they're not high growth. And so they might have very small budgets. Right. And so that is an issue is that they're constrained. So has that come up and, and how are you picking your ideal customer profiles when, hey, you know the publications I used to run, whether it was Engadget or Autoblog or the ones that Vox created themselves, like the Verge, etc. A lot of those publications are having challenges these days. Right. And so how do you think about that? Because you're picking publishers which are a group of people whose businesses are, might be flat, they might be contracting, they might be slow growth and they may not have budgets. Yeah.

So we offer two solutions for enterprise publishers. We have a SaaS platform where publishers can pay a SaaS fee and we, we also have a free with ads version. Publishers have to have a minimum size to be able to qualify for free with ads. But we basically inject ads into all our modals and that essentially pays for the deferral program, including the rewards.

Got it. So there's nothing to lose for those publishers by using this tool. And on the enterprise version, I assume I can opt out of like you owning the profiles and they're just for me if I pay you enough money.

No, the whole program is across publisher rewards program that's sort of backed by Bonbon. And one of the things we do is we issue, basically we issue the rewards out across publishers.

Okay, that's clever. So hold on, that's an important thing to pause on there. If you're going to give away this OLED TV that costs $5,000. It that cost of $5,000 will get abstracted across 10 publishers. So it's really net, net $500 each. Or if it was 100 publishers, it would be $50 each, which is super clever. Okay, I understand.

Yes, we have the benefit of that in that we basically can amortize the cost across publisher. Number two is sort of, we put the privacy guarantee, we allow the users to come into our system and opt out of any publisher they want. We know generally speaking, consumers don't like to do that. They just don't want to mess with the settings. But it is a really key, important part of our value proposition.

One of the other nice things here is, you know, the user can participate or not participate. So what I like about what you're doing is you might have somebody like me who looks at it, you know, you know, at a, as a 53 year old now, and I'm like, I don't want anything for free. I don't want to be part of any sweepstakes. But I would might want you to know a little bit more about me as a publisher because I want to get certain mailings, etc. So I have an email for just when I'm shopping that I use for my shopping websites and I don't mind them knowing about me because I would rather get sales for men than women. Or you get the idea. Or I want tickets to Knicks games, not, you know, I don't know, Miami Heat games or Chicago Bulls games. So yeah, that kind of personalization is a benefit because you don't waste my time. And then when I was younger I might, if I couldn't afford the OLED TV that's $5,000, I might very much want to join that sweepstakes. Thank you very much. So I think that's a very interesting approach too is this data will help us personalize content for you. And then there's, hey, this data might let you get into a sweepstakes. And you didn't mention the gamification, but that one was a highlight for me as well. So do you have an example of gamification yet or that's on the roadmap?

Yeah. So one of the things we do is after people register, we sort of say, hey. And we deliver this through the email campaigns. We sort of send them a weekly email that basically says, hey, here are some articles you could read that are personalized to you that earn you extra points. And so we're essentially starting to sort of like enable gamification through, through our newsletter program. And we do also offer point bonuses for doing things that go to this one publisher and play one of their games and that will earn you 100 points. So we're early on in the gamification process, but I do think that that's going to be some of the critical things as sort of my promise of publishers is like, hey, not only can I get you a lot more registered users than you could on Your own. But these users are hyper engaged because through communication we can sort of sculpt traffic, direct them back to your site to spend time on the things that you care about and ultimately reward them for that behavior and that activity.

One thing that's tried and true is inviting a friend or a friend member, bring a member kind of gamification. So you might find some interesting, you know, engagement there from publishers. If you could say, hey, I'm already a member, I think my brother should know about this content. You know who else should know about this story? I enter somebody's email, it emails them, they click, they register, I get points. So you, if that's tried and true, you see that in Robinhood where you gift a stock, you get a stock Uber, you give a ride, you get a ride Dropbox, you give some storage, you get some storage. So I think you got a really interesting tool based and network based business here. Two ways to win. And so what's your question for me? Any challenges right now in the business?

Yeah. So you know, in today's fundraising environment, you know, ever since the market turned, publish rather VCs sort of, they behave similarly and they'll sort of the mantra now is like I want to see revenue and all they care about is revenue, revenue, revenue. Now this is a network effect business and we win by building distribution. Right now we're building users at zero cap. And when you think about most rewards businesses, they're paying you know, five bucks, 15 bucks per user, we're not paying for users. And so our go to market really should be 100% focused on building distribution, getting more users. But at the same time VCs and the market wants to see revenue, revenue, revenue. Me putting pressure on publishers to essentially pay us is going to slow my business down. So how would you navigate that?

Yeah, sure, this is a great question. Network based businesses do have a carve out in our industry when it comes to monetization. So if you're a SaaS business, if you're superhuman and you charge a dollar a day for an email product or Slack and you charge 8 to $35 a month depending on which plan you are per user, yes, of course people want to see the number go up in your business. However, if you could prove that if you, you have these 60,000 members and if you said hey, we're giving away the iPhone 16 and we just want you to take a survey about mobile phones and visit these three reviews of, you know, we're going to present you with 10 pages every time you visit one of these 10 pages in our network and you scroll to the bottom and you know, Click on the iPhone 16. You get entered in. Now you'll get a bunch of sweepstakes jerks who, you know, kind of just do this in a scammy kind of way. But you also, if it's good content, might get people to actually who are interested in iPhone 16s and technology to visit 10 pages, that's good for the publishers. And it costs you $1500 to set up the sweepstakes and give it away. Now you could prove to people, if you could prove to people that when you do that, the publishers get page views and the cost is spread for the iPhone 16 across 30 publishers. So it's only 50 bucks a publisher. And you just run those experiments without having the publishers tell you, I'll pay for it. You just, you're running a fifteen hundred dollar experiment. You say, look, we sent. Of the sixty thousand people, 3% of them, you know, went and actually did something. Imagine if we had 6 million people and 3% did something now we'd really have a business. So that's up to you to prove it to me, the investor that you could do these little experiments and whoa, look at the interest that this created. And they had to click on an email to confirm they were in it. And if they put it in their friend's email, they got, you know, $10, they got 10 more entries and their friend got double the entries. So there are little experiments like that where you could show growth in the user base and the page user driving and be able to correlate it. So that, that's really up to you to run those small tests and prove to us you can grow, you know, 5 to 10% week over week. That would be what viral growth would look like in your business. Now for sales and for SaaS products, you show me a SaaS business growing 10% a month. I'm interested you show me a business like yours growing 10% a month. I probably would think not very interesting. So you got to get to that 5 to 10% week over week. That's a lot of tests to run. And you know, but you could actually do that with very low dollar amounts.

Yeah, yeah, yeah. I think, I think what you're sort of pointing at is a sort of a unique unit economic stor that we need to tell to investors like you to basically say, hey, we're getting these users for free. And then we run these experiments that actually show that we're getting revenue traction. The revenue traction is Building or the activity traction is building on a per unit basis.

Engagement. Yeah, engagement is the name of the game. So you could say to me, you know, if you came to me with these 60,000 and said, okay, it cost us, you know, six months and you know, we burned a half million, we burned $100,000 in six months getting to 60,000. So we're going to need to spend a million dollars to get to, you know, what we think is 3 million because we're getting better at it. It's a fixed cost business, whatever. But if we did this many prizes we would get to, you know, 10 million. And then every week or every day we're going to run an experiment with a million of those 10 million so we don't burn them out. But every 10 days they're going to get some sort of offer to engage them and then we'll start charging the publishers because they'll be addicted to it. But you know, you have to get the flywheel started. Right. And so getting the publishers to agree, getting the VCs to agree, all this stuff takes work. So you may have to just invest small dollar amounts to show it on a micro basis.

Yep.

And then say imagine we add two zeros.

Yeah.

And here's what it costs to add two zeros to the velocity I'm showing you.

Yeah, yeah.

Makes sense of sense.

Pretty straightforward. Yeah. And that's like your investment. But I, the good news is I think you can show this for very small amounts of money. How many people working on the team? I'm curious how many developers you have?

Yeah, so we have a product person, a couple engineers who are really, really good, are kind of working kind of.

Core on outsourced engineers or full time employees. Offshore.

Offshore part time basically is what the team looks like. Yeah.

So you're in, you're in year one, you're an early stage startup, I take it?

We're pretty early. Yeah, we're sub two years. Yeah.

Okay. Yeah. Stories, how much your risk?

1.4, basically.

Okay. Wow. So you've raised a decent amount of money.

Yeah.

You should be able to get that network effect going. Just invest in giving away whatever the product of the moment is that aligns with this. I would also start thinking about, you know, the virality of social media and TikTok, you went after publishers, which is where attention is going away. So they need your tool, obviously they really need it, but their businesses are contracting and you may be selling like, you know, deck chairs to the Titanics. Now what business is growing? TikTok shorts, video podcasts, There's a bunch of things that are growing in the world. So I'd also think about these tools and say, hey, if we were to get 20, if we had 60,000 TikTokers and people making shorts and they were engaged in the network, what might this look like? Because sometimes, you know, a surfer is only as good as the waves presented to them. But when you go big wave surfing, right?

Yeah.

You're, you've got these little tiny waves, three, four, five foot ones. Okay, fine, you can surf them, but it's not like you're going to be going to the North Shore and do it, you know, hitting a 20, 30 foot wave where like, whoa, it's impressive if you can catch one of those waves. So I encourage you to, to really think deeply about the beach you're surfing at. The beach you're surfing at. It does it. The tide might be going out faster than you can build a business. I, I love looking at, you know, and I've, I've done this myself, right? I caught the blog, I caught the magazine zine wave in the 90s. I caught the blog wave, I caught the podcasting wave 14 years ago when I started still riding that wave and caught the angel and seed investing and incubator wave. Sometimes there's waves that are bigger than you and if you catch them right, man, that's why I'm, I'm doing more thinking about live and shorts. You know, TikTok shorts or. Well, TikToks aren't called shorts. I think YouTube calls them shorts. But anyway, I've been thinking about other alternative formats outside of podcasting and I've been thinking about live a whole bunch. So I encourage you to think about that as well. You have an interesting business. I mean, I would have if you had presented this business to me to come to our accelerator. I've been like, yeah, let's do it. But I think you're in a weird position having raised a bunch of money, not having violent product, market fit or market pull yet. So you really got to get some market pull. And I think it might be that you might need to find another beach where the waves are a little bit bigger. So I would think about that. E commerce is another one. Direct to consumer. The people in direct to consumer were getting slaughtered. Slaughtered work. Trying to get Facebook to work. And Facebook worked for a long time, then it stopped working. Yeah, the ones who went to TikTok and social media and podcasts did make it work. So sometimes it's just like it's not you. It's the environment you're applying your skill in. So I think you're very talented.

Yeah, yeah. And I, we are. I can't disclose what we're doing, but I think some of the things we indicated are areas where we're testing right now. Doing some experimentation. Good.

Yeah. I'll leave it at that. I don't want to tip your cards. Great job. And for everybody listening, who wants to learn more, give us your URL again. I know that you're a tech, but remind everybody of your domain name.

Yeah.

Bonbon Tech.

What a great domain name. Bonbon Tech. If you would like to get one of these great tech domain names, just go to get tech and get one of those great tech domain names. Tell them your boy jcal sent you. We'll see you all next time on Jam with Jake Allen. This week in startups.