From Open Source to a Global Business - (Confluent) Jay Kreps & (Sequoia) Matt Miller | Slush 2022

Good morning. It's like coming out in a rock concert.

I know it.

So I'm so thrilled to be here today with Jay. I first met Jay 7 years ago shortly after he started Confluent and went to go see him at his office with a colleague that was at the time this small room in the back of a dentist's office in Mountain View, with a bunch of folding tables and folding chairs, and everybody working on code and building the beginnings of Confluent. And started very similar to how many of you are starting your journeys. And just 7 years later, Confluent is a global business with thousands of customers around the world, offices all over the world, a very nice office just a few blocks away from that dentist office. And the only thing that Jay has had to pay for is he has a lot less hair than he used to have 7 years ago. But on a more serious note, Jay has built an open source technology that has changed the world and built one of the most successful enterprise technology companies in the last 10 years. So thank you for letting us at Sequoia have the opportunity to be your partner. And thank you for having the chance to work with me. One of the things that is not well known is This was my first board seat and the first company that took a chance on me to let me join their board. And so thank you for giving me that chance to be your business partner and to work with you, which has meant so much to me in my career. So thank you.

Thank you, Matt. It's been an awesome journey.

So let's start by taking, taking everybody back to the beginning. You know, as the host mentioned, the foundation and the early ideas for Confluent came from an open source project. You created while at LinkedIn. You know, what was the problem that you were trying to solve then that started all of this?

Yeah, I was working as a data engineer and, you know, this is kind of the early days of social networking and big kind of at-scale web systems. And so a lot of the work was kind of from scratch and it was building technology to manage data in different ways. And one of the things we realized was Most of what we had was technology to store data. It was like a place where data could go to sit, you know, and you could look up bits of it at the right time. But ultimately, you know, a big system like LinkedIn was all about how data flowed between and how we would trigger the right actions and do the right things at the right time. It was kind of a very dynamic, you know, living thing. And there wasn't really good infrastructure for that. And so, you know, it's— I guess whenever you approach a new problem, You kind of assume somebody has solved it, and at first you're just looking through what's there and expecting that you're just using it wrong or you don't understand. And then eventually you realize, no, like this hasn't really been solved in a good way and in the way that would be possible now. And, you know, that was the origin of the open source technology, you know, what was really solving that problem. How do we get all the right data to all the right applications? How do we act on it in real time instead of, you know, putting it in some Hadoop cluster and running some batch process at the end of the day. That was kind of what motivated us to do this.

And you released this open source technology, and pretty soon this was the backbone of some of the most successful companies like Netflix and Spotify and Uber that were being created. What was it that gave you the push to quit your job and go start Confluent, the company, after seeing the success of the open source out in the world?

Yeah, well, it wasn't quite like that overnight. Like, the initial open source we thought was going to be a big deal. And we'd done other open source projects which were popular, but it was initially released to resounding silence. Nobody really understood what we were trying to do. And yet we thought it could be like a really big deal. And so, you know, the first few years was really just trying to get people to understand the ideas around streaming and try and get that out there. And we felt like, hey, there should be this huge opportunity for it. And it took us a while to really figure out how to explain it. And then as that As that happened, yeah, it started to get adoption in tech. Nonetheless, we felt like, "Hey, if we're really going to take this all the way, there's no way we're going to be able to do that working from the basement of a social network. There needs to be some product around it that'll make it easy to consume, and some company that can really invest in it and drive it." That was the origin of Confluent. You know, it was a little questionable at the time in some ways. Like, there wasn't a lot of enthusiasm for open source companies at the time. There was a lot less enthusiasm for kind of the data infrastructure layer as a place to build a company. But we felt like, hey, there is a lot of value here and there's something that could be, you know, very powerful for companies. And that was what motivated us to kind of take the plunge, as it was. And, you know, it actually really was, you know, really helped. I think just being fully focused on this and being out there in a way to meet with customers partners made us move so much faster in anything, you know, than we could have done just working in open source.

Amazing. And you went through this transition where you were an engineer for most of your career to being a CEO and founder. What was that transition like, and what was the things you needed to learn and navigate as you went from engineer to CEO?

Yeah, yeah, it was honestly pretty brutal to start out with. You know, it's very different jobs. And, you know, I think especially that step into being a CEO, you immediately realize, hey, your peer set kind of includes like Steve Jobs. So I went from being, at least I thought, a very good engineer where I had kind of built skills in each area. I could, you know, write good code and I could talk about it and I could design systems. And, you know, I went from that to being, you know, effectively the, you know, a very inexperienced CEO of a 3-person company. And it was very hard, right? Because you're suddenly pitching an idea that it's hard to explain.

Yeah.

You know, a lot of what you're doing is now really more of sales. You know, you're selling the company, you're selling the vision, you're, you know, trying to hire employees, you're trying to bring in customers, and you don't have much to go off of. You're in, you know, a small dentist office. Your revenue is zero. Your product is as yet unreleased. It turns out that your many friends who you assumed would immediately join are a little skeptical of the new company. And so, so yeah, it was actually quite, quite difficult and, you know, really a different skill set to learn. And, you know, I had, I'd almost forgotten what that was like. To be so bad at something. But the good news is, I do think if you're going to take on a CEO role, I do think starting the company is the easiest way because logistically a 3-person company is not that complicated. And so then you just kind of have to at least grow with the job as, as the business and the company, you know, itself grows. And that's like a little bit more each day. Yeah. And that makes it, I think, you know, at least approachable.

Yeah, which by the way, everybody, from a Sequoia perspective is amazing. If you think about it as a human to go from writing software to running an organization with thousands of people and thousands of customers, the growth that you've had as a founder has been incredible and something that very few people can do. And it's something we look for in the founders we back is this mentality, this outlier mentality, people that see the world a little differently, people that have a, have a chip on their shoulder to go do something different and, and will fight relentlessly to solve that problem and grow with these challenges of, of transforming yourself to be able to lead a large organization and go after these problems as you have done. Having been through the founder journey, what are the key attributes that you think make for a very strong founder?

Yeah, I mean, it's a cliché, but I do think it's the dogged persistence of just kind of not giving up on things. You know, when they're difficult and, you know, kind of continuing on. And, you know, I think that's probably more important than being, you know, smart or having particularly deep insights. All of that stuff helps. But I do think the main thing is just, you know, kind of keep rowing and, you know, see if you can get there. And I think that becomes, you know, increasingly important in any kind of more difficult time where, You know, which I think every company has some of those. And, you know, without that, I think it's just hard to stick with it.

Yeah, no, I mean, grit really is what it's all about. And these many outliers also have like early beginning outlier stories, and you yourself have some of those going all the way back to when you were in high school. Like there were things about you that stood out, like dropped out of high school as I understand. Like tell us a little bit more about your youth and like the journey, like the What created Jay?

Well, okay, I was a very idealistic teenager, and I felt that it was just very inefficient in school, that you didn't learn much each day, and I thought, okay, I could just teach myself everything I need to learn in school, and then I would have all the extra time to do something else, right? And so I left after 1 year of high school, and actually with a whole group of friends. We basically were like, okay, we're just going to teach ourselves. Turns out that's not the best idea in the world. It's actually much harder to teach yourself everything you would learn in high school yourself. Some subjects, English, a little easier. Chemistry, a little bit harder to just do out of a book. But it was at least an original path. And you definitely see other things in life if you have the extra time. So it was a different experience.

Yeah. Well, thank you for coming here all the way to Europe to be with us today. How important has Europe been to Kafka and to Confluent, and what are some of the early successes that you had here?

Yeah, it's incredibly important. So, actually, some of our earliest employees were here even before we had a sales team. We had, you know, some folks on the engineering team. I've been really impressed with the startup scene here. You know, I spent this week visiting our customers, and there's just incredible things happening. And the energy is, you know, I think in many ways at the same level as Silicon Valley, but the feel of it is actually more fun. And I don't— I couldn't say why that is, but, you know, I think it's a really cool thing to see. And really creative new companies coming around. And, you know, that was one of the reasons I wanted to come out here. We have a Confluent for Startups program that we—

Yeah, you just announced, right?

That's right, that we just announced. And, you know, it's really targeted at these early companies. If they're getting started in the data streaming world, this is a way for them to kind of get going on the cheap, which is what you need when you're a startup. And, you know, our motivation from that was we were just seeing this kind of global rise of these small tech companies. And, you know, especially right now, there's a lot of pressure on efficiency. You know, a little bit less money in the system than there used to be. And so looking for a way to do things faster and cheaper and easier is definitely on everybody's minds. And we felt it was a good time to kind of focus on those early companies.

Yeah, that's what Carlos, our sales leader for Europe, is jumping out of his chair.

Yeah, yeah, that's right. That's right.

That's right.

We can go visit Carlos at—

I think we have a booth here to get you started with the Confluent for Startups program.

That's right.

And, you know, we talk, we talk, we talk a lot at Sequoia about crucible moments.

Yeah.

And these moments in our journey when there's a really difficult decision to be made, when sacrifices need to be made, and they can be really arc-bending for the whole company. And as I look back on the journey of Confluent, one of those crucible moments was the decision to shift to the cloud.

Yeah.

And it was a hard left shift that you took the company on to go to the cloud. And for everybody's benefit, the business was just crushing it with this on-prem self-managed offering. And Jay took the whole company in a massive transformation to make cloud-first. Tell us a little bit about that, how you made that decision, how you worked through that, leading the company through this big change, different constituents like sales and engineering moving different directions, trying to get everybody going to transform the company.

Yeah, it was, you know, the first piece of advice they give you in a startup is don't try and do a second product in an early company. And that turns out to be very good advice. Unless you really have to. And so for us, we knew early on we would have to, you know, operate in all the environments our customers had. That's just the way our technology works. It has to be there with your applications. And so we kind of debated if we wanted to do a software offering or a cloud offering. And there were pros and cons to both. But we started with a software offering. And we quickly realized we were going to need to have a managed service. And we'd come out of the business of kind of running these big data systems at scale. So we knew that was where You know, that was going to be the best way of delivering this. But it was really a pretty significant undertaking. And so we started building our cloud offering. And early on, you know, it just was not good enough. And we really were at kind of a fork in the road with, do you pull back from that and just focus on the thing that's working, or do you just kind of push on through and try and, you know, make it work? And, you know, it was a very difficult one because we understood we were going to have a whole new set of competitors. In the public cloud, including the cloud providers themselves that have services. So it was going to be a very difficult undertaking, and it was kind of a bet the company initiative to focus everyone on it. But I think that kind of thing ends up being very good for companies. I think the best companies actually have some kind of difficult trials you have to go through, and this was definitely that for us, where it was first on the engineering side, you know, really bringing a global cloud service up to scale and, you know, the kind of operational capabilities. Then on the go-to-market side, really rebuilding around kind of a more SaaS go-to-market. You know, each of these was these big transformations in a company that was, you know, quite young and immature in its own way and a lot of pain to get that working well. But, but I think ultimately was very important for where we wanted to be. And, you know, I do think once you've done a few of these very hard marches through a difficult problem, you're well set up with whatever comes next. As you march into the next hard challenge, at least then the team feels like they can do difficult things together and get through it. And that gives, I think, a little more resilience to the people in the company than if it's just kind of purely up and to the right and easy sledding the whole way.

Yeah. And the move turned out to be genius, even though it was really hard to go through it. Totally changed the company, opened the envelope of customers that you could work with.

Yeah, yeah.

It's always great when the crucible moment, when you come through it and you've chosen the right path, which—

That's right. That's right. Yeah. Yeah. It was some years of very hard work, but that ended up being, you know, really the bulk of the new business coming in. And of course, in many ways, the future of the company and for our customers where their new applications are going.

Yeah, exactly. We're, we're in a time now where lots of companies are going through crucible moments. there's layoffs happening, people are cutting spending, we have a very different macro condition, interest rates are rising, we have a war going on. What are some of the steps that you're taking right now to help adapt to this environment at Confluent?

Yeah, yeah, I think the most important thing is just to realize the magnitude of the shift in the environment and what it means. The tendency, I think, is to always go off, you know, norms that were set in the past. And when there's some kind of very significant adjustment, you need to really look forward and think, okay, what is the talent market going to look like? What is going to be expected of me in the future with a set of expectations that actually doesn't look like the past? And so that was an interesting thing for us. We went through this, you know, as a public company, which, you know, is much— I think much more painful than as a private company because, of course, you know, the whole market is shifting around you and your employees are watching the stock price day to day. And so You know, it's definitely a tricky one. But I think this is kind of coming, you know, through the private markets as well. And so it's a sizable adjustment towards efficiency and profitability. I think, you know, virtually every stage. And I think it is going to change how we build companies. You know, we put effort into this internally. You know, we had always had, you know, a pretty disciplined framework for growth and profitability. But even on that path, we, you know, really looked hard at all the areas where we were investing. And just tried to get very real about, you know, could this be done, you know, better, faster, cheaper? Is there a way to be more efficient in this area? I think that was a very healthy thing for us. It wasn't like we hadn't done that before, but that, that extra level of inspection really helped us. And I do think it gets you, you know, license to be clear about what the priorities actually are and what really matters and what has to happen and where you have to invest. And what can wait, you know, what are the things we can take on next year, which is probably something we should all be doing anyway in running a company, but the environment really gives that extra push to do that well, and I think it's an incredibly important thing.

Yeah, Doug was here on stage yesterday, and he was talking about how, you know, the playbook of the last 10 years is not going to work for the next decade.

Yeah.

And moments like this are actually very positive because they force us to think more scrap, to be scrappier in our thinking, to think lean, to think efficiently, and the result is we build better companies. And there's a result, and so it's actually never been a better time in that sense, in that the guide rails of this market environment will force you to build something that will be more enduring over the long, long term. And so it's exciting to see us doing those things at Confluent.

Yeah, I totally agree.

One of the things that we both love, one of our shared passions, is poker.

Yeah.

And you you know, just talking about leadership, you've often given lots of analogies around leadership to poker, and one of the quotes that you've shared is, "The goal for a poker player is to make good moves. Sometimes when you play well, you still actually lose the game because you just got bad cards." Yeah. Tell us more.

Yeah, yeah, this was something, you know, that struck me, especially early in my journey. I was, you know, coming out of a world of being an engineer where, in many ways, You know, it's a little bit more like chess, like there's a right answer to most of the problems. If you think hard enough, you can come up with the best solution. A little more research and analysis is usually the right answer. And then in running a company, most things are not like that. You have these big, messy, kind of often impossible-seeming problems where it seems unlikely that you'll succeed, but you need to kind of pick the best path. And, you know, it's actually kind of daunting to go after something like that. One of the things I felt was what I need to do is concentrate on playing the game well. That doesn't actually necessarily mean that the outcome will be good, but that's the part I control. That means the things I am doing day to day, if I do that well, and if I feel like I have a good trajectory, then that is the best I can do. If that— the hope is, of course, that will turn out spectacularly well. It might not, but it's so easy to be worried about this larger thing that you're responsible for that you actually stop playing the game well and making good moves. You're too tied up in, you know, the final outcome. And I actually found that was very liberating for me, and that made me just focus on, okay, you know, the parts of my job. How do I do this better? How do I do it more effectively? The things we're doing day to day, how do we accelerate those? You know, and, you know, making sure that does line up to some good outcome, but knowing that in the end I can't really control much of that bigger outcome. I can't control what my competitors do. I can't control what the larger market reaction will be to my product. You know, I can control just the things that we do and only so much of even that, right? And so that was interesting for me to be able to think about and kind of shift into a mentality where that was my focus. And I thought it was ultimately helped me be just a lot happier in doing my job. Because you can let go of some of the things that are outside of your control. You may hope for the best, but of course, you know, there are things that you can't control day to day.

Yeah, I mean, it helps a lot when you're both lucky and good. But really, at the end of the day, all you control is being good.

That's right.

And by focusing on being good— That's right.

And then hopefully the luck comes around eventually.

Exactly. And then the luck will finally hit you. One way, one of my friends has this quote, "Unlucky in Vegas, lucky in life." Right. And so if you're unlucky, eventually the luck will come your way. And get it in the right moment in time in your life.

Yeah.

So this conference is filled with literally thousands of people who are looking at or trying to or part of building a company to change the world. Many are— have started companies with an ambition to change the world in some way, or many are considering doing so. You know, as a parting words, what would be your advice to the founder looking to start their business today? On what they should do in building their company?

Yeah, yeah, I would say go for it. I mean, it's a daunting time in the world. I think that's always true in some respects. You know, I think, at least for me, my feeling was it was kind of an incredible journey. And so into the entrepreneurial journey, I felt like, hey, even if this all fails, I felt like I would have grown from having I got to do a whole set of things I wouldn't have experienced otherwise. And so, you know, I think kind of enjoying that journey for what it is, you know, makes it more fun along the way. It kind of makes some of the trials and tribulations more enjoyable. And so, yeah, you know, my advice is, you know, kind of aim big and go for the thing that you would really want to be the outcome. And, you know, I guess the cards kind of fall where they do, but, you know, But that's what makes it exciting.

Awesome. Thank you, Jay.

Thank you, Matt.

Thanks, all.