Baseten | Self-Serve Apps for ML Teams

Hi everyone. Welcome back to Gray Matter. I'm Sarah Gua, a general partner at Greylock. It remains very difficult to build end to end products that leverage machine learning for business impact. And base 10 makes it fast, seamless and serverless to build scalable user facing workflow integrated apps powered by AI. We've been working with the base 10 team since inception and are so excited to announce the company's $20 million seed in Series A financing today led by Greylock and open availability of the base 10 platform. I'm here with Tuhin, co founder and CEO of base 10. Welcome to Hin.

Thanks Sarah, nice to be here.

Can you just start by telling us what the base 10 platform does?

Base 10 reduces time to value of machine learning efforts in businesses. We enable teams to incorporate the machine learning models into production grade applications within hours instead of months. With base 10 data science and ML teams can easily pre process data, serve their models and build ui. So you can end up with a full stack machine learning application all within a single end to end platform.

That sounds like a lot of the functionality in the machine learning tooling and applied ML space. A space that has a ton of startups and existing tools. Where does base 10 sit amongst all that? And just being aware of the landscape, what convinced you that there was something important that was still to be built here?

So I think it's worth just stepping back and thinking about everything that goes into shifting a machine learning model from I have a problem to I have something that's integrated into the business is adding value to a business. And so you first obviously make a problem statement, but then you prepare your data, you train your model and that's what data scientists have traditionally been very, very good at. After that you need to somehow get it hosted somewhere. And that's where tools like SageMaker come in. In the past people have used stuff like Flask and fastapi. Some data scientists can do this, but it starts to get pretty hairy really quickly, especially when you start dealing with a lot of infrastructure or you need to think about high performance applications, but that's really just a model behind an API. Once you have that, you need to start to actually integrate the business logic. So that might be pulling the data from somewhere, doing some transformations, writing it back to some other third party service, or integrating it with the system of record. And then if it's a workflow application, you oftentimes need to build a ui. And so there's a ton that goes in from zero to value in building with machine learning. Yes, we do some machine learning things, but we're much, much more focused on going from model to business value as opposed to model to behind an API. And so the question you asked was like, how do we think there was something, especially in such a noisy landscape, why was this something worth building? It's because we saw white space and kind of this idea of productizing models is trying to think of machine learning as a product function as opposed to a research function. And what would be the tooling that unlocked that?

I do think that there's a really odd dynamic in the machine learning landscape overall, where the research functions of a very small number of largely private companies, less so even academic institutions these days have made these amazing contributions in model architecture and new techniques to the world. And a lot of that has ended up in open source. But the realm of actually applying any of that innovation to real business problems is like massively underserved and not really focused on by much of the industry versus, for example, hitting a new benchmark or an improvement in model architecture. And our assessment of the space at Greylock is that the vast majority of companies, even those with significant machine learning investments in data science teams and their own product engineering teams, were very, very early in that journey of the application, much less having their own research effort. And so definitely saw eye to eye with you guys on that. I also think that if you ask people today, I asked the portfolio companies that we get to work with to talk about their machine learning strategy. Very often it ends up being a second step in their company strategy, which is, oh, when we can afford a team of a bunch of data scientists and we can really resource this, then we'll invest in it. And I think I'd love for you to talk a little bit about your background and the experience that you guys have had with Gumroad and with other companies you've worked for.

Yeah, absolutely. So I have two co founders, Phil and Amir. Phil and I actually grew up together in Australia. We've known each other since we were 12 and we met Amir in 2012 when we were amongst the first few employees at a company called Gumroad, which is like a creative marketplace. And we were kind of tasked with this problem. We were a really small team. We were getting smashed by fraud. There was more fraud than there was actual volume for a while on the platform. And then people were uploading all sorts of weird content and we're like, oh, this seems like a pretty machine learnable problem. And as a data scientist and someone with a machine learning background, I was like, okay, yeah, I can get a model up and running to do these things quite easy. And prior to gumroad, I actually worked at a research lab in Boston where we were using machine learning to predict the diagnosis and prognosis of neuromuscular disease. And largely in academic exercise, we were publishing papers more than we were actually building value. And I didn't really have that engineering skill set. So we, you know, lo and behold, I spent a few weeks putting together this data set, came up with this great model and I went to Amir, who was a CTO at the, like the de facto CTO in Sahel, hey, can I have some resources? And they're like, what does that mean? And so I basically realized that I had to learn to become a full stack engineer to actually get any value out of my machine learning model. So I went and I knew Python, but I didn't know anything about servers. So I learned how to build a flask app. I deployed it to aws, took me way too long. And then I was like, okay, cool, now I have my model behind an API. I was like, oh, can I have some front engineering resources? And they're like, you know, there's no retool at the time. And like, no like. And I was like, okay, well I guess I become a front end engineer now. And so within a few months to get value from these models, like I had to become a full stack engineer. And I quickly realized that while there was a lot of work that goes into preparing your data and training your model, and you don't want to discount that work, there's just as much work, if not more, that goes into actually connecting that model to actual business value. And you know, for me this was great. You know, I wanted to be a founder. I got to learn to be a full stack engineer on the job so I could build whatever I wanted in the future. But if you think about it from a value proposition perspective to Gumroad, it wasn't great. You know, it took out a person. And it was really clear that for every model we came up with, we'd have to pair it with an equivalent amount of engineering resources to make it work. I think right after that, you know, me and Phil actually started another company, Good Shape, that was acquired in 2018. Amir went to work at a company called Clover Health where he was managing data platform and he saw the same thing. So Clover Health, you know, they were trying to provide these smart diagnoses at the point of care to doctors. So they would take a medical record, then mine it. They take insurance claims and be like, hey, you should check for X, Y and z. Again, the data scientists come up with a great model and you know, they're like, okay, how do we actually get this in front of doctors? And you know, obviously they didn't have the resources we were all constrained for engineers. And they came up with this great idea to put it in a spreadsheet and the doctor would pro row look it up really with the output of a table. And obviously the barrier here for that workflow in this case the knowledge worker is actually the doctor. To actually do that work to be able to interact with the model was way too much. There was no ROI seen and those efforts were scrapped. And so it became really clear to us that data scientists weren't equipped and machine learning weren't equipped with the skill set themselves to be able to kind of ship these full stack solutions. And nor did they oftentimes have the resources internally. And as a result teams were coming up with a problem that was clearly solvable by machine learning, coming up with the initial model but not being able to show value. And so as a result other stakeholders, management would lose faith and as a result those efforts would get nixed. And to us this seems like why was this big. It's like if we can reduce the barrier to these companies that clearly have a business critical way to use machine learning, we think that's the kind of the future that we're going in, that all these tasks today where there are humans making decisions, first augmentation and then automation will come with tools like Base 10.

The story of both the sort of engineering learning curve that you went up yourself and then the stymied efforts of Amir and his team at Clover Health, I'm sure also mirror a lot of the conversations you had with data scientists as you were exploring this idea. It's clearly not what any of the companies that are recruiting these machine learning teams are out to do. Right? And the likelihood that any of them is going to recruit high quality machine learning people who are also going to learn back end and front end engineering and DevOps and string together a service that way single handedly just feels low. It feels like an impossible ask if like what we're aiming for is the democratization of ML, nor should they.

Right in the sense that is this the best use of very, very smart Ph.D. from Stanford to go and learn how to make react apps? I don't think so.

Yeah. So let's talk a little bit about just like you went through this, who did you end up deciding to build base 10 for and who's using it today.

Now we were ourselves, like Ices or Album Hats, startups that were going through some growth and didn't necessarily have the resources to have all that infrastructure in place so we wouldn't have to build it ourselves. Turns out that we think that's 90% of the market right now. When it comes to machine learning teams, I think when we think about who we target, it's exactly that. These hustler type of machine learning and data science teams that really have the expertise to train a model don't have the support. And what we've seen is that when you give kind of access to these ICs in this bottoms up way that we've started to target in the last year, all sorts of weird examples come up. And it's really cool. Whether it's at the larger scale. We have teams like Patreon and Pipe who are using IT for content moderation, underwriting assets, and they've also started using IT for data labeling and transcribing audio. And what we saw with those larger companies was that once you gave folks access to something that unblocked them, it kind of got the imagination going to what else they could do. And as a result of base 10, there was actually more machine learning at these companies. On the other side of the spectrum, we've seen the really, really small teams of startups or nonprofits where they might not necessarily have the resources to operationalize these models. People are just using base 10 for all sorts of use cases, so for user verification. So one of my favorite examples is this company called Primer, where they're using it to detect if someone's an adult or a child in these audio rooms, which one can talk to each other. We have nonprofits using IT for translating language models to translate coded materials and put them on the ground in Africa. We have a climate company using IT in Europe to figure out where to place energy grids offshore. And so to me, what's really amazing is that by targeting the ICs unlocked all these different use cases that we didn't think about.

I know you guys are focused on being an applied company versus primarily a research company, but that still sounds like a researcher's dream to be seeing how people are using all these different data sets that you often don't really have access to in academia versus, I don't know, for example, ad performance data on media. I love hearing about the different use cases. So you mentioned you've been working on base 10 for over two years now. And you brought people onto the alpha release just about a year ago who were some of the early adopters to help you develop the product and figure out what to focus on.

We experimented early on with all sorts of people. One of our earliest customers was a large fintech company trying to build their fault underwriting design with the basic. They didn't have the resources to build it internally. We did some cycles with a large airline and we also did early iterations with a bunch of ICs who were building prototypes. I think for us what was key there was that we wanted to be a horizontal company and we still aspire to be. We're not wed to anyone anyone use case of base 10 if anything this time is the ethos of government carried forward empowered creators were told and see what they would do. And so we kind of cast the net really rewind. One thing we noticed really really quickly with all these users was that data scientists, they know Python, they're actually more product thinkers than you think in terms of customer problems as opposed to engineering problems. And they don't take infrastructure going back to that customer journey. What is going to be the tool that allows them to think in terms of the customer needs? How do we allow them not to have to worry about infrastructure and how do we what we know data design to machine learning really well which is and I think that's really what led us to building what is base 10 today. Understanding of like let's lean into this technical Persona. Let's make sure that we are optimizing for what they want to do which is solving problems and lengths make sure they don't do the things that they hate doing.

How did you think about maturity and scope as you built this out and can you talk a little bit about the iterations and journey to get here?

I think this actually goes beyond just data science machine learning. I think today the general developers expectations of product quality is higher than ever. That's amazing for the developer. It makes it really daunting when you're starting to build a horizontal product around all the things you need to build. I think one thing that we were really focused on was we wanted the product to be strong enough that it can deliver value, but usable enough that that value could be self attained by the data science and machine learning team that we didn't want to do a bunch of hand holding. We've gone through so many iterations of base 10. We built our first version of the product and we were kind of targeting the enterprise for a bit. So we didn't really focus on usability at all. We got abstractions wrong along the way. One of the things that we wanted to ensure that we had were these abstractions that made the easy things really easy. So, sure, you shouldn't have to think about Docker and your YAML files, but when you want to add a GPU to your model, that should be really, really easy. And what this meant was that we had to have these abstractions that supported this wide array of easiness, but also allowing customers to build on top of us. And so the first version of our backend was wrong and we rebuilt it. We're about to release very soon, and I think customers really struggled with going 0 to 1. And we were able to use that time to prove out value by helping customers. But we did have to rewrite big parts of the product multiple times over and over again to ensure that it was usable. And I think this is going to be a work in progress and it's going to be a journey that we're on. I think the product's in a really good spot, but we're nowhere near where we want to be.

Was there a specific user story or experience that that someone had with the product that made you feel like we finally are on the sort of golden path? Because I know you and the team have been incredibly excited about the sort of cumulative efforts of grinding away on usability over the last months.

Sometime last month or so, we realized that people were building something without us knowing. That was a really big step for us in the sense that people were getting into the product. We had onboarded them months ago, we thought they turned off and they came back and we had this group of crypto developers, were running a consultancy on base 10, and you know, we don't talk to them and all of a sudden one day they built all this UI out. I think like the big, big moment for us was like when they started talking and out. And what I mean is that, you know, we have these concepts of views, you know, backend endpoint, which we call workloads and mod deployments. And when a user calls you up, starts telling you about, hey, this workload is not working for me, kind of magical moment where you're like, I made that concept up. That's been like a really great.

That's really cool. And they're learning those concepts by themselves from the product and it's worthwhile. And so I think that is a huge unlock. So we had long discussions about where the value was and what type of company you wanted to build. From a go to market perspective, you went through a few iterations and then just very much committed to building a product that a single engineer or data scientist could self serve. And now you're seeing the magic of that, like, why is it important to you? And like, you know, if it's one user at a time, how does an organization adopt base 10?

Yeah, that's a good question. I think why it's important to us is like, you know, we went through a lot of iterations as you said, and I think for a while when it was just me, Amir and Phil, we just decided at some point that when if we're going to build a company in the long term, we had to be really excited about the go to market motion and it had to match who we were as individuals. And you know, I think one of the things that I talked to other founders about early on in their journeys, which is that whatever you do, it's just going to be really hard. I don't know if there's an easy way of building a company. If you know Sarah, let me know. But there's two things that we decided early on was that if we're going to do it, we're going to take a big swing. And if we're going to do it, we may as well work with the customers that we like, people like us that we really enjoy talking to and spending time with. And that's been really amazing, working with engineers and data scientists. Our customers are really smart. It's honestly a privilege that we get to spend time with them. We learn from our customers constantly, which is amazing. I think how organizations adopt based 10, that's a really good question. What we've seen is that usually it's one data science engineer, they get in, they start playing around. It doesn't really matter what the use case is. It can be with or without machine learning. I think there are just so many applications that get built from the point of I have no data or I have data to I have something shipping and adding value, I want to iterate on it. It doesn't really matter where they come in. They come in, they start building, they build a tool, they usually start to invite either other data scientists or people who will use that tool onto base 10. And then what we see is that once they're in, they just, they say, well you guys do X, Y and Z as well. So you know, Patreon started with a data labeling tool and they're like, oh, but you got to do model deployment too. We should give that a Shot, I think same with Pipe. Exactly the same journey. They started with this really lightweight digital labeling tool and then like, oh, actually we have this model. Can we deploy it? So usually what we see is that it doesn't matter what they come in to do, but they come in to build an application, they bring other people in, and they just start to build.

As somebody who invests in a lot of collaborative tooling for technical audiences, I feel like this is part of the magic of this type of business, which is incredibly hard to get right from a product value and usability and aesthetic and distribution perspective. But when you do, like, the amazing thing about technical people is they choose tools for much of the organization and they take the tools they like with them. And like, it's. It's amazing to see you guys empowering that. The company has accelerated hiring dramatically over the last, like, 612 months. Can you talk a little bit about where you guys are as a company and, you know, who the team is and how you collaborate?

We are a group of 20ish people. We have a small office in San Francisco, but we are essentially a remote company. The office is more of a social gap as a gathering spot as opposed to a place where people do work. We didn't set out to build a remote company, to be frank. I was actually quite against it. I was like, I love being in the office. What are you talking about? I live in LA and I fly off to BNSF to be around people constantly. But what we realized once the pandemic hit was that, you know, it was actually golden opportunity to hire all sorts of folks who otherwise wouldn't consider working at a company like base 10 because it wasn't in the geography. And as a result, you know, you know, we built this remote culture. We have people in, you know, San Francisco, Los Angeles, Seattle, Montana, Quebec City, New York, Boston, Armenia. And it's been fantastic just to see, like, the different types of talent that remote work has unlocked for us in terms of what that means for us. I think just like every other company that I think we had one employee punkage before the pandemic hit. But for every other company that's matured during this period, still a work in progress. We're figuring it out, but we do our work. We spend a lot of time on zoom, unfortunately, but at the same time, we're figuring out how to make that effective. But I've been really, really lucky to be able to hire some really, really senior, kind, thoughtful folks who have a really constructive mindset to work a lot of that's been from our network. But I'd be. I can't not give a shout out to John and Duane of the Greylock recruiting team, who have just been fantastic in terms of adding value. It's pretty unreal, I think, in this hiring climate that is just ridiculous. And these are high quality people who have shaped our company and we wouldn't be where we are without them.

We're rooting for your success. And John and Duane, if you hear this, we love you. This is actually a pretty broad product and now you have a certain level of product maturity and you have production deployments, you've still decided to call the platform release a beta. Why are you doing that and where are you excited to continue deepening and extending the platform?

Yeah, two things. We still think we're at the earliest stages of learning. The product feels mature and we have production use cases, but we're still figuring out a lot of the usability kinks. I think the reason why we want to call it beta is just to invite people into the product with having high expectations of what they'll build and really surprise them once they're in. I think it also gives us a bit more wiggle room in terms of like making sure all those abstractions are right. We just have so much to build and we hope that base 10 over time can power a lot of value and hopefully we can capture a small amount of that, of that value and build a really valuable company. But overall, to get there, it's gonna take a minute and we're not there yet. There are three parts of base 10. It's a pretty wide product, right? So like there's the model deployment engine, we have the workflow builder, which is basically an API endpoint builder. We allow you to embed a model and surround it with all sorts of code so you can actually build business logic alongside that model. And then we have a UI builder. I think given that we have these three pretty intense components, let's be honest, there are three mini companies or mini products within themselves. There's a lot to build and what that means is that everything's at slightly different maturity. One of the reasons why we're releasing this as a beta is because we want all those three things to be up to parity before we have a go ga. I'd say today the workplane and the model deployment piece are doing really well and I think we've seen lots of value being added there and the UI built a bit earlier in life and we're going to continue to iterate and learn from customers. And as part of this beta, the goal is to get that up to the same level of maturity and then we'll go ga. So the cool thing about having these three different product pillars is that actually gives us three places to build this platform and ecosystem. And so on the model side, you can deploy a state of the art model with base 10 very, very quickly. Today we have I think maybe over 30 state of the art models that you can deploy with one click, whether that's wave 2 vec0 classification. We have a GPT3 integration. You can use the state of the art models and it gives us a way, I think today to build this ecosystem there. I think on the worklet engine side, you can almost think of the way of assembling this business logic as blocks and we can build an ecosystem around the different snippets of code you might surround your model with. If you're a data scientist and you want to write your prediction back to Salesforce, do you need to write that integration yourself or can base 10 do it for you? And I think the third part is the views where we have these views of these, these individual components, a drag and drop builder. And you know, we've already had customers ask us can we contribute back to this so we can reuse some of these components? I think the even more exciting thing is when we, when we take all three of these things and layer in templates and I think we can have an ecosystem of templates that uses a model with some worklets with a view for a particular use case. So if you're a company with a content moderation problem, can base 10 take you 0 to 1 or 0 to 80 or 90% of the way? There's with this pre built template that uses a specific type of model, has all the queuing logic and verification logic built in, and then has a UI for reviewing those predictions.

Yeah, that last piece sounds like very holy grail for many machine learning applications because I think the, you know, most organizations like, let's say they can find the two hin and Phil and Amir, like Unicorn, that's like, yes, like I personally know data science and full stack engineering and I'm motivated to build it all together. Given that they're still kind of stuck between I'm going to use some sort of off the shelf application or API against my use case or I'm going to build it all from scratch myself. I think what people really in many cases want is the ability to use a state of the art model and to tune specific parts of that workflow. And that could be, as you mentioned, pre processing or it could be the UI for the analyst in a fraud use case, but not like take the entire lift of building that application from scratch and get all of the performance and benefit. So the glorious future, it's coming in.

Terms of just like with all three of those pieces, like we do think about them in a modular format so you don't have to use all of base 10 to get value from it. I think you can use just the model deployment piece or the workload builder or just the view builder. And we're built for you to integrate in and out of base 10 across those three pillars.

You've talked about base 10 being a very mission oriented company. If base 10 succeeds, what change do you hope to see in the world in the next decade?

If we're successful, we're going to reduce the costs of shipping. ML iteration cycles will be quicker, more people will be collaborating around models and I think overall, if there's anything you can do to increase collaboration and quicken iteration cycles, you'll see an effectiveness in the overall process. And that's really what we hope is that as a result of base 10, we are going to increase the effectiveness of machine learning efforts. And as a result of that, really that's going to result in more models, more businesses, because there'll be more investment because we're seeing how ML is moving the needle for real organizations.

Yeah, I absolutely believe that. Tuhin, thank you so much for doing this. Congrats to you and the entire Base Gen team on the platform launch and we're thrilled to be working with you.

Awesome. Thanks, Sarah.