The fastest way to build ML-powered apps

The thing that I'm seeing more and more is that the thing that is most valued is people who are able to take some data, write some SQL, create this simple data set to train a model, and then if you can get it behind a flask API working decently performantly, not amazingly performantly. Honestly, you're in the top 1% of data scientists. I think for me, the trend that I'm seeing is that people are valuing those people more. I think like 5 years ago people were looking to hire data scientists and more and more it's like, hey, we want machine learning engineers. There's a difference there. And data scientist is in my mind kind of a flawed title in the sense that it just represents anything from someone doing BI as an analyst to people building models to ML engineers. And we're kind of separating those things out, which I think is great. But I think the trend that I'm seeing is that more people value those machine learning engineers. Unfortunately, it's very, very difficult to find those folks. I think it'll take a bit for everyone to catch up to that.

Big thanks to our partners Linode, Fastly and launchdarkly. We love Lino. They keep it fast and simple. Check them out@lino.com changelog our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by LaunchDarkly. Get a demo at launchdarkly.com this episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal tech books and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos and all their conference talks. Plus you can learn by doing with live online training courses and virtual conferences, certification practice exams and interactive sandboxes and scenarios to practice coding alongside what you're learning. They cover a ton of technology topics. Machine learning, AI, programming languages, DevOps, data science, cloud containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there. If you need to keep your team or yourself up to speed on their tech skills, Then check out O'Reilly's online learning platform. Learn more and keep your team skills sharp@o'reilly.com changelog again, o'reilly.com changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@gamejog.com community and follow us on Twitter. We're racticalaifm.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm joined, as always, by my co host, Chris Benson, who is a principal emerging technology strategist at Lockheed Martin. How you doing, Chris?

I'm doing great. How's it going today, Daniel?

It's going wonderful. You got any cicada broods brooding down by your parts?

So for any listeners that have listened to us for a while, they know that I do these weird wildlife things. And so, yeah, I'm getting tons of calls. It are like totally outside the AI thing. Our nonprofit that does the animal stuff. We're getting all sorts of calls about Copperheads and all that and people are afraid and really it's probably too late by the time people listen. But it's not an issue. There are no more Copperheads than there ever are. So I know that has nothing to do with AI and ML. But, you know, there you go. There's my public service announcement for. For the circada.

Yeah. I mean, it is sort of a fascinating thing that I hear people talking like, you know, of course, like with climate changing and other things, how the broods are sort of a bit off. But some people, you know, it's hard to know a little bit because also more people are reporting data via all sorts of apps and channels and all those sorts of things. And now, I mean, you're probably familiar with these apps that.

Yep, there's several of them.

I think it's Inaturalist and other things where people report different species and that sort of thing, and those are fed into models and all sorts of goodness. So, I mean, it is interesting.

I love how you just tied the wildlife thing in with the whole AI ML thing. That was really good. That was good.

You know, it's just one stream of consciousness for me. I'm really excited. Well, Chris, you know that we're very practical on this show. At least we try to be. And relevant to our listeners.

Indeed we are.

And I think that this is definitely going to be a very practical discussion because we're going to talk a little bit more about some new AI and ML developer tooling.

Excellent.

And deployment tools and all of that stuff. Today we've got Tuhin Srivastava with us, who is Co founder and CEO of Base10. Welcome, Tuhin.

Hi. Thank you. Thanks for having me. Chris and Daniel.

Yeah, it was great to meet you. We had a chat last week and I was learning a little bit about some of the things that you're doing and had to get you on the show right away. So, really excited to hear about that stuff. But before we dive into all that, could you just give us a little bit of info about your background, how you got into doing what you're doing now?

So my background is actually in electrical engineering. I studied electrical engineering in college, and after college, I decided that the right thing to do was to go work in finance in New York for a couple years. It had nothing to do.

That's the rumor. Academia that I heard was like, you can sell your soul and go make a ton of money in finance. Not that you have to sell your soul to go into finance.

Yeah. Well, I got to work on the fun problem of privatizing toll roads.

Ah, there you go.

Which is fascinating in its own way. But after a couple years of it, I decided that this wasn't for me and I decided to go back to engineering. So I actually moved to Boston from New York to work at a weird academic lab at Beth Ezrael Medical center, which is part of Harvard Medical School, where there's a professor there who'd won a prize, coming up with a non invasive way of tracking the progression of als. And he was spinning off a startup as part of that that was very, very data driven. This is like 2011, and I didn't know anything about machine learning or stats. I'd done a bunch of electrical engineering and information signal processing in college. I was like, I could probably convince him. Just take me on. And Boston startups are a weird breed in that they're kind of very research focused, they get a lot of grants, and frankly, they're quite cheap. And so I was like, I'll go and tell them to pay me hourly. I'll go work there. So I went there and I got to work with this guy and these three MIT PhDs, basically figuring out if they can predict the prognosis of neuromuscular disease. Yeah.

Do you remember their names? I'm just curious because I've read up on ALS lately.

Yeah. So the guy's name is Stuart Rutkove, and the technology that he created was called eim, which is Electrical Impedance Biography. And this is really interesting and it makes me excited, so I'll tell you really quickly what it is. We use EMG today to track how muscle health is progressing when folks have neuromuscular disease. It's a pretty horrible thing to go into every three months and have a bunch of needles poked into You. So his idea was rather than poke needles in and track the electrophysiological properties, to shoot a small current, a microcurrent that no one can feel across the outside of the muscle, measure the impedance drop and see if that correlates with muscle health. And you know, it does. He's been working on it for 20 years. I happened to catch him at a time when he was commercializing technology. You know, a really fun time. You know, I got to work on all sorts of things. I learned statistics again, I really got into machine learning. And so really, you know, the first thing we did was at the time we were applying support vector machines. This is like 2011, 2012, when support vector machines were all the rage. Support vector machines to see if we could separate out healthy muscle from diseased muscle. That was really fun. Published a couple of papers and kind of really got. I was like, okay, this is pretty cool. You know, it seems like there's not that many people doing this. I could focus on this for a few years. And so I did that for a couple of years and my wife was. My now wife, then girlfriend I guess was a PhD student at Berkeley. I was like, I need to move out to California. So moved to San Francisco and started working at this company called Gumroad. You know, Gumroad is a lot bigger now than it was then and I was one of the first few employees. It was open ended e commerce platform. So anyone can sign up and start charging people, anyone can get paid out. Great idea for creators, you know, really, really horrible from like a payment security and fraud perspective. You know, especially in the early days, you know, we used to get hammered with fraud. And so we just started first applying heuristics and then we built out an entire kind of like ML pipeline there to not only cash fraud, but also the tooling to be able to deal with it once something had been flagged. So did that until 2015 and then kind of had the entrepreneurship bug and founded our first company which went through a bunch of twists and turns. Ended up being an operational analytics company where we would capture data from a bunch of different operational tools, create a BI layer on top of it. Not really Mr. Related, more data related. Sold that company in 2018 and then after working at the acquirer for a year, decided that, hey, I really wanted to go back and work on ML tooling. I built a bunch of this stuff. I think a lot of ML engineers and data scientists end up building their own tooling even today. And you know, it seems like the beginning of something a lot bigger and you know, don't know where it's going to take us, but really wanted to work on this idea of how do we get more people doing more machine learning and the tooling that will enable that.

So I'm curious, as you were sort of in that progression, you know, you were sort of thrown into the fire working with statistical tools and machine learning at various stages in different companies throughout that history. Was it mostly sort of you as a like data scientist or ML type person working with a bunch of developers? Or was it sort of a bunch of developers trying to kind of pull in machine learning tooling where they needed it for like the fraud detection and stuff? What sort of patterns did you see as you were working at those various places?

Yeah, that's a really good question. I didn't actually start as an engineer, I started as a data scientist and machine learning engineer. And when I was in Boston working at the healthcare company, it was okay because it was very academic and a lot of it was like in Matlab and we could get a lot of the stuff done in Matlab and throw it over the fence to someone who would happily take it out of that. When I joined Gumroad, it wasn't like that I was a data science and engineer. It was an early stage company. I ended up hiring another data scientist and engineer who was a really good dear friend of mine from Australia. You know, we didn't have the resources to help us productize the stuff we were building, our production, so we just had to learn it ourselves. So it was very much coming at it from a data scientist perspective. But I don't think I was ever lucky enough to have, you know, a team of devs ready to go being like, hey, we'll happily productize this or productionize it. It was very much, I was expected as a data scientist or machine learning engineer to do that myself or be resourceful about that.

So do you feel like in that process, where was your time spent in that process? Like as opposed to like before when you were at the company where most of your time was developing these sort of prototypes in Matlab, like and now you were having to bring in this engineering side of things, like how did that sort of split up your time?

Yeah, that's interesting. I think initially it was obviously in the model development part of it. What I realized really quickly was like a lot of other data scientists, you end up being perceived as somewhat of a research function as opposed to a product function. I found that really demoralizing. And so what I found was that it was in my best interest and the company's best interest if I worked on getting my stuff out of the Jupyter notebook and in front of other folks, as opposed to keep optimizing my model. Because you can do that forever, right? The model is never going to be 100%. What I realized pretty quickly was that for me and Phil, who was the other data scientist I was working for, that, hey, we already got the highest order bits out of the model. What was the most important right now was to make this practical for the company and have it plugging in back into the business process and really have other folks, whether that be the CEO, whether it be other engineers, customer support, really seeing the fruit of what this model can enable for the company, for them.

It's a bit of a timeless problem that you're describing there and that I know in the organizations that I've been part of over the years, I've seen that same set of problems, problems recur over and over again, as well as those same biases and perceptions that people have. You talked about people assuming that the data scientist just has a research function, whereas the three of us are talking here today. We understand the value of applying data science into the daily operations of business, and yet so many people have failed to do that. It's a fairly wide chasm to get all of the things together, to be able to not only do the data science, but to be able to communicate that out to the audience in an effective way, in a way that is accessible and usable to them. How did you approach that problem? Because, I mean, that's a massive problem that many people have tried to tackle and I don't envy you that. So how did you set about going about that?

Yeah, I think I was lucky in that I've talked to a lot of customers over the last year and a half, and one thing we realized is that a lot of engineering functions, and this is just a kind of a nature of the beast of where a lot of data science and machine learning engineers are coming from. A lot of engineers don't want them writing code. They're like, hey, it'd be best if you didn't do that. And I was lucky enough to be in a place where it was encouraged to write code. And so honestly, for three to six months, I just became a Rails developer. I was like, the company's Monorepo was in Rails. I was like, I'm going to figure out how to do this. I'd already done a little bit of Django Programming on the side. I was like, I'll learn Rails, I'll understand kind of their language, and then I'll be able to bridge that gap. I think that was really good. It's been really great for me because as I've gone through and built products over the last five or six years, being able to be an engineer has been great, but at the same time, I don't know if that's attainable. I don't think a lot of great, really intelligent data scientists who have great intuition when it comes to models want to be engineers. I don't think they want to learn about unit testing. I don't know if the best use of their time, but that's how I went about it. I think I've heard this story enough that there's a lot of leverage to be gained if you can somehow give data scientists engineering skills. And there's not enough folks who have both of those skill sets. And has that been your experience as well? It has.

I mean, I know that Daniel and I both came into the AI world with lots of programming experience and different backgrounds. We actually did not meet each other in the AI ML world. We met each other in the GO programming language world and kind of transitioned over. And so I think my own perspective on that is that no one can master everything, but it makes sense to have some areas of strength and then extend your tentacles in other directions so that you can collaborate with other people who also have multiple areas, but maybe their strengths are in a slightly different area. And so that's how I've kind of tackled it as I've worked with different groups over time. I'm always super excited to hear about how people are addressing it. And really, the thing that you're doing that I'm really impressed with is being able to scale that out to many people, whereas I'm struggling to do it on my own and to work with different teams and to make those skills overlap. And so I'm pretty excited to see, you know, kind of what you came up with in your solution here to enable that for other people such as myself.

Yeah, I think that I do see this changing, and maybe this is what you're referring to, Hien with sort of things changing in the landscape before when I saw many tutorials for people getting into AI and other things really just focused on data prep, training and evaluation.

Yeah.

And I think that, like, there's just this sort of common perception that, like, okay, I'm evaluated, like, end of story, like, I've done my job and some People I talk to just even this concept that, like, you can save a model, like you can serialize it to a file and then use it later, like, that's something they're sort of not quite used to yet. I find that very, like. Yeah, I mean, even if someone's not going to be like, you know, a very engineering focused person, like, having some level of understanding of the integration points and how, like, our work feeds into other things, that integration point is very important. Was that what you were referring to when you're sort of talking about, like, trends you were seeing or other things?

Yeah, totally. I think the thing that I'm seeing more and more is that the thing that is most valued is people who are able to take some data, write some SQL, create this simple data set to train a model, start to train their model, and then if you can, get it behind a flask API, working decently, performantly, not amazingly performantly, honestly, you're in the top 1% of data scientists. I think for me, the trend that I'm seeing is that people are valuing those people more and more. I think, like five years ago, people were looking to hire data scientists, and more and more it's like, hey, we want machine learning engineers. There's a difference there, I think. And your data scientist is, in my mind, kind of a flawed title in the sense that it just represents anything from someone doing bi as an analyst to people building models to ML engineers. And we're kind of separating those things out, which I think is great. But I think the trend that I'm seeing is that more people value those machine learning engineers. Unfortunately, it's very, very difficult to find those folks. I think it would take a bit for everyone to catch up to that. And so that's kind of like the trend we're seeing. And that's one of the trends that we kind of latched onto as well.

So I'm very curious. We've led up to it without talking, without diving into it yet. But I would love to hear the backstory about how the idea for base 10 came about, how you moved into it, how you arrived at the start of that process.

Absolutely. So, you know, I think we've kind of gone through the context here, which is, hey, you know, it's pretty hard to get a model working. The hardest part of that, or maybe the most important part of getting a model working, from my perspective, and you know, quite frankly, from the name of this podcast perspective, is making that model practical, is making it plugged into some process or in front of real users who can give feedback on that. Again, like as I mentioned, we noticed this, we learned that skillset. So I have two co founders. One is Phil, who I've worked with a bunch. And our third co founder was kind of the head of engineering at Gumroad who went to work at a healthcare startup which was very, very data driven. And from his experience as well was that there's so much tooling that needs to go in once you have that model. So whether that be the deployment monitoring tooling, but more importantly like the stuff that happened afterwards, which is how do you build, how do you integrate that into existing system or build an application out of that model. And you know what happens today that we notice over and over again was people were kind of building this flask app. Sorry, I don't know if I took it.

It's okay, keep going.

There's a lot of them out there. I've seen them with my own eyes.

Yeah, so you know, like flask apps that they're definitely not going to scale. It's pretty hard to get them deployed. You know, everyone starts by pickling their models, but really quickly, as soon as their model gets any significant size, you can't fit in memory anymore. So you can't keep pickling and unpickling things. You know, we used to do this stuff ourselves. Like I'm not hating on it at all. It's just like, you know, we built these really bad systems where we took a model and especially at Gumroad, we then deployed it. So we built a flask API. We were pickling and unpickling it on the fly. Not great. But then we were building all the application logic that sat on top of that model. So hey, model gets run. So we compute the features, model gets run and then the decisioning part of it, if the model exceeds the threshold, do X, Y and Z. Otherwise put it in front of a human to go and review that transaction. What we did was we went and built all those things. So we deployed the model, we built the backend service to support that model and then we also learned front end engineering to go and build that kind of like Q like interface for folks to do that. And you know, my co founder had a very, very similar experience at a healthcare company where they were trying to predict diagnoses from a medical record. And you know, they wanted to put that in front of the doctors so the doctors could get feedback on it. The way they did this was they ran the predictions in batch, they piped into some data warehouse, ported it out, put it into Excel. I'm not joking. They sent the Excel file with like the patient ID and the predicted diagnosis to the doctor and asked for feedback. Obviously, the doctor's not going to like, act on that, right? They're going to be like, this is too much work. And so I've kind of led up to this where I'm saying that, you know, if you are that top 1% of data scientists or machine learning engineer who can kind of do all those things that we had to learn how to do, this is great, you can do that. But for the rest of them, can we give them the leverage to become that data scientist? And what does that leverage mean? And so what we went and built was a way to lower the barrier to usable, productized machine learning. And so today, with base 10, you can deploy machine learning models with a couple lines of code. And so what that means is that you go import base 10 in your Python library, you base 10 dot, deploy with your model binary, and we take care of everything. So we docker, right? We create a Docker container, you know, we load it all up, you get an API that's ready to go, that's nice and scalable and really, like, don't worry about Flask, don't worry about the infrastructure, the servers you need behind this to make it work. Do you need GPUs? It's literally, you know, an argument or a check of a box in our UI to put it behind, power it with a gpu, and don't worry about versioning. So you can keep deploying over this and we can do like version rollout and you can switch back. What's most important though, is that, that to us is the integration cost. This is really exciting that you can do that with a couple lines of code, but that's not really where we see our value. Our value is the stuff that comes after it. Can we make it so as a data scientist and machine learning engineer, you can also write all the business logic on top of that model. So we kind of give you a serverless framework to be able to write logic on top of that model. And so you can build APIs, you can run things from Cron, you can even have forks, so you can have decision points within that business logic. But a data scientist or machine learning engineer can do all these things without learning any infrastructure. All they really have to worry about is their Python code. And that complexity of the infrastructure doesn't vanish, it just shifts. And we think we can take care of that. And all you need to worry about is your code, your Python and your model. Lastly, something that we loved, you know, at times we're kind of told that we're trying to boil the ocean here, but that's fun to us. The last thing, the thing we do that we love is, you know, we always wanted to put a face on our model and interface on that model so that folks could either give feedback on it or operate off the output of the model. So if you are building a recommendations model, can you really quickly assemble a UI so someone can input something, run that model that you deployed, see the recommendations and give feedback on each of those recommendations. So the data scientists can take that and really get that iteration cycles going. So you can build that in a drag and drop way without learning any JavaScript, HTML or css. And that's really where we arrived for now what we again hope to be able to do is just give more machine learning engineers and data scientists that leverage so they can kind of ship full stack applications. And hopefully as a result of that, what you'll see is that people see results of machine learning faster and as a result they will invest in machine learning more because they'll see what it can do for them. Sorry, I just spoke for a long time.

No, it's good.

No, that's great. Yeah, really good. To get that whole context, one of the things that I find interesting is like I see sort of a trend on my end in terms of ML AI in that world. There's sort of this trend to kind of no code solutions and that sort of thing where things plug together. What I find interesting about sort of what you just talked through is it's not no code. So the data scientist is still sort of working in their Python world, but then it lowers the barrier to these other things. So there's like a data scientist. I think if we assume they're working in Python, it's not that far. Like you say to maybe have them think about a flask app or something like that, but then as soon as they try to start getting that out into the world, they're like, oh, how do we scale this up? And other things. And then they find out, oh, there's these various layers of abstraction that I need to think about, like, you know, containers. And then how does that scale up? Well, now I need multiple instances of my service behind a load balancer and where all those containers gonna run? Well, they're gonna run in Kubernetes or some container orchestrator. It's like it starts to develop in their mind, this sort of like inception, dream within a dream within a dream, like they just sort of keep going down that thing and things start becoming just less clear and really strange. So I think like it's cool that you could still let them operate. From my experience, data scientists, you know, and people working in this area, they love opening their Python editor 100%, writing out their code, really having kind of fine grain control over the pre processing and fine grained control over how they build their model architecture and all those things. That's sort of where they want to live. So not like saying, sort of taking that away from them and saying no, don't do that in code anymore, but sort of letting them still enjoy that part of their code, but then adding on these other things I think is a really interesting way of thinking about it.

This guy has become a friend of sorts. His name's Theta Stitch. He had this great tweet over the weekend which was what I want. Even more than no code is yes code, but literally you never have to think about infra at all. And I think that is from our perspective like the data science, like the great articulation of the data scientist issue, like Python is a really, really valuable tool. You know, you take pandas away from me, I'm a third of the person I was. I want to write Python and I want to be able to use SQL as well, you know, but I don't want to think about Docker kubernetes. I don't want to think about, you know, if I deploy a model, how do I switch versions, how do I ab test things, how do I monitor things, how do I deal with downtime. You know, code is great, infra is hard.

Yeah. Before I ask the next thing I gotta say, you have some really good one liners in there that I'm gonna steal for later on. Please do the boil of the ocean. Well, that's what I like to do is an excellent one because I hear.

That all the time.

Every day of my life.

Yeah, yeah, you've had several there. So I'm stealing your stuff, dude.

I appreciate it. I wanted to ask.

So one of the things that it sounds like you've done a really good job on is by taking those levels of abstraction, as Daniel referred to it, the dream within the dream. You can only so many layers of abstraction, you can't do it all. And it sounds like by taking control of some of that, that Somebody can use base 10 to integrate in with an existing environment by using the APIs and so they get a big payout. And they don't have to worry about all of the things, you know, whether, you know, from containers, you name it, all the things that Daniel just talked about and yet can get the benefit that way, which is quite beautiful because I know that it's kind of where people have been trying to drive to, and it sounds like you've gotten a very nice, elegant solution for achieving that. Is that fair? Is that kind of how you would think of using it?

Yeah, I think, you know, we're very early on and, you know, early users will tell you that there's a lot more elegance to be added, but there always is. That is, you know, from like an abstraction perspective. Absolutely. You know, like, I think one of the things that we did early on was, like, we kind of defined kind of the principles of the architecture that we wanted to build. And I think, like, two of them that really stand out here are like, the principle of least astonishment. You know, the. Like, take it away, but don't make it magic. I think that's like, you know, it's like, hide it from me, but still give me details of what you're doing.

You have a blog post that includes that information, I noticed, which you could talk a little bit about here, because I was going to bring that up as well.

Yeah. So the principle of least astonishment is really important to us just because, again, like data scientists and machine learning engineers, we're kind of like these pseudo engineers. Right. We're somewhere in between. We know what's going on. Maybe we don't know kind of all the intricacies. So all the abstraction we've tried to keep quite similar to, firstly, what data science and machine learning engineers understand. And two, even building an application for an end user, there's a back end, there's a front end, and you see these concepts in base 10, and we separate them out and we allow the interplay of them, but we're not hiding them all together. I think the second thing, which I think you're kind of talking about is this is something that we saw from one of our early engineers. It was like, easy things are easy and hard things are possible. And to me, that's such a great line.

It is.

Which is we want to 80 or 90% of the things to be simple and you don't have to think about and they just happen. But that shouldn't come at the expense of control and visibility. And I think that's one place where a lot of kind of lower code or no code abstractions get dinged, was that they hide things away and you're kind of like, well, how did that happen? That kind of scares me. We are exposing these things and so you can go deeper if you want with each of our concepts because we are built on Docker Knative postgres. You know, you can write postgres queries, you can kind of inspect those technologies. Go deeper if necessary.

This episode is brought to you by Snowplow Analytics. Snowplow is the behavioral data management platform for data teams. Maximize the value of your behavioral data using Snowplow Insights, a managed data platform that's built on leading open source tech leveraged by tens of thousands of users. Capture and process high quality behavioral data from all your platforms and your products and deliver that data to your cloud destination of choice when marketing needs to make data informed decisions, when product needs next level understanding, and when analytics needs rich and accurate data. Snowplow is a solution for data teams who want to manage the collection, processing and warehousing of data across all their platforms and products. Get started and experience no plot data for yourself@snowplowanalytics.com again snowplow analytics.com.

So with your solution, we've talked about several different things. So you talked about the sort of, you know, importing a few lines of code, deploying your model and also this sort of UI builder. Could you kind of describe like for someone maybe that's listening to this that doesn't have a webpage up, could you sort of, you know, just describe a bit like the developer workflow in terms of what they would need to do with base 10 versus in their code to take their model from local model to an API and then also maybe to build an one of these UI apps around it. What does that workflow practically look like at this point?

That's a really good question. So you start the workflow from where you do your work. So from a Jupyter notebook I mentioned earlier, you can import base 10 and base 10 deploy your model. That's it.

And that would be the rest API sort of framework.

Yeah, exactly. So basically we work with Pytorch, Scikit, Learn and tensorflow models. Right now you can also write a custom model. So if you just create a class with load and a predict method, basically every time the model gets run, that predict method will get called. The load is kind of the deployment step. Just again, easy things, easy, hard things possible. Like if you want to do more, you can do those things. But once you deploy it with that single line of code or a couple lines of code, what you end up with is like a REST API. You'll go to base 10CO, you'll be able to see the status of your model all the times it's been called. But then you get instructions on how to call it from your own server straight away. Once you're there, you can easily move from the realm of the model to the realm of the application. And that's what we're really excited about. And so, you know, for that model you can then write all the pre processing code and post processing code within base 10 in the browser around it. So think of it kind of like a. You know, we're still iterating on this and figuring out the exact right experience, but right now it kind of represents a DAG that you're used to in airflow. So you can write the code within base 10 is represented in a graph like structure. Again, there is a shared state in context between each of those nodes in that dag. So you know, you can reference things from the pre processing node and the post processing node, but you can do other things as well. You know, if you want to call the Twitter API, it's a python like environment so you can bring in whatever libraries you want once you've done that. Now that entire thing, we already gave you the REST API for that model, that entire thing is callable by an API. You can also trigger it from a CRON job. So we've got built in support for Cron or in line or you can call it from a streaming data source if you have a Kafka queue. We built those integrations as well. So you've really gone from just a model in your notebook to something that was deployed with an API really quickly. And then in just a little bit more you can start to write the kind of pre processing and post processing code and logic within base 10. Put that behind an API. And so really what we're seeing is stuff that would take three to five days of kind of wrangling and getting it set up without really even thinking about how to get it on AWS with an API. I don't like exaggerating, but it really is like within 30 to 40 minutes we've had live services running with machine learning models. I think the last thing now, as I mentioned earlier, is that now you have kind of like, so you have this model now you have these API endpoints, then you can start to build UI within base 10 and link actions within that UI to those API endpoints.

So with this deploy method, one of the things I'm thinking about just from your perspective in terms of developing this infrastructure is all of the crazy stuff, that varied frameworks and architectures and everything that people are using. So they have all sorts of crazy dependencies. Some people are using, like you're saying TensorFlow, some people are using Pytorch right now. How do you approach that sort of dependency stuff? So you mentioned supporting like TensorFlow, PyTorch models, SciKit Learn models. What about the sort of other dependency stuff? Is it a matter of pulling that from a person's like virtual environment? Or how do you go about handling that side of things?

Yeah, again, as I said, like with The PyTorch and TensorFlow and SciKit learn stuff like, you don't have to really worry about that with the custom model stuff. We do need that. We do need to know what requirements you need. So really when you deploy that custom model, you also passed us a requirements txt and then we basically set up the environment with that. Ready to go. Cool.

Yeah, yeah. And I'm trying to. So at the beginning of this conversation, I think this conversation has clarified a lot for me and sort of where I'm distilling things down is that it seems like if you look at this sort of landscape of developer tooling for AI people, there's sort of some things kind of over in one area around ML Ops and like experiment management, like logging experiments and all of the thing, you can run your jupyter notebook and log your experiments and put your job in queues for GPUs or whatever. And then you sort of got maybe on the other side of things like widget and prototype type builders, you got sort of like Streamlit or Gradio or something in the middle. Maybe you have maybe some serving frameworks that people use. Maybe it's like TensorFlow serving or something like that. But this sort of like, path from, like left, you know, from one side of that to the other, just like involves so many different, like, jumps between different sort of systems and spinning up all sorts of different kind of infrastructure and all of that. And I think all of those tools have obviously some really interesting features, but it almost seems like you're sort of not all of it, but you're gluing many parts of that together in sort of a consistent workflow for people that really could sort of accelerate them through that process from experimentation to deployment. Am I characterizing that right? I'm trying to think of how people might sort of view this with reference to other things in the landscape they might be familiar with.

Yeah, I think That's a really good characterization of it. I think we talked a bit about this last week on our course. This pretty wild how the MO Ops tooling has that's evolved over the last couple years. There's so many great point solutions out there and that's kind of like the ML upside on the streamlit hex side. There's some really great kind of notebook based application builders that are out there. Putting all these things together is quite difficult. And also having to jump between three or four different tools to kind of have that end to end thing working is pretty hard and it's complex. It takes time, patience, budget and most of these tools, including us, we're in beta still. You can't just sign up and you're ready to go. So it is difficult to do. What we do want to be able to do is again when you're Aruba and you have literally millions of requests per second, going through baseline is not the right tool for you. You need something specialized when you're working on something that in its infancy or it's critical but you know, it doesn't need that type of latency or doesn't need that sort of specialization. What we Hope is that base 10 can kind of fit the bill of that end to end solution. So you have your model and rather than waiting for DevOps to come and deploy your model, waiting for a front end engineer, product engineer to come and build the things that make that model relevant for like 80 or 90% of the use cases. We think base 10 can be that end to end solution to one, get your prototypes and toys out the door and two, in the end build those full stacks mission critical applications as well.

As you're looking now, you know you're in beta, you're, you're approaching release. What do you have in mind? What needs to be there that may not be there yet that you're envisioning for release? Are there any new features or is it solidifying the things that you already have?

Yeah.

In base 10, how are you seeing that 1.0 release in the short term?

Yeah, totally. It's a really good question. So we did like a launch last week which was well received and you know we got a ton of inbound from that, which has been great. But really what we're trying to figure out in this next phase is, is usability. Again there's the user and the use cases. They're intricate and they're complex and figuring out hey, how can I get you to value, how quickly can you have your aha moment and be using this in something real. We want to solve that and we want to solve that world before we open this up to kind of the world and say go wild. Because we know that a lot of those decisions are easy to iterate on and change once you are working with a small core group of users where you know the values to be added.

As you're having all of these discussions with users, obviously right now you're talking to a lot of people, you're getting feedback. So you're sort of seeing a cross section of the industry to some degree as you're having that perspective. And maybe looking at the industry more generally as we close up here. Is there anything in terms of the AI ML industry or maybe the developer tooling industry that just sort of generally excites you in terms of where the industry is headed or maybe capabilities that are being developed or other things? What's on your mind? Yeah, I mean you're obviously focused a lot on base 10 right now, but sort of what are those things that you're thinking about towards the future?

Yeah, so there's two things that I keep thinking about which make me really, really excited. So the first one is one of our investors has a really good campus recruiting program and so we're able to spend a lot of time with new grads. What's really exciting to me is that, you know, nine out of 10 software engineers, not data scientists coming out of, you know, the best schools in the country or honestly any computer science degree, have exposure to AI and ML. They're taking a couple courses. At the very least they can phrase they can frame a problem in terms of how machine learning can solve it. So and that to me is amazing because that means that the amount of evangelization that needs to be done within a company is lower because people already appreciate what can happen with that. I think the second thing which should be considered in line with this or like together with this is the prevalence of pre trained models is going up. You know, Hugging Face has done so, so freaking well over the last couple of years. And you know, every company, almost every company that we talk to who's starting to some NLP initiative is starting with Hugging Face, maybe with some fine tuning step. But to me that's really exciting because you have two things going on, right? So you have one thing going on which is more and more people know what ML can do and two is that there's all these ways to have that model ready to go. Like you don't need that two or three month effort to get that first model up and running. Put those things together and I think you got something really special, right? So you have people understand what the tool can do and more powerful tools in the form of pre trained models. And I think that's one thing that we're kind of leaning into as well, even with our user base, is that we're not only considering data scientists, machine learning engineers as our user base, but are you an engineer who knows what ML can do for you? Because if you are, we hook up all those pre trained models and give you a model zoo. So if you want to have a zero shot classifier, you don't really have to think about anything. You can have it embedded in a workflow in an application with base 10 as long as you know how to frame the problem. And so I've tied it back to ourselves, but from a high level, they're the two things that I'm really, really excited about.

That's awesome. Yeah, those are really exciting. It is really cool to see most conversations now that I'm having with companies you don't have to sort of get over the hurdle of like we expect AI ML to give us some value. It's more like entering into the follow up discussion to that, which is where is there value? Where should we focus and that sort of thing. And so technical teams are really thinking about this a lot, which is cool. So thank you so much Tuhin for joining us. This is really exciting. Congratulations on your launch. It's awesome and we really hope that our listeners please check out what base 10 is doing. We'll have links in our show notes so check them out. Talk to us about some of your opinions about this in our Slack channel. You can always find us in one of our communities on LinkedIn or go to changelog.com community join our Slack channel and happy to chat with you about base 10 and other things. So yeah, thank you so much Tuhain and we'll let you get back to work.

Thank you guys so much. You guys are really nice and it was really nice being on the show.

Thank you for listening to Practical AI. We appreciate your time and your attention. If you enjoyed this episode, help us out by spreading the word. Think of a friend, think of a colleague, somebody who would benefit from listening to it and send them a link. We'd really appreciate it. Practical AI is hosted by Chris Benson and Daniel Whitenack. It's produced by Jared Santo with music by Brakemaster Cylinder. Thanks again to our sponsors Fastly Linode and launchdarkly that's our show. We hope you enjoyed it, and we'll talk to you again next week.

Sam.