1.8% Engagement Lift Across 30M+ Users, Fueling MAU Growth w/ Kumo Models | Enginears Podcast

Ladies and gents, welcome back to another Engineers Podcast. Today we're joined by Viman Deb, who is founding engineer and head of product engineering at Kumo AI. Kumo uses artificial intelligence, AI, to accelerate the creation and performance of machine learning models. And we're bringing Viman on who's going to tell us all about the how they're doing it. And they've got a pretty star-studded founding team. So I'm looking forward to getting into some of the details around what these guys and girls have been building over the last 3 years. So first and foremost, Viman, thanks for coming to join us. How are you?

Good. Thank you for having me on your show, Elliot.

Absolute pleasure. Absolute pleasure. We always start with an intro into your background. So that's just as star-studded as your founding team. Talk to us a little bit about your background, where you've come from, from an engineering perspective.

Yeah, I would love to briefly go over my journey. I started in the Bay Area in 2012 at Google. I was working on Google Wallet and its integration with GAIA, which is Google's centralized authentication and authorization service. In 2014, I moved to LinkedIn where I spent about half a decade working on a wide range of roles and projects. So I started in the team responsible for companies' data and company pages, and the CEO of LinkedIn back then, Jeff Weiner, had this amazing vision of creating an economic graph consisting of LinkedIn members, jobs, skills, companies, which would be interconnected and data would flow beautifully, allowing us to build very rich applications. One challenge was that we didn't have enough companies' data. So we went ahead and we found a lot of open datasets, company registries, and so on and so forth, about 60 such different datasets. But then there would be a lot of duplication across them. So that's when I got my first exposure to machine learning. We built an unsupervised clustering model which didn't work that great. Then we built kind of a union finding algorithm with, you know, each set denoting a single company. And then the question whether a company belongs to a set or not was a binary classification problem, which we built a random forest classifier for. So I was completely blown away by this technology because there was no way we could handwrite a solution to that problem. So just let it learn by data and automatically build a model which spits out the results and outcome we are looking for was completely amazing for me. And I, you know, I could see it very clearly that this would change the world. And, you know, the rest was history. The next 10 years is, you know, it's come— AI has completely taken over the world. So yeah, I was very fortunate to get early access and exposure to build a practical application using this. We were very successful. We went from 3.5 million organic company pages created by LinkedIn members to over 20 million, and we had a much richer economic graph. So I moved to hiring platform after that. I worked in creating LinkedIn's first ATS, which is applicant tracking system, for small and medium businesses. I spent some time there. I also played a number of leadership roles at LinkedIn. I was part of the data model review committee, which responsible for approving new APIs at LinkedIn. I was also the chairperson for the RFC committee for LinkedIn Talent, Talent Org, which is based on more than 300 engineers. Then I worked briefly on the jobs recommendations model at LinkedIn, which is, you know, again, like very data science, machine learning heavy. From there I moved to Uber, and at Uber again, I was very fortunate to be part of the Uber AI Org, which is the central org for helping building foundations of AI and machine learning models for a variety of our stakeholders. The beauty of being in the Uber AI org is that you get to work on projects which is kind of impacting pretty much all of Uber. So I started working on the black box optimization platform, which was used by pretty much, you know, every team to hyperparameter tune their models. And Uber models in 2018 were largely tree-based. There was still a movement to move them to deep learning, and Typically for tree-based models, tuning has a lot more significance than for deep learning models. So I had a great time there. I worked on embeddings platform after that at Uber AI, and one of the notable models you would see where my team contributed significantly is the Uber Eats home feed ranking. So if you open up your Uber Eats app and you see a bunch of restaurants, that recommendation is coming from the model which my team was building. So all of that led me towards Kumo AI, which is basically a platform which we allow customers to build machine learning models on. And that's how I started. I started as a founding engineer. It was a small startup. I think I was the 9th hire, but we are growing so fast that I joined 3 months after signing the offer, and I was already the 16th hire from signing to joining. And then, yeah, so since then we've been growing phenomenally well. I think we've amassed some amazing customers. We have solved some really, really powerful real-world use cases. And yeah, that's, that's where I've been for the last 3 years.

Love that. LinkedIn and Uber, two products and a number of different teams that I know really well. Two names that are associated with doing some incredible engineering stuff at some serious scale. So good for you. No doubt a really good learning ground. And obviously you're taking that into Kumo. Do you want to give us a bit of insight into and almost like the elevator pitch of Kumo AI and help us understand more about what the business are doing?

Sure.

Yeah.

So Kumo is an enterprise AI company. The net enterprise is built machine learning models on top of relational data. Um, the beauty of Kumo is that the traditional data science and machine learning workflow is not required anymore. So no feature engineering, no training table generation, no comprehensive data transformation aggregation pipelines, no training infra, no serving infra. So all of this abstracted behind two lines of a simple query. Which you run directly on your relational database. And this query is written in a DSL which we have created. It's called the Predictive Query Language. It's very similar to SQL, except that with SQL you can only query the past, but with PQL you can also query the future. So think about writing a SQL query with dates into the future, and we train a model underneath and deliver you those predictions. So it's, I think it's phenomenal. And initially when we started, our goal was to make machine learning easy. But the underlying technology we use, it also allowed us to not just make it easy but also build really, really high-quality models. And, you know, we talk about that briefly. So that is kind of very unexpected because you can either build a product which is very easy to use and delivers reasonable and moderate value, or you can build a product which delivers very high technical value but is very hard to use and it requires advanced experts to navigate their way through it. But we've managed to kind of do both together. So we've had customers where they bring in their models which they've been working on for many quarters, and they bring their data to Kumo and train their first model. And within like the first 2 days, they have a model which performs better than their existing models. So it's not just a productivity tool, it's actually a tool which just lets you build models which are like really, really accurate and state-of-the-art. Right, so that's where we are. Our goal is to make machine learning easier than analytics, you know, and this is like really a revolution in predictive AI which we are chasing, and I'm confident we'll deliver on it. You know, we have a great founding team and we have some amazing talent here, so we are very well strategically placed to deliver on this, and I think we're on a great path.

Yeah, good. That's the thing that I love, and I can hear that loud and clear— the creation, the speed, the performance of machine learning models, removing Removing the abstraction, you touch on that note, is really important to help teams come up to speed. I think it would be really good to dive into a couple of use cases, because beautifully put on your website, gives you a really good indication as to some of the great work that you've done. Help us understand a little bit more about the use cases for teams and businesses and where they can actually start using Kumar?

Sure. So, you know, the beauty of the platform is that we are not tied to a specific kind of machine learning model. So we have, you know, customers who are large social networks, we have customers who are into grocery and, you know, food delivery, we have customers who are large-scale hospitals, and then we have customers who are large fintech banks, right? So as long as you have data represented in relational form, which is pretty much, you know, the entirety of the data in the internet, right, largely, uh, you can basically build any model by writing a specific query. Uh, now, uh, where we have really shined in the past is actually these kind of 4 categories, which is recommendation models. So think about you go on a website and then that website is trying to sell you certain products., and they have a large inventory of 2,000 SKUs, and they have to determine what are the top 10 items they should be showing you, so which maximizes the probability of a purchase. So we really shine for those kind of use cases. We are doing exceedingly well for fraud use cases. So if you have a large history of customers and then some of the customers have been fraudulent, our model can do really, really well. To predict which of your customers would be fraudulent. It's a huge financial impact for fintech institutions. Those are definitely two really, really high-power use cases. We have been into advertising use cases where customers have taken our embeddings for their entities and tried to optimize their ROAS, which is return on ad spend. That has been working great. So, you know, like I said, we are not bounded by specific areas. We did in the very early days when we're trying to find the product-market fit, had to build a couple of foundational models. But, you know, the good thing about our machine learning engine is that it doesn't need to really be specifically tuned towards a use case. It's very, very generic but still builds very specific and high-quality machine learning models.

Okay, can you actually help us understand by breaking this down for us how AI can actually accelerate the performance of machine learning models.

Sure, yeah. So let's jump right into, you know, kind of understanding how Kumo does this, right? So when we say we are using AI to build machine learning models, the AI is really a graph neural net. And the benefits of graph neural nets is that you don't need to perform any feature engineering automatically. Right, so let's try to learn how that happens. So, uh, traditionally when I was working at Uber or LinkedIn, you know, like, we hired a lot of very high-quality data scientists. A lot of their, uh, job in work— working on building machine learning models would basically be feature engineering, you know, where they end up with this training table where you have a large number of features and a label, right? And these features would be like, let's say if you're building an Uber Eats home feed ranking model Then there'll be features like, you know, number of orders you made in the last 60 days, number of Italian orders you made in the last 20 days, number of, you know, orders you made on a Tuesday. It's, it's pretty arbitrary. And I think like good data scientists use intuition to understand what kind of signal to the model could help improve the performance, and then they implement that signal, and then they would try to test that model offline, see if they get a performance boost. If they do, they'll basically ramp up an experiment to launch that experiment online. It's a very long cycle to try to find the best set of features. And a lot of the machine learning teams, this is what they're doing essentially, trying to find the next set of data representation which can improve the performance of their models. Now, the way Graph Neural Nets works is it works directly on top of your relational data. So you don't need to perform those aggregations, those transformations, that single feature table. As part of training the model, the Graph Neural Net will actually go through your entire graph and try to retrieve data which best represents your model. So as part of training, backproping from the loss function, we sample your graph and we try to see what data is causing the loss function to improve. And as part of this aggregation, we use Transformers, which is the same technology which GPT uses for language models, but we're using this for relational data. And it automatically learns those aggregations and in a nonlinear way, which is actually even better than writing features like 60— number of orders in the last 60 days and so on and so forth. So the end result is that as you're training the model, the model is automatically performing feature engineering on your entire topology of the graph. So it learns from the data, it learns from the number of edges, topology, the entire graph properties. And that's why sometimes when you're doing manual feature engineering, you're obviously limited by the imagination of the data scientist. And, you know, we have some great data scientists in Silicon Valley, and they, they've learned the art and they can do this really well. But with Kumo, you basically find the best set of features on day one when you train your model. So you don't have to go through those iterations quarter after quarter to, you know, try and find your best model.. So that's really where I think we are able to differentiate ourselves as a machine learning platform. And what we have done is we have built a bunch of peripheral platforms to support the GNN training and the PQL abstraction I spoke about earlier, which makes using Graph Neural Nets very, very easy. So it's not easy to train a large-scale Graph Neural Net model. So many of our customers actually use us for simply Graph Machine Learning as a service, even though our value prop is just making machine learning so easy. Because Graph Neural Nets, they work phenomenally well on relational data. That's how it uncovers how we are able to build these great models. Another analogy I really like to give is if you think about what happened in the computer vision era, we initially had people build in these small features, like, you know, trying to write algorithms to find 3 connected pixels which denotes a line and small circles. And then they would write more features on top of that to try and connect those things to find a bigger shape, and they'll classify images. And then we had convolutional neural nets, which basically would slide a window through the image and automatically perform the convolution operation and find those small shapes. And then, you know, in increase the window size and then find bigger shapes. And we had massive performance boost on model performance for image classification. So we are kind of doing the same thing for relational data. So we slide a window over your relational data and, uh, you know, we automatically find the best set of features to train your model. So that's, that's where really, uh, you know, uh, that's how we really distinguish and build like really, really high, uh, state-of-the-art models.

Love it. Has the mission of the business deviated at all as you found product-market fit and you found how you've been able to provide so much value back to your customers, probably as you've continued to build the product? Has the mission deviated at all? Was it really a crystal clear vision of, actually, we believe that we're a better case and companies should eliminate data science, do you feel, in your journey there?

Yeah, so I would say the mission hasn't deviated. Um, the first product-market fit we had, we have basically been there, fortunately. And this is surprising because I work with a lot of startups as an advisor or investor, and, you know, typically startups see 3 different companies in 2 years, right? So we've been here for 3 years, and we've kind of like stuck with the same mission because it's so powerful. And if you deliver on it, I think it's extremely high value. Uh, the TAM for predictive AI is really, really large. It's actually significantly higher than GenAI at the moment, right? So, uh, the mission is still the same. Uh, we have obviously as a startup pivoted, uh, based on customer feedback, and sometimes they tell us, hey, we need some more control over what we are doing. So for example, we had a UI product in the past, so you could connect your data in S3 Parquet or Snowflake or Redshift or Databricks, BigQuery, Azure, and then you could create your graph and then you could write predictive queries. And then they told us that, hey, we need some more control. So we started to offer model planner settings, which basically allows you to customize some of the aspects of the platform. Then our customers told us, hey, we would love to do a transformative EDA. So you want to train the model, get the results and we want to go back. So for some of our advanced users who are, who understand GNNs and they want to, you know, play around with our platform more and get more access and control, so we build the SDK, which is basically, you know, it's a very simple and intuitive programmatic access to Kumo. So you can connect your data again, build the graph, train your models, look at the data in the same notebook, and go back and, you know, start from cell one making the required changes you want to. So I would say most of our effort has been towards, you know, just catering to our consumers, to provide more different aspects of access to Kumo. It's really not changing the core of the product. You did mention one thing which I do want to, you know, definitely address. So we are not eliminating data scientists. I think what we are really doing is making them very powerful, right? So as a data scientist, if you're building one model in a quarter, now you can build 5 models. You know, like many companies, you know, maybe Barring Google, Meta, and some of those big tech giants, they don't have budget to hire like 5 data scientists for each of their use cases, right? But they do want machine learning to power the bulk of their website, right? So that's what we are really going after. So you have a lean team of machine learning or data scientists, machine engineer, data scientist. We can basically empower them in a way where they can create significantly more number of models and more accurate models. Than what they would have done using the traditional way for predictive machine learning. Right, so that's, that's basically what our real pitch is. And yeah, I think again, like, we will keep listening to our customers. I think that's what startups need to do and iterate on that. But the core engine has been constant.

Yeah, that's actually a very smart frame and smart business play, you know, building an offering where you can actually get 5, in essence, really talented people for the value of 1. Help me understand a bit more about— you spoke about core engine, and not necessarily even Kumo's core engine, and we're trying to understand some of those product and engineering challenges. I'm really trying to understand and would love to understand what engineering looks like at Kumo, because we're talking about some really interesting challenges that you're solving for businesses. When you get under the hood and understand Kumo, what are we likely to see, and what problems are you guys solving on a day-to-day basis?

Sure, yeah. So let's break down the architecture a little bit, right? And I would also like to tell you about the different deployment modalities, which will kind of answer your question on if you break down Kumo, what do you see underneath, right? Where are the technical challenges? So we start with customers' data. So we have a data platform which basically will help us bring in your data and seamlessly transition it into a taxonomy or structure which Kumo would understand better. Like any products which works with external customer data, so data is in a variety of shapes and forms. It can be sometimes very dirty, sometimes in a weird structure which you have never seen. Every company which is working with customers' data has to go through that journey where we have to iteratively process one block at a time. And just kind of get a very comprehensive coverage of what are the different datasets you can probably encounter. So many of our initial design partners and even initial customers, we had this journey where we would provide wide support to them to understand what are their data requirements. So we have a very comprehensive data platform which is built in a very versatile way now to process even data situations which we haven't encountered in the past. And a lot of this has been our learnings just working with customers. So that's the first component. The second component is we need to convert it to a graph which Kumo can understand and train a graph neural network model on. So we basically perform data materialization, which also happens on the same data platform. And it basically like actually converts to certain extent encode features and, you know, kind of position us for training. The third component is basically the training infra, which is responsible for actually training the models. So we have a feature store, which is basically kind of optimized towards providing features for a sampled graph, which we use when we generate mini-batches for training. And then this feature store works with our trainer module to allow easily and in a very fast manner subgraph sampling and providing the data to train the model. Then we have our standard training infra. We run on one GPU. We don't have a multi-GPU situation right now. Practically, we found that even customers who have terabytes of data do not yet need multi-GPU. We are pretty fast with training in a single GPU, so we haven't had to complicate our architecture further there. Then the actual model is, uh, you— is basically built by a research team which is responsible for building some of these foundational models. And like I mentioned earlier, we actually don't have, uh, you know, one model per use case or one model per type of use case. We basically have maybe a couple of different variants, and that's about it. So that team is constantly working on, uh, you know, building this like core engine, which is the GNN module, and which, you know, is adaptable to different kinds of use cases. So the training infra obviously uses those models to train the models, right? And then when you're making batch predictions, we basically, you know, would generate your model, and then we have similar kind of structure to, you know, go through the serving infra, which is again on PyTorch. And on top of all of this is basically what we have, what we call the predictive query language. So we can basically define your query. We have a grammar, and then we parse that query and we convert that into understanding the requirements for your problem. And using that, we generate the training table using the same data platform. So that's the core application. Now, uh, we are deployed on AWS for our SaaS stack. So if you basically, you know, come and ask Kumo to provision you an environment, you can go to our website, ask for a free trial, we provision you by default to AWS. But we had many customers who would come in and they have specific data governance requirements. So they don't want to leave the data. Let's say their data is in Snowflake. They don't want their data to leave Snowflake. And they want it to operate within the boundaries of the Snowflake data center. So we have a native app offering inside the Snowflake Marketplace. So think about the way you would go to App Store, install an app, and start using it. Similarly, you can go to Snowflake Marketplace, enable Kumo, and start using it right away. We have a free trial for about 30 days and your data never leaves Snowflake. And you can also pay us using your Snowflake credits. So you don't have to go through the cycle of getting approval for a separate application. We have similar integrations with Databricks. You can basically push down compute onto Databricks and run your application primarily in Databricks. So because of all of those deployment modalities, the way we have architected the system is basically it's not tightly coupled to a single implementation. So Databricks and our SaaS offering, which runs on AWS, uses EMR, which is basically Spark-powered. So Databricks and our SaaS offering is very similar, but Snowflake uses a different implementation for data processing. They use Snowpark services. So our implementation is abstracted out from our interface. So we have a same interface but a different implementation. For Snowflake, and still the entire end-to-end system works seamlessly. We are also working— recently we signed a deal with a customer who's bring your own compute, so we run entirely inside their own VPC, which is also, I think, I feel, I believe they're also deployed in AWS, but they don't use our AWS stack, right? So a lot of the engineering has been focused on providing those kind of multimodal deployments across, you know, different different data planes, different containerized services. So all of this combined together with this multifaceted architecture where we have like the core architecture which I described earlier and then deployed into these various modalities, right? So that, that's pretty much, you know, how comprehensive the whole technology is.

Love it. Okay, and obviously everything in the AI GPU space is quite volatile at the moment. Does that impact or affect your business at all?

Yeah, that's, uh, that's a very interesting question. So, um, you know, like I said, we don't have to use multi-GPUs, so GNNs are not as, uh, data hungry as LLMs. So, uh, we can actually learn very, very quickly and build a very good model. So we mostly use single GPUs. Uh, we haven't had an issue because of shortage of GPUs so far. Now, you know, this landscape is kind of a demand and supply marketplace, and obviously large language models, you know, there are a lot of companies and very powerful companies being built around them right now. So the GPU landscape is kind of unpredictable, but so far I would say that we don't have such a strong need for large number of GPUs to train models. So we have kind of not been affected by it so far. Um, that's in a nutshell, that's where— that's the scenario.

And, and for what you can share, where are Kumo placing bets on what space you'll navigate towards in the next 6 to 12 months? As in, like, problems you anticipate you'll be solving in that period of time as a business, where are you placing those bets at the moment, for what you can share?

Sure. So, um, one challenge with Kumul is, um, you know, so our product is very, very easy to use, but if you think about testing a PoE, which is a standard process in selling, uh, you know, SaaS product, there is a minimum amount of work you have to do to really establish proof of value. And for a solution like ours, sometimes it is not that easy because You have to build a model. Many of our customers would— I mean, we would show them great improvements in offline testing on their previous historic data, but they would want to deploy the model and kind of A/B test and establish clearly that, hey, if you deploy this model, then we are going to get great performance. So that has been one of the reasons why our sales cycles are longer than some of the other typical SaaS companies which have simpler products to deploy. So obviously there is like a lot of effort towards, you know, helping build tools which can fasten that process. We are gonna invest in online serving. So Kumo has been an offline batch prediction company right now, but we are going to move to, you know, allowing online serving for some of our customers like fraud, where they have an incoming request and they want to resolve quickly. They don't want offline predictions because by then the fraud server would have already committed damage. So we are working on that. I think it's a very interesting piece of engineering challenge for us, and especially for GNNs where really you have to, during serving, you don't have your kind of flattened feature store. You have to basically still sample from the graph, and we still want to provide reasonable latency for a reasonably high QPS. So that is another area of investment. We will continue to invest towards being more flexible with our deployment modalities. Although I think we are already in a space where more than 80% of the time, if you bring in a requirement for how you want to deploy Kumo, we are going to be able to surface that. Then we are also investing in agents to allow Kumo to be able to easily used by individuals who don't even want to learn predictive query language. So you can come and connect your data and tell us what you want to do and we'll generate your notebook or even train a model. So those are some of the high-level strategic areas we are investing in. And yeah, like I mentioned earlier, we'll basically play it by the ear. We keep listening to our customers and based on their feedback, we'll keep iterating and improving.

Agents and their architectures and their workflows, everyone's talking about that at the moment. I think that's a really interesting space, and actually it's quite, it's quite nice to hear the unique way in which Kumo will try and use it. I'll definitely see in 6 to 12 months where you guys and girls are up to. Help me understand where you might be building the team over the next 6 to 12 months for people listening that are really interested in your mission, really interested in solving some of the problems that you're solving. Where will you be growing and with what types of people?

Yeah, so we have our head office in Mountain View in Silicon Valley, or now called the San Francisco Bay Area. We are setting up a short office, a small office in Texas as well. We have some presence in the East Coast in the United States. And then, you know, we have like several of our, you know, amazing engineers in Europe as well. So we have some in UK, then we have, I think, a couple of really like bright engineers in Germany. So we are not limited by, uh, kind of geography. So we are pretty much, you know, if you're good, then we'll, you know, the mission aligns with you, then we can basically make it work, you know, together. So, uh, we also had an engineer in India in the past, and we brought the engineer, uh, to Mountain View recently. Uh, we still make one— part of my team still works in India, uh, who are building the UI. So we are not limited by geography in any way. So, you know, if, uh, there's a good fit, then we can make it work. Uh, if you want to move to the US, I think that's a, uh, we can always make that happen. Uh, it's obviously subjected to the visa situation. Um, but if we can, you know, we have a very strong immigration team which can basically help you you know, get your way across. In terms of alignment, I think, you know, if you are passionate about machine learning, if you think enabling this or using this technology will really change the world, that's what we need. And of course, you know, we have a standard interview process. We'll make sure that your and Kumo's goals are aligned, and that's it. I mean, I don't think we have specific ask on, you needing to know certain technologies. We are mostly hiring like really good engineers who are problem solvers. So yeah, that, that's what mostly we look for.

Love it. Good. I think for people listening, if some of what Viman's spoken about today aligns with you, uh, you are a good engineer, you've solved some really interesting problems, go and check the guys and girls out on their website, which is below, or ping Viman. On LinkedIn. Sorry, Viman Deb. And get in touch with him to understand a little bit more about where they'll be hiring. But I hear the message loud and clear, which is something that you really wanted to drive home to people listening, that Kumo is this new amazing way of building machine learning models. And I think you've done a really good job of articulating how the business actually enables this. So I want to say a massive thanks for coming to join us on the podcast and really breaking down Kumo and where the business will be going.

Yeah, thank you, Elliot. I think great conversation. Would love to share more details to anyone who's curious on— hit me up on LinkedIn. I'd be happy to connect and answer questions.

Yeah, absolutely. There's some really good content below which I'm going to share in the description for people to look at, which helps understand some more of the use cases and talks about GNNs and even more of some of what Viman Deb's been talking to us about. So again, a massive thanks from everyone at Engineers, and of course the likes, shares, the subscribership, it goes a really long way to continue to contribute to this ecosystem. So a massive thank you. And Viman Deb, again, massive thanks for coming to join us.

Thank you, Elliot.

Hey guys, thanks for watching this episode. Massively appreciate you listening and checking in with us. If you want to find out more about us and what we're doing, please check us out on social media. What we're trying to do at Engineers.io is build a community to drive knowledge sharing and experiences. On Twitter, we can be found @Engineers_io. It's no underscore. We've also got a website which is engineers.io. These links will all be posted in the description. Any feedback and comments are massively appreciated. We're always looking to improve on where we can. Thanks, guys.