#313 Developing Better Predictive Models with Graph Transformers with Jure Leskovec, Pioneer of Graph Transformers, Professor at Stanford

Generative AI is transforming industries at an unprecedented pace. But as AI changes how you or your team work, one thing is clear. Your skills also need to evolve. At datacamp, we offer everything you or your team need to adapt and thrive with AI. Whether it's business users looking to get the Most out of ChatGPT and Copilot, or developers and data scientists looking to fine tune models. You can learn the entire AI skills spectrum on Datacamp. Power your AI transformation today. Start learning@datacamp.com. What we have seen is that the best way to make decisions is through making them via predictions, via forecast. And then based on that forecast, you make a decision, right? So I think going on the demand for predictions, demand for forecasts, demand for decisions is only going to increase power of data. Private data will become even more important. Right? Because if everyone is using the same pre trained model, then the only way we make differences among ourselves is by the data we have.

Welcome to Data Framed. This is Richie. For the last few years, Transformer based large language models have been the biggest story in AI. They're great for generating text, but unfortunately they're not so useful for machine learning, AKA predictive AI. What predictive AI needs is a tool with power and ease of use that we see with generative AI chatbots. Today's guest claims to have built such a tool. He's been building a new type of foundation model for relational data based on Graph Transformers. Now, the word game changing gets thrown around far too lightly these days, so I'm going to avoid that. But I will say that Graph Transformers are perhaps the most exciting technology for machine learning that I've seen all year. Our guest is Yora Leskovets. He's the chief scientist at Kumo AI and a professor at Stanford University. He spent seven years researching graph neural networks while working as chief scientist at Pinterest. And his innovations were a major factor in turning around the fortunes of that social network. Following that, he founded Kumo to commercialize his graph Transformer innovations. Let's find out about Graph Transformers and find out how you can make use of them. Hi Jorah, welcome to the show.

Yeah, hi, glad to be here. Cool.

So there are a lot of great foundation models already. So can you just tell me why are you building another one?

That's a great question. Definitely. There are foundation models like the large language models. We also have protein language models and DNA language models, kind of on the biology and a bunch of things on the computer vision. But what we are fundamentally missing is a foundation model for enterprise data, for Business data and that type of data. There are two differences when you think about how a foundation model for enterprise data would look like. The first thing is that enterprise data is usually organized in this kind of structured, semi structured way of having different data in different tables. And then we have kind of pointers, primary foreign key relations between these different tables. So that's, I would say the first big difference is that this is not now a sequence anymore, a sequence of tokens, but it's a much more complex structure. And, and we'll get there during the podcast, I assume. And then I think the second difference is about what kind of tasks you want to be performing on top of this type of data. And in enterprise data, the key task to perform is decision making is right. We are collecting this data to be able to make decisions. Now, if you say, what is a decision? Decision is something that affects the future. So the best way in some sense to make an optimal decision is to anticipate, predict, forecast the effect of the decision today on the future. So you need to be able to make what we call predictions. Right. So that's why you need another foundation model, because this is different type of data and different types of tasks than, let's say that large language models that have been amazing breakthrough in technology are able to do. Right. So this is different. LLMs cannot do these things. Okay.

So I think the idea of predictions are going to be very familiar to anyone who's done machine learning before. So there's lots of different kind of models for machine learning, including neural networks, but also to four vector machines and random forests and all these kind of things. So talk me through, what's your new model type and how is it different from these traditional machine learning approaches?

That's a great question. Right. And maybe another way to say is to say the following. Right? And I'll get to your answer to the question is the AI revolution has kind of taken us by storm. Right. Computer vision is never going to be the same because of convolutional neural networks like learning directly on top of pixels. Right. Large language models, you know, natural language processing will never be the same because now we learn directly from tokens and kind of all kinds of tasks and abilities emerge. Right. But if you think about predictive modeling, about machine learning, that hasn't really been touched by the AI revolution, that still kind of stuck 30 years back. Right. Like decision trees, I think were invented in 1980s kind of, or even earlier, before I was born. Right. Perceptrons, you know. Sure. Even earlier and so on Right. So the machine learning, the predictive part hasn't really been touched with the latest wave of AI. And what I'm excited about is to bring kind of the AI revolution to machine learning. That's why we talked about, okay, why we need another foundation model. Because we need a foundation model for machine learning. Right. That's the exciting thing. And then how is a foundation model for machine learning different from, let's say the classical ML algorithms that you talked about is two separate differences. First difference is that it can learn directly on your raw data so you don't have to do manual feature engineering. And the second difference is that it comes pre trained so you can just point it to the data, specify the task and you get a prediction. So you don't even have to train the model for a specific task.

Okay, so no training, no feature engineering. This all sounds like, I guess a dream come true for the machine learning scientists. So I'd love to get into the technical details of like how the workflow changes and things like that later. But you talked about some of the enterprise and business use cases. You just make it concrete. Just give some examples of what these new graph transform models are suitable for.

I would say any kind of predictive task and then you can, you know, put a different kind of, when you train a different loss function on top of them. But this would be anything from classic forecasting models to fraud models to customer behavior models about lifetime value churn, any kind of risk models around. I know in, let's say in financial insurance industry, what kind of risks are going to happen, Sales lead scoring, next best action, personalization, recommender systems. We see them used in healthcare to predict risk of readmission. So kind of all this kind of classical, classical, let's call them machine learning predictive problems from, you know, predicting a binary value, which is classification, multiclass, multilabel things like that, any kind of regression, time series, temporal type problems and then any kind of set prediction ranking problems for personalization, search, result ranking and so on and so forth.

Okay, so this really is like the full suite of like anything you can do with sort of traditional machine learning. It's like you can then do with.

These new models pretty much at kind of, I would say at the first order of resolution in some sense. Definitely at the second order there might be some nuances.

Okay, all right, nice. Can you explain to me what the business benefits are that you might be expecting? Is it just about being able to create models quicker because you don't have to do feature engineering or is it about you going to get better accuracy? What should be looking for to see what are the improvements?

Yes, good question. And I will just acknowledge that here I'm kind of promising a lot and the audience should be almost like, hey, what's going on? This guy sounds crazy, right? So if people are listening carefully, they should be skeptical, right? What are the benefits we see? I would say there is maybe two nuances to separate out. One is the benefits of the pre trained model. The benefit of the pre trained model that doesn't even have to be pre trained on, let's say customer specific data or user specific data is that you are able to get a prediction for any kind of predictive problem in less than one second. Right. So this means that you don't have to spend time feature engineering, creating training data sets and all that. Right? So you go from months, weeks of work to 1/2 worth of work, connect to the data, make a prediction and then if you now decide to say okay, actually, but I really care about. And in this case with the pre trained model, the accuracy is comparable to what senior human engineers, data scientists, are able to build manually. So you will get a human level accuracy model with accuracy prediction with a pre trained model. Now if you want to push that further, you can then fine tune the model on your data for your task. And if you do that, then you get another 10, 20, 30% more accurate. And that means basically you get into the superhuman regime of accuracy. And that's very, very valuable. You know, before we were asking about use cases, right? If you are doing fraud detection, direct business impact, if you are doing personalization recommendation, imagine your direct business impact. If you are doing ranking of products, direct impact. If you are doing any kind of ad matching like user ad matching, we see great results where you know, people are, or engineers are able to get kind of years worth of improvement in a couple of weeks.

Okay, yeah. So certainly those are some bold claims of getting superhuman levels of performance and being able to answer questions within like a couple of seconds. Do you have any examples from real customers who've been solving real problems just to back this up.

So for example, where, where we've seen great results is on, on ad matching. For example, user ad matching. So predicting what ad will be relevant for what user. We have been working with Reddit, which is, you know, humongous large social media platform and I would say that in a matter of a couple of months they were able to make about three years worth of modeling progress. Right. So what do I mean by that is they, they have an Amazing team. They've been building this ad click prediction model for, you know, for, I know, several, several years. It's kind of the flagship type of model and you know, every year they're able to increase the performance of that model for, for some delta for a couple of percentage points, right. By basically taking this technology and training the models using this new technology, they were able to kind of push forward for about three years worth of progress in a matter of weeks. We've seen similarly amazing results on fraud detection use cases, on let's say sales lead scoring use cases. This one is public as well. So actually in collaboration with the databricks sales team, they are trying to predict which of the sales leads is actually going to convert. I can actually say their definition of conversion is will I be able to meet with that potential customer? Right. Will I be able to schedule a meeting and meet in the next one month? And if you think about as a data scientist, right, it's like what features predict whether you and I will be able to agree on a time to meet and you know, you and I are going to show up. It's kind of unclear, right? As a human data scientist, you're like what features should I generate? Right. So the benefit of this approach is that it can go all the way down to the data and basically discover those signals that turn out to be very predictive for whether you and I are going to be able to meet in the next 50 days, right. So that was a really good use case and yeah, I probably think of more as we go along.

That's impressive stuff and especially like working with databricks. All the people I've worked with at databricks, very smart. So like these are top quality data scientists and if they're wanting, like if those humans can't figure out the features, then having the machines do it better, that that's a pretty impressive benchmark.

I guess I can explain, right? I would say maybe, like I'm not saying when we go and discuss maybe a bit more about the technology then then it would be clear how this is different. You ask me about how does it change the workflow, right? The data scientist is needed and data science jobs are not going away, right. I think where the difference is that the goal of a data scientist is to really kind of model the data and have the business impact and you don't have to be bogged in, in data cleaning, feature engineering and creating training data sets to get to your impact. Right. With this new graph transformer based technologies and platforms, you as a Data scientist can get to that goal of business impact to having a performant model much, much faster. And you can get there faster with better accuracy. So the. But the human kind of the data scientist, modeling the data, thinking about how to take a business problem and translate it into a predictive problem, what data is needed, how is the problem formulated, I think is crucial. And today data scientists are maybe spending 5% of their time doing that, and the rest is kind of doing groundwork. So that, I think is the way the world changes is being able to learn over more data with a higher fidelity kind of much faster. Right. So that models can be built in a matter of hours or days rather than weeks and months.

Okay. Certainly like having the speed up seems pretty incredible. So, yeah, you mentioned that it would be helpful to understand the technology a bit. So I want to get you to explain what a graph transform is. Maybe do you want to build it up? Let's start with just like what are graphs, networks and why they're useful.

Yes, yes, great, great point. Right. So generally when we think of a graph or a network, the first association we get, it's a social network, right? Like we have people and they are connected by certain type of edge. You and I have sent email to each other and you and I are now talking. I know, maybe we are connected on LinkedIn and things like that, right? So you have this set of notes and then relationships between them. And this graph learning technology has been super helpful and useful in this to model this kind of interconnected data. One type of interconnected data that we all have, but generally kind of don't think of it that way, is a database, right? Enterprise data, as it results in multiple tables, is also a graph, right? You could have my customer table, I would have my products table. And then I have let's say transactions table, right? So a user through transactions buys a product. That's kind of the simple three table schema. But the schema could also have another table that let's say is website clicks. So a user can click on a product, a user can purchase a product, right? And now I have this let's say, schema of four tables. That is a graph, right? So I can take that and represent that as what we call a temporal heterogeneous graph, right? Every user is a node, every product is a node, every click is a node, and every purchase is a node. So now a user is connected to the click that is connected to the product. And of course the click has a timestamp, right? And every user is also connected to the purchase that also has a timestamp and a bunch of other attributes back to the product, right? So now you have this heterogeneous, because you have different node types. You have the user node type, the click node type, the purchase note type, and the product note type, right? And these things are interconnected. But now what is the benefit is in traditional machine learning, what we are doing is we are joining these tables. I would take the user table and the transactions table. I join it, and then I say, and then I need to aggregate over the user. So I say, how many transactions did they have? How many in the morning, how many in the afternoon, how many over the last five days, six days, seven days, on holidays, on odd number days, even number days, it can go forever, right? But the idea is that rather than needing to join tables manually, they are now interconnected through this schema, through these primary foreign key relations. But now, when you, when you train or when you build a graph transformer on top of this, a graph transformer is a transformer architecture that we all know how it learns over a set of tokens. Transformer architecture, we say it learns over a sequence of tokens, but that's false, right? What does it learn over is a multiset of tokens, right? So basically it's tokens, and there is a multiset, meaning each token can appear multiple times inside that context window. All right? So it's learning over a set. But then how do we know what the sequence is is? Because we have positional encodings, and for every token we describe in that set, what is its position in the sequence? Right? So now what is a graph transformer? A graph transformer is a generalization of that where you can learn over an interconnected graph. And the way you learn over that is you say, okay, I have now a set or a multiset of nodes, but then my positional encodings are much more complex because they tell me where in the graph each node is. And of course, building, building the infrastructure that allows you to do this is like, hard because text is linear and it's easy kind of to pump it through. But graphs you cannot shard. It's harder to work with. So that's what's called a graph transformer. So now what did we say so far? We said the database is a graph. And now that I have a graphical representation of a database, I can apply a graph transformer on top of it. And then you could say, so why should I care? Why does it matter? There are two reasons why this matters. First is that now you basically can apply in some sense deep learning. You can apply deep neural networks to learn directly on your raw database representation. Right? So the same way as a convolutional neural network learns over the pixels. Now this graph transformer, or even like a graph neural network, which is a bit different type of models, but for people who are familiar with those, they can learn directly on top of your of your raw data. So you don't have to do feature engineering, you don't have to be joining tables manually anymore because I can now learn over a set of interconnected tables. So the process of building the model or tuning the model is much faster. And then the second benefit is that kind of the fidelity of this attention mechanism over the graph is so much more fine grained than what we humans are able to come up with with our joins and aggregations that generally it leads to a higher accuracy. Right. Because it's kind of attend. Rather than saying let's take the customer table and transactions table and join, you are now basically saying let's attend over all the customer transactions. And that attention mechanism can figure out that early things and late transactions or morning transactions or whatever is the pattern, it can kind of extract it from the raw transaction. So rather than basically joining and creating one signal, you are now attending over all the events. And you can not only do this at one level, but you can kind of do this recursively. So you go from the user to transactions to products, back to transactions to other people to transactions to products. So you can learn all kind of collaborative filtering effects and so on and so forth. Right. The model has that capability. Sorry if I'm talking a lot, but maybe I have one more important thing to say that is that, you know, in the old days we had a lot about AutoML, right? And I think data scientists have been kind of burned historically by this automl. But this is fundamentally different than Automl. Right. Automl was about let's just join everything with everything, aggregate, generate a gazillion of features, kind of throw them against the wall, see what sticks. And this doesn't scale is kind of brute force and all that. Right. The approach I'm advocating for is fundamentally different. It basically follows this modern AI trend. Have a powerful neural network learning directly on the raw data. The neural network is so kind of powerful and capable that it can extract, detect new types of patterns that we as humans are maybe not able to do. That's maybe, you know, a short, long introduction to this.

Okay, It's a nice explanation actually. I have to say the thing that surprised me most is the idea that just any standard SQL relational database that can be thought of as a graph. And I guess, yeah, I've spent many times just staring at those schema diagrams, not really thought, oh yeah, okay, this is a graph here. But yeah, it really is. The question here is around scaling then. So if you've got a network of like your whole company database, lots of different connections, so does the do graph transformers scale then? Well, it sounds like it's gonna be very computationally expensive to think about this whole network in Mongo.

That's a great point. Right. And you know, I would say nothing is magic, nothing comes for free, right? So maybe saying, oh, now you can bring your I know, thousand table schema with bazillion of records in and learn over that. That's a bit too optimistic. Right? So when I say modeling, I really mean, you know, we've been able to scale this to let's say 50, 60 tables, tens of billions of records. Right. If we think about the scale of Reddit, we are doing fraud. Another for example rscale use case is some food recommendation with doordash type food ordering data which is also massive. Or some fraud detection on the entire bitcoin blockchain is a project we are working with Coinbase. But even there, right? This is not all the data you kind of select what as a data scientist, as a data modeler, what tables you want to use, what date ranges you want to use when you build this model. So there is some iteration to really discover what data is useful, what is not useful. Of course, if you throw everything in, in principle we can learn over all that, but it will be very computationally expensive. The inference time will also be bigger because a lot of data will be used. So maybe it's not the most optimal thing to do by quickly iterating. Basically you or a data scientist can build much more accurate models than before.

Okay, so really it's like if you're trying to solve a sales problem, you just put the sales tables in. If you're trying to solve like a logistic problem, you just put those operational tables in.

For example. Exactly, for example, Right. Or you can. Many times we see people start with purchasing data, then they say, oh, let's add in also website behavior data, let's add returns data. And then they see how the model performance is climbing up as they integrate more data and then they can make a kind of informed decision, trained off to see, you know, what's a sweet spot for them in terms of data size, data volume, model accuracy, cost of training and things like that.

Okay, nice. So I'd love to talk a bit about how you get started with this. So suppose you decide, okay, I want to try one of these graph transformers, I guess. Is there anything you need to prepare for? Do you need to do anything with your data in order to make it ready for use with these models?

Yeah, so there are a few options I think that people have here. One is go the open source, kind of the open source route. The approach here would be to use pyg. So pytorch geometric. Pyg.org is a library. It has, let's say, medium level of scalability. I wouldn't necessarily say that it's kind of production ready and requires quite a lot of expertise because the graphs have to be manually created, the training data sets, the labels have to be manually created and then the models are built. Another, another option possibility here is to use a platform called Kumo. It's industry scale, easy to use, robust, where basically all you have to do is connect to your database. Essentially you kind of just register the tables you want to be using. You specify the predictive problem in a SQL like language. So you say, oh, I want to predict this label for this entity and then basically from that the system can generate the labels. You can then kind of tweak that as a data scientist, how much you want and then kick off the model training. These models are not compared to large language models. They're actually not humongous and they are able to train in a matter of hours, let's say less than a day.

Okay, all right, out of the way. You just casually mentioned Kubo as an example. Of course, since you're building the platform, I expect it'd be the one you'd recommend. So it seems like the main idea is that you've got to translate this, your relation database, into the graph format. And you can either use the open source Python tools or you can use your commercial platform for this.

Yeah, to be able to do this right, you have to take the data, translate it into the graph. If you have to build the graph yourself manually, that's a lot of work. The second thing is scaling up graph learning is very, very hard. This kind of, let's say commercial graph databases and things like that, they've been built for a different use case. So because when you are doing graph learning for a given node, you need to kind of sample a local neighborhood, like a little circle around it. And being able to do that very fast, add the node attributes the features to it and send that to the gpu in order for you to do that so fast that the GPU doesn't start getting bored. That's quite hard. And the hard part is that it's very hard to shard graphs, so you cannot hide kind of throughput by sharding. So in my career, I was a chief scientist with Pinterest for six years and I built two generations of graph learning platforms there. It took us like three years to build it before it was really performant. But it had the material impact on Pinterest business across, you know, a large number of different use cases. So here at Kumo, we took that time, we took that effort to build a platform to put in all the innovation, all the latest architectures to really allow people to train, let's say, predictive models directly on the raw data. And what is nice is you can either get predictions out of the models or you can actually get embeddings which you can then use for any kind of downstream analysis or tasks or even like features in more traditional models and things like that.

Do you need to have like one model for each of your different use cases then? So you mentioned the idea that you probably don't want all of your corporate data in one single model because it isn't scalable. So have you got to build lots of these things for each use case?

Great question. So maybe at the beginning we kind of glossed over a bit over that. Right. So I would say historically in machine learning we've been building one model per task. Kumo kind of comes in and disrupts this space by saying, you can still build one model per task, but just point me to the raw data, specify the task. I will build a neural network that learns directly from your data. I will save you time of feature engineering and you know, I will save you time, like reduce the time that you need for data preparation. There is still some data preparation, but cut that down by 95% and I'll give you 20, 30% more accurate model. Again, depending on the data, depending on the task and things like that. Right. So that's the first part. That's what we've been kind of discussing now. We have just announced a new, which I would say a true scientific breakthrough, which is a pre trained foundation model. The pre trained foundation model. The idea there is that you don't even have to train the model, you just point it to your data, you specify the task in some kind of declarative language and one second later you get an Accurate prediction. And if you think about this, this now allows you to do ad hoc predictions, right? Like not to say oh you want to estimate churn probability. So let me go away before I would go away for a month or two months to build a production ready churn model. Now with Kumo maybe I say I will go away for two weeks to train or a few days to train a model. Now with foundation model you can directly get a prediction by just kind of asking the foundation model. And that's really I think super cool and especially important in the kind of the AI agentic world where you need agents to make decisions, non hallucinated decisions over private data. And that's where this starts to shine and shows kind of its complementarity to kind of human level reasoning. Large language models that we have today.

We'Ve gone from the traditional machine learning thing of one model per task to the sort of, I guess the you've got standard graph crunch home. This is like one model per data set. And with the pre trained foundation model you basically just got one model for everything.

That's correct. And that was very surprising to us, right? Like when we saw this working in practice, it kind of blew us away, right? Like it wasn't clear that you can basically build a data set agnostic model that is able to make predictions and you could even say how is that possible? Because data set, right. And it's not that it's a single table, it's a model that works across multiple tables and tables can be different from very narrow to very wide with different number of rows and columns and different information in rows and columns. And that could be users and, and purchases or it could be posts and clicks or it could be you know, drugs and phone calls and patients or whatever it is, right? So that was the impressive thing. And what we have built, it's called Kumo rfm. Like for relational foundation model. It's available and people can go and try it for free at Kumo RFM AI. The exciting thing is that basically you connect to your data in a SQL like language, you specify what's the target, you want to predict for what entity. So I could say predict sum of, let's say transaction price between today and 30 days in the future and give me this prediction for user id27 and it's going to predict the sum of transaction prices. So kind of a lifetime value of a customer over the next 30 day window. Now somebody else can come and say oh I don't like 30, I want 27, we can do 27. Somebody can say, I don't want transactions price, I want just count. So you can say count. And now we are predicting number of transactions over 27 days. Right? So that's a beautiful thing. And what happens in the backend is that basically the private data is represented as a graph. This predict based statement generates a number of in context examples that are then the historical in context examples plus the unlabeled example are then sent through the foundation model that basically in a single forward pass of it kind of in its neurons essentially builds a machine learning model based on those in context examples and gives you a prediction. Right. So the amazing thing is that transformer in a single forward pass gets a little training data set plus unlabeled set of examples and is able to, you know, I know in its brain or I know how else to say, build a predictive model that gives you accurate prediction for those unlabeled examples. So it means you can throw in any kind of examples, any kind of labels, a set of unlabeled examples and in less than a second figures it out. So there is no, no training, no gradient updates, anything like that. Right. Like sometimes I would demo this and people would be like, wow, you really sped up training. It's like, no, no, there is no training. The model is frozen. What it gets in is a set of in context examples and then the unlabeled examples and it generalizes from in context to the unlabeled.

Okay, it sounds like powerful stuff and it also sounds easier to do than a lot of traditional machine learning. So does this mean that, so for example, commercial team members could use it? So if you, if you've got a sales problem, you want to do some sales predictions on your pipeline, then is that something an account executive could do themselves?

Exactly. I think with this technology we are truly kind of democratizing access to machine learning. Right. First, it's much faster, much less costly, much cheaper in a sense that we don't have to now build humongous teams and wait for a long time to get that model. So it means today I would say high end Silicon Valley type technology companies are able to find the talent and build these predictive models to huge benefit to them. Right. You know, I was building them in Pinterest before and so on to huge benefit of these organizations. But not everyone is so lucky or so fortunate to be able to have access to that talent and also be able to invest so much in this. But now with, let's say these pre trained foundation models, we are making this much More broadly accessible to, even to people who don't necessarily have a PhD from AI or machine learning and also allow for faster kind of experimentation. So it means that organizations can really kind of iterate on top of their product, on top of solving business problems, rather than say, oh, we are now stuck somewhere down trying to figure out how to make accurate recommendations. Right. The, the foundation model can do that and even a business person can then use this to actually have the, to have the impact.

Okay, nice. That seems very cool, the fact that basically anyone then can ask questions about the data, like if they've got questions that require predictions and then go beyond just like, oh, I'm looking at a dashboard or doing some sort of a simple analytic. If you want to get started with this, what's an example of a good like simple first project to try?

Yeah, so for example, a simple first project would be like a very simple nice data set is actually a Kaggle data set from H and M, the clothes retailer. It has this nice schema of customers, transactions and products with a number of columns, right. Descriptions, images, all that. And it's very easy to build all kinds of models over that very simplistic schema. You can be doing. You can be predicting whether customers are going to churn, you can be estimating their lifetime value, you can do product sales forecasting, you can do recommendations, you can do any kind of attribute inference, for example, trying to predict genders of customers and things like that.

Okay, that sounds cool. Yeah. And certainly I think retail data is fairly simple or it's at least something a lot of people know about. Everyone's done some sort of shopping, so just trying that something with a small scale weight, you've got a reasonable intuition for what the answer should be. So that sounds good. I'm curious about where things can go wrong. So you mentioned before, large language models have this big problem with hallucinations. They make stuff up. Are there any equivalent problems then that these relational foundational models have?

Good question. So maybe a few ways to respond. Right. So the first thing is this technology is very complementary to large language models. Right? Large language models are amazing at human like reasoning, but actually are not good at predictions or making decisions based on predictions. So machine learning as such and predictive modeling as such, you know, is here to stay and is very important with Kuma, it's getting even easier, even cheaper and much more accessible. Right. More accurate and so on. So now what are some caveats? Right? Like one thing, for example, that is important in these models is that many times in Business data. Basically you have time streams of events and it is very important that your timestamps are accurate, right? Because a classical problem in machine learning, in the old way, when you do feature engineering, is that your feature is stale, right? Maybe you update the feature once a month, you know, on the first day of the month, but then you are predicting something on the 25th of the month. Now the feature is three weeks old. It's like it's out of date. It's not informative anymore. It will work terrible, right? So this out of date problem or staleness is problem in classical machine learning, or that you update the feature on the last day of the month, but you're actually making a prediction on 25th. So then, you know, predicting future from the future, it's a much easier task, right? So maybe one thing to say is now in this relational foundation model approach, you need accurate timestamps because the model is learning from time and the model is very aware of what is the time of now. And because it sets kind of what we call the anchor time, the time of now, then it knows what is in the past, what it can look at and what it cannot look at now. If your timestamps are wrong, the model is kind of going to look into the future, not by its fault, but because of the data. So that, for example, is one problem. The way we are solving this is that the explainability capabilities of these models are really good because they allow us to go all the way down to raw data. So you can ask the model, what are you looking at to make this prediction? And the model can say, I'm looking at this table, I'm looking at this column. And as a data scientist, right, it's important to understand your data. And if the model says, hey, this column here is amazingly predictive for this label, then the data scientist can say, oh, you know, oh, something is wrong here. This shouldn't be so predictive. Let's go and investigate any kind of information leakage. So that's, I would say, is one example of caveat. The other example of caveat is that these models really shine when your data is complex, right? If you have all your data kind of crammed in one single table, then we already know how to do ML over a single table, right? So the delta won't necessarily be that large. But if you have data that's kind of nicely spread across multiple tables, then this is when this technology truly starts to shine.

Okay, that seems important to know that timestamps are incredibly important in terms of data quality. Data quality is a huge thing in most areas of machine learning. But are there any other data quality issues that people need to be aware of? Is it different to traditional machine learning?

So, good question, I would say, because these models learn over the entire relational context, actually they are very robust to issues, errors, null values, missing values, and kind of, let's say, shaky data. Right. So the amount of robustness we see is actually better than in classical machine learning. I think the other thing that happens when, when we do classical machine learning is that we spend so much time feature engineering that through that process we may discover quite a lot of other unintended things that we are then able to fix in terms of issues with the data quality. But I would say in this approach, what is really important is to know that a user ID in one column is a user ID in another table in another column. So having those relational information well mapped out is the key data can be kind of incomplete. That is not so much the issue, but having a good relational structure, proper primary foreign key relations that are not corrupted or that are present, that really then makes the approach to shine.

Okay, so if you screwed up the schema, you've not got like sensible primary keys or secondary keys, whatever, that's going to break the graph and therefore that's going to reduce the model quality.

Exactly. Then you basically break the flow of information across these tables. And if you break the connection from one table to another, then the information from that other table cannot flow and it's all for nothing.

So as related to this, do you need to do data governance differently if you were working on, I guess, traditional machine learning problems or analytics problems to affect your approach to this?

That's a great question. I would say it's. It is a bit different, and it is different in a sense that you really kind of need to understand what are the row tables, rather than kind of doing governance, almost like at the level of features you now need to be doing governments at the level of raw data, having well annotated kind of correspondences between tables, knowing what columns point to what other columns and so on. You don't need. What is maybe interesting is you don't need semantic information, right? You don't need. In this, the model doesn't care, this approach doesn't care about column names and column descriptions and things like that. You don't need that. All you need to know is you need to know kind of the semantic types of columns and have reliable, basically have reliable timestamps, primary foreign key relations, and some semantic types about the Columns. That's about, that's about it. So if you can do data governance at that level, life is good.

Okay, all right. It sounds relatively straightforward. So one thing I'm curious about is how you go about interpreting these models. So for example, something like linear regression decision tree is really easy to interpret. But the fancier you get with most model types, the harder it is to interpret what's happening underneath them. How do you go about interpreting any predictions made by this foundation model?

There are, let's say two types of interpretability, right? You can ask for this kind of global level model interpretability or model explainability type questions and you can also ask about individual prediction type questions. And the way we have innovated this predictive capability, sorry, explainability capabilities is I would actually argue that these models are actually more, more explainable than these traditional feature based ones, right? Because in the feature based ones all you get is some stack ranking, some shape values, whatever of those features, right? So if you missed an important signal, you will never know it exists, it will never be part of your expansion. So instead what we do is we do basically this kind of back propagated gradients where we can trace through the neurons of the model all the way down to row columns and row tables. And then basically we can ask the model for this type of prediction, what data are you looking at, what tables, what columns? And also for individual level predictions, we can ask the same thing. And here we can trace this all the way down to individual rows, individual columns. So the idea is that now the model tells you, this is what I'm looking at, right? It's very similar. For example, if you ask computer vision models, how do they explain a computer vision model that maybe let's say is doing human detection, right? If the model says, I'm looking here and then, you know, it's doing the right thing. If the model is saying, oh, I'm looking, you know, just above your head, because I'm expecting a light to be turned on, on the ceiling, you know, the model is doing something wrong, right? So these types of models are kind of at the same level. They point down to the raw, to the raw data that was used that was influential to make that prediction. And from this you can really kind of see. And then again, as a data scientist you can ask yourself, is this actually, are these the right signals? I would expect that this match my intuition. What if I remove this, can the model recover and things like that, right? So again, I think the data scientist is very important. The knowledge of the data, the Knowledge of the of the domain and being able to model it is not going away. We can just do that now. Faster, easier and better.

Okay, so that sounds pretty good that you do have some kind of level of explanation. And Chris, do you have to do things like cross validation on your models just to check that they really are working?

The foundation model is pre trained so you don't need to do cross validation. You can give it some unlabeled examples and it will give you back the prediction. If you now fine tune or build custom models per task, then yeah, then automatically kind of the training data set get generated. Some part of it is used for model training, the other one is kind of for validation and then there is holdout. So that is still fundamental. The goal of ML is to do out of sample generalization. So how do you check generalization is through cross validation so it doesn't go away. It's a short answer. Okay, nice.

No cross validation needed. That sounds great. So okay, at the risk of this all getting too nerdy, maybe we just bring it back to. Do you have any sort of final advice for people who want to try out these models?

Yeah, I would say like really, I think it's very exciting time, right? Like at the beginning of the, of the conversation I said right. It seems like kind of AI has skipped machine learning, right? And then people got kind of oh, what happens now? Is machine learning still needed? What will happen? And I think what we know now is machine learning is not going away. It's going to be even more important, right? Especially with, let's say agentic workflows. With our lives industries getting even more digitized, collecting even more data, being able to make business critical decisions on those data is of vital importance. What we've been, let's say doing in the past or what people have been doing in the past is, you know, running some SQL queries that analyze the past and then somebody tries to kind of make a common sense decision how the future might look like. On the past, what we have seen is that the best way to make decisions is through making them via predictions. Via forecasts. You want to forecast and then based on that forecast you make a decision, right? So I think going on the demand for predictions, demand for forecasts, demand for decisions is only going to increase. The second thing I think that is going to happen is that the power of data, private data, will become even more important, right? Because if everyone is using the same pre trained model, then the only way we make differences among ourselves is by the data we have. So I think that's another important thing is like how do we make decisions on our private data? Not on the data that you know, that the LLM has read on the Internet, but something that is truly unique to me, to my organization, to my business, to my customers, because every organization kind of right. Is different. So I think the importance of data is going to go to be even more. And then I think the third thing is that now with this technology that we talked about, relational foundation models, relational deep learning, as well as, let's say Kumo as an implementation of some of these paradigms, I think the kind of the power of AI is finally coming to machine learning. Predictive, predictive modeling, predictive decision making as well as it's truly exciting and I hope we embrace this new world soon and get the data scientists basically the impact that they deserve. Right? Like no data scientist I would say is dreaming. Oh, I wish I could do more data cleaning or you know, wakes up in the morning and says, oh, let's do some data cleaning today, right? Or right. They say they don't, I, they don't wake up like that. They wake up and they say, I want to have some business impact, I want to solve some hard problems, I want to build some cool models. With this technology you can focus on start building models starting to have business impact rather than say, oh, I have to go feature engineering.

It does sound pretty amazing, the idea that you can just build models a lot faster, just, yeah, make more cool models. It's a bit of a machine learning dream. Nice to wrap up. I always want recommendations for people to follow. So can you tell me whose work are you most interested in at the moment?

I would say very excited about the work that is generally happening in the field of graph learning. You know, organizations that been at Forefront have been like people at DeepMind as well as at other organizations. There is a very cool conference on learning on graphs that I think is very exciting. The PYG community is very vibrant and there is a lot of super cool contributions and I would say a lot of support from Nvidia and other organizations. So that's another place that I follow because I truly, I'm truly kind of excited about, you know, this new generation of AI where we have this kind of human like decision intelligence and then we have this private enterprise data predictive intelligence foundation models and how the two can, can join hands and really have the impact, positive impact on the humanity going forward. Wonderful.

Yeah, lots of exciting things happening. So some great recommendations there.

All right, wonderful.

Thank you for your time. Yura, great having you on the show.

Yeah, thank you, Richie. I enjoyed it a lot, Sam.