#286 Data Science Trends from 2 Kaggle Grandmasters with Jean-Francois Puget, Distinguished Engineer at NVIDIA & Chris Deotte, Senior Data Scientist at NVIDIA

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey. On datacamp.com.

We'Re going to see more use cases of using these LLMs assisting the data science workflow and few shot.

Learning is becoming a thing. It lowers the need for gathering data. One revolution is bringing GPU acceleration everywhere.

Accelerating all these algorithms on GPU can allow us to build new hybrid models.

And I see the use of LLMs is just yet another level of abstraction.

Welcome to Data Framed. This is Ritchie One thing about working in data that's both brilliant and terrifying is that the field is changing fast enough to give your skills a fairly short half life. That means that staying up to date with trends is essential. Today I have two guests who are Kaggle Grandmasters. That is they're world experts in competitive machine learning. Naturally, that means they've got some great insights into the latest techniques and processes for making models that predict well. I'm curious as to which of these translate well from competitions to day to day business. Our guests both work at Nvidia, so I'm also keen to find out what the latest developments are in using GPUs for data science. First up is a returning guest. Jean Francois Puget is a director and distinguished engineer. He manages the Kaggle Grandmasters of Nvidia or KGMON team. He started his career as a researcher in mathematical optimization and published 70 scientific papers before discovering machine learning first at IBM and then Nvidia. And he's a three time Kaggle Grandmaster. Chris Diot is a senior data scientist and part of Jean Francois's KGMON team. He's also gone one better than his team leader, having achieved Kaggle Grandmaster status four times. Chris is also a keen educator and is currently ranked world number one on Kaggle for the quality of his notebooks. He honed his machine learning skills while working as a researcher in math and Data science at the University of California, San Diego. Super so two top experts. Let's find out what's new in data science.

Hi there Chris and hi, welcome back. Jean Francois, welcome To the show.

Thank you. Thank you.

Thanks for having us, Richie.

Brilliant. So, to begin with, let's start with the big one. What do you think is going to have the biggest impact on data scientists this year? Chris, do you want to go first?

So I think we got an exciting year ahead of us. We're all seeing all these chatbots and copilots and AI assistants. So I think this is going to continue and we're going to see more use cases of using these LLMs assisting the data science workflow.

Okay, yeah, that certainly seems to be like a very big topic at the moment, and I'd love to get into that, into more depth. But yeah, that seems like a solid choice for what's going to happen this year. Jean Prancer, do you have any predictions on what's going to have the biggest impact?

I kind of agree. I believe there is also another impact which is happening already, moving in many case from supervised learning where you aggregate a data set and train a model. So you aggregate data set, have humans create the answer to the question you want for every row in the data set and then train a model. And few shot learning is becoming a thing. It lowers the need for gathering data. You only need few examples and especially for text classification. Say you want to create a spam classifier. Before you would have to collect thousands of spam and train a model to these. Now you can say to a big LLM, here are a few spam examples. Now tell me if that new one is a spam or not. And a stack specification. It works quite well. So it's minor compared to what Chris says. But replacing supervised learning with few shot learning works as well.

Okay, yeah, certainly. I think for data scientists that's kind of huge because supervised learning has very much been like a mainstay for just decades and decades. So the fact that you need less data now because the models are more powerful, that's a very interesting twist.

On that note.

So Chris, your example was around generative AI and it does feel like in the last year OR 2, Generative AI stolen like a lot of the thunder from data science. Like it's just captured all the hype. So do you think innovations are still happening in data science? Jean Francois, do you want to go first this time?

So no surprise from an Nvidia employee. One revolution is bringing GPU acceleration everywhere, and in particular in data science. In tabular data, there are two or three packages that everyone use, Pandas, Polas and Seciternation. And we ship over the last year GPU accelerators for Pandas and Polars. So you can take a Pandas code or POLAS code and with 1.9 move it to GPU. So I think this is a game changer because your code running faster, you can experiment faster. For Scikit learn, we have an equivalent called qml. And overall all these GPU accelerated machine learning and data science is called rapids. So easing the move from CPU to GPU code, as I said one liner, I believe we will make more impact than what we did before. That's definitely an innovation on the algorithms. Another innovation is same, it requires gpu. It's the use of deep learning models on data science or tabular data. Every other week there is someone claiming a new SOTA model for forecasting or tabular data. Maybe they are not sota, but they are interesting.

Certainly the first one about making easier to use GPUs. I'm sure you've got a slight vested interest in talking about that, but I definitely want to get into that more. We've known for a long time that GPUs are much better at doing computations. For a lot of data scientists, there's been a bit of a hassle in terms of getting them set up. So making it easy for them to be used, that does seem like a pretty important innovation. And yeah, I guess LLM sort of, or at least neural networks are sort of eating every problem. So that's interesting that they're even invading the sort of tabular data space as well. All right, Chris, are there any other innovations that you wanted to talk about?

I'm observing the same ones that JFP said, but I think so. I've worked at Nvidia six years and I think one of the biggest things I've seen in these years is accelerating all these algorithms on gpu. And the significance of that is, besides just obvious speed, is this can allow us to build new hybrid models. So I've actually recently in some competitions seen people fusing deep learning models together with a machine learning model. An example is maybe you're doing an image regression problem, but you use an image model as a backbone to extract embeddings and then you regress the embeddings using a support vector machine. But previously a support vector machine running on CPU kind of wasn't fast enough to keep up with the, the iterative cycle of deep learning. So because of the speed, now people can kind of incorporate knn, you know, KNN to compare embeddings to find similar images or similar texts. So you're seeing a lot of More uses of machine learning. So that's exciting that these new ways to approach problems.

All right.

Yeah, I like that.

Mixing models together. Okay.

I can even add a second thing. So also you're also seeing more automation. So in feature engineering or AutoML, once again, because you're speeding up the process of data framing on gpu. Instead of being selectively picking a few features to try out with the increased speed, you could just have a for loop, just iteratively try out hundreds of, I mean, literally overnight hundreds of thousands of features and then just have it evaluate and find which ones work. So the speed's allowing us to do faster experimentation and just automate some of these search procedures.

Okay, so that's really interesting, talking about feature engineering, just being able to, I guess, brute force it now by just trying lots of different combinations. So I guess one thing we were teaching for years on Data camp is that actually when you're doing feature engineering, you need to think about like what the problem is you're solving. But it sounds like that's just going away. Then you just need to just try more things. Is that about right?

So they both have value. I mean, I use a lot of intuition in my work and when I pick features and it's a lot of times that's how you improve the accuracy of a tabular data model. It comes down to finding that sort of, we call it sometimes the magical feature. But more and more I am using automation. I recently took first place in a tabular data playground competition on Kaggle. And what I had done was instead of thinking about the columns and which ones should I combine and then targeting code, there was these, it was categorical columns instead of, you know, think. I just ran the code overnight and on GPU on some A1 hundreds, it was so fast. It tried out tens of thousands of combinations. I woke up in the morning, it gave me a list of 100 and it had boosted the model accuracy a lot and just jumped me to first place. So I use both. So intuition counts for a lot. And I, that's how I found a lot previously. But you also have to use the brute force method to just, just even search things that you wouldn't have thought of.

So you mentioned the sort of brute force method for feature engineering. Jean Francois, you were talking about these sort of combining different models together as well, like combining LLMs with more traditional machine learning. You sort of both talking about this sort of stuff. So is this just an increase in computational power that's allowing this then? So you make use of the GPUs, you can crunch more things together. What's happening here?

Using deep learning on something else than GPU is not a great idea in general. So definitely using GPU enables, using transformers and LLMs for time series, for instance, there is a lot transformer kind of models that appears to be quite powerful. So that's one on traditional models I just concur with Chris. I see machine learning and data science as an experimental science. You have a baseline model you want to improve, so you design an experiment, like adding some features or different parameters or assembling with another one. You run your experiments, you evaluate the results. So you need to have a reliable way to evaluate your model. And that's what we learned on Kaggle, I would say. And if your experiment is successful, your new model becomes a new baseline. If not, you look what can explain why it's worse and you learn from it and you design the next experiment. That's how I work. Then the faster you iterate, the better because you try more things. So that's why GPU acceleration is useful, even if your problem may not be that big. And training a model takes only half an hour. Well, if you replace the half hour by a minute, in one hour, you can try 60 things instead of two. So even if it's not computationally demanding, if you accelerate, you will benefit. So it's really, really anything that can make a better use of the data scientist. I would say that's the scarce resource. That's the thing. You can't compress the time you have. So the more you do in a given time frame, the better.

So I'd like to go back to something you mentioned earlier on, Chris, is that AI agents sort of one of the big trends of this year. So what I've seen so far is like a lot of these things, they are not really data science focused. They're all like focused on like particular industries or just kind of more general business use cases. Do you think AI agents are going to take off for data science this year?

I think so. I mean, I would say that they already are helping. So in my daily workflow, I'm actually constantly consulting ChatGPT as a copilot to help me with my coding. You know, if I have a function to write, which is just some basic function, I might just say, hey, write a function that sort of lists the numbers and it spits it out. Or if I have an error in my code, I'll just copy and post the error. So it's already in a lot of ways an Assistant to me. I mean, I'm working together with it and it's, I mean it's saving me so much time. So that's already happening. And I just noticed that they're getting better and better. I noticed that when they're providing code, they're making fewer and fewer mistakes. I think now it sort of has some chain of thought maybe or it's checking its work. It can even have access to tools to verify. So it's, it's continually getting better and therefore I'm relying on it more and it's being more useful. And I, I see this trend just to continue to grow to the point where it's almost like I almost see myself being, you know, a manager of a little office and I'm sending out my agents on little jobs to do and then they come back and they give me some code or some EDA or this or that. And that's kind of what I see happening.

I do like the idea, and certainly the idea of a copilot, like something to help you write code. Very, very important already. Do you see yourself being taken out of the loop where you can just leave the AI to go and solve some data science task and then you come back and maybe just review it at the end?

So my take is quite optimistic. So I'm old enough to have seen. I never used punch cards, but when I started computer science we were using compiled languages mostly and before they were using assembly, then people moved to interpreted languages and then to frameworks. And each time there is a level of abstraction and I see the use of LLMs is just yet another level of abstraction. You still need to tell the LLM what you want. So it's just programming at a higher level. So designing a software system still stays. It's just that instead of having to write lots of code, you automate the writing, but you still have to design the system and the other bit. So it's more software engineering than data science. It's debugging and maintenance. I see people amazed by all this. App code was generated and sometimes creating some machine learning and deep learning models automatically. Sure. But next, if there is a bug, what do you do? I'm not convinced yet that coding assistants are good at debugging and maintenance people who think they no longer need to understand coding. Well, I don't want to be there when their application has a bug.

No.

It's kind of interesting that we often talk about the idea of you want AI to automate the boring things that humans don't want to do. But if you're getting AI to write the code in the first place, which is kind of the fun bit, but then the human has to debug it, that's just a worse situation for everyone.

I think AI automate is the most fun part because it's the easiest. So at the same time it's normal. Automation starts where it's easier.

Okay, so in terms of automation, are there any things that you think really shouldn't be automated by AI that they always ought to be done by a human?

Model evaluation. For me, defining how a model should be evaluated, that's tricky, and I said it once already, but I believe what distinguish people like Chris, me and Kaggle competition grandmasters from most data scientists is we evaluate models reliably, we have to win competitions. If we don't pick the right model, we don't win. So we have to learn and sometimes we are wrong and we drop in the ranking. So I don't think this should be automated. It's dangerous.

So I guess talk me through what's going through your head when you're evaluating a model. Like what's this sort of human process that you're doing that you think is important?

So I think the most important thing is just to do it without bias, without leak. So you just have a local validation. So ultimately you want your model to make predictions on new unseen test data. That's what you're trying to do. It's gonna be deployed in production somewhere and see new data. So when you, so you, you need to set up a robust local validation where you're training on one set of data and then, you know, validated on some unseen of data. Typically we use a cave cross validation. So I think it's just important to set it up correctly, make sure that there's no leaks, you're not using any information from your validation folds. And then, yeah, you just do this and then you compute that local validation score. And then if you've set things correctly, if it mimics sort of the unseen data it's going to see in the future, then oftentimes you'll see a very great correlation between this validation score you're computing and then how it will perform unseen data. And then after you have that set up, that's the only number you look at. You don't fudge things. You don't say, oh, let's just try it on the test data. Oh, let's change that. It's helping the test score. I mean, you have your validation score and you just very, you know, basically you do an experiment, if it increases that you keep it. I mean with a level of, I guess I will point out there's always a level of variance in validation scores. Maybe if you change the seed, it might, it might change plus or minus a little bit. So you have to kind of determine that. But once you do an experiment, if it shows to be statistically significant that it improved your validation score, you keep it. And if it doesn't, you don't keep it. And that you follow that rule and then you do it over and over and that the expression on KAGGLE is trust your cv. So many people, if you trust your cv, you will consistently do better and place better on the second unseen leaderboard. People that are too concerned with watching their score on the first leaderboard and only keeping changes or doing tricks to increase that first leaderboard, then there's, there'll be a huge shakeup and their score will drop. So trust your cv, use that local validation score, make decisions based on it and then you'll, you'll do good.

Are there any other skills that you think are important or becoming more important at the moment?

So both Chris and I are fan of Kaggle, but we have to admit it does not exercise all skills to be successful. So KAGGLE is probably the best way to learn how to build a model and evaluate it. No doubt. But what happens before? So data curation at large. What do you train on, what is the target and what happens next? How do you deploy is important, but I believe the most elusive skill is how do you convince non machine learning people that your model is useful?

Yeah, the art of persuasion.

Yeah, because saying oh my AUC score is 2 more than before. Well okay, so if you don't know what a UC score is, you don't know. So I see people are impressed by numbers. So you can, it's easy to just create the wrong expectation by storing numbers that are not validated properly. So I remember I had to argue, not at Nvidia but before that, no, the score on the training data is not relevant to measure the model, all this kind of stuff. Another senior data scientist was having better numbers than me. He was evaluating on the training data and it was almost impossible to convince management. So that's what consultants do. Well, normally. But I believe data scientists have to learn how to speak in terms of business, KPIs or whatever, but not just machine learning.

That does seem incredibly important, that communication skills not just to other technical people, but maybe to people who have more of a business interest or a less technical Focus. Actually, do you have any tips for that? Like did you learn how to convince your manager of things?

I would say what's helpful is if your model can exhibit some explainability. So I've worked on these projects and been up against just what JFP describes. Whereas I have a model, I have a very good validation score and I've worked with so many models that I know that I could trust that if you set up the validation correctly, it's representative of the unseen text data, data, then you can confidently say this is how it's going to perform, like it's going to do good, we should use it. But you know, to a business person it may look like a black box. Well, why did it make, why did it make that prediction? What's it thinking? Is it going to make a mistake tomorrow? I have a natural trust, I know that it'll do what it says it does. But anyway, what helps in those situations is if your model can give some level of explainability. If you sort of say, why is it, you know, maybe it's making some forecasts, why is it predicting an increased sales at this one store but not this other store? So if you can do shop values or other things or, or if you've got some feature importances or if you can just show how things affect the model, but anything that kind of like explains a little bit why it's doing what it's doing, or if you showed a few, if you look at a picture of a decision tree. Because if a manager can say, oh I see, like if you were to have like a picture of a decision tree, you know, if the store has a drive through window, we will sell more stuff at 9am oh that makes sense. So this store, that's why this store will sell more in the morning, you know, so if you can kind of explain why it was making some of the decisions again then they say, okay, I see, I understand that. So anything that helps their understanding builds their trust and then they're more likely to use the model and rely on.

It, I would add. So showing prediction on their data since I worked on predictive maintenance to predict problems before they occur in some compute environment. So we just compare to the reality. So using actual data and using playback with a model pretending it's new data and comparing the predictions to the actual, if the prediction quality is good, this is extremely convincing.

So in general it's like, well, you need these sort of better communication skills. Some of it's just about giving people more trust by explaining stuff. It sounds like in General, it's just like providing more evidence for what your argument is. Okay, all right, I like that. And so beyond the communication skills, I guess the flip side is are there any skills that you think are becoming less important now?

Yeah, I would say so. You know, I've been asked this question kind of many times by my friends, you know, will my job be replaced or what skills are more important and less important. And I guess I have a simple picture in my mind. I think that data scientists and employees will all be more like managers and will be essentially having a team of AI. So anything that that incorporates some of the roles of managers. So basically with regard to data science, I would think like individual tasks, just maybe manually making features or EDA or iteratively chaining models, finding hyper parameters that can easily be automated. But the overall things like the purpose of the project, I mean the goal of the project, the choice of metric, how we deploy it, I mean sort of like what a, kind of like what a project leader or a manager would do. I think those skills will be more important because that's sort of how I see the relationship between the worker and the AI. The AI, you can send them to a whole bunch of roles and tasks, but then you have multiple agents. This guy did eda, did some of this and then you have to synthesize it together in sort of the bigger picture. So that's my view of how you have to be more of a multitasker, a generalist, a manager, a project leader and know how to put it together, assign roles, assign work, and then those skills I think will be beneficial.

That's absolutely fascinating the idea that you need to become a manager of your AI. Actually imagine have like a performance room review meeting with your AI, but oh yeah, good partner. Okay, that's kind of cool. And so do you think that the data scientist role is still going to be around in 10 years time?

I believe so, but it really evolved. As Chris said, look at what data scientists did 10 years ago. If you're still doing it now as you did 10 years ago, you need to update and then maybe the small data problems would still require a lot of human intervention. When you have lots of data, people will just throw generic deep learning, whatever agent but on edge cases. Also maybe in medical health care people will resist automation quite a bit. So maybe in regulated industry, but now there are areas where, I don't know, E commerce probably it's already quite automated, recommender systems, etc. So there are probably many industries where data scientists won't be needed. But I still believe there are pockets where human judgment would still be quite important.

So we talked before a little bit about how GPUs are making data scientists able to run their code faster and therefore be more productive. So I know that Nvidia is involved in the RAPIDS project for data science tooling. Do you want to tell me a bit more about what is Rapids and how is that changing this year?

Yeah. So RAPIDS is a whole suite of libraries and each one does different things. The central theme being they're all GPU accelerated code to accomplish different things. The two I work most often with are QDF and qml. So cudf, it's similar to Pandas. It's the same API as Pandas. It can handle all your data framing needs. The API is similar, but everything happens on gpu. So all of your operations, if you have a bunch a data frame with columns and you're calculating means and group by aggregations, I mean it happens in a blink of an eye. That's qdf. And recently they released some zero code change versions, qdf, Pandas, cudf, Polars. Which means if you already have your code written in Pandas, instead of rewriting it in, you just turn on a flag at the beginning of your code and then all of the Pandas code will behind the scenes use CUDF and be accelerated on gpu. So that's cudf. That's awesome. Another one I use often is qml. That's all your machine learning models. So the API is the same as scikit Learn and it basically accelerates all your machine learning models, KNN support vector machines, linear regression, dimensionality reduction, like UMAP or T sne. So all of that stuff is accelerated on gpu. And once again, if you run the same model CPU versus gpu, you could see hundreds of times speed up. And that's exciting because it's allowing me to use new models, as I mentioned earlier, that I didn't previously use in competitions or on projects because it was too slow to train. I can combine them with deep learning and I can also iterate experiments really fast and then that allows me to tune them and make them highly accurate.

I would add one which is not really part of RAPIDS technically, but it's developed by Rapids. It's XGBoost. So Nvidia developed the GPU version. XGBOOST is what we call gradient boosting. And so one size fits all almost algorithm for tabular data. I say almost because linear model, SVM and linear regression are still useful by XGBoost. And there are others, cat boost, like GBM. So these are really also interesting to have on GPU.

Okay, so these are all just maybe, I guess, some of the most popular Python packages then for data science. So yeah, you mentioned Pandas, you mentioned SciKit, Learn, XGBoost, all incredibly popular things. And I like the idea that anyone who's a Python data scientist is going to know how to use these packages. And therefore, yeah, it's just a question of setting a flag and then you can go in, run your code on gpu, which is nice. I like learning new things, but I don't necessarily want to learn like almost the same code again and have to like tweak things if I'm switching to gpu. So I guess related to this, you've both been doing Kaggle for a long time, you're both grandmasters. I'm curious as to how your process has changed for competitive data science recently.

That's a great question. So I've been competing on Kaggle now for six years, the last five, and with Nvidia. And how have things changed? So I think the competition has gotten much more competitive. It's much more difficult to get gold medals and win in the top. And I think that's because I think previously if you just knew how to do something correctly, by the book, you could do well. So for instance, when Laura Parameter efficient tuning of LLMs first was released, if you could just get a LORA adapter, train an LLM, process the data and kind of do everything and set up a correct validation and get your training schedule and if you could just pull it all off, you could typically jump in the top 50, top 20. I mean, just kind of knowing how to, knowing how to use the, the different models and, and machines and validate and stuff. But nowadays that's just kind of a prerequisite, you know, and these comps, you have to know the basics and then you have to add an innovative thing. So more and more, I think because the user base is increasing, there's now 20 million people on Kaggle, so pretty much kind of everybody knows about it. And this is researchers or practitioners at companies. So, you know, it's the best of the best. And I think what's changing is that you have to know more than just the basics and specifically you're going to have to use all the advanced tricks that you learn over time on Kaggle, like maybe pseudo labeling or stacking. And then you also have to add innovation, read the latest research, but add a kind of a new trick, maybe Modify an architecture, invent sort of a new training technique. So it's really, it's pushing you to do more and more. I think you know, each new year we continue to compete.

Okay, that's very cool that the sort of baseline level of competitive ability in machine learning has been increasing. I'd love to hear about some more about like some of these advanced tricks that you have. As you mentioned, pseudo labeling and stacking. Do you want to talk us through what those are and why you might care about them?

When I started Kaggle, stacking was a big thing. If you knew how to do it, you were getting gold medal. So stacking the idea is it's a way to ensemble models ensembling. It means you have several models and you combine somehow their predictions to this works better than each model. The simplest way is to average or if it's a binary prediction, you vote, you take the most frequent answer if it's classification. Stacking is the idea of you train a new model on the output of your first set of models. So you train several models and to find the best way to combine results, you train another model could be XGBOOST or whatever or the narrative regression or a deep learning model. And to do it correctly you have to be careful because the prediction of your models depend on the training data. So if you train again on the output of a model, you use the target of the training data in your training. So you have to be careful and use what we call out of fault prediction. It's a bit complex. So there was a barrier country but now it's common knowledge on Kaggle so everybody does it. Another related technique is called target encoding which is very powerful. Chris used it. Maybe he would say something after me. It's the same idea to use a target to create features, but you have to do it carefully. Once you master it, you have yet another advantage over other people. So there are more and more tricks like this and now they are understood by more and more people to distinguish you need to well to find something new. There is also a shift on the nature of competitions nowadays. No surprise, a lot are LLM gen AI stuff before they were image a lot of image segmentation. And so the nature of competition evolved as well. So we need to evolve.

Okay, that's very cool. I like the idea of. I mean it sounds like stacking gets quite complicated because you got a lot of different models and then you're using another model to see which one of the original models was any good. But it sounds like you've got all these techniques then, to really take you to that sort of advanced level of model performance, are these just used for competitive data science or can they be used more generally? Like, if you've got a business problem, would you ever make use of this sort of thing?

Yes and no. So they, so they clearly work. I mean, they clearly make models more accurate, but they obviously make the model more complicated. So the downside is you increase latency. Instead of inferring one model, you're inferring a bunch of models, you're combining them in some fashion. So often in production you have latency concerns. Things need to be quick. And also in a competition, you want that fourth decimal place, right? You want an extra 0.00001. But maybe in your particular application, that's not important. But if you're dealing with something, maybe in healthcare, maybe it's very important to be very accurately diagnosed, something. But maybe if you're in some other industry, you're building widgets and it doesn't matter if they're a little bit off. So I think you have to look at your use case and you have to balance. How important is it for you to get a more accurate model if it's only a tiny, tiny bit? And how important is. Well, the complexity of maintaining the code. But you need employers to constantly train and maintain the code base. And then that's two, and then three. There's even in test time or inference time, the latency of making the predictions, you have to kind of weight all those factors. And I would say that I have used some of these, but I've never used them to the level I do on Kaggle. I mean, I've had some Kaggle solutions, I'm almost embarrassed to say. So sometimes the solutions, it's either a code competition, means you submit your code and then it has to run within constraints that include time and resources. There's another competition which you just upload the predictions so you can do all the computation as much as you want locally, and then you just upload your predictions in a CSV file. So in that second type of competition, I have literally sometimes ensembled a hundred thousand models, because you can. And it's not that I'm just staying up all night, but you work for the whole month, and every day you might add like a new model, and then you just save the model weights. And every day you're making new models. So every day you're logging another dozens of hours of compute. And then on the final day, you average everything you did for the Last month and the result is the average of, I kid you not, I've exceeded thousands of models and it's given me that edge to win. But you would obviously never ever use a model in production that literally takes a month of training time. Right. So you see the trade off of accuracy versus all the other resources you're managing.

Yeah, absolutely. I can see how you try and run something in production. You don't want it to take a month to give you a response of like, of. Of what's happening. So but that's very cool that there is this sort of difference like in terms of like exactly how precise you need to be with your predictions or how accurate you need to be with your predictions. But yeah, some of those techniques are still going to be useful, maybe just like to a lesser degree in the business context.

I'll add one quick comment. There are some techniques like pseudo labeling, which is an advanced technique used during the training process. But afterwards you still only have one model. So ensembling and stacking, you might be making lots of models and making things more complicated for inference, but there's other techniques that, and we could list dozens more which just help you make a more accurate model. But at the end of the day you still have one model. So these you definitely use. Why not? Right? Because you can do these things. So I will mention there is a lot of other techniques you learn on Kaggle which help model accuracy and don't make the end result more complicated or increase time. So I don't want to make it sound like, you know, you're learning all this weird stuff that you never use. So a lot of it you use. But there are some things that you do have to balance the trade off.

I would add one that is quite recent. It's test time training. So for instance, for forecasting competitions you submit a code, but every month it's evaluated and every month you have one more month of data. So in your inference code you can update the model on the new month or there was a competition, the ARC AGI Prize and same. I participated and the key was to have a model that could be trained on the test data in addition to pre training. So as Chris we are pushing, the community is pushing the envelope and it's harder and harder.

I do like the idea that, you know, there's still being progress made in what seems like, well, you know, people are doing these competitions for a while now, but there's still things going on. So it sounds like for listeners who are interested in kind of learning some new techniques to get to advanced machine learning levels. Sounds like pseudo labeling then and test time training, maybe model stacking. These are things that people should look into. Are there any other things I've missed there?

Yeah, I would add a target encoding. This is a technique used to generate features so, you know, make new columns in the tabular data set for the purpose of making a model more accurate. And probably, yeah, the most powerful technique and it's very effective is target encoding and I'll even say a little plug. We're actually teaching that at the upcoming Nvidia's hosting a GTC conference in March and a bunch of kgmon, my coworkers and some other Nvidias were given a workshop and we're teaching exactly how to do this targeting coding technique and some other techniques and where I even show you how you incorporate that into a model and we demonstrate how it makes models more accurate. So it's a hands on workshop. You follow along with the code. So want you to know about that and invite anybody who wants to join. I think you can join it both virtually or in person in, in San Jose, California.

So yeah, that's been a lot of fun. I have to say. The last two years the GTC conferences has clashed with datacamp's Radar conference. This time we've made sure to have it on different weeks so people can attend both. So yeah, Radar's the following week after GTC this year. Brilliant. Okay, so I'm curious now as to whether there are any things that you think of like just recently become possible or are just becoming possible in data science that previously were very difficult.

So one thing very fresh is reinforcement learning at a low cost and effective. So deep seq R1 model that just came is a good example of a technique that was published by same Deep SEQ a year ago. But the model itself is interesting. But the recipe used has been used by many people just to improve models in all sorts of situations. I have yet to see it applied to tabular data, but maybe this will happen. So we see reinforcement learning applied for the first time in a number of places effectively. That's really recent. It's like last week.

That's interesting because I always feel like reinforcement learning is one of those. It's like the unloved cousin of supervised learning and unsupervised learning. It's very useful, but it's sort of the third branch of machine learning that gets a lot less attention. So that's interesting that deepseek sort of bringing it into a bit more of the public eye. All right, and so just to wrap up, what are you most excited about in the world of data science?

I'm really enjoying all of these LLMs and chatbots and stuff, and I think I'm excited to just continually see them getting better. So. And I'm noticing that, but I'm looking forward particularly I'm enjoying multimodal. So that's where nowadays, in addition. So let's say you're asking ChatGPT or some other thing, a question. Instead of just typing in text, you can upload images or upload a PDF or upload documents, and, you know, it can ingest those documents. And then furthermore, it has the ability to use tools to process things. And then when it gives its answer, it doesn't need to be just text. You know, generative AI, it can make images or plots or graphs, and obviously you can incorporate voice, you know, audio input or audio out. So I think I'm really excited to see all these different modalities of the inputs and the outputs and just allowing the models to do much more. And I guess it's so exciting for me because the reason I'm in data science is as a small kid, I've always seen so many patterns when I look around the world, right. As a small kid, even before I could talk, you'd see a lot of people entering a room, and then I'd even be like, oh, I know those people are going to leave the room. You just sense things, right? The body language, this, like, life is all about recognizing patterns. That's what it's all about. And it's exciting. I just want to see how powerful will these AI models get? Will they be able to watch me, my face, and say, oh, Chris is going to get angry in the next 10 minutes because they saw me twinkle my eye. I'm like, really looking forward to see how many patterns will they see in the visual world, in the audio world, in this world. And I think that they're gonna teach humans a lot because they're gonna think and see and process patterns different than we are. And I'm really looking forward to kind of seeing the insights and the connections and everything that they share with us.

That sounds very cool.

Yeah.

Just being able to learn a bit more about yourself just from looking at it through a different lens to a human one. And Jean, Pronto, what are you excited about?

Coming back to your question, will we need data scientists anymore? When Deep Blue won the chess, well, beat the best chess player, Kasparov was quite a long time ago. People could think, okay, Chess is done. No, no more chess. But what happened is the opposite. Professional players started using computer, so it was AI and it still is AI to train. And human players became much stronger than before. And professional chess competition are more interesting. So it's the opposite. And I expect the same to happen in data science and in software engineering and so on. The quality of the output will improve and we would still have human in the loop. I'm really convinced, so I'm really looking forward. So there is a focus on coding and not as much focus on data science, but I'm looking forward to see data science assistants coming and maybe coming from us, who knows?

Okay. Yeah. Certainly the idea of having assistants to help you out with the job. Everyone's a manager now. That does seem like a promising future. At least if you like managing things or like managing.

I'm a Metatron.

You just need to make sure the AI doesn't have the opportunity to complain when you. When you're harsh in it. Harshly managing it. Okay, cool. All right, final question. How do I get my hands on one of Those fancy new 50 Series GPUs you've made?

Ask me the same question.

All right, cool.

Yeah.

Thank you so much for your time, Chris and Jean Francois.

Thank you.

Awesome. Thanks a lot, Richie.