#130 The Path to Becoming a Kaggle Grandmaster (with Jean-Francois Puget PhD)

You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization. Join us for in depth discussions with data and analytics leaders at the forefront of the data revolution. Let's dive right, right in.

Welcome to Data Framed. This is Richie. Today's guest is Jean Francois Puget, a distinguished engineer at Nvidia. In this episode, we've got two topics to cover. That means we'll be talking about doing data science with the Nvidia Stack, meaning computing with GPUs, but we'll also get into the software that accompanies them. Jean Francois's other claim to fame is that he's been in the top 10 of the Kaggle Machine Learning competition leaderboard for the last few years. So we'll delve into the world of competitive machine learning to see how to become a Grandmaster. Hi Jean Francois, thank you for joining me today. Just to begin with, can you tell the audience a little bit about what you do and what it means to be a distinguished engineer?

People believe that to get more influence, better compensation, whatever you need to move to management. While in some tech companies, including Nvidia, it was true at IBM, my previous employer, they let people grow as individual contributors. So distinguished engineer means I'm a good individual contributor. I manage a small team of individual contributors, but it does not take me a real time. So what I do mostly machine learning models, be it as part of machine learning competitions and Kaggle, for instance, to showcase Nvidia Stack and Nvidia as a brand maybe, but also for internal projects or Nvidia partners. But my job is to build good, possibly good machine learning models.

And can you give some examples of different projects you're working on? So what are you building machine learning models about?

So for instance, on Kaggle, the last few competitions I did, I can discuss openly Kaggle because it's public. The internal projects I'm working on some very exciting ones, but people will have to come to GTC soon to learn about those projects. So recent machine learning competition, for instance, was to predict some protein property from the protein sequence. And for this we use models like AlphaFold and other models that were quite hyped recently. So it's a breakthrough in computational biology. Previously it was a natural language processing, not the kind of chatgpt. It was more text classification for specific topic for medical examination. So it can vary a Lot. I also worked on diagnosing from medical images, from radio images or microscopic images. Could be also temporal forecasting. So time series forecasting, sales forecasting, or for instance here there is a competition sponsored by GoDaddy. We have to predict basically the number of business websites hosted on GoDaddy by US County. So that's time series forecasting. So you see it's very, it varies a lot. It's across industry. So the only common piece, and that's a bit surprising if you read the same mathematical technology. Machine learning can be used across a lot of use case and industry.

Okay, so it seems like you're doing a pretty broad range of machine learning projects just all over the place in terms of different technologies and different industries that you work. So you mentioned Kaggle and I'd like to get into a bit more depth about that. The top rank of competitive abilities, I believe at Kaggle. So can you tell me how you got started using using Kaggle?

I started my professional life when I was a student. I was fascinated by AI in particular machine learning. So I did a PhD in machine learning. It was a long time ago, so before deep learning, wave, etc. I always was interested in machine learning. After my PhD there was an AI winter to adjust confidential. So I moved to a startup doing something else, mathematical optimization. And back in 2008 or so, IBM acquired the company I was at and they noticed I had a background in machine learning, wanted to invest in AI and data science. So I was asked to do more than mathematical optimization. And I was looking for a way to refresh my knowledge of machine learning. So I started rereading papers, academic papers, but especially at the time, it's less true now. But academia was a bit remote from practical use. So I looked and I found Kaggle, a site where people could compete and people were using whatever tools they could to get the best possible result. And there was no preconception, as long as something works, it was used. So I started reading Kegel Top Solutions and using it for my job at IBM and by looking at what tools people were using. So I saw the emergence of Keras, for instance of xuboost, which is now very popular. But it started on Kaggle. I witnessed deep learning frenzy there, so it was useful for my job. But after a while I said maybe I should try myself. So I remember entering my first competition. I say, you will see what you will see. Guys, I'm a picture PhD in machine learning. I would crush you all. So I was doing quite well until the results on the Hidden test set or private data set on Kaggle and I dropped from top 100 to like 2000 rank. So I said, okay, I need to learn. My theoretical knowledge is not really practical. So I started learning, I enjoyed it and after a while I became one of top 10 on Kaggle under Kaggle Grandmaster. So I keep doing it even today.

That's a pretty impressive achievement being top 10 in the world out of, I'm not sure how many is it hundreds of thousands of people who participate in these Kaggle competitions.

Kaggle has 15 million users. Not everybody enters competition. So people who got the rank on competition, I think they are in the tens of thousands, which means many more entered but got no results. So yeah, it's quite a lot of.

People, but that's a very impressive achievement. And so I'd like to hear a little bit about the secret to your success. So how have you managed to get to that high ranking position?

I would say it's a combination, it's to have a scientific approach. I was trained as a scientist, I was good at physics and math in France, I achieved the best possible math result. That's good. I even go to physics Olympiads representing France. So I have a good scientific background. And the scientific method is, I could say in a nutshell, is you check your assumptions always. So you have assumptions, you design experiments such that the result will tell you if your assumption is right or wrong. And I do. I strongly believe machine learning is a science. It's an experimental science, like physics, like parts of physics. And so I approach a competition with a scientific approach and everything is an experiment. So for instance, if I have an idea of a new data processing, a new feature or a new model architecture or whatever, I must have a baseline, something I know I trust. And then I run an experiment with a new data processing with a new model architecture, change, whatever. I run my experiment and then I look at the result and is it better or worse? Sometimes I dig a bit in the output to understand where it is better, what is worse, and from there either I accept the change and it becomes my new baseline, etc. And for this you need to have a good, what we call a good cross validation system. So the bread and butter of practical machine learning is cross validation. So can be K fold cross validation. If it's time series, it's temporal cross validation. But the point is you split your training data, you keep some of it to validate your model, you train on the rest and when it is trained, you predict on the validation and you compare with the ground truth with the target and the validation. And K fold means you do this k time with k different splits. So there are variants, but really you don't evaluate your model on the training data. That's the most common error I see. It's surprisingly common. And that's something Kaggle teaches you where to evaluate model square. So the real point is evaluate your assumptions, evaluate whatever modification you make to your code. So make one modification at a time. That's also something I've seen people modify three things, it's better but maybe one is detrimental. Or they modify three things, result is worse, they will discard those three things. Maybe one of them was good, but it's offset by the other. So you have to be consistent to run experiment. And if you run experiment correctly, record the settings, you can reproduce what you do. That's also important.

I really like this idea that you should treat machine learning as an experimental science. I think quite often you find that people, they learn about things like AB testing in a statistics class and then machine learning class is separate and they don't reply those ideas that actually I should be doing experiments when I'm machine learning. So I really like that idea. And you mentioned that things like cross validation are really important.

So there was a course I was recommending a lot Andrew Ng course on Coursera, but it's outdated now it's with Matlab. But still he was teaching how to evaluate models and I saw people just forget what he says because they were using a different kind of model, say a deep learning model. I said, you've taken this course? Yes. Why don't you use cross validation? Oh, does it apply to deep learning? Yeah, it's a methodology. It does not depend on the type of model.

That is interesting. Once you switch from just regular machine learning to deep learning, people forget all the stuff that they learned in the original machine learning models. Do you have any tips for how you go about winning at Kaggle? Like what are the techniques or things that you use in order to get better predictions?

Sure. If you have a good cross validation setting so you can rely on what you do, the next thing to avoid is overfitting. Even if you use cross validation, if you use it a lot, same splits over and over again, you end up overfitting to it. So you need to use to be conservative. Make sure you don't select something just because it was lucky. So there is a tendency of people Kaggle offers you public leaderboards. So it's a trend test split fixed one across the competition and people rely Too much on this fixed trend test split so they will overfit to the public test. So I almost never use a public test on Kaggle. Almost never. Or let's say I use it as an additional fold. If I use a five fold cross validation on my training data the public test is a six fold but no more. And the second is to have no preconception. And so quickly create a baseline, a linear model usually if it's tabular data, a simple CNN if it's a computer vision just running a pre trained transformer if it's nfp very quickly have a complete pipeline where you can submit and create a solution and then you improve gradually and you have no preconception. Always wonder, well in all I've said, oh, I have this parameter fixed. Why not try to vary it? Why not? Don't be shy. I see people, they ask in the forum, do you think this could work? Don't ask, just try it and see what happens. You will learn something always. So it's really good. Performance is just from a solid use of scientific method. Sometimes people have a great idea that nobody else has. It happens. It happened to me as well from time to time. But that's less and less frequent because the level of people is increasing the knowledge. For instance, I did well in NLP competition because I started using prompts. There were some papers coming with prompt engineering. There was an NLP competition at the same time. So I just did some prompt engineering before it came really known. So that was a good advantage. But I would say the key is to perform the right experiment. And what does it mean? It depends on the competition and you get some. That's a knowledge. The training we get from kl I would say so practice, don't be shy. Test your hypothesis and be conservative.

That's really interesting. The idea that it's very difficult to predict which theory is going to give you the best result. So the only thing you can do is just try lots of things and see what does the data show. Are the results actually any good or not? Because that seems very different to a lot of sciences. But I'd like to talk a bit about the Kaggle Grandmasters of Nvidia. So this is your team of Kaggle competitors I presume. So it's not just you, it's a group of several of you who are competing together.

Yeah, we are called the kgmon. So our CEO found the acronym and it sounds like Pokemon. So it's not by chance. So our Kaggle Grandmaster of Nvidia. We are eight of us, plus me. So nine. So it's not a big team. There are about 150, I believe, competition, Grandmaster. So there are not many companies. There are a few companies having a Grandmaster team as well. So it depends on what you want to achieve. But I do believe in small teams of very good people and they all do the same as me. It's like people having a PhD. So I will come back to this, but Kaggle Grandmasters, they know how to work effectively, otherwise they would not be Grandmasters. So having good work habits is key, which means they don't need much management. So I don't see myself really as a manager, coordinator maybe, but most of my job is individual contribution and they all do the same as me, a mix of competitions and internal projects. I want to come back to PhDs. I believe the one thing people learn during a PhD is autonomy. Good PhD does not need to be told what to do every day. And they know how to complete a complex project in the end. And Kaggle competitions are the same. There are complex projects, time box, usually three months. And to do well, you need to complete your project on time. So that's also something that is good about Kaggle Grandmasters. They work fast and they meet deadlines.

So I think a lot of people listening are going to be thinking, that sounds like a cushy job, being able to just participate in Kaggle competitions while I'm at work. And they're going to want to know how do I get this for myself? So can you talk about how you persuaded management that you do this as your career?

When I started kaggling at IBM, maybe I was spending on average one or two hours of work every day on it, which is already good. But most of my Kaggle time was evenings, weekends, holidays. It was a hobby, a passion. So it's like people going to casino for gambling. I believe it's the same. It's a legal drug, except you don't lose money here. And to become a Grandmaster you need to spend a lot of time. It's fierce competition. A lot of people, they became Grandmaster because they are students, PhDs in machine learning usually. And once they are Grandmaster, they get a job and they stop gagging because they don't have enough time later. And when I was hired at Nvidia, I remember I was seventh on Kaggle. So I became good at Kaggle before it became my work. So I would say just invest time. If you have the skills and the motivation, become a Grandmaster and then you will find jobs like mine at Nvidia or at few other companies. So obviously I answered the Nvidia job ad with Kaggle Grandmaster as prereq. But I see on LinkedIn for instance, we see from time to time companies asking for Kaggle Grandmasters.

And what does Nvidia get out of you being a Grandmaster? Like what's the benefit to the rest.

Of the company in a competition? Someone shared a notebook that accelerated Pandas and Pipeline using Polar and this and that. And I looked at it, I said let's see what we can do on gpu. So I use Nvidia so Cuda Data Frames part of Rapids. I used QML knn. So I used Nvidia or Nvidia Stack, recorded our notebook and got I believe 17x speed up. So as a result people know that if they use GPU they will get better performance. If I had not done it, people would say oh, if Pandas is too slow, just use Polar. Which is an interesting advice. And yes, Polar is more efficient in general than Pandas, but QDF is even faster. Another thing we did, there is another competition, medical imaging. So DICOM image, it's a standard format in medical image and in the competition only people were not using GPU to decode the images. But Nvidia had a toolkit. So some people on my team, they tried the Nvidia toolkit so that it did not handle some of the formats. They worked with Nvidia product team and last month they released a notebook with an early access of the new toolkit. And as a result images can be decoded on GPU in Kaggle and same the speed up is at least 10x. I believe it's more than that. So we showcase Nvidia tooling.

That's really interesting and actually I'd love to talk more about the Nvidia tooling. So of course GPUs are perhaps your biggest line of business. So can you just tell me a bit about what sort of machine learning tool problems are particularly suitable for a gpu?

I would say so if deep learning is the way to go, think of gpu. That's your first advice if you're in computer vision. So image classification or object location, video tagging, whatever, NLP, since burp paper, since transformers took over, it's again deep learning with some pre processing called fast Fourier transform. It's unable to computer vision models. So for these three class of data, which people call unstructured data, usually using an accelerator and especially GPU is a way to go compared to CPU for tabular data. So say you have Cells, you have past sales and you need to predict the future. I'd say depend on the size of data. Sometimes you have people, they have 100 location, they have five years monthly data. So it's like 60 times 30 data point, 60 times 100 data point 6000. If you use XGBoost for instance, you may not need GPU, that's fine. So small data, use whatever you want. But for tabular data, for instance, a recent competition, it was recommender system, we had 18 million user, 1.8 million products and 100 millions interactions running. So doing data processing and modeling Bitex, Zuboost or something else using rapids on GPU, the speed up is enormous. It's 50 or more. So again, if we go back to what I was saying, the key is to perform experiments quickly and effectively. So as soon as you can accelerate with gpu, you will run more experiments. So within a day you will test more hypothesis and you will make progress much more effectively.

So it seems like most of these examples you gave where the GPU is going to be faster. These are examples where the code can be easily parallelized. So you're doing multiple things independently, is that correct?

Well, GPU does, the frameworks do it for you. So for instance, let's say even for data processing, if you use Pandas and you want to compute, I don't know, one column as the aggregate, but you do a group by. For instance, you want the mean of the spending by user. In Pandas you would group by users and it compute the mean, but it will iterate through the the users one by one. So it's sequential. With CUDF and GPU it will be run on parallel for you. So you don't need to write a parallel code. The code is parallelized. So this way you can get 100 times speedup just because it will process hundreds of users at a time in parallel. So that's how you get the speed up for deep learning. The bulk of computation is matrix multiplication. Tensor multiplications and GPU are designed to do this in parallel. So they chunk the memory and do the multiply of two parts of the matrix in one cycle. Cpu, they do have some parallel, but GPU are massively parallel. So when you can use massive parallel, GPU is a great idea. XGBOOST is the same. Most of the computations can be paralleled on GPU. So you just select GPU hist. It's one parameter in XGBoost. And your code runs on GPU using the GPU parallelism, but you don't need to change your code.

That sounds like a pretty useful thing is like not having to write completely different code when you're switching to GPUs. So the Nvidia software stack for doing all this data science and gpu. So that's Rapids. And can you just tell us a little bit about what you can do with Rapids and who it's for?

So I'd say Rapids is fairly comprehensive and the motivation was to get a GPU accelerated version of Pandas and scikit Learn. So you have a package called CUDF CUDA Data Frames, which is similar to Pandas, except the data processing is done on gpu. But the API is really similar to Pandas to the point that now when I have a Pandas code that is too slow, the first thing I do is import CUDF as PD and then I run my code and most often runs as is. And we are working with Rapids team to reduce a case where behavior is different. And for Scikit Learn, the rapids equivalent is called Quantum. So CUDA machine learning. So not every algorithm is implemented yet, but a lot. And the API is really similar to the point that QML documentation refers to scikit Learn documentation. But the goal is really that it's easy to translate pipeline that uses Pandas and Scikit Learn into a pipeline that use PDF and query. And then over the recently many other packages have been added to Rapids like Cusignal, Coolgraph. I have less experience with these, so they are a bit more specialized, but they are tooling. So always the same idea is to see what is needed to move pipeline from CPU to gpu. For deep learning, there is no package, no framework From Nvidia because TensorFlow, PyTorch and others did the work correctly. So we support these frameworks. There is a backend called CUDNN that this framework use, but users don't need to worry about it. Personally I use Pytorch, I know it uses CUDNN under the hood, but I just use Pytorch. So for that reason, given the deep learning framework, we're already on gpu, Rapids itself is not dealing with deep learning, but we know and it's part of the feedback we gave the Kaggle grandmasters that often it is useful to combine deep learning with other machine learning models. So work has been done and recently UDF team has released a way to share memory between QDF and Pytorch so you can prepare your data with QDF and when it's ready, it's used by Pytorch without Memory copy and it's all on gpu, so the full pipeline is on gpu.

All right, so it seems like QDF is perhaps the most interesting part of RAPIDS for data scientists and machine learning scientists. So it's a high performance Pandas alternative. But there are about a dozen of these different high performance pants alternatives around. So how does QDF compare to things like Vex and Modin and Koalas and all the others?

Well, some of these are distributed, so they get speed up by distributing computation. Because for people who listen to us, especially data scientists, some may not know yet they soon learn it as soon as you use Python, that Python is monothreads because of something called global interlock Gil. So Python is mono thread, which means that if you want to use parallelism in Python, either you call say a C or C library that does it for you, like Cuda, or you implement multiprocessing or you distribute across machines. So there are some distributed data frames. And you could have mentioned Spark as well. Our preferred way at Nvidia is called Dask. So Dask is a distributed computing system, a bit similar to Spark, but it's more Python friendly I would say. And there is a Dask UDF for those who want to distribute. And one reason to distribute is when your data is too big to fit in one machine memory. And GPU memory is increasing, but it's limited still. So Dask UDF is a way to distribute data processing across multi gpu. And then when it comes to benchmarking, as I said, each time I tried PDF it was faster than anything else because really GPU are so powerful, so massively parallel that it's really hard to compete. The only thing that would limit application is the fact that the GPU memory is limited. So for this looking at that SPDF is a way. But that's I would say if you can fit in the GPU memory in the memory of your GPUs using DASQDF, it's hard to beat.

So it seems like QDF is a pretty high performance thing and maybe worth checking out if your Pandas code is running too slow. But I want to circle back now to talking about your Kaggle competitions and how it relates to more standard machine learning work in a business context. So do you find there's a difference between competitive data science and machine learning at work?

Yeah, sure. And it addresses some valid criticism of Kaggle, which is that at work you maybe not just a data scientist, but the company, the organization using machine learning must cover a full life cycle that starts with framing a problem as machine learning, gathering data for it. Since most of the applicable machine learning is supervised learning, you need to annotate this data to get training data. Then you have data procuration, modeling, model evaluation. And once models are evaluated properly, you put them in some production system or behind a dashboard or whatever, you connect it to an E commerce site for recommendation, whatever your use case is. And then you need to monitor the model in production, detect if performance is going down, which means may mean you need to retrain because something has changed in your environment. There is a full life cycle and Kaggle does not cover all of it. When you in a Kaggle competition, you have curated data, you have annotated data, you have a metric, so the problem is already defined for you. And once you've trained a model, you submit predictions to Kaggle or you give your prediction code that it's applied to to test data and that's it. So you don't deploy, you don't need to worry about downstream. So Kaggle is only part of the machine learning pipeline, but for this part it teaches you the right methodology, which is what I explained before, experiment based, et cetera. So I would say Kaggle is great to learn about modeling and model evaluation, but it's good only for this. To someone who never worked in real life and only on Kaggle is not a full fledged data scientist. People need to get experience in, okay, how do I even apply machine learning to this business problem? Where do I get the data to working with people, how do I annotate it, how do I get labels and downstream as well? So downstream is more understood. I would say there is this ML engineer profession that has emerged that can operationalize the model so we find more and more ML engineers. But I would say the upstream part, framing the problem as machine learning, getting data, reliably curating it, et cetera, it's still a kind of art and maybe underestimated at this point.

So that is interesting, the idea that the competition only focuses on the sort of the middle part of the machine learning lifecycle, around making predictions. But you don't get the start bit about framing the problem, collecting the data, and the end bit about how do I deploy this or how do I actually use the model. So it seems like a big part of this is about not having to align your model with some kind of business goal. So do you have any advice for people on how to do that? Like how do you make sure that the machine learning work you're doing is going to align with some kind of business objective.

That's a great question. And actually when I'm asked to help on a machine learning project, if I'm not at the start, I ask people, suppose imaging, assume that your model is perfect, it makes perfect predictions. How do you use it? And for instance, if it's forecasting, you can play back. So assume you had perfect predictions. How would have this impacted your business? How to use a perfect prediction? You predict exactly the target, what would happen. And not surprisingly for me, but most of the time people have no clue. So I said, you need to design your business process, tooling, whatever, so that it can consume the output of your model. It's straightforward. For instance, I've seen once I used to be active on Twitter, but I remember once saying they work at a pharmaceutical company, they don't say which one. And they worked based on feedback about one medication produced by that company to predict when the medicine was most effective and they did a good job. So with their machine learning model based on a patient features, the model could predict if it was worth using the medicine or not. So it could be a good help for medical doctors. When they presented the result to their management, the project was shut down. So I guess it's because they prefer to sell the medicine even when it's not effective. So I'm not going to discuss the pharmaceutical industry incentives, but I want to point that the people working on the machine learning project should have asked, should have present, should have asked the stakeholders, what if we succeed, how would we use a model? Is it worth doing? And maybe someone would have said, no, we have no interest in doing this. Instead they spent one year with team, so just check that you're doing something useful when you start, not when you're done.

So that leads to an interesting point about like, how do you measure the success of a machine learning project? So I think like the Kaggle ideal is machine learning works best when you have the best predictions, but in real life that's not always the case. So can you talk about what constitutes success for machine learning?

Yeah. So in Kaggle, most of the time what matters is how good a metric can become on the test data. And this leads sometimes to complex solutions with lots of models being ensembled and several stages and whole stacking and it's too complex to be used. So KEGR is trying to limit the complexity. But in short, you want to balance the quality of the predictions with the cost of maintenance, the cost of implementation. So you want, you prefer to have one model that is a bit less performant than the complex ensemble you could get on Kaggle, but which is simple to implement, simple to retrain, you can maybe automate everything, etc. So complexity of the model, complexity of training the model is a key factor. The other point is the metric is a proxy to the business problem. So it's not because you get a good metric that you improve your business. So let me give you an anecdote that I read. I don't know if it's true or not, maybe it's too good to be true, but someone I know claimed to have worked on a support organization and did a customer churn problem for a subscription company like a telco or TV or cabin or whatever. So his model was predicting which customers were most likely to not renew subscription. So what they did, and this is a classical example you see in many machine learning textbooks. So then they say, okay, let's run it on the customer base and we'll have the call center, the support team, call the people most at risk to propose them an incentive, a rebate or what have you. The problem is many calls went like this. So hello, Mr. Customer, I'm from Company X.

Great.

I wanted to cancel my subscription. Let's do it. So in fact they accelerated the subscription because they targeted the right people, but not with the right answer. So this is an extreme, but it really highlights what I said. Assume your model is good. How do you use it effectively to improve the business? The other thing is to measure. If you don't know up front you have or do a B testing as you cited before, say you run the previous process for alpha whatever thing you apply to your users, your machines, your whatever and the other half. You use the process with machine learning and you monitor and you see if there is a difference and in which way. Hopefully the part with machine learning works better so you would use it more. But always keep a small fraction without machine learning so that you can see, you can detect if to point. The machine learning system no longer works well. And this can happen if the underlying conditions are changing. So monitor what happens. I've seen a presentation in an industry forum, someone right before me and they were describing, I believe it was a recommender system and the results were not good. But they only discovered it like nine months after deployment because they were not monitoring. And as soon as they started monitoring they noticed that the sales of promoted items were not increasing. And in fact they did not include promotions in the training data. So the model was insensitive to promotions. So it predicted that some products were popular for the wrong reason. They were popular because they were promoted. But the system did not have the data, so it invented a reason for which that would explain why the product were popular. It was overfitting. Once they noticed it, they retrained the model with past promotion data and all of a sudden sales started to increase. But they had to monitor and see in practice. So it's same as I said, always check your assumption. You assume you have a good model and you have good reason. You have done cross validation or all of that. Check that it is really good in practice.

Okay. And those are two really great stories of machine learning disasters. The one in the one about Churn in particular is really terrifying to me. At datacamp we're primarily a subscription business, so customer Churn is something that we live in terror of. So the idea that you could do a machine learning project and then make it worse is absolutely horrifying. So I'd like to talk about productivity a little. It seems like, particularly with your competitive machine learning background, you've got good at doing models very quickly. So do you have any productivity tips for how you do machine learning faster.

Key is to have a modular pipeline that is easy to maintain, to modify, to log. So now I'm used to log things, to have something modular, controlled by configuration file. It's so it's. I would say it's standard software development practice. But data scientists are not developers, so that's something I really believe is true. And for those that have no experience in software development, they have to learn it. And unfortunately it's a bit the hard way. There are no programming course for data scientists that really. So people need to be able to version their code to have a clear distinction between configuration and baseline script. So all sorts of things. But the goal is to automate as much as possible. And then there are tools like Weight and Bias or NEPTUNE AI that help you track experiments, for instance. So there is more and more tooling that comes. But really the goal is to automate most of the things and focus on your idea. You have a new idea, you should just write a bit of code or change some configuration, run it, get the results logged, easy to compare with other experiments. So the key is really to remove the need to do manual work. Manual meaning typing, to get a new result. So for instance, I start with notebooks, they are very good, but as soon as possible I move to Python scripts with configuration files because it's faster to iterate.

Okay, so removing any kind of manual tests, trying to automate as much as possible. That seems Like a great productivity tip. And do you have any advice for if you have to do machine learning projects in a very short time? So how do you do like very fast projects that just a couple of days or maybe a week or two.

So if it's only few days. So if you're clear, it's amazing what you can do in four hours really. If you only have few hours, I would use an automated tool. So for instance I've tried one called Auto Gluon, I think it's from Amazon, it's quite impressive but it's not the only one. So AutoML, if you have few hours, I would go with an AutoML tool. If you have weeks, then you beat automl because you can include additional data for instance that is relevant. You can include domain knowledge that an automl system cannot devise so you work more on the data, etc. So it can be automl. But if you only have few hours I would run an automl and if it's tabular data I would even start simpler. I would run a linear regression or logistic regression. If I have a bit more time I would run exiboost. If you have few hours you can do something complex, so use a simple model.

So start simple and if you have more time you can always make things more complex later on. I'd like to talk a bit about collaboration since that's a big part of productivity. So do you have any tips on doing machine learning as a team?

There are two cases. One is when we have to deliver a common code and the other is when we have to deliver predictions. The second case is more for Kaggle where you don't care about productionizing your model if you only need to ship predictions. We collaborate by exchanging data. So data sets, prediction on these data sets. If it's a common code, we have to use something like GitHub, GitLab and use software engineering technique. For communication I often use Slack because of time difference my team and Kaggle teams and I always work since with remote teams. For instance, on my team I have one person in Japan, one in Germany, two in France, one in Brazil, three in the us. I hope I'm not forgetting someone. I would be crushed but you see. So time difference it's hard to have everybody on a web conference. We do it but not often. So we rely on asynchronous communication. So put commit, upload, download in a directory and Slack Slack people can use other but the point is it's asynchronous so we write our IDs our result. The other guy comes and read later respond. So it's like a remote development team, like an open source project. It's quite different from some dev organization where everybody is in the same office have been used before the pandemic. So after the pandemic more people are working remotely, but that's what I've been doing. That may be why I did not need to relocate to Silicon Valley, being able to work remotely.

Yeah, it does seem that communication is just a huge part of productivity. And having this idea of asynchronous communication where things are written down is incredibly important, particularly when you're in a remote team in different time zones. All right, before we wrap up, I'd like to talk about conferences. So we've had a bit of a clash since both datacamp and Nvidia have data science conferences going on with a partial overlap in dates. So the datacamp conference is called radar, that's on the 22nd and 23rd of March and Nvidia has a rival GTC conference with a few dates overlap. So can you tell us a little bit about what's going on at gtc?

Yeah. So GTC is a semi annual, it runs once in spring, once in fall. That's Nvidia conference. So we have keynotes by our CEO, he usually announces new products, services. Then you have tons of sessions by industry, by use case, more or less technical, ranging from research to very applied. And we do have a couple of sessions from us from kgman. So if you're using GPUs attending GTC. So to avoid clash with concurrent sessions are always available in replay. Well, first gtc, the main conference is free and you can register watch when you're ready. So for instance, being in France, I can't watch everything live. I just use replays for what I'm interested in. But it's really how to get the latest news. And it's not just on data science. As people know, Nvidia is great for gaming and other use of gpu. So whatever your interests, if you use gpu, that's the configuration that actually seems.

Like a good sort of practical diplomatic approach if you stuck between trying to decide which conference to go to, since they're both virtual, you can register for both and then watch whichever sessions appeal to you on the recordings later on. So just to finish, what are you most excited about in data science and machine learning?

The meta excitation is it's evolving so fast you have to learn all the time. So I like it. I don't know what will be hot next year. I don't know what is doable. So there is a frenzy about generative AI, etc. So I'm less into that. Not sure why. So what excites me. It's a progress. I spoke a lot about rapids, but this year I used it more than before and the speed up are incredible. So that's one thing, the other is the ability. When I started there was a clear divide between statisticians, classical code, unquote, machine learning, deep learning. And now this is the barriers are being removed. Maybe because everybody moves to using Python. So it's great when we see unexpected use of one technique to a place where it was not used. For instance, a colleague of mine won an image classification competition without training any deep learning model and running SVM regression, Super vector machine regression. So he ran super vector machine regression on the predictions of deep learning models without training any deep learning model. That's surprising. So I know I will have surprise all the time. That's what I like.

That's a great answer. And I do think that having people pushed into different situations, like people moving to Python from a different language or coming from statistics to machine learning, they do show up. Lots of interesting opportunities and innovation. All right, super. Thank you for your time. Lots of really great insights and yeah, thank you for coming on the show.

Jean Francois, thank you for inviting me. Enjoyed it.

You've been listening to Data Framed, a podcast by Data Camp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment and share episodes you love. That helps us keep delivering insights into all things data. Thanks for listening. Until next time.