Five Pillars of a Successful Machine Learning Project with Nebius at SaaStr AI Day

um, each unhappy family is unhappy in its own way. Actually, this, this could be used for the project as well. So all successful projects, they actually have common thing, you know, they have a common things that they really looks the same. But they're all failed projects, they failed for their own reasons, and there are a lot of reasons might be for that. So let's stay on the reasons why the projects became successful. And I was actually doing machine learning for a long time, 6 years in AWS and now in Nibius. And for all those time, from, you know, in that time, all those years, I found like 5 most important things that usually make projects successful. So looking at the agenda for today, it's really short. First of all, let me introduce Nibius because I believe A lot of people don't know who is Anibius and what we are doing. And then the main topic where I spend most of the time is 5 pillars of successful machine learning project. And that's just a really, really short overview of the Anibius Cloud. So let me start from the who we are. As Anibius, we have a mission that we frame for ourselves that we want to create platform, and we want to provide AI practitioners with the most efficient cloud to build, tune, and run AI models and applications. And this is what we are working right now, and this is what we are delivering to our clients. And we actually have quite nice clients with us. You can, you can, you can see them like Mistral, JetBrains. So most of them, they're AI-centric companies and the companies working in AI space. But what, what actually is Neebus? So first of all, it's a really geography-spread company. So we have a headquarter in Europe, in Amsterdam, but we also have offices in the US. We have offices in other places in Europe, and we have a data center in Finland. We have data center in Paris, and we are building the data center in in the US. So this is about the footprint and about who we are in terms of the people. So we actually like more than 500 software engineers from writing full-stack cloud, and not only software, also hardware engineers. If you, if you look on what exactly we are doing inside, so there are 5 most important things that we're building. First of all, we are building data centers. So we're building from scratch. For example, data center in Finland, uh, is not just a data center. There is a lot of science behind it, and one of the results is actually the data center is hitting the village nearby. Then we're building the hardware. We do not sell the hardware. We're using that hardware to build our platform, and we are doing it also that in a unique way. We design everything from the firmware to the hardware. And we do a lot of physics to simulate how the air flows inside. If you want to learn more about it, we have a video on YouTube about how we design Solus, and we actually open source design. Then third, we are building the cloud platform where everyone, regardless the expertise, could get things done. Do you want just to run a model, or you want to train a model, or you want just to get a model through the API? You can find it there. And we have really big AI expertise. We actually have our own R&D team who is building the models and building interesting approaches, and we open sourcing them. And then we are actually big team, as I said, of the software engineers, hardware engineers who are building different things. So this is Anibius in a glance. As I promised, I took just 5 minutes. And now let's jump to the main topic. Let's jump to the 5 pillars of successful machine learning projects. So in general, those pillars applied to any machine learning project. So it's not just GenAI. It's actually— it was true in the time when the XGBoost was main algorithm, and it's still applied when we're talking about the GenAI. So we have 5 of them. First of all, be owner of your data. Second, have a skilled and collaborative team. The third, you need to have a good communication with stakeholders. Fourth, you need to select project wisely. And the fifth, you need appropriate technology stack. So let's jump to the first, the data. You know that, that I'm pretty sure you heard that, that in ML space, there is a golden rule. Which says garbage in, garbage out. That means if you don't have data, if you don't have good data, you probably won't succeed because any machine learning, as previously analytics, big data, starting from the data. And in the data, we say that you need the dataset. But here's all, you know, it's often when people asking like, where you can get a dataset. And here I could say that you can ask yourself 3 simple questions about the data. This is where you need to start, like what kind of data you have today. The second, if you don't have data, what could be easily collected to find? Maybe you can find them in open source data. Maybe you can find them in open, already prepared dataset and just filter it. And combine them. And the third, I think one of the most important, if you don't have data today, but you already see that you probably want something, try to understand what it is. So you can, you can ask yourself what exactly I could start to collect today. So when tomorrow will come, I will have data. That's, that's 3 questions that you can ask yourself. And when we're talking about the data, There are, let's say, there are 3 main features, characteristics of the data. So first of all, the data should be available when you start the project. It means that you need to have sufficient relevant data to train an effective model. The second is about the quality. So it's not just that you need to have like terabytes of the data, it's more about that you need to clean, curate, and label data if we're talking about, you know, training when you need to label data. And actually, latest, you know, models in LLM space prove that, that you can train with less data, but if data is higher quality, you'll get a better model. And the third thing that is important about the data is also that the data should be diverse. So if you want to have a strong model, you need to think about that you need to have multiple classes. So you, data need to be diverse. Otherwise, data will just learn specific pattern and will not be able to work in a continuously changing world. So the model should be generalized and to build that such model, you need to have diverse data. Few more things about, about the data. You know, depending on the size of the company, it's actually, you might have a data in a different place, different department. Some data could not be used by everyone. So the reason, the way how you can solve that, that challenge is actually you can build the data exchange platform internally to exchange the data between the departments and share and maybe pre-cook some data, pre-clean some data so everyone could reuse. And actually, that will speed up the process. And, you know, the last thing that I want to mention about the data, it's not a single, you know, single time process, not single, you know, it's not a single task, single one. You cannot run it just once. The data is changing. The world is changing continuously. The data around us is changing. That means you need to continuously collect the data, continuously clean it. And what is really important, you need to monitor. There are a lot of ways how you can monitor. For example, statistical, you can find like data drift. If you see that the data is changed, some metrics of the data is changed, that actually means that you need to retrain your model. And that, and this is the one way how you can actually reduce the cost. If you will retrain the model not just once a month, but if you will retrain the model when data change, that will be more cost efficient. So that's, that's about the data. Just want to give you like really one short example about from the, from the real life about the data. So, as I said, we have a team who is actually building the models, building the systems. And recently they were thinking about creating the agenting system for the automated software engineering. So It is not about that, you know, the model will just finish the code. It's a little more complex. You want to be able model to solve issues in a code. That means the model needs to understand the problem, the issue in a code. It needs to be able to propose the solution to write the code. And then it has to test the code in a specific environment to validate that the, you know, the patch, the code that model produced actually solving the problem. So it's a little bit more complex. And one of the things when, when they started to build that, they realized that there is a lot of data exists already, but all of this data, most of this data is actually missing comprehensive context about what's the execution environment, what was the result of execution after this patch was introduced. So they need to find the data. And this is what, what we, they started to do. So they become the owner. They decide, okay, we need the data. And they start to find a lot of open source data, clean it, patch it, concatenate. If you want to learn more, as a result, by the way, they actually achieved 40.6% on one of the benchmarks for the, for the, for the such system for the code generation system. And this is And that in that time it was state of the art for the, for the open source models. So they achieved that with just open source models without any commercial model usage. If you want to learn more about how they cook the data or how they train them, the system, I put 2 QR codes to the 2 blogs, articles in our blog. So if you want to learn more about it in details, please feel free. And by the way, we open source the data. If you want, you can download those datasets on, on Hugging Face. So the second thing is, is about the team. Here is several important things that I want to share about the team. First of all, it's about, it's about the main expertise in the area where you're working. So imagine that you decided to build a model for the creditworthiness or parametric identification system or any other type of the model. I mean, like, you as a company, as a business, it's not enough just to find a good data scientist because in the end, uh, people, data scientists or data engineers, they have no idea how business operates. So they have no ideas about, for example, financial rules or rules that you, the company and bank might have. So they need some domain expertise to actually prepare the data in the right way. And this is the first thing that you need to think about the team. So you need to have a collaborative team. So your data scientists, your data engineers must talk to business and to the people who actually owning the business process to understand how it works. And the second thing is about actually expertise, machine learning expertise. So usually, with, you know, when you do not just MVP, when you're not just trying, when you actually want to build a really good project, you usually need at least 3 people in a team. One should be data scientist. The second should be data engineer. And the third should be person who right now we call it MLOps engineer or AI or machine learning engineer. So the people who could operationalize the model, who can move the model to production. The data scientist will make sure that the project actually still achieving business goals with the available data. The data engineer, he will help to transform and obtain data to make the model stronger. And the MLOps engineer or the person who move, regardless how we call it, call him or her, who will move the model into production will make sure that the actual model is working and able to, you know, to serve clients. Because moving model from the Jupyter Notebook or any other place where you build the model to production with the real traffic, with the real SLAs is absolutely different stories. And that's why you need a separate person for that. But usually there might be a question. So if I don't have those skills, if I don't have those people, where I can get them? Honestly, when we say team, it doesn't mean that the team is strictly inside of your company. It could be external team as well. So you can extend your team to, you know, by including other people. So it could be partners, it could be cloud providers if you use cloud services, because some of them have solution architects who would help you. Some of them have professional service that could help you. And don't forget about like different academies and the learning portals, though, learning platforms. They could actually help you to grow team, to grow the skills of the team so they will be able to deliver the project. And here I probably also want to share one example from the life. There is a company called Recraft. Right now they have one of the best model according to the, according to different benchmarks. So they're building a tool for the professional designers, illustrations, and marketers. And we were working together. And actually, I just want to highlight the full story. If you want to read also in the QR code, you can read the full story. But I want to highlight just two outcomes that most important thing that mentioned in this blog. So the first one is to actually build a model, let's say to build a product. It's not only about the hardware, but it's also about how you could solve the issues about this hardware. So regardless where you get your resources from the local data center, your own data center, or from cloud providers, you need to have a connection with this team. Otherwise, if something doesn't work, no one, you know, you need to find a way how to solve that. And the second thing, you know, there could be a lot of problems. And if you If you, if you read the Recraft blog, you'll find what kind of problems we have during the training of the model. There was a problems with the software, different things, but how we managed to solve it, we actually were working with the same team. So the, our team was connected with the same communication channels with the Recraft team and working as a single team. And that's the reason how they were able to solve everything successful and really fast. So this is about team. Skilled collaborative team means you need to connect people and find right skills for them. Let's talk about the third, the stakeholder communication. So with a stakeholder communication, it's more about that you need to have an open line, open line communication with all stakeholders and the team who is building the project. And I could say that there are two really important things that you could take from the stakeholder communication. The first is expectation management. So when you starting the project, you need to set realistic expectation to prevent misunderstanding in the future. In other words, you need to have a well-defined business problem that you want to solve. So when you have that, it means that you will, you will answer the questions. You will have an understanding, you know, how this project will increase the revenue or impact my customers, increase customer satisfaction, or bring new functionality. That's the first thing. When you have a connection to stakeholders, the ML team, a team who's doing ML project, actually could always verify that they're on the track. To the things that should be achieved. And the second is more about the internal communication from the leadership to the entire organization. Because ML space, while we see that a lot of problems, a lot of tasks could be solved with ML, still, there is a huge chance that something will go wrong, that experiment will fail, that the targets will not be achieved, business targets. And here is really important to have a communication that, you know, that machine learning adoption could be, it could be important that it could change. Because of that, people feels that they, they need it, that, you know, that the company need those projects. And if something goes wrong, there will be another project and they will have internal power to continue and to do the next project. But, you know, we were talking about communication skills and data. So, and I multiple times said the project, and the project selection actually is super important because you need to find a really good balance between selecting the project that makes sense for business, that is not really difficult and not really simple. And there are multiple ways how you can select a project. I saw really nice framework for that, and I want to share that framework with you. It's, it's really, really, really simple framework. Oh, yeah. So what you can do, you can take all your projects that you have in your mind, and then you can put them and evaluate them from the 3 perspectives. The first is data availability. Do you have data for that project? Second is the impact on the business. And the third is machine learning. You know, was this project released, solved by machine learning in other areas, in other companies, or in open source? And you can put like from 1 to 9 for all of that, and you will have a table. So there are several rules. You know, first of all, it's obvious that you want to select the project with the highest total number. But you need to think, but you need to be careful with a, you know, border situation like Project 1. There is a lot of data and there is really good ML solution for that. We can find a lot of examples of the such project. But for your business, there is no impact. It doesn't mean, you know, it means that you will build something that doesn't make any sense from a business. So you will be successful, but no one will understand why it needed. And then vice versa, like you might have like project 2 where you have really good big impact, you have a data, but this is really complex thing, probably no one solved it before. You probably don't want to select it as well. So in this specific example, there is project 3, which have more or less everything on average, but it means that you can, you can already achieve the target. You can find impact while solving the problem. You have a data and it's already been somehow solved on in a, in a, in the world. So that's a really simple framework how you can select the project, but I found it really helpful. It helps many times, at least for me and my clients. So That's, that's the way how we can do the selection of the project. And the last pillar is a technology stack. So one of the, I mean, like, the most important rule in a technology stack is that you should not do non-differentiating things. If someone already created a tool, created a framework, you can just reuse it because when you start a project, you want to achieve the target in terms of the business goals. Not to create state-of-the-art technology stack. That's why, you know, if you will create state-of-the-art technology stack, you probably— your clients will not notice that. But if you create a nice functionality or nice metrics, business metrics, everyone will notice that. So that's why do not waste time on non-differentiating tasks. Look in the market, what is— what's already available in software libraries, cloud solutions, and just take whatever is needed to achieve your business goals. And when you achieve your business goals, when the project is successful, you can start to think about optimization, you know, changing different library, changing different software, hardware, and etc. And depending on the, you know, on the market, it's already a lot of things, you know, you can build it by yourself, you can get GPU from the cloud, you can get something from API models for the API, software as API. You can find a lot of in open source and you can find commercial products. So all of that will, and some of them, I mean, like you can select whatever it fits to you, but do not spend the time on working on non-deflation things. Yeah, so there is a 5 pillars that I saw continuously repeating in all projects and all successful projects. Have all of them working. Looking at the time, we actually have just 5 minutes for questions and answers. And just let me just give you like 1-minute brief overview what we have built as Anibius Cloud, and then we jump to the question. So in Anibius Cloud, we build a full-stack cloud where you have hardware, you have a lot of managed things like Kubernetes clusters, different type of storage. We have applications like Spark for data processing, MLflow for experiment tracking, and other things. We also build the Nebu CI Studio, which brings model as API. So you do not see the infrastructure to manage that. You just send APIs and get a response. And everything is covered by friendly UI, self-service, on-demand resources, and pay-as-you-go. So if you want to start using Nebu, just open Neeviews.com and there will be a button create account and that's it. You don't need to even talk to us to use all of that. It's available on demand to everyone. Yeah, thank you so much and let's jump to the questions.

Yeah, thanks Alex for going through all that and walking us through it. So we've got a question on the live stream. Depending on how you answer this, it might be our only question, but that's okay. You've been doing AI and machine learning for a long time. They want to know, where do you think this is heading next for SaaS specifically? Just with everything going on, even in the last 24 hours and week, any predictions?

That's, that's really good. And definitely, I think that right now, we have a big We have a big development of the Genetec system. So, you know, the system could not just write something or not just propose something, but actually could do complex things. Like, for example, you know, I could ask, where is my order? And that system supposed to find who I am, go to the database, query the database, find my order, check another API to find, like, where is, like, in the delivery. Company, find where it is, and tell me, hey, here is it. And this is just example of the, of the, you know, shopping experience, but it could be anything. I could say, uh, who are my biggest clients in the CRM system? The CRM system is supposed to find it by itself, uh, without, you know, me providing anything else. So those are genetic systems that could make a decision about actions that they need to take. So I think that You know, this is, this is what is rising. This is that we see huge adoption and development in this area. And I had the feeling that a lot of software as a service platforms could integrate it to make user experience faster and actually to solve the problems much faster because, you know, instead of have a static workflow developing in, in a way, if and else, you could actually have a system who can adapt and react based on a changing world around it. Yep.

Yep. Makes a lot of sense. Good prediction. And then just final question is, how can folks get in touch with you if they have a more technical question? LinkedIn or email? What's best?

Both. So we have a— we, you know, you can always find me on LinkedIn. You can write me email. You can just find the Neebus. We are trying to be as much as possible visible to everyone. And we actually have a Discord. So there is a lot of way how to communicate this. So feel free to find any of us. Yeah. And we'll be happy to talk.

Cool.

Yeah.

And we will— thanks for doing this, Alex, and staying up in Europe. Hopefully, we'll see you at Sastra. I know Nibius will be there. So thanks, everyone, to those who joined. We have one last Sastra AI Day session. So we'll see you there. But thanks again so much, Alex, for doing this.