Five Pillars of a Successful Machine Learning Project with Nebius at SaaStr AI Day

Each unhappy family is unhappy in its own way. Actually this, this could be used for the project as well. So all successful projects they actually have common thing, you know they have a common things that they really looks the same. But they all failed projects, they failed for their own reasons and there are a lot of reasons might be for that. So let's stay on the reasons why the projects became successful. And I was, I was actually doing machine learning for like long time. Six years in AWS and now in nebulous and for all those time from, you know, in that time all, all those years I I found like five most important things that usually make projects successful. So looking at the agenda for today, it's really short. First of all let me introduce the Nibius because I believe a lot of people don't know who is an ebius and what we are doing. And then the main topic where I spend most of the time is five pillars of successful machine learning project. And that's just a really, really short overview of the Nebius cloud. So let me start from the who we are as an ebs, we have a mission that we frame for ourselves that we want to create platform and we want to provide AI practitioners with the most efficient cloud to build, tune and run any models and applications. And this is what we are working right now and this is what we're delivering to our clients. And we actually have a quite a nice clients with us. You can see them like Mistra, Jetbrains. So most of them the AI centric company and the company is working in AI space. But what, what actually is an ebus. So first of all it's a really geography spread company. So we have a headquarter in Europe in Amsterdam, but we also have offices in the us we have offices in other places in Europe and we have a data center in Finland, we have data center in Paris and we are building the data center in. In the US. So this is about the footprint and about who we are in terms of the people. So we actually Moderately more than 500 software engineers from writing full stack cloud and not only software, also hardware engineers if you look on what exactly we are doing inside. So there are five most important things that we're building. First of all we are building data centers. So so we're building from a scratch. For example, data center in Finland is not just a data center. There is a lot of science behind it and one of the result is actually the data center is hitting the village nearby. Then we're building the hardware, we do not sell the hardware we're using that hardware to build our platform and we are doing it also that in a unique way. We are designing everything from the firmware to the hardware and we do a lot of physics to simulate how the air flows inside. If you want to learn more about it, we have a video on the YouTube about how we design solos and we actually open source design. Then third, we are building the cloud platform where everyone regardless the expertise could get things done. Do you want just to run a model or you want to train a model or you want just get a model for the API. You can find that. And we have really big AI expertise. We actually have our own R and D team who is building the models and building interesting approaches and we open sourcing them and then we are actually big team as I said of software engineers, hardware engineers who are building different things. So this is an ebius in a glance as I promised it took just five minutes. And now let's jump to the main topic. Let's jump to the five pillars of successful machine learning projects. So in general those pillars applied to any machine learning project. So it's not just Genai, it's actually it was a truth in the time when the XGBOOST was main algorithm and it's still applied when we're talking about the genai. So we have five of them. First of all be owner of your data. Second, have a skilled and collaborative team. The third, you need to have a good communication with stakeholders. Fourth you need to select project wisely. And the fifth you need appropriate technology stack. So let's jump to the first, the data. You know that I'm pretty sure you heard that that in ML space there is a golden rule which says garbage in, garbage out. That means if you don't have a data, if you don't have a good data, you probably want success because any machine learning as previously analytics big data starting from the data and in the data we say that you need the data set. But here's all you know. It's often when when people asking like where you can get a data set? And here I could say that you can ask yourself three simple questions about the data. This is where you need to start. Like what kind of data you have today. The second, if you don't have a data what could be easily collected to find. Maybe you can find them in as an open source data. Maybe you can find them in open already prepared data set and just filter it and and combine them. And the third I think one of the most important if you don't have a data today, but you already see that you probably want something. Try to understand what it is. So you can ask yourself what exactly I could start to collect today. So when tomorrow will come, I will have a data. That's three questions that you can ask yourself. And when we're talking about the data, there are, let's say there are three main features characteristic of the data. So first of all, the data should be available. When you start a project, it means that you need to have sufficient relevant data to train an effective model. The second is about the quality. So it's not just that you need to have like terabytes of the data, it's more about that you need to clean, curate and label data. If we're talking about training when you need to label data and actually latest models in LLM space prove that you can train with less data. But if data is higher quality, you will get a better model. And the third thing that is important about the data is also that the data should be diverse. So if you want to have a strong model, you need to think about that. You need to have multiple classes. So data needs to be diverse. Otherwise data will just learn specific pattern and will not be able to work in a continuously changing world. So the model should be generalized. And to build that such model, you need to have a diverse data. Few more things about the data. You know, depending on the size of the company, it's actually you might have a data in a different place, different departments. Some data could not be used by everyone. So the reason the way how you can solve that challenge is actually you can build the data exchange platform internally to exchange the data between the departments and share and maybe pre cook some data, pre clean some data so everyone could reuse. And actually that will speed up the process. And you know, the last thing that I want to mention about the data, it's not a single, you know, single time, not single, you know, it's not a single task, single. You cannot run it just once. The data is changing, the world is changing continuously. The data around us is changing. That means you need to continuously collect the data, continuously clean it. And what is really important, you need to monitor. There are a lot of ways how you can monitor, for example, statistical, you can find like data drift. If you see that the data is changed, some metrics of the data is changed. That actually means that you need to retrain your model. And this is the one way how you can actually reduce the cost. If you will retrain the model not just once a month, but if you will retrain the model when data changes. That will be more cost efficient. So that's about the data. Just want to give you like really one short example from the real life about the data. So as I said, we have a team who's actually building the models, building the systems. And recently they were thinking about creating the Agentix system for the automated software engineering. So it is not about that. You know, the model will just finish the code. It's a little more complex. You want to be able model to solve issues in the code. That means the model needs to understand the problem, the issue in the code. It needs to be able to propose the solution, to write the code and then it have to test the code in a specific environment to validate that the, the, you know, the patch the code that model produced actually solving the problem. So it's a little bit more complex. And one of the things when they started to build that they realized that there is a lot of data exist already. But all of this data, most of this data is actually missing comprehensive context about what's the execution environment, what was the result of execution after this patch was introduced. So they need to find the data and this is what, what they, they started to do. So they become the owner, they decide okay, we need the data. And they start to find a lot of open source data, clean it, patch it, concatenate if you want to learn more. As a result, by the way, they actually achieved 40 and 40.6% on one of the benchmarks for the, for the, for the such system, for the code generation system. And this is and that in that time it was state of the art for the open source models. So they achieved that with just open source models without any commercial model usage. If you want to learn more about how they cook the data or how they train them the system. I put two QR codes to the two blocks articles in our blog. So if you want to learn more about it in details, please feel free. And by the way, we open source the data. If you want, you can download those data sets on hugging face. So the second thing is about the team. Here is several important things that I want to share about the team. First of all, it's about the main expertise in the area where you're working. So imagine that you decided to build a model for the credit worseness or biometric identification system or any other type of the model. I mean like you as a company, as a business, it's not enough just to find a good data scientist because in the end people, data scientists or data engineers, they have no idea how business operates. So they have no ideas about for example financial rules or rules that the company bank might have. So they need some domain expertise to actually prepare the data in the right way. And this is the first thing that you need to think about the team. So you need to have a collaborative team. So your data scientists, your data engineers must talk to business and to the people who actually owning the business process to understand how it works. And the second thing is about actually expertise, machine learning expertise. So usually when you do not just mvp, when you're not just trying, when you actually want to build a really good project, you usually need at least three people in a team. One should be data scientist, the second should be data engineer and the third should be person who right now we call it MLOps engineer or machine learning engineer. So the people who can, who could operationalize the model, who can move the model to production. The data scientist will make sure that the project actually still achieving business goals with the available data. The data engineer, he will help to transform and obtain data to make the model stronger. And the MLOps engineer, or the person who moved, regardless how we call it, call him or her who will move the model into production will make sure that the action model is working and able to serve clients. Because moving model from the Jupiter notebook or any other place where you build a model to production with a real traffic, with a real SLA is absolutely different stories and that's why you need a separate person for that. But usually there might be a question. So if I don't have those skills, if I don't have those people where I can get them? Honestly, when we say team, it doesn't mean that the team is strictly inside of your company. It could be external team as well. So you can extend your team by including other people. So it could be partners, it could be cloud providers, if you use cloud services, because some of them have solution architects would help you, some of them have professional service that could help you. And don't forget about different academies and the learning portals, learning platforms. They could actually help you to grow team, to grow the skills of the team so they will be able to to deliver the project. And here I probably also want to share one example from the live There is a company called Recraft. Right now they have one of the best model according to the, according to different benchmarks. So they're building a tool for the professional designers, illustrations and marketers. And we were working together and actually I just want to highlight the full story. If you want to read also in the QR code you can read the full story. But I want to highlight just two outcomes that most important thing that mentioned in this blog. So the first one is to actually build a model, let's say to build a product. It's not only about the hardware, but it's also about how you could solve the issues about this hardware. So regardless where you get your resources from the local data center, from your own data center or from cloud providers, you need to have a connection with this team. Otherwise if something doesn't work, no one, you need to find a way how to solve that. And the second thing, there could be a lot of problems. And if you'll read the Recraft blog, you'll find what kind of problems we have. During the training of the model, there was a problems with the software, different things, but how we managed to solve it. We actually were working with the same team. So our team was connected with the same communication channels with a Recraft team and working as a single team. And that's the reason how they were able to solve everything successful and really fast. So this is about team skilled cooperative team means you need to connect people and find right skills for them. Let's talk about the third, the stakeholder communication. So with a stakeholder communication, it's more about that you need to have a open line communication with all stakeholders and the team who is building the project. And I could say that there are two really important things that you could take from the stakeholder communication. The first is expectation management. So when you starting the project, you need to set realistic expectation to prevent misunderstanding in the future. In other words, you need to have a well defined business problem that you want to solve. So when you have that, it means that you will, you will answer the questions, you will have an understanding, you know, how this project will increase the revenue or impact my customers, increase customer satisfaction or bring new functionality. That's the first thing when you have a connection to stakeholders, the ML team, a team who's doing ML project actually could always verify that they're on the track to the things that should be achieved. And the second is more about the internal communication from the leadership to the entire organization. Because ML space, while we, you know, we see that a lot of problems, a lot of tests could be solved with ML, still there is a huge chance that something will go wrong, that experiment will fail, that the targets will not be achieved, business targets. And here is really important to have a communication that you know that machining adoption could be it could be important, that it could change. Because of that people feels that they, they need it, that you know, that the company need those projects. And if something goes wrong, there will be another project and we will have internal power to continue and to do the next project. But you know, we were talking about communication skills and data. So and then multiple times said the project and the project selection actually is super important because you need to find a really good balance between selecting the project that makes sense for business. That is not really difficult and not really simple. And there are multiple ways how you can select a project. I saw really nice framework for that and I want to share that framework with you. It's really, really, really simple framework. Oh yeah. So what you can do, you can take all your projects that you have in your mind and then you can put them, evaluate them from the three perspectives. The first is data availability. Do you have a data for that project? Second is the impact on the business. And the third is machine learning. Was this project released solved by machine learning in other areas in other companies or in open source? And you can put like from 1 to 9 for all of that and you will have a table. So there are several rules. You know, first of all, it's obvious that you want to select the project with the highest total number. But you need to think, but you need to be careful with a, you know, border situation like project one. There is a lot of data and there is really good ML solution for that. We can find a lot of examples of the such project. But for your business there is no impact. It doesn't mean, you know, it means that you will build something that doesn't make any sense for my business. So you will be successful but no one will understand why it needed and then vice versa. Like you might have like Project 2 where you have really good big impact, you have a data, but this is really complex thing or no one solved it before, you probably don't want to select it as well. So in this specific example there is project three which have more or less everything on average. But it means that you can, you can already achieve the target. You can find impact while solving the problem. You have a data and it's already been somehow solved in the world. So that's a really simple framework how you can select the project. But I found it really helpful. It helps many times, at least for me and my clients. So that's the way how we can do the selection of the project. And the last pillar is a technology stack. So one of the, I mean like the most Important rule in a technology stack is that you should not do non differentiating things. If someone already created a tool, created a framework, you can just reuse it. Because when you start a project, you want to achieve the target in terms of the business goals, not to create state of the art technology stack. That's why you know, if you will create state of the technology stack, you probably your clients will not notice that. But if you create a nice functionality or nice metrics, business metrics, everyone will notice that. So that's why do not waste time on non deferation tasks. Look at the market, what is what's already available in a software libraries, cloud solutions and just take whatever is needed to achieve your business goals. And when you achieve your business goals, when the project is successful, you can start to think about optimization. You know, changing different library, changing different software, hardware and etc. And depending on the, you know, on the market it's already a lot of things, you know, you can build it by yourself, you can get GPU from the cloud, you can get something from API models through the API software as API you can find a lot of in open source and you can find commercial products. So all of that will in some of them, I mean like you can select whatever it fits to you, but do not spend the time on working on non deflation things. Yeah, so those are five pillars that I saw continuously repeating in all projects and all successful projects have all of them working. Looking at the time, we actually have just five minutes for question and answers and just let me just give you like 1 minute brief overview what we have built as an EBS Cloud and then we jump to the question. So in the Nibius cloud we build full stack cloud where you have hardware, you have a lot of managed things like kubernetes, clusters, different type of the storage. We have applications like Spark for data processing, Emma Flow for experiment tracking and other things. We also build an EBUE CI studio which brings model as API. So you do not see the infrastructure to manage that. You just send APIs and get a response and everything is covered by friendly UI self service, on demand resources and pay as you go. So if you want to start using nibius, just open nevius.com and there will be a button create account and that's it. You don't need even talk to us to use all of that. It's available on demand to everyone. Yeah, thank you so much and let's jump to the questions.

Yeah, thanks Alex for going through all that and walking us through it. So we've got a question on the live stream. Depending on how you answer this, it might be our only question. But that's okay. You've been doing AI and machine learning for a long time. They want to know where do you think this is heading next for SaaS specifically? Just with everything going on, even in the last 24 hours and week, any predictions?

That's really good. And definitely I think that right now we have a big, we have a big development of the Gentic system. So no, the system could not just write something or not just propose something, but actually could do complex things. Like for example, you know, I could ask where is my order? And that system supposed to find who I am, go to the database, query the database, find my order, check another API to find like queries like fro in the delivery company, find where it is and tell me, hey, here is it. And this is just example of the, of the, you know, shopping experience. But it could be anything. I could say who are my biggest client in the CRM system And the CRM system supposed to find it by itself without, you know, me providing anything else. So those are genetic systems that could make a decision about actions that they need to take. So I think that, you know, this is, this is what is rising. This is that we see huge adoption and development in this area and I had the feeling that a lot of software as a service platforms could integrate it to make user experience faster and actually to solve the problems much faster because, you know, instead of have a static workflow developed in a way if and else you could actually have a system who can adapt and react based on a changing world around it.

Yep, yep. Makes a lot of sense, good prediction. And then just final question is how can folks get in touch with you if they have a more technical question? LinkedIn or email, what's best?

Both. So we have a. We. You know, you can always find me on LinkedIn. You can, you can write me email, you can just find the new views. We are trying to be as much as possible, you know, as much as possible visible to everyone. And we actually have a discord. So there is a lot of way how to communicate this. So feel free to find any of us. Yeah. And we'll be happy to talk. Cool.

Yeah, and we will. Thanks for doing this Alex, and staying up in Europe. Hopefully we'll see you at saastr, but I know NVS will be there. So thanks everyone to those who joined. We have one last Sastra AI day session, so we'll see you there. But thanks again. So much, Alex, for doing this.