The Future of AI Infrastructure with CoreWeave

Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Bluesky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now on to the show.

Welcome to another episode of Practical AI. We are the podcast that tries to make AI practical, productive, and accessible for everyone. We get to talk to all sorts of cool people in industry. And today I want to introduce you to Corey Sanders. Who is the Senior Vice President of Product at CoreWeave, which we've been seeing in the news quite a lot lately and looking forward to learning more about CoreWeave and the various problems it solves and the position in the AI world that it's fulfilling. Welcome to the show, Corey.

Thanks, Chris, for having me. I was gonna say, you got to talk to all kinds of cool people, but today you're stuck with me. So here we are.

Oh no, no. That's fine.

No.

No, I gotta say, you have a really interesting background. I know, like, with things changing as fast as they are, I know you did a long period at Microsoft before coming to CoreWeave. Yeah. And you have seen so much of the cloud landscape change. Like, I can't think of anyone better to address that in terms of going, kind of like, all the things you did in Azure and then coming to a little bit of a different problem set as I'm looking forward to hearing from you about. And so anyway, yeah, if you could kind of start us off and kind of what problem are you— what problems are you interested in and kind of paint a picture of how you see the world from where you're sitting?

Yeah, no, it's a great question. And yeah, I mean, absolutely right. I did 20 years at Microsoft before I joined CoreWeave and worked in the early days of Azure. And so it's been a trip now to kind of take a step into CoreWeave and think about many of the same approaches we had with Azure in the early days now applying with an AI-specific lens. And that's really sort of the right way to think about what I think CoreWeave is doing and sort of my view on sort of the world and the industry right now is We are seeing the opportunities for AI-based applications to evolve and grow into a similar way that cloud-based opportunities back in the early days grew and evolved, where it starts off with one or two things and suddenly people are expanding and growing and building. And then seemingly overnight, obviously it took many years, and I think will take many years this time, every single application, every single service, every single thing that we sort of touch and experience experience in tech has AI as a significant component of it. And that, you know, I think that pivot is a critical part of the places that I'm working, right? Which is, with that being a pivot of how applications and how services are built, the need for AI-specific capabilities, AI-specific, AI-centric application models are paramount, right? Sort of asking people to apply that AI concept to their legacy way of building applications will end up slowing them down, right? End up creating challenges. And so the opportunity and what I get to really focus on every single day is asking the questions, you know, great, I understand this is how it was done, but how could it be done different with an AI-centric perspective or AI focus? And how are we building these services to make it so that people who want to go build AI have a place they can do it where that's the entire focus of the platform?

When you, when you talk about that kind of AI-centric approach that you're taking, with, with a lot of people, uh, and we certainly are guilty of it on the show, like people here for a long time, we were talking about the latest models coming out and we, in that, and the last year it's been a GenTech everything and, and stuff like that. And so can you talk, like, there's a perspective shift here, I think, where, you know, where people, when they're thinking AI and those first things that pop into their head, can you talk a little bit about like what what was missing as like, if you were coming from kind of the old way of doing cloud, what was missing that needed to be there instead of just as an add-on? Because that's what happened early on is like, oh, hey, you can get a GPU in our cloud and everything. But now with this kind of different, you know, approach that you've just talked about, which goes all the way from infrastructure to application, what has been missing that needs fulfilling? How are you thinking about that problem? And like, what are some of the details around what infrastructure should be going forward?

Yeah, absolutely. Well, you know, maybe as a starting point, it would help to explain maybe the different, the sort of different streams of AI, right? And you sort of mentioned Agentic. And so maybe I'll start there and talk because I think it helps a little bit of a frame. And the streams have separated and then there's, convergence now that we're heading towards that's also worth talking about. So look, I mean, at a very basic level, there are kind of two big aspects of sort of what, of how AI is used or leveraged. One is on the training side, right? One is the actual creation of these models, right? Creation of these weights, right? The making, sort of doing the learning in a very similar way to the way the human brain works, but doing the learning to actually create the output that we then engage with, whether it be OpenAI, whether it be Anthropic, whether it be, you know, Meta, sort of all of those have that sort of background in training. And that requires very specific infrastructure to be able to deliver. And the infrastructure is very expensive, right? As we all know, and it's very, you know, deployed in sort of large tranches and very interconnected, right? One of the biggest requirements for many of these training workloads is they're sort of these large tranches of deployments that are all deeply interconnected, so they're all working together. And, you know, as part of that training area, some of the things that, you know, CoreWeave identified and, you know, certainly even the broader market identified is the potential places where that work slows down because the GPUs are so expensive, right? Any sort of slowdown in being able to get the job done is impactful. And so that includes things like failures in the GPUs, that includes things like storage being loaded into the GPUs, right? And so all the way down at the infrastructure infrastructure level, what's going wrong with the GPUs, are there failures, into all the way up to the orchestration over which job is running on which infrastructure to be able to optimize the output, right? And so this is where I think AI-focused direction, that's a very different set of requirements than just your standard public cloud with a whole bunch of compute racks, right? It requires a bunch of both design and how the hardware is being laid out and specifically how it's interconnected. That's very unique to AI workloads, all the way up to then how the jobs are being orchestrated with knowledge over what's being run down at the core level of the platform. And this is a bunch of places that I think, you know, CoreWeave has a bunch of differentiation on things like our observability platform, our storage platform, right? There's a bunch of places where we offer unique services targeting that type of AI massive-scale training workload that then is just very different than your, again, your classic cloud workload. So yeah, go ahead. I'll pause there because the second half of the story I'll get to. No, but go ahead, Chris. I'm ready. You had a follow-up and I'll talk for, I'll talk 45 minutes uninterrupted if you let me. So let me pause and let you kind of jump in. Go ahead.

No, no, it's fine. And don't lose the second half of that story because you had me, which is why I had the follow-up on there. I am curious as you're talking about that transition, like if you'd bring, like for you personally, —when you were—before you came to CoraWeave, when did you have that realization? Like, as this is—as whatever this epiphany is occurring, you know, was it all at once? Did it happen over time? Like, when did you realize, I want to do something different? And don't lose that second half of the story that I was cutting off a second ago.

Yeah, no, no, no, right? And the second half of the story is very similar to the first half, just a different sort of outcome, actually. But yeah, I mean, for me, look, I got the honor of my final years at Microsoft getting to work on sort of a a lot of the AI infrastructure aspects in, in the platform. And it was, it was kind of a funny thing because it was sort, sort of not my day job. Like, my day job, I was working on industry solutions, so like financial services, um, you know, retail services, etc., trying to think through how these, uh, large verticals could leverage AI in their workloads, which actually is the second part of the answer. But the first part of the answer, I ended up getting pulled into a lot of the discussion over how at Microsoft were we optimizing the deployment of that infrastructure, right? And that, that realization really struck me there, which was like, you know, um, the prior to that, a lot of the, you know, a lot of the big cloud strategy was you end up having a bunch of space and power, and as the needs come up, you deploy what you need to, right? So you need more compute in a given location, you deploy it, and within, you know, days, uh, you have a huge amount of additional compute. Uh, you need a bit more storage in a location, you can go deploy a bit more storage, and you you can kind of add. And that sort of commoditization, that sort of fungibility, is a big value add for, for the big clouds at the scale that they're running. But it turns out you can't really do that when you're dealing with sort of AI workloads, again, all interconnected, all wired together with, let's say, InfiniBand or RoCE, very specialized network behind it, very specialized storage underneath it. Suddenly it's like you've got to pre-plan things, right? You've got to, you got to be thinking about it ahead of time. And so It definitely was a— it definitely for me was an experience of learning, right? It started there. And then as I moved over to CoreWeave, I think the realization of how many places from, again, caching, specific caching for AI workloads to specific approaches to Kubernetes, which is of course a standard orchestration solution, but can deliver focused AI enablement for bare metal-based deployment to really squeeze out all the juice of that GPU, right? Um, all the way up to, again, the orchestrator being sort of aware. That realization of like how many aspects of the platform needed to be, could be, and then needed to be customized for AI-specific workloads, that, that was really— I feel like I didn't fully get it until I came to CoreWeave. Um, but I, I saw it and the beginnings of it when I went at Microsoft. And I don't know whether that's a Microsoft or CoreWeave thing or more just a point in time thing for me personally. Like it's it's more probably that learning over time that you start seeing more and more of this customization realizing, wow, like this is because of how much value and cost there is behind this, it is worth it to do this customization because people will pay for it to get the most out of those very expensive GPUs. So you raise a great question.

If you are, if you are, it's you or you, you're coming with the background that you have and the experiences that you had professionally. And you're making that adjustment and you're seeing that you can come to CoreWeave and see the capability. How do you, like, as a problem, like you've identified a particular set of problems that are part of this evolution and you're doing it from a great vantage point given professionally where you've been. So all the other AI practitioners that are out there don't have the benefit of your background. So how do you teach the, like, how do you teach them that there is something that they're not seeing that is a set of concerns that they need to address that they don't even know that they have? How do you approach that, that education piece? Yeah.

I mean, look, it's very, it's true. It's very hard, by the way. It's very, it was, it's very hard for me. I mean, I think, uh, even so, like, I, as much as I appreciate your sort of lead in that, that, uh, I'm obviously uniquely capable of doing this. It's actually, uh, uh, you know, uh, that experience, that sort of big cloud experience, while it's super, super valuable, it also ends up actually being a limiter because you come in with a bunch of assumptions, right? To your point, you come in with a bunch of like, well, this is how we built it. Like we learned that over 10 years and this is why we built it. And you don't necessarily question all the things you should question over like, well, but maybe this is a different world, right? And so I do think it's, You know, in some ways it's innovator's dilemma sort of at the core, right? It is basically, you know, what, what, and I say this a lot, like what got us successfully here today does not necessarily get us successfully to the next wave, right? And that to me is such an important just life lesson, but it's certainly a tech lesson. And so for me, like it really is like asking those questions, right? It's asking sort of the pointed question of like, okay, so, so That's true, but, but that's true based on 10 years of commoditized, you know, general purpose public cloud assumptions. What if we assume this is different here in AI? Or what if we assume that the data requirements are different, or the container's image sizes are different, um, or the overall flow of, of AI innovation is different from typical software development? Now what problems do we need to solve? So in some ways, it's all about asking the right questions and it's, um, it's also one of the things I always like to push on as being, I think, one of the most protected aspects of humans when it comes to the innovation of AI is that creativity, asking questions that maybe are different from the way things are. As we all know, AI is all based on the way things are, right? It is deep learning and regurgitation with a ton of amazing magic, uh, in what exists. Right? Um, but sort of the ability to now say, well, but maybe what exists is wrong. That's the difference. And I think that's where, when we approach AI and frankly, when we approach a lot of tech innovation, that's the questions that need to be asked. Um, and then I would argue, as much as I think I'm decent at it, hiring really good people who are amazing at it is actually my secret sauce. So I look for those people in the market, uh, uh, and, and, and basically find, try and find the people who are going to challenge me and say, Well, but Corey, maybe your baseline is wrong. Like this is what you did in Azure and it worked in Azure, but maybe it doesn't work here anymore. And it's like, oh wow, that's good. Harsh, but fair. And so, you know, that type of approach I think is—

Okay. So one of the things that you mentioned a few minutes ago kind of caught my attention and I wanted to go back to it and that's having to do with training. And you know, when you have, you know, training jobs fail and you talked a bunch about like where in the system that could happen. You kind of identified some of those points, but could you talk a little bit about what actually goes wrong in the systems before an AI practitioner or researcher or something has identified that things have gone wrong and that they're not making progress along the line that they're trying to work toward? I think there's a little bit of, I don't know, you know, just black cloud mystique about like where that can happen. Can you talk a little bit about what you've seen and where in terms of where that problem arises and how and what it is, what the implications are on that? Yeah, absolutely.

I mean, it's, yeah, there's, it's one of those special areas where there's both science and art to it. Right. I think. And, and that's, you find some of the best people in the industry who do this. It is that sort of instinct. And sort of knowing how things work. But like, look, I think that the concept of a failure in a job can mean a whole lot of things, right? Um, there's certainly sort of failure from an infrastructure perspective. Um, you know, uh, the, the infrastructure fails, uh, the sort of a node slows down, things are running slower than you'd expect. Um, and that can be sort of a plethora of things, right? That can be, um, storage slowdowns, that can be GPU slowdowns, right? One of the things that we've recently launched, which is a surprisingly, surprisingly enthusiastic capability, is our GPU straggler detection, right? This concept of like GPUs slow down for a variety of reasons inside their infrastructure. And then when your job is hundreds or even thousands or sometimes tens of thousands of GPUs, finding that one slowed down and you're The output of that is maybe your job is slower, but like which GPU slowed down, right? Like which GPU isn't performing? And so that type of detection and observability is actually quite challenging. And so, you know, a bunch of that observability, this is a big part of, I think, where that sort of AI-centric work, that sort of, you know, we believe our secret sauce, but certainly everybody's working on that sort of like AI-centric focus on sort of being able to detect those types of hardware failures, the hard failures and the soft failures, the sort of slowdowns and, and so on that can really dramatically impact the job. So that's kind of more on the like pure failure hardware impact. There's also the other side, which is, you know, a big part of this. It is research, right? I mean, and we call it research for a reason, which means you run a ton of experiments, right? You run a ton of experiments with a bunch of different parameters and different controls, right? And then you, and then you need to compare them all and sort of study them and understand which ways went what, and sort of now based on your output, what should the next experiment be and so on. And this is where a product like Weights and Biases, which is the top of our platform, really focuses on that. But one of the biggest is sort of that experiment tracking and comparisons and so on. One of the biggest things we've heard feedback on and recently sort of announced, hopefully a response to this is very new, is it's really hard. Your point about sort of the art versus the science, like it's really hard that detecting and finding and then learning from that experiment what the right next step is. And so we've recently launched what we call ARIA, CoreWeave ARIA, which is short— it's a clever name— short for AI Research and Iteration Agent. And its entire job is to try and do this sort of continuous analysis on experiments that are being run and what the right next step should be. And so, look, I think the simple answer is right now it is a lot of experience and expertise in this area that's driving that type of work. But as is so common in technology, a big step of platforms like ours and, and, and others in the market is to enable that democratization. How do we make it so that many more people can do that type of iterative model improvement work? And, and, and that's— and then, and using AI itself to do that, I think, is sort of a clever, clever approach.

With, with you talking about Aria, could you talk a little bit about, like, that I presume is introducing a kind of new workflow, new capability. It gives new insight, but there's new workflow there. Could you, could you talk a little bit about like how that changes the practitioner's day-to-day activity and like what's introduced or taken away and, and kind of, you know, talk about the benefit of that?

Yeah. Yeah. I mean, look, I think if you take my, my full full sort of view, full extension of time. I think this concept of portal experiences and console experiences and sort of standard views, I think they're all going to fade away. I think the experience is going to be an interaction with an agent. Right now, how that interaction actually sits, is it in a website? Is it on a console screen? Is it in a, you know, my, uh, VS Code, console, like, I don't know, right? Like, I think there's a lot of different places, or all of the above, frankly. But like, I think the idea will be, hey, here is the type of improvement I want in my model based on what I'm seeing from my traces. Um, you know, what's the best way for me to run some experiments? Great, run them, right? Uh, and then, you know, on my mobile app overnight, I see, okay, the experiments ran and the agent's popping in and saying, hey, you know, the experiments ran. It turns out this worked, this didn't. I think we should do another run here. What do you think? Yes, let's go do it. And so then that run kicks off, right? And then it comes back and says, oh, this improved by quite a bit. But based on this data, perhaps it actually would be helpful to go and iterate and iterate and iterate and loop and loop and loop. And so this sort of like approach to AI loop, I think will be that type of that development experience. It will still be the brains of the researcher. Like it will still be the, well, I don't know if that's going to work. Let's try this instead., but it is going to be heavily guided, uh, by this sort of agent analysis of the data versus you go into your console, you look at the line charts of your 6 runs, and you compare your success rate, uh, to each other and your performance rate to each other, and you visually look at it, which is, by the way, state of the art today, but not the way that the world's going to move. Um, and, you know, I'm excited about that direction. I think that that agent-led is, is is the right direction. And candidly, I would argue, I think that's going to be the case for all of our interactions moving forward, no matter what we're working on, whether it be banking or buying clothes, right? It's going to be that type of interaction model versus click on a button that says buy, which I just, I think will be antiquated in 5 years and sometimes even laughable in 10. But we'll see sort of how far it goes, how quickly.

I know. Well, I'm on the older side of things and can remember back. And yeah, when I think about some of the things from a few few years ago, you're like, wow, we got through it. The, one of the things I wanted to ask about is, is you've mentioned agents a few times. And I think, you know, typically people are thinking about agents in terms of, okay, I have my model and I'm going to, you know, figure out in my application, you know, how many agents are assigned to the model. Is it one-to-one or one-to-many? And I'm going to give them the tasks and give them the interactions. But I'm, I'm wondering, is it a little bit different on the infrastructure side? Like, how do you see You've kind of already made reference to it. So like, how do, how does agent architecture work in an infrastructure environment like that? What do they mean to you and what does it bring? And can you talk a little bit about like, this is what we get on our backend as a provider versus this is what you as a customer would get on the frontend. Can you talk a little bit about that infrastructure agentic? Absolutely. Yeah.

Yeah. I mean, the— so I'll go back even to, I think, the question I answered, I feel like many moons ago, where I said there were two paths, right? And we sort of went down the first path. The second path is this inference side of the house, right? Which is basically the AI application, right? You know, I like to call it AI application because I do, to your point, I think you said like everything's— everyone's talking about agentic., and I think that's, that's fine. But like in some ways it's all about an application that's AI-centric, right? And Agentic happens to be one type of that. But all of it will sit on some aspect capabilities of an inference call, basically leveraging all those models that we just built and asking it questions or asking it to do an analysis or asking it to simply spellcheck something, right? Like all these things could be on the backend there. And look, I think to your point, as we get more and more complex and there's there's in some ways two different approaches to AI applications. There's certainly the productivity internal-facing ones, the, the Copilots of the world. And then there's the, what I like to call the sort of mission-critical, the business-critical applications that are either, you know, external-facing for a customer, so like a customer's customers, or sort of critical to their business workload, like a gene folding application for a pharmaceutical company. Now that, that second category, uh, and now kind of returning back to your, your, your, your, your most recent question, that second category I think will be a very complex, um, uh, application model, right? Like over time, right? Um, you know, today I think there's a lot of this concept of like, I have a problem and I'm calling a frontier model and I'm getting an answer, right? And that's my app. But over time it is going to be, I think, a consist of many agents. Or many models, many inference calls of many different types that come together to build an application, right? So in some cases, the very deep analytical sort of the biggest, hardest problems may go to those frontier models and call into them and ask them to kind of do that sort of analytical work. But like translating, someone asks a question in Japanese and you want to translate it to English, like that doesn't need that level of depth that may not, that may enable sort of a different, smaller, cheaper model. And I think as you look at some of the very specialized workloads, the gene folding example, it may need a very specialized approach to a model versus, again, this general purpose feels like a big giant hammer for maybe a screw, right? And so you can certainly push it in, but it doesn't quite work. And so the opportunity, I think, for some of those workloads is to think much deeper about how their application's going to work. And this is to your point, what, how is it going to leverage the infrastructure, right? You know, what is the, with the infrastructure being sort of a significant component of the cost, how is it leveraging it? Is it fully optimized for that infrastructure? Is it fully infrastructure aware? All the things that I mentioned before about being infrastructure aware and storage caching and so on, all of that makes the inference workloads work better, work faster, work cheaper. And then I think the other question is, is it leveraging the right model, the right parameters, the right prompt? And that to me is opening up a new approach to AI application development, which we like to call this AI loop, where you're never done making it better, right? You're never done making the prompt better, or you may want to take a model and make it a little bit better and do a little bit more training on it and it feeds back into that training component. And so this concept of doing inferencing, running an AI application, having 30, 50 different models all interacting with each other, and then finding the part that needs improvement or could be cost, could be a cost benefit or, or could have a little bit better results or could run a little bit faster. And then running that through the system and using products like Weights Biases to take that output, run it through the system, use something like Aria, to learn and make it better and better. And so I think that's our future for those types of applications. And that's all going to be built on the best infrastructure all the way up to that workflow that I think is going to be crucial to, to building the best apps.

It seems like the amount of web pages, landing pages, et cetera, that we have to create every week just keeps growing. And often those turn into a pile of tickets and handoffs amongst team members. And that's why I love what Framer is doing, one of our partners who has a pro website builder platform where teams can collaborate in real time and iterate on the same page and publish instantly. And you can even tie in agents into this work where agents and humans work in tandem. Agents bring speed and scale, people bring taste and judgment and control. So I would encourage you, if you are starting a new website from scratch or needing landing pages or relaunching a website, check Framer out, learn more how you can get more out of your site from a Framer specialist, or get started building for free today at framer.com/practicalai for 30% off a Framer Pro annual plan. That's framer.com/practicalai for 30% off. framer.com/practicalai. Rules and restrictions may apply.

Okay, Corey, so as you were talking a moment ago in answer to that question, I've got a whole ton, a whole ton of questions. I'm sorry. No, it's good. This is what we're here for. But no, it's like, so As you know, when people are out there and they're thinking about their own workflows and they're looking at, you know, the, the, the, the, the current incarnation of agentic engineering is kind of loop engineering and getting all your loops going and, and everything like that. And you made reference to that in your answer. And so as people are trying to constantly level up in that way, uh, and there's a learning curve for them to, to do that, what with, with CoreWeave, if they go, I'm moving, they've They've listened to you today. They're like, I definitely want to move to this platform. What changes in their agentic or loop engineering that, that they should be thinking if they're already trying on whatever platform, maybe Amazon, Azure, Google, whatever they're on, they're already kind of trying to do that as they come and, and, and, and get onto CoreWeave. And you've talked through our conversation about all these advantages, all these different capabilities that you're bringing in. How does that change, uh, what they're setting up in their prompts and how they're setting up agents to address? How do they take advantage of that in their current workflow?

What, what adjustments are made? Yeah. I mean, I think that one of the key things is, um, being able to quickly execute and learn from that loop without a lot of, uh, you know, additional noise or disruption, right? And what I mean by that is, and this is one of the things that I'm pretty excited about, what I get to kind of work on here at CoreWeave, is how are these, how are these, how is that loop all basically playing out in interconnected services, right? And, and, and how do you basically execute against that loop? And so by running your inference service and having it then leverage our Weave platform for traces and running evaluations, basically saying, okay, in production I'm now seeing that with these types of questions or these types of, uh, operations, I'm go either going down this wrong path. So I need to figure out how to fix that. Um, uh, or let's say it's taking too long, right? And so I need to think about, am I trying a different model, right? And so like the tracing from production allows you to sort of understand where you need to make these improvements, right? And with the agent, sort of the Aria agent helping you sort of find those, right? And say, oh, well, gosh, this looks like it's slower when it has this type of question, to then being able to directly execute, okay, great. If I change my prompt, does that fix it? If I change my model, hey, this new QIIME model is out. Wow, like, let's— this I'm hearing is really, really fast. Like, let me change my model and see if I get impact. And then I can quickly set up an evaluation that then gets executed, right? You can use our sandbox product to be able to just run that evaluation, say, okay, now I see these two together., and I understand now, okay, the performance did get, get improved on this one with KIMI. Let me make sure that the pricing isn't sort of a big difference. Okay, it's okay. So now let me kind of swap that in and, and, and get it back into production. And now let's take that loop again, right? So the loops could be fairly small, right? Like every single modification. In fact, most modifications I'd say are not, let me put this into my training workflow and actually do some reinforcement learning or do some, you know, fine tuning. That is a possibility. Like in some of these cases, like, okay, gosh, this is great, it's working well, but like my accuracy is just a little bit off in these types of questions and I don't think a new model's gonna help. Let me actually do some fine-tuning, right? Or let me do some reinforcement learning since I maybe don't have the data to do my fine-tuning. Um, and you can run those services again, all integrated into Cori with again the Weavetraces to see did it work, run your evals, it did, plug it back in, right? And so Like, I think what we're going to find is that developers, at least what I believe developers are going to look for, is one, being able to do much, if not all of it, all in one place. Because just the simplicity of being able to track those traces, having a registry, which CoreWeave offers for lineage, like every updated model, every updated parameter, et cetera, being able to try, oh, that version was 3 yesterday. OK, that's the one I want. With then an agent that sits on top with the knowledge of like, oh, that's what you need to go try, or you should go work on that. To me, that's, I think, the real power here. And I'm excited that like, you know, these components like Corvi delivers a bunch of this, but a lot of this is also we're continuing to, you know, to make it super easy for that flow to happen at a much more accelerated pace than it even happens today on Corvi. Gotcha.

With, you know, as you're talking about like that flexibility to get different models. But in the same conversation, we've been talking about kind of just all the benefits and the virtues of the performance wins you get from that kind of vertical integration of your stack. Is there any kind of tension between having about kind of supporting an open approach on one side and having it deeply optimized in terms of portability, you know, for a customer and things like that? How do you You know, how, what's your position on the tension there in terms of portability versus performance optimization?

It's an awesome question. And, you know, I'll give an answer that I've actually believed for quite some time. I had the luxury of deploying the first Linux infrastructure on Microsoft, on Windows. It was called Windows Azure at the time. So it was Windows Azure hosting Linux. And so, you know, my belief system from then to today, and I think the core we believe system is We need to expect and build for multi-cloud. Like this belief that CoreWeave is going to be the only cloud people use for their AI applications, I think is false. Like I think, and so I talk about this loop and I'm enthusiastic about the loop, having all the components on CoreWeave to deliver on this loop. But with the strong realization that people are not, that many people are not gonna use every service that CoreWeave offers for the loop, they're gonna use some other services. They're either already have an evaluation service that they really love, right? Or, they already do their training somewhere else and they want to run sort of the inference here, or they already do their inference somewhere else and they want to run their training here. I also think in a, um, so, so I think being very open and, and, and, um, delivering upon consistent services, um, uh, is, is a key part of our strategy, right? And that's so like when you think about things we offer, Sunk, which is Slurm on Kubernetes. Slurm and Kubernetes are both, you know, open source projects, right? And so, you know, we take a lot of pride in believing we deliver the best. And that really, to me, is where I think the differentiation and value comes in is, can we take some of these open source, multi-cloud, cross-cloud capabilities and make people want to use us because we offer the best services? And so lock in with love is sort of the joke I typically make. And so, you know, I think that really is how I think about it. And so it's also why we take some of our Sunk Anywhere we recently announced. You can actually take our Sunk offering and deploy it on other infrastructure. Like, we think it works best on our infrastructure. Yeah, go ahead.

Go ahead and go there. No, I was just going to say, go there. You've mentioned Sunk a couple of times along with Slurm and Kubernetes. And like, go there and tell me a little bit about it. And add in also, you know, for people who are using, you know, Kubernetes in their old world, the old cloud and stuff, like what's different? What's it doing well? What does Sunk bring? Can you kind of package that up so people can Can I kind of follow you there?

Yeah, absolutely. Yeah. So, so, um, uh, you know, put simply, like, I think Kubernetes is such a market leader, and, and, and, and there's deep awareness and understanding on how to operate and orchestrate infrastructure, right, and workloads on infrastructure, right? The concepts around Kubernetes, sort of the, the, the platform that's been built on top of it, uh, certainly, uh, has, has, has effectively, uh, dominated that market for, for years, right? Which is awesome. And then Slurm, which is kind of a job scheduler orchestrator placement engine that has, I would argue, been a dominating force in research-based workloads, right? In sort of AI-based research. It's not the only one, right? There's quite a few out there and we could have probably a full hour show debating the merits of each. But many, many AI researchers know Slurm and are comfortable with Slurm. The challenge is, is that the Slurm on top of infrastructure is actually tricky to manage, right? Slurm on top of infrastructure, particularly for sort of AI researchers, is challenging, sort of managing ups and downs of infrastructure, failures in infrastructure. And these are the things that Kubernetes is awesome at, right? And so what CoreWe built was Sunk, which is short for Slurm on Kubernetes, although it's with a U. But, and it is about bringing the best of both worlds, right? Enabling the sort of the power, the capabilities of Slurm job scheduling with the orchestration power and ease that Kubernetes brings and sort of jamming them together and creating sort of a platform. And this is sort of a key, you know, key product that we offer directly on our infrastructure. But like I said, we have now recently announced that we offer that to be taken anywhere. It's called Sunk Anywhere. It's a clever name. And so you can deploy this on other clouds too and take that, take the power of that Slurm plus Kubernetes support.

Onto any cloud. So really, really interesting. And like one of the questions I really wanted to get in for, I'm taking a slight curve on this, is as you are servicing and providing more and more capability to people, people are always talking about the cost of things. I don't mean the cost of any given provider, but just in general, to be able to do meaningful experimentation and research at at the frontier, um, is so, so capital intensive in terms of the compute and stuff. And like, how is that something that could be addressed? Is, is it— how do you guys approach it? And how do you— and in a broader context than just CoreWeave, how do you think the industry at large needs to approach the problem? Because it, it kind of— there's a, there's a, there's a special few in, you know, in a sense, that, that have a lot of resources, and a lot of others are saying, how do I compete against that?

How do I do that?

Yeah, a lot of those folks are listening right now and watching the show and like, what, how, what, what do you, what do you, what would you say to them? How would they approach that? And what can CoreWeave do?

Yeah. I mean, I think the part of the goal with this AI loop concept, part of the goal with this sort of integrated story, um, is to enable people to do more with less, right? Or, or, or do, um, do the same with, with less, uh, I guess depending on how you look at it. So, and so what I mean by that is, Um, you know, optimization can mean a lot of things. So it can mean being able to choose a cheaper model that gets the work done well enough, right? And so it requires less infrastructure, requires sort of a less deployment model. Um, uh, it may be, um, uh, you know, doing, um, uh, model improvements and fine-tuning to reduce the size of a model or enable it to be more effective with a smaller sort of scale. Um, part of the goal with Aria, uh, and things like Aria are to enable less trial and error, right? Enabling faster movement from experimentation to production, which is actually a big component of that cost construct, which is like, great, like I want to go build this in production, but how expensive is it going to be for me to get there? And with something like Aria, with something like the AI loop, you, you, you have sort of this flexibility to, get things out, get them to start be revenue generating, sort of get them to start building on your business and then experiment and learn as you go. And with the help of like, hey, I wouldn't spend the money to fine tune this. I think you could just shift your prompt and improve it, says Agent. Uh, and so like, I think the opportunity to sort of get that guidance, uh, to, to help direct that is built on the back of what, you know, the learnings we've gotten with all of those massive labs still working. By the way, those massive labs are still on the very cutting edge frontier where like an agent that talks about historical knowledge may not be as helpful. But for everyone who's kind of one step behind the frontier labs, which by the way, I think every enterprise and every sort of large company would want to be there because they're like here now, right? That is where I think some of these capabilities coupled with sort of the metal-based optimizations that I've already talked about, where I do think this type of platform could really, could really be meaningful. That's really cool.

With, with another thing that, uh, that is, you know, so big this year and exploding outward, being kind of embodied agentic engineering, uh, at the edge, you know, robotics is just taking off, you know, with AI integrated. Um, what is CoreWeave's story going forward in that space as you are getting, you know, where you're not strictly based on cloud, but there's some hybrid between cloud and edge platforms that are out there with different levels of embodied intelligence and agentic capabilities.

Yeah, I mean, there's a few places that we really focus when it comes to, you know, robotics and, you know, let's say some of the scenarios around robotics, industrials and manufacturing and so on. You know, one is I talked a little bit about, you know, our models, our experiment tracker technology, right? And I kind of talked talked a little bit about looking at the different charts and so on and so forth. You know, one of the things that we've recently added to our Weights Biases platform is a specific robotics focus, right? And, and, and something even kind of, um, as trivial, but is a huge, huge value, is shifting from those sort of, um, line charts showing success to visuals showing success, right? Showing the actual robot and how it's moving with all of your experiments and being able to detect what went wrong and what didn't. Obviously this is pre-Agent where I do think then add on top of that Aria and you sort of have all that benefit of that robotics sort of training and iterative work adding up to being able to sort of drive a bunch of the robotics workloads. And then, you know, the other area that I think is quite interesting is work with Monolith, which was a company we acquired a few months back. And, you know, one of the things that we've realized in many of these cases is the need to go engage directly with customers on the types of scenarios they're building. Right, and where we can go help. And so that's where, you know, I think a lot of this sort of focused engineering resources going in and sitting with customers and saying, let's talk through what you're building, let's talk through what problems you're trying to solve, and let's figure out how we can help. And, you know, we've had this for a while as part of our overall Mission Control offering. We called it Direct to Expert, which is really where we kind of lean in with resources and expertise. That sort of additional model is brought some of the industrials, the manufacturing expertise to bear here, um, and I think it's a big factor in sort of helping people build out their solutions. And we'll continue to look at other verticals as well. And then the last point, you know, as we mentioned, I think, um, more and more of the capabilities that we're building, we are enabling to run on other infrastructure, right? And so I mentioned Sunk already. Our caching service for our storage platform can run on other infrastructure. And so Most of these are based on just Kubernetes operators, which means an edge environment, an on-prem environment, like these also have the opportunity to leverage these same services and take advantage of them in those types of environments. And so, you know, even our services, our software solutions, our weights and biases solution, all can be brought to bear in those environments as we see some of that evolving and maturing. That is super cool.

As we start to wind up, longtime viewers and listeners will know that we ask you to to start waxing poetic a little bit in the last question. And that is, we always like to get your— what we describe is like, it's the day's over, you've chilled out however you chill out at the end of the day. You know, you're kind of getting in bed, your brain spinning on different problems and what might be. And if you can share a little bit of your vision, your dream about where you think things are going It might be for CoreWeave, it might be for the broader industry or both, but just like what's in your head for like, for, you know, several years out to where it's not on your product map. You know, I'm not asking for what's your next release. I'm asking for like, where are things going in the large in your view? And recognizing that just as, as I will do this and get it wrong regularly, it's fine to get it wrong. But I love to hear people's thinking at any given moment about where things might go, whether you get it right or wrong. What's yours? The, the, the I'm chilling out in bed before I go to sleep.

Yeah. I mean, um, I'd probably say, and I talked a little bit about it already, uh, early on the call, but like, I'll maybe double down on it. Like, I think the progression that we saw with public cloud, um, to the point now where every enterprise, every company in the world has a public cloud engineering team, like it's like, in fact, they've They've far exceeded their, you know, on-prem engineering team in most cases, right? There's obviously exceptions to that. But for the most part, like everybody, and you're sort of, you know, every single application or experience out there has some backing in the cloud. I think we will go through the same progression with AI and probably faster to the point where we will have AI engineering teams at every single enterprise company and they will at, at, at, at probably sooner rather than later exceed the amount of people who are on the sort of cloud engineering team, right? Cloud engineering teams will be legacy and the AI engineering team, and they will be the supporting team for the AI engineering team who's getting to build the AI applications. And AI applications, I think, will far exceed our current interaction model with AI applications, right? Like, I think, and I'll use sort of again, the, uh, the, the, the cloud as my example, like, like the initial wave of cloud was click a button, click a button, click a button, right? But now we record podcasts on cloud-based experiences, right? We watch, you know, World Cup games on cloud-based infrastructure. Like I prefer that over going to my TV and being like, gosh, what channel is this on, right? It's like, and so that transition, like it will be much more than just I chat to an agent, right? It will be, it will be, visual response to the face you're making to your screen when something comes up. It will be, you know, human interaction models that will far exceed what we experience today and will replace what we know of as modern today. And I think it's going to happen much faster. Cloud took what, maybe 20 years, 15 years to kind of really get there. I think we'll probably half it, right? I think within, you know, 5 to 7 years, that's going to be our world. Websites with, you know, buttons that you click are gonna be, are gonna be legacy. Like, I can't believe they haven't, they haven't AI'd this yet, right? Or whatever the verb may be. Um, and so, uh, that to me is the direction and the requirement there for a company like CoreWeave, but frankly for a company, uh, all the companies in this space is how do we enable that progression by democratizing what's required to get there? Because right now those enterprises don't have those people, they don't have those frontier lab researchers. Right? They don't have the talent that they need to be able to be innovating in this way. And so they're limited by that. And so just like they didn't have the cloud people in the original world, right? And so they will ramp up, they will train, they will skill, they will grow, they will hire, right? That's going to happen. But in parallel, just again, like the cloud wave, we have an obligation to democratize the services so that more people can deliver upon that value. For less. And that to me, like, I think of why I'm here. Uh, and, and I don't even mean here, CoreWeave. I mean in the industry right now. That is why.

That is very— I really appreciate that insight. That was, uh, not only I do, I do, I definitely think that you're onto something there, but, uh, it's interesting to listen to and fun. Thank you so much for coming on the show. Uh, it was a great conversation. I learned a lot. So, uh, Corey Sanders of CoreWeave. Hope to have you back again.

Appreciate it. Thanks for the time, Kristin. I'll be— come back whenever you want.

All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Bluesky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner PredictionGuard for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to BrakeMaster Cylinder for the beats, and to you for listening. That's all for now, but you'll hear from us again next week.