EP 545: How to build reliable AI agents for mission-critical tasks

This is the Everyday AI show, the Everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

It seems like the entire business world is sprinting to implement agentic AI, but what's crazy is there's no real playbook per se, right? Because not only is the concept and even kind of the definition of agents changing, but so too are the models that generally power them. So I think it's an important conversation for us to have today to talk about, well, how do you just build more reliable AI agents for mission critical tasks? And luckily I said, even though there's not an official playbook, because this technology and the developments in AI and large language models are so fast moving, we at least have today a great guest, the co founder of Galileo, who they're kind of writing the unofficial playbook, at least for how so many enterprises are using agents in agentic AI. All right, I'm excited for today's conversation. I hope you are too. What's going on, y'?

All?

If you're new here, my name is Jordan Wilson. Welcome to Everyday AI. This is your daily live stream, podcast and free daily newsletter helping us all not just learn AI, but how we can leverage it to grow our companies in our careers. It starts here with this unedited, unscripted live stream and podcast. But if you really want to leverage what we're going to be going over today in this conversation, make sure, if you haven't already, to go to your everydayai.com, sign up for the free daily newsletter. It's that newsletter. We're going to be recapping the best insights from today's conversation as well as keeping you up to date with everything else that's happening in the world of AI. Because it doesn't sleep kind of like me. All right, so if you haven't done that, make sure you do that. And if you're looking for the AI news, sometimes we start the show with that. Today, not so much technically pre recorded, we're debuting it live. So make sure if you want the AI news, just go check the newsletter. All right, enough from me. This is a conversation I think you need to hear. All right, we're going to talk about a lot of things, but just where we're at with AI agents, where we're headed, how to increase trust and reliability. But luckily you don't have to hear me ramble on about it. We have a great guest. I'm excited. So livestream audience, please help me welcome to the stage there, we got them. We have Yash Sheth, the co founder and COO of Galileo. Yash, thank you so much for joining the Everyday AI show.

Thanks Jordan. Thanks for having me here.

All right, can you explain to everyone what is Galileo? What is it that you all do?

Yeah, so at Galileo we are building the reliability platform for agent tech development. And so basically think of Galileo as the AI reliability platform for helping developers build, ship and scale their agentic applications reliably. We started the company about four years ago. That's a while before ChatGPT even came out. And that's because you know, me and my co founder, you know me and my co founders actually realized, and we were working on LLMs back at Google and we realized that it's going to be extremely hard for enterprises to adopt these models for machine critical tasks. And I love the topic that we have today because we're finally here, we're here where AI is being adopted in the mainstream. We really want to see what is the extent of the ROI that we can extract from this exciting new technology. And large language models are amazing at doing so many things. But when it comes to enterprise applications, these models as we all know, may not have ever seen the data that we are working with or the systems that we are trying to automate in our enterprise. And then as we can talk more, we would love to even shed some light on what we're seeing there, Jordan. But that's in a gist of what Kalio does.

Yeah, and yeah, I do want to dive into a lot of aspects here, but I kind of maybe want to start at the end and then we can work our way backwards. But ultimately, and I know it's not an easy, easy answer, right, because it's probably the billion dollar answer of 2025 when it comes to AI implementation. But how do companies build more reliable AI agents? Right. It seems like no one quite knows. There's no official playbook per se. How do companies do this? What are the correct steps they should be taking?

Yeah, and that's, that's great question because we're thinking about this all the time, Jordan. And I think this is really top of mind for everyone building agents today. I mean for, I think I'll start off with at least why we are doing that just so that at least everyone understands what's on the hype about agents. Right. You know, we are all seeing how ChatGPT, Gemini and others can be actually super and cloud as well and can be super helpful in our day to day tasks like can be great assistance. Right. But you still have a human in the loop when it comes to really large scale tasks that are being done. And in order to see real ROI from LLMs from AI, I think the big shift or the big excitement is all about can we automate a lot of things that we do today without even involving a human in the loop? And can these systems get, you know, just leverage the true intelligence that they have to understand the right intent behind what a human is doing, talk to systems behind the scenes and get actually end to end workflows done. And so that's all, that's why the world is so excited about building agents. And if I were to put it bluntly today, the world of software, we talk about having microservices and small software services that do specific things, even websites. So we are moving from a world of microservice based architectures and software to micro agents and micro agentic software where you'll have each, each part of the, the, each component of the software we build today just will become more intelligent and, and smarter and more independent at doing tasks than us having to hard code as, us as developers having to hard code heuristics within it. So that's why I, that's why I wanted to start with like why is, you know, you know, why is all the hype of being ages actually is there? And, and you know, why is, why is it so important for us?

Yeah, and I, I think even that you know, like zooming out even more is maybe even helpful, right? Because I think as the actual large language models themselves, especially via like an AI chatbot interface as they change. Right? So as an example, you know, you, you mentioned Claude, you, you know, with ChatGPT, Google, Gemini, obviously now the BAS have, you know, agentic capabilities, right. You know, these reasoning models, they'll start to plan, you know, they'll say oh wait, I need to go look at more documents. Oh, I can tap into all of your company's dynamic data. So it, it, it's almost like the lines are blurring between traditional AI chatbots and you know, more quote unquote, traditional agents. So like where do you even draw the line, right? Like what is technically like oh, I'm just chatting with a, you know, an AI chat bot that has agentic capabilities connected to my data versus like, oh, I'm working with a full blown agent.

Yeah, I think the, I mean just going to go into some of the more academic definitions of agents. Like typically a chatbot ends with an answer as the final action. An Agent typically has three stages and I would think at least the top, the first two is like, firstly there's a planning phase where the agent is trying to be more, is trying to be curious about what you want and like getting you as a user to the right spot before it actually takes the action for you itself. And then even after taking the action it kind of, there's a reflection step typically on like, did I do it well or is there anything missing? For example, like if, you know, as a human I, if I am ordering something for you, maybe if I'm like, let's say if, even if I'm like, you know, we have a waiter in the restaurant, someone is actually ordering food for you, even after they complete the task, they do ask, like, is that all? Do you, would you want anything else? Like, you know, there's actually, you know, a feedback loop happening all the time. And that's the big difference between a chat bot that is a, you know, and you can easily relate to that as a, you know, me as a consumer, I can relate to that very easily when I'm, you know, when I'm talking to ChatGPT these days as a user and then now it's starting to do some actions like it can take over my screen, it can do stuff for me. So that's authentic in nature. But when you build these capabilities into your apps, that becomes an agentic experience.

So where would you say, where are we at today? Right, because obviously as someone that talks about AI every day and I'm lucky enough to get to talk to, you know, smart people like you, sometimes I feel like I'm in a little agentic bubble, right? Like are enterprises at scale actually using agents today? If so, how? If not, why?

Yeah, that's a great question. So I think, you know, Galileo works with, you know, hundreds of enterprise teams and organizations to be able to see agentic development across the board. Whether it's financial services, healthcare, retail, telco and even, you know, cutting edge startups, right? If, and there's a whole spectrum of adoption, you know, there are so many teams and I mean obviously startups are all, all going to be like, all about agent, agentic workflows. But even we're seeing a lot of investments from enterprises as well. Now we have, we already have several customers who have agents that are live in production and we have customers who are even thinking of like this year is just going to be about us productionizing rag applications and chatbots and agents are going to be next year because these are heavily regulated industries and Basically chatbots and rad applications are basically a stepping stone. It's like the crawl, walk, run analogy. They're still in a crawl phase and next year with agents is going to be walk and then multi agents is going to be the true run. Right. But it's so exciting to see the adoption curve is extremely fast. It's something that we've never seen before, honestly. Even with the regulated industries and organizations, a lot of teams are already building agentic applications and they will be productionized this year. So it's incredible to see the arc of innovation that even regulated organizations, large organizations, are able to do with this technology now. So that's where we've seen agents that can actually preempt Internet outages in production. We've seen agents that can really manage the data platform for an organization. We've seen supply chain agents that can look at multiple warehouses and automatically place orders. And so that's. And these three are just like the starting point. Right. And all three use cases, as I mentioned, they're far away from being a chatbot style use case.

And one thing that I wanted to dive into a little bit deeper there that you said is talking about some of your, you know, companies you work with that are in highly regulated industries. Right. So whether that's, you know, finance, healthcare, etc. But when it comes to agents working on mission critical tasks, right, Aside from, you know, certain regulation that, you know, you might not be able to out or you know, over engineer, you know, around regulation. Right. But aside from that, what are those most important steps for enterprises that do have that in their control to be able to start unleashing agents on those actually mission critical tasks versus just the low hanging fruit, you know, content creation research. Right. Areas where I think agentic AI has already done very well in 2025. So what are those most crucial next steps to actually get to those mission critical tests for tasks for enterprise?

Yeah, and like the last three examples I gave you were actually mission critical. They're actually mission critical tasks that run businesses. Right. And if anything were to like go wrong there, like, you know, you have real world systems that are going to fail and that's. And we've seen enterprises actually, you know, be successful in deploying these mission critical agents. And this brings us back to your very first question, Jordan, on you know, again, how do we make agent AI reliable? Right. And that's because that's the number one thing. And I'll talk about like there are other things as well and we can go to where, which will truly make multi agent systems successful. But the number one thing for agents right now is trust and reliability. How can we make sure these when these agents and mission critical tasks means that these agents have access and control over real world systems. They can make API calls to tools, backend databases, ticketing systems, to update them, to change them. And if these agents were to behave incorrectly, you know, and to preface this as well, the reason why we keep talking about if these agents were to, you know, go wrong is because we are entering a world of non deterministic software and enterprises as a whole need to adapt to to a world of non deterministic software. It's a new world. None of us have built these agents at scale before and so reliability, setting up a reliable pipeline for building, shipping and scaling these agents is absolutely critical. And happy to share more there.

Yeah, and I think it would be a good time to talk about the agent leaderboard. But real quick, before we do, first a quick word from our sponsor at Google.

This podcast is supported by Google. Hey everyone, David here, one of the product leads for Google gemini. Check out VO3, our state of the art AI video generation model in the Gemini app which lets you create high quality 8 second videos with native audio generation. Try it with the Google AI Pro plan or get the highest access with the ultra plan. Sign up at Gemini Google to get started and show us what you create.

All right, let's, let's dig in a little bit on this trust, this trust and reliability aspect. Right, because that, you know, even if, if we're talking AI chatbots, right, obviously trust and reliability is still huge. But when we start talking about agents it becomes even more paramount. So for our, our live stream audience, I'm sharing Galileo's agent leaderboard. That's on a hugging on the hug face space here. But Yash, maybe if you could walk us through what is this agent leaderboard and what does it help people understand at least when it comes to trust and reliability.

Yeah. So before we walk through this Jordan, I want to at least preface this by like just by creating the understanding that reliability in agents come through like you know, come through a foundation of really test driven development for these agents. Having high quality evals for an agent and building agent evals is not just like evaluating an LLM. It's more about creating unit tests for an agent and an integration test for an agent. Just like software engineering best practices. When we built these agents, the eval tests that we are building, they're extremely catered towards the use case that you're building for it includes the metrics that understand what the agent is doing and what is the expectation from the agent. And it includes a data set that is actually applicable to that use case. Now once you start with that strong foundation, you can apply these eval tests in real time as well. Because in production your agents are going to be non deterministic. So these strong evals power really valuable observability. And then even using these strong eval tests, you can create strong guardrails that can prevent the outcomes at all. Imagine if your agent started hallucinating, started making the wrong tool calls, API calls, and if you can prevent it in under 300 milliseconds, that is super real time, then you have the end to end reliability pipeline in place. And what we're doing here with the agent leaderboard there is actually helping teams understand just how do we start evaluating LLMs and agents. And so with the agent leaderboard, our teams at Gallerio, we've selected many, many models and actually applied them to several use cases. There are data sets, there are agent prototypes and we run them across different LLMs and then we evaluate them for the task that they're doing. So we have the entire GitHub repo link there. Anyone can go in and start seeing that and actually start doing their own EVAs with the GitHub repo and coming up with their own sort of ranking of like how, how can I architect the best, most accurate agent for my use case? This leaderboard is extremely popular. You know, we've, we've received millions of his hits on this and this just shows that developers and teams, they, they really want to go in and see how these LLMs apply to real world agents and not have some academic benchmark, you know, rank these models for them that don't represent real world use cases.

So you know, as an example, maybe help us simplify this for our non technical audience. But there's a data set here, right, that kind of tells users a little bit more about what's actually going on under the hood. Let me see if I can get it in the, in the correct window. There we go for our live stream audience. But you know, as an example, if I scroll down here in the data set, you know, one of these conversation pieces is, you know, who won the last World cup in football, right. And then there's a way to test if these different models in an agentic framework are able to retrieve the information correctly. So number one, is that how it works? But number two, maybe can you just tell us how it works with this data set and then how you're able to tell, hey, which models are maybe most reliable or trustworthy in these agentic settings.

Yeah, and I mean, I'll answer your last question first because this is like an ever. This is an ever like moving target, right? Like new models, new capabilities are coming out so fast and you know, our teams are even having a hard time to keeping up at that, honestly. But which model is the best? That answer can only be like gotten by going to the leaderboard and seeing what is the live, you know, comparison like. And we keep updating these fairly regularly in terms of the data set. Right. Like in you'll see how agents. The most common pattern for an agent is to make a tool call. Right. And that's where even, you know, we've seen how popular MCP has become. Like the expert protocol from Anthropic, because it really reduces the overhead of an LLM calling a tool. It just simplifies that at the most basic level. Right. And that makes agent building a lot more exciting and easy. So in this data set we're seeing agents make a bunch of tool calls and the most basic requirement we want is from an LLM to not make the wrong tool call, to understand the action of tool calling really well. So that's what we're seeing in this data set. And then even the evaluation metrics are based on tool selection quality for this particular data set.

And then as we talk about, right, you kind of went through this crawl walk run, you kind of mentioned there multi agent setups or scenarios. You know, what should business leaders even though, yeah, for some industries it might be a while until we get there. But what should companies be keeping in mind when it comes to multi agentic AI and making sure that they have reliable agents?

You know, very soon we're going to get to a world of like having small agents talking to each other. So you know, the kind of like micro agentic architecture that I talk about which is truly multi agentic systems. For example, if I wanted, and this is a very simple example that like, you know, a lot of people would have heard about but for those who haven't is let's say there's a travel agent application like it's helping me book like create firstly create my itinerary plan for my vacation and then also help me book things end to end and then coordinate any logistics for me if I had to ship things when I'm traveling, etc. Right. So it can be an end to end thing. Even a supply chain agent that I Talked about, which you know, one of our customers have, have implemented is literally like you know, looking at the inventory in their warehouses and automatically placing orders. And so there are multiple tasks that this entire workflow is doing. Right? Right. From planning to making reservations or making orders and then execution of those orders. Right. So that's our tracking of those orders. And these can be implemented as individual agents that talk to each other and, and yes. And over time the planning agents capabilities include, just can grow. It can not only help me book hotels and flights, but also restaurants and excursions. And for that it has to talk to other agents that are, that are, that specialize in booking, making reservations for those things. So now in that world of multi agent existence, there are three things that need to be really solved for. First is again trust. When an agent is talking to another agent, how can it trust that other agent? How can it, what kind of observability can we get from the other agent to say like look Jordan, right now I can see you, so I know it's you, but if, if it was some bot that I was talking to that I wouldn't trust it as much as I can trust you because I can see you, I'm getting that observability feedback that it's you. Similarly, and you know, there's also other challenges beyond trust is authentication. Like how do I know that it's, you know, an agent passing, handing off the task to another agent can authenticate me as an end user. And the third thing is in communication. And these three are kind of the top priorities for most of us in the agentic space. And communication is solved by a bunch of things. You'll see there are MCP based patterns that are emerging. Google's announced the A2A protocol. Galileo has been part of even founding this agency organization, which is a truly open organization that is making multi agentic systems easier to solve for. And you'll see that overall the world will kind of converge on one protocol where any heterogeneous agent built in, any system built using any LLM is able to talk to each other.

So Yash, we've covered a lot in today's episode. But as we wrap up, what is the one most important takeaway that our audience needs to know when it comes to building reliable agents but for specifically mission critical tasks?

Yeah, thanks Arden. I think the one thing as we head towards a multi agentic world, the one thing is that let's get our single agents to be more reliable. Now you know, it's very important even if you're not launching them in production. There is a whole stack, there's a whole playbook being created on how to build, how to launch. Like just the CICD aspect of things. How do we make them reliable in production? Like there's prevention, there is mitigation. How can we prevent bad outcomes from happening? How can we mitigate them? Because in a non deterministic world of software, again, good evaluations followed by good reliable preventions and mitigations is going to be absolutely critical to be successful in the world of non deterministic software.

All right, some important information, not just for the future, but things you need to start doing today to prepare for the multi agentic future of tomorrow. Yash, thank you so much for your time on joining the Everyday AI Show. We really appreciate it.

Pleasure being here, Jordan. Thank you.

All right, now as a reminder, y', all, that was a lot. If you missed anything, don't worry. We're going to be sharing it in our newsletter. Some resources that Yash talked about, the AI Agent leaderboard. It's all going to be in there. So if you haven't already, please sign up for our free daily newsletter@your everydayai.com. so thank you for tuning in. Please join us tomorrow and every day for more Everyday AI. Thanks y'.

All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.