EP 471: Inside Multi-Agent AI - Rethinking Enterprise Decisions

This is the Everyday AI show, the Everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

It's no secret that agents are already the buzzword of the generative AI world in 2025. But what if we zoom out and even just think of agency decision making process? What does that mean? Should we be handing over as large enterprise companies, should we be handing over that agency to AIs, to large language models? And not only that, what happens when you do that and you get some success and you want to scale and you want to be working in multi agent environments because whether you've realized it or not, at your company, that time is today. So today on Everyday AI, I'm excited to have a guest with a great wealth of experience in this area who's going to be help walking us through and inside that multi agent AI in rethinking enterprise decisions. So welcome to Everyday AI. If you're new here, my name is Jordan Wilson, I'm the host and we do this every single day. It's a daily livestream podcast and free daily newsletter helping us all learn and leverage generative AI to grow our companies and our careers. If that sounds what you're doing and what you're trying to do, this is your new home, your other home, that's our website, your everyday AI.com. so there you can go sign up for our free daily newsletter. We will be recapping today's conversation as well as keeping you up to date with literally everything that you need to know to be the smartest person in your company at AI. And you know, we have hundreds of episodes with some of the brightest minds literally in the world on AI and today's conversation is no different. All right, so enough of my chit chat. If you're here to, you know, normally we go over the AI news, this is technically pre recorded, debuting it live. So if you want that AI news, it's going to be in the newsletter. So make sure you just go check it out there. All right, enough chit chat from me. I'm excited to bring on our guest for today. So please help me welcome Babak, the CTO of AI at Cognizant. Babak, thank you so much for joining the Everyday AI show.

A pleasure, love to be here. Thank you for having me.

This is going to be a good one. So before we get into your background in AI, which is extremely impressive, can you walk us through? I'm sure most people are aware of Cognizant but for those that aren't, can you just tell us a little bit about what you all do and what your role as CTO of AI entails?

Yeah, Cognizant is one of the world's foremost technology services companies. We're based in the US but we're everywhere around the world. We're more than 360,000 employees and we are an AI first company and pioneering in the fields of AI and multi agent systems and working with clients in all sorts of different verticals. And yeah, you know, I joined about six, seven years ago and it's a very large company but it has a culture of a startup. I'm a startup guy myself so that's part of the reason I've stuck around here. It's, it's really cool to be working here.

Yeah. And speaking of you know, agentic AI environments, you know, can you tell us a little bit about Cognizant Neuro? It's been what, like a year and a half now? How is that going so far? What does it entail, you know, for those that aren't aware?

Yeah, it's going really, really well. We launched Neuro AI as you said about a couple years ago and it's because we recognize the fact that people are moving from, you know, give me the insights and I know how to make the decision to, you know what, give me some recommendations. Like it's not just the insights, it give me recommendations because making decisions in the presence of a lot of data and in a multi objective way is very, very difficult. And so that was the premise for creating Neuro AI and you know, it was, it's a very technical platform and you know, not too many people know how to build decisioning systems. So it had a whole certification program and so forth. We would quote about 10 to 12 weeks to create a POC. What happened last year was that we started identifying the platform. So agentification, you can think of it as, you know, taking the modules in your software and replacing them with agents that talk to each other. And believe it or not, that POC period of 10 to 12 weeks is now down to 10 to 15 minutes.

Wow.

So it's, it's simply amazing and just talks to the power of identification. And then we've of course now started looking at how you could expand that beyond just software and whether, you know, an organization itself could be more multi agent based.

So before we get into the details, which I want to, this is something I'm personally giddy to talk about but let's hit Rewind. Like what the heck is an agent? Right? I, I think it's a word now everyone throws around. I feel it's like when two or three years ago public companies just, you know, said the word AI as many times they could in their earnings call. But now everyone's just saying agents everywhere. Like what the heck is an agent? And what's the difference between, you know, a traditional like large language model and an agent?

That's, that's a great question. You know, we were, some of us AI folks were at a roundtable at Davos a couple of weeks ago in Switzerland and we were talking about how last year at Davos you would walk down the promenade and everybody was talking about Gen AI and we figured, you know what, these CEOs are going to go home and say, you know, I want Gen AI, by the way, what is genai? And now it's the same for agents. Like you walk down the promenade and everybody's talking about their agents. And the CEOs are probably now back home asking their technical folks, you know what I really need? This agent thing, what is it? So, great question. So agents have been around. I worked on multi agent Systems in the 90s and the fact is that if you take an AI system and give it some tools and some level of autonomy in deciding when and how to use its tools, we can then call it an agent. It looks as it's at its environment, at the task at hand, at its job description, and then decides how to use its tools to fulfill whatever responsibility or task it has been given. The reason for this resurgence today in popularity is because by using a large language model as the brain of an agent, that allows it to reason using the reasoning of these large language models and also it understands language. So you can just simply express to it what its responsibilities are, what the scope is, what you expect it to do. So intent driven and then it can decide and more often than not it makes good choices as to how to use its tools. So the modern day agent has a pretty powerful brain in a large language model. When you think of a large language model, that's just a model, you give it some inputs and it produces some outputs. It doesn't really do stuff. It's pretty general purpose usually, but it doesn't really do stuff. The moment you recast it as an agent, you write some code outside of that model that allows the system to decide, so it actually enacts the decisions that the large language model is asking it to do. One example I use is we all know large Language models are pretty good at writing code. We have all these git copilots and so forth. If you ask a model to write some code, it will write some code. If it's not too complex, you can take that code and run it and work fine. If you have an agent and you've consciously decided that that agent will have as its tool set this ability to run code in a container and see the result of that code, then the same task of writing the code suddenly becomes much more tractable because the LLM can say, hey, I want this piece of code that I wrote to be run. And then the external code within the agent runs it, shows it the result. Maybe there's a runtime error or something and it can correct that and iterate on it before it comes back to you and says here's the piece of code that you looking for. And by the way, I actually ran it, here's a sample run. So right there you can see same large language model, but one of them is more powerful because it has agency, it can actually use tools.

So Vivek, you said that you've been working on multi agent environments since the 90s, so I guess why now? Why is this, you know, at the all, you know, at the all time high? Is it just because the, the advancements in large language models? But why today and in the very near future is the multi agent environment going to be a huge driver in the enterprise?

It's just happening organically and that's because, you know, the whole structure, the underlying requirements are all there. It's just all come together. When you look at the late 90s when we started working on agents and multi agent systems, we had a lot of missing pieces. Like we, we didn't have the processing capacity. The, the world wasn't like a bunch of API and microservices like it is today. You didn't have the luxury of a model that could understand your natural language. So an agent was much, much weaker and more in the lab than what we have today. But why today? When people started looking at large language models, immediately they're like, oh what we can do copilots? We can have these large language models help us edit stuff or do stuff for us. The moment you start thinking about doing things, in other words, allowing whatever the large language models suggest to actually be actuated in the world, you have an agent. And so over the past year and a half, two years, organizations have started to build agents already without even calling them agents. There are these one off large language models that do certain stuff for us. You know, one of them is extracting data out of unstructured, you know, documents. The other one is, I don't know, editing something or filling out a form for you. The reason why I'm saying it's organic is because it's very natural for you to want to have these agents to talk to each other. Because why would you now have a rolodeck of agents that you've implemented in your organization, have to know which one to go to talk to. Like why don't they talk to each other to figure out which one of them or more than one is responsible for handling some, you know, input that you give them, some, some inquiries and command. So it's kind of happening organically right now and we're starting to see companies, you know, demand multi agency for their operations.

And, and I think sometimes people think that this is a, a highly technical endeavor and in many cases it is.

Right.

But for, for listeners of this podcast, I did this recently, right. I did it live here on the show. I had chat GPT's operator go out and use Gemini and then send an email. Right. I had it do some tasks that I would normally do. And so, you know, very simple, but multi agent environment. So, you know, as we talk about doing this at scale and, and at level, what are both the promise of the multi agent environment and then what are some of the challenges or the things that, you know, business leaders really need to consider?

Yeah, the promise is breaking the silos, making things much smoother and operationally more productive and efficient. Because, you know, regardless of the entry point, you have a whole organization of agents, each responsible for some task within your organization that can listen in on your requests and given whatever authorization you have and so forth, they can come together and do something complex that would otherwise have taken you a long time. Lots of forms filled, picking up the phone and trying to find people. I'll give you an example. Like at Cognizant, right now we're identifying our intranet, which is a collection of many, many apps. You know, we're so large that our HR department has its own IT department, you know, our finance department. Right. And each one of them has come up with their own agents. So we're actually incrementally plugging these agents together and creating this multi agent intranet. Now, Interesting. There are certain things you can do here that you can do very easily. I can go in and say, hey, my laptop has an issue or I need, I need copilot for this. That how can you provision the. And Very quickly it finds the agent that's responsible for that copilot provisioning. It checks your authorization and gives you a form and pre fills it and you can click, okay, great, that's easy. But what if I had like a life change event just recently my son turned 26, right? I didn't know what to do, where to go in the system for that, so all I had to do is type in my son just turned 26. Just imagine typing that into a search engine on your intranet. I mean, you're not going to get anything useful. But the system kind of looked around and it's like, okay, your payroll is going to change, your benefits are going to change. And hey, by the way, congratulations, if you want to take some time off to celebrate your son's 26th birthday, I can provision that. Like I can take that time off for you as well. Things that, you know, off the top of your head you're thinking, wow, you know, yeah, some of these are very useful for me and they're coming from disparate sources from different departments. So that's the promise, that's the amazing promise. The peril though is you don't want to leave this to happen organically. I mean it is right now, but you really want to have control over this. Remember, with agents comes this deferral of autonomy. So you're actually asking a large language model based system to take some decisions autonomously on your behalf. So you want to make sure this is done responsibly, safely and in a manner where if stuff breaks, you can actually fall back to a rule based, you know, mechanism. So while it's not very difficult to connect agents to each other, it is rather dangerous because very quickly you lose control over what's happening in your organization. And so exerting that control, making sure it's designed. One last thing I want to say here is with agents and multi agency, we're moving back into an engineer discipline with large language models. We kept thinking, okay, this is one model that does everything. So I just express my intent and it'll do something for me. The moment you break it into smaller agents and for each agent you give it some microservices, some API, some data to be responsible for. You have to consciously decide what is the subset of tasks I'm going to give to this agent and where am I going to defer to it to do something autonomously and where am I, do I want to be part of that process? I want it to ask me that kind of engineering, that kind of design Needs, you know, thinking means for us to know what the process is. It's kind of a holistic perspective and safety and responsibility is super, super important here.

So speaking of that safety and responsibility. So you know, you said in the instance where if something fails, you need to be able to fall back to something more rule based. Right. And I keep thinking of, you know, quote unquote, early in the days of large language models, right? Like this is decades ago. But you know, at least, you know, after the chat GPT moment everyone's talking about it. There was so much of an emphasis placed on hallucinations. Right? Now I don't know, maybe, maybe I talk about AI too much. It doesn't seem to me as if there's that much attention being paid to what happens now in that multi agent environment. Right? Because everyone was worried about hallucinations, but I don't know, is it the acceleration is too fast, is the opportunity too great that maybe people aren't worried the same way about like agent alignment that they were initially about, you know, hallucinations from a single large language model?

Well, they should be. The hallucinations have not gone away. You know, with progress in building these large language models, we can reduce them. We can never guarantee that we are eliminating them. So one of the things we're doing is we're basically giving away consistency in our engineered systems in favor of getting robustness. So it's consistency versus robustness and we gain a lot by getting that robustness and the autonomy that we get from these systems. But we have to be very mindful of the fact that you might actually ask the same query from even an agentic system and get different answers. Like 9 times out of 10 it'll work the same. 10th time it might be different. There's a reason why there's a regenerate button on ChatGPT and Claude and everywhere else. That's exactly because of that. Now remember though that if what you're doing is like what OpenAI is doing with operator, like one LLM that does everything, then you're much more subject to confabulation or what's commonly known as hallucinations and the inconsistency. Whereas if the same task is broken into smaller tasks, like I've got one agent that's helping me, you know, look at hotels, another one that's helping me look for, you know, various different destinations, another one that's looking at pricing. Now I've actually reduced the scope of each one of these agents and by reducing the scope and being more explicit about what I expect the LLM to do, I'm reducing the inconsistency and hallucinations as well. So multi agency actually helps us reduce that. Again, we're not eliminating, but we are reducing. But there are other techniques as well. Actually our lab just came out with a technique. We had a paper at Neurips just recently where we can actually measure uncertainty in the output of a large language model. This is a huge breakthrough, right? And so you can actually ask this system to give you like 10 responses and tell you what its confidence is for the same input on those 10 and pick the one that has the highest confidence. That significantly reduces the inconsistency and hallucinations, but it still doesn't eliminate it. We, there's no guarantees.

So one thing that, you know, I'm always thinking about, and I've talked about it with a couple of guests before on this show, is, you know, let's say an agent, a Single agent is 1 degree off target, right? In if, if that goes for a while unchecked, that one degree is very far off the destination. But generally in one human, one agent scenario, the human is hopefully paying attention and doing their good human in the loop job, right? Making sure that that 1% goes as close to zero as possible. What about the compounding factor though, when you have a multi agent environment, right. I almost think of it as, you know, the someone in an air traffic controller and there's all these planes flying at once. They can't possibly give enough attention to all of them. So how is that going to work? And how should the CTO CEOs be addressing this issue of kind of this alignment or this compounding? You know, if an agent is 1% off and they're working in a, you know, a swarm of agents, right? So how do you see that playing out? And what should business leaders be doing?

Yeah, you know, the way we deal with that with humans, I mean, humans are these black box intelligence systems that do. I mean, we don't call it hallucination, but they are inconsistent sometimes they are erratic, they do things that we didn't expect. And so how do we deal with them in human organizations? Redundancy, like you actually have more than one person checking each other's work, double checking, triple checking. And so that whole error compounding that you're talking about, that actually helps reduce it because you have multiple agents checking each other's work. So we can do that. When large language models came out, some folks even within the AI community were like, oh, there's this compounding issue because it outputs one token at a time and there's no way that it's going to remain consensus after a while. That's a fallacy though. Remember that as it's moving its window forward, it might have actually come up with one token that's kind of off, but then when it actually looks at that token along with all the other tokens and gives you the next output, it can correct for that. And so that's something that they didn't think about. The same kind of goes with multi agent systems if you want your system to be safer. Beyond the fact that, as I said, you can, you can add rule based fallback and you can have human in the loop. All those things are granted and great, but you can also build redundancy into the system. That does make it cost more and it might slow things down. But there are certain processes where, yeah, you would willingly pay for that extra cost to make the system, you know, more resilient that way.

So another thing that I think is worth talking about in these multi agent environments, I don't know, I feel in, you know, maybe 20, 2023, maybe 2024 even you had teams, right? Small teams were, you know, working with their group of 10, 20, 50, and trying to find the, the one, you know, AI solution or the one large language model that could really help propel them, you know, so team of a lot working with one. Are we now going to be flipping that on its head now? Is it going to be, you know, one person working with 50 agents or is it still going to be, you know, groups of individuals working with groups of agents? How might that work out?

Yeah, it's hard to say and I think it depends on the use cases. I do think that we would all benefit by, by getting into the habit of, you know, creating and using our own little team of agents that would do stuff for us. Let me give you a quick example. We have this, this thing we do at, at our lab called the FedEx Day. It's like a 24 hour, you know, FedEx. We deliver basically like a hackathon. And I was thinking, look, I get a lot of email that comes in and I play the router, the email comes in and I'm thinking, you know what, let me send this to my product manager, send this to my head of research, let me get their opinion and stuff like that. Can I automate that? So what I did was I actually replicated our lab like the hierarchy of our laboratory in agents, right? And then I had, as the email comes In I would get the email to the top agent, the top agent would check for spam, would check for urgency, and then it would actually distill the email and send it down to the hierarchy. And it was as if my team is looking at the email and they would come back sometimes even, you know, giving me like the text of the email response that they thought I would be sending back. So even at a personal level at work, I think that one to many, as I described right now, will make sense for a lot of stuff that we do right now so that it frees us up to do the real work. But having said this, there are processes where we will have to have the human in the loop on an agent per agent basis. The agent cannot make the decision, it can give recommendations, but it can't actually move ahead with the process without actually making sure that the human is taking a look at that, if anything, because we need a responsible party in this process. We need someone that takes responsibility for the action. Even if we believe that the AI would take a better decision than the human, right now, society is really not ready to defer completely to, to machines for many types of decisions.

Yeah. And speaking of, you know, decision making and machines, you know, I'd, I'd kick myself if I didn't ask you this. And for, you know, our audience that is isn't aware. So you were the, you know, co inventor of a technology that essentially led to series development.

Right, Right, Yeah. The natural language technology behind Siri.

Yeah. So, you know, so you have many decades of experience. Right. I, I look at this transition that we've kind of been going through over the last couple of months. Right. Or at least in, in popular technology terms. Right. But how we've kind of gone for this robotic process automation, the rpa, to the, to the qa, you know, computer using agents. But I don't know, to me it still seems almost like archaic. Right. Like it still feels to me almost like RPA where it's like, oh, I'm having to click this quote, unquote, record this. Even if I'm recording something, you know, through a large language model, is it going to get to the point where we're just like this, using our voice? Right. Natural language. Right. Natural language processing to, you know, a multi agent kind of workforce. Is that where we're going and if so, how long or what needs to be accomplished before I'm just talking to a group of agents that are helping me accomplish my day to day tasks? Are you still running in circles trying to figure out how to actually grow your business with AI. Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too, just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen AI.

That's, that's a really good question. I don't know what that mode of interaction is going to be, and it might actually be different depending on the use case, I think. I mean, having been involved with these, you know, conversational systems for many years, one of the things I found is that while we seem to be okay talking to our car or talking to our dogs, we don't quite find it natural to talk to our, like, I don't know, a cylinder in the corner of the house, you know, so anthropomorph, anthropomorphizing seems to be okay as long as that being is like moving around and has eyes and, you know, just subliminally we consider it an animate, you know, intelligent being. So, and there are many work situations where, you know, I don't know if it really makes sense for us to build that kind of humanoid sort of companion just to make the interface more natural for us to just talk to it as if we're talking to someone else. It might be. And there is a line of thought that says we might want to consider not doing that. And actually always. There was actually something yesterday I was reading about how we should regulate so that robots don't speak like humans, they actually speak mechanically like old science fiction movies, just so we know that we're not talking to a human. And so there is this line of thought that says, yes, use the intelligence in the box, but there may be interfaces that, yeah, have text in them and you can express intent, but there might be, you Know, in the old days, when we were in the pre series days, you know, in our, you know, youthful exuberance to, to bring intelligence to the world, you know, we had the system that could work the TV set on, the DVD and the, and the lights and stuff. But it felt so silly to say like turn the lights on and volume up, volume up, volume up, volume up. And that's, that's just stupid, like just turn them up, you know. So there are certain things in the interface that, you know, we've come a long way since just you know, simple text based interfaces.

So you know, you mentioned two great use cases or examples of kind of a multi agent environment. So you know, kind of the Internet example and you know, HR agents kind of helping you, you know, handle a query and then the email example that you just gave. What do you think, whether it's internally or things with clients or things that you've seen, what do you think are going to be some of the first or best use cases that kind of go quote unquote mainstream in the enterprise world of this multi agent system?

Yeah, I mean there are some obvious ones which have to do with the entry point. Maybe a consumer talking to a business in a B2C setting like, like support line and so forth. These are very obvious cases where we've been working on this forever and these agentic systems can even make them more pleasant and more productive. There are internal use cases where an employee needs to get access to something in the organization. The organization is huge with many moving parts. You can't simply know everything about what everybody is responsible for and doing. And so that's the intranet example. There are other processes. There are cases where you basically look at the existing organization and the nodes, the organizational chart literally and you go, okay, here I have various different responsible nodes that are human driven. How can I augment them and then connect them just like they're connected in the world right now and then give access to every single node in this organization to be able to use their sort of buddy agent as the entry point into the entirety of the enterprise. So I think we will see a lot of that coming in. But I think the first set of use cases are going to be the more natural use cases that grow out of areas that we've been exploring already and we'll see a lot of productivity out of that just by creating these connected agents together. Let me just say that we are sitting on a bed of microservices and API already. So the tools are there, the organization and the processes are there to be discovered, but they can be identified and then you have. So in all of these cases, we're talking about fully aligned multi agent systems. So, you know, it's your organization. So all the agents better be working together and be friends. But if I have an agent network that on my behalf, a consumer or B2B or something is supposed to go talk to another business, that's where, you know, that alignment kind of breaks down. And so we will move into a case where agents are actually agents. Like they're representing us or our organization and communicating with other agents representing other people in other organizations. And you know, just recently I, I did a, an article on this. I actually created this multi agent system similar to operator but, but multi agentic that helps with decision making. And I connected it to agent networks for some travel sites and it was amazing just to read the interagent communications because it's all English. You can read what the agents are saying to each other. And so it would come back and say, hey, you know what, I got this deal from Airbnb that's beating what you're giving me for a weekend stay in San Francisco. Can you do better? So this is like my agents talking to the, I don't know, Expedia agent and trying to come up with a better deal for me.

Yeah.

So that's, that kind of gives you a sense of where this thing might be going.

Yeah, it's like, it's like have, I'll have my agent go contact your agent.

My agent is going to call you.

Yeah. And like I was actually having a very similar conversation with someone the other day about like human in the loop. And I'm like, I think it's going to turn into like agent in the loop where you have, you know, certain agents that are kind of like your direct. Right or their direct towards you. And those agents are talking to other agents and you're just checking in with one. So that's not a crazy idea that, right?

No, I don't think so. I think like, it's only crazy in that we're not used to it just yet. But I think we will get to a point where we're, you know, this is a knowledge worker in a box. Like, how cool is that? You can set it up to do whatever you want it to do. So yeah, I think we will be. And you can program it using your own language. Like the barrier to entry is very, very low. So we should all be thinking, you know, what kind of agent would I love to have right now for What I'm doing in my work and my daily, I don't know, my hobby, my entertainment. Let me go build one or a few agents to do that for me. Like, why not? I think that will come. Maybe it's generational, maybe it'll take a while, maybe the LLMs need to be a little bit more powerful, but it will come. Yeah.

All right, so we've covered a lot in today's conversation, but maybe here as we wrap up, what is your one most important takeaway that you think business leaders need to know when it comes to, you know, the future of enterprise work in this multi agentic AI system?

Yeah, I think we want to liberate our employees to really spend time thinking about how they want to use their tools versus actually filling out forms and, you know, that whole, you know, grind of, you know, figuring out who to talk to and what to do. So I think this actually will make organizations much more agile and much more fun, quite frankly, to work on in organizations like that. So if you have that as your vision of where you want to take things, I think you need to start thinking of agents not just for developer productivity or for your call center. You really need to think about it at an enterprise level and strategically.

So good. Vivek, thank you so much for joining the Everyday AI Show. This is an instant. This is one of those instant replays. So if you're listening to this now, you got a bookmark. Come back and listen to this again in three to six months. I guarantee you it is still going to be a gem. So thank you so much for taking time out of your day to join the Everyday AI Show. We really appreciate it.

My pleasure. Thank you for having me.

All right, a lot that we covered there in a little time, so don't worry. Maybe you were at the gym or on a walk and you missed something that we covered there. Don't worry, we're going to be recapping it in our newsletter as well as a whole lot more. So if you haven't already, go to your everydayai.com sign up. We're going to be recapping today's episode. I can't wait to re listen to it. I hope you find a ton of value. So thank you for tuning in. Hope to see you back tomorrow and every day for more Everyday AI. Thanks y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.