Ep 492: Microsoft Copilot’s New Agents: Insider tips how to make them work for you

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life.

It's been a. It's been one of those weeks in AI development where, you know, you're like, did all of this just happen over the course of a couple of days? It seems like all of the big players have offered some something new for us all to take advantage of this week. And I think Microsoft maybe had some of the biggest announcements that I don't think enough people are talking about because this is tools now, new AI agents that we can all use. So today I'm excited to talk about and have a great guest on the show, returning guest, by the way, to talk about Microsoft Copilot's new agents and give you some true insider tips on how to make them actually work for you. All right, I'm excited for today's conversation. Hope you are, too. What's going on, y'?

All?

If you're new here, my name is Jordan Wilson and this is the Everyday AI Show. This thing, it's for you. It's your daily live stream, podcast and free daily newsletter helping us on not all just keep up with AI, but how we can use all these new tools and all these new advancements and new large language models to get ahead of, to grow our companies and our careers. So it starts right here by listening to this live stream Flash podcast. This is where you learn, but to leverage it, you need to go to your everydayai.com. sign up for the free daily newsletter. We're going to be breaking down the most important takeaways from today's conversation, as well as giving you a lot more information that you need to know to actually take advantage of what we're going over today. So make sure you go do that at your everyday AI. All right. If you're looking for the the daily news, that's going to be in the newsletter as well. We got to take advantage of every second we can with today's guests. So please help me welcome to the show. Let's bring them on. There we go. Ray Smith, the VP of AI Agents at Microsoft. Ray, thank you so much for coming back. Second time to join the Everyday AI Show.

Thanks for having me back, Jordan. Clearly the first time went okay, if that's why you have me back on. So I'm glad to be here.

Yeah. And like, obviously, right? Like, if you can talk to the VP of AI agents at Microsoft, like, we gotta have the conversation. So many people are asking me, you know, hey Jordan, what's new with all these agents in Microsoft? So I said like all right, let's ask, let's ask the man himself. So Ray, what the heck is new? I mean there's so much new inside Copilot Studio with these new AI agents. But walk us through some of the hot off the presses announcements.

Yeah, I mean think when we last spoke back I think was back in November. It was around the ignite time frame, there was lots of releases and as you touched on it just seems week after week things are just moving fast and I don't see it slowing down. So this is going to be week after week, there's going to be feature releases, there's going to be new capabilities and actually I think we're at this new era where the customer engagements, the real production use cases is pushing the tooling across the full Agentix stack more and more. I think this week we announced deep reasoning capabilities to really kind of bring that kind of analysis, research, report generation into typical workflows. We also kind of announced around agent flows which is a new way of bringing kind of what would have been kind of traditional RPA but bringing some of these guardrails and real deterministic behavior as a key tool into how we build these agents. So this mix of deterministic and non deterministic. So we asked the agent to reason over the tooling we but maybe choose its path and for certain parts of that path we always wanted to behave the same. So that's kind of agent flows. And actually what we also announced is that the autonomous agents capabilities has gone to general availability. So that's in the last couple of days. So yeah, so lots going on and I think when we last spoke I think we were talking about 100,000 organizations using Copilot Studio. That's now at 160,000 and there's like, I think the stat is like over 400,000 agents were built in the last three months alone. So we're, I would say we're still at early innings but it's only accelerating. And the best part of my day is meeting customers and they're like do you think you could do this and could do that? And I'm like technology is no longer the barrier, it really is the focus on these use cases. And we'll see across every level of the Agentix stack new capabilities around testing and evaluation frameworks, new tools, improvements around rag and orchestration. But the net net is just for end users and for everyday people is it's going to be easier and easier to build these apps in this new world. The biggest programming language is going to be just natural language. It's not going to be the code that we are all familiar with. It's just going to be describing what you want and you're going to interact and iterate to build these solutions. So it's a very exciting time for business leaders, domain experts to say hey, I don't need to take a ticket with it or I can work more easily with it at least to, to build these solutions quicker.

And if those two things right there that you just heard from Ray didn't excite you, right, like so, you know, obviously everyday AI, you know, we're for non technical people. Those two things, technology no longer being the barrier and the most important kind of programming language now is natural language. Those two things. I'm like I'm excited to dive in more so but before we do, I first want to zoom out a little bit Ray. So you know you said now more than 160,000 organizations using CoPilot Studio. Amazing. But for those that started using Copilot Studio, first let's just give everyone a quick overview. What the heck is Copilot Studio? How do you access it? And then I'm really excited to talk about these two new agents.

Yeah, so it's go to copilotstudio.com and you can set up a trial and get started pretty easily. What it is is effectively it's a low code agent or app building in this new AI world framework or solution. So it's trying to abstract away all the complexities of model selection frameworks, how we add tool, we kind of bring knowledge sources together. So all with kind of enterprise grade controls, observability and governance. So it's trying to bring this enterprise grade solution building or agent building framework but also make it really kind of easily for the average kind of almost business user to describe what they want and to iterate through that process.

And I know it's even access has changed a little bit. But is this available if everyone has Microsoft 365, you know, seats for everyone in their organization, is this available to them? And then second part, if it's not, can they still use the, you know, kind of the pay as you go pricing to take advantage of these two new agents in Copilot Studio?

Yeah, so there's, there's probably a number of ways. Obviously first of all it, you know, easy to go in and just get a trial set up but in terms of beyond that trial and beyond that, whatever it is, 30 days. Then it's really a case of there's a number of ways either you buy M365 copilot licenses and you get certain entitlements or you put in your Azure subscription and it draws down against your Azure commitments or you prepay and buy effectively message packs or kind of a buy packs that draws down on that meter. So we're trying to make it even easier for people to get started because in this new world it really does take people just get hands on with the tools. It's like you build your first agent then you get bitten by the bug and you're like oh I'm going to build the next 10. And the first one is always the hardest because it's the kind of the concepts around how rag and how you use actions and connectors and triggers to make it autonomous. But after you kind of get that first one then it's kind of I get the conversation with customers. It's like we heard what you said, we liked it. But then when we built our own the penny really dropped and we now are just like we want to agentic transformation or AI first transformation in their businesses.

Yeah. And, and I do think the usage based pricing was super smart because yeah, I even remember walking around, you know, talking to people at Microsoft Ignite and I'm like why? You know, for, for people that aren't using this. Why? And you know at the time people were like oh, you know, we're not sure if we want to you know, roll out hundreds or thousands of licenses. So I think that was a smart move by the way. But let's get into the good stuff. Right. Let's talk about the deep reasoning agent. Like, like what the heck is this? I'm excited about it.

Yeah. So foundationally these new class of models are based on kind of reinforcement learning and really what it is that difference is is that it's like a model that can kind of verify itself and be trained based on the output. So it's kind of like we call it think think deeper or kind of like think longer or kind of self analyze. So it's like it's able to look at itself these models based on the output. So if you think about that it has to be a verifiable output. So when you are creating code or creating an analysis or some sort of research or generating a report, it's something that can be easily evaluated compared to just let's say flowery language. So it's a different type of model and it's all based on that reinforcement learning. And we're familiar with OpenAI, so they had 01, then O3, Mini and Pro. So there's just different flavors of it. There's obviously deep seq. So we think there's going to be a number of these reinforcement learning or deep reasoning models that will emerge. Obviously our view at Microsoft is to make sure we make all of these different models accessible to our users, both in Azure, AI Foundry and also within Copilot Studio. Because ultimately it'll be around picking the right model for the right job that's optimized and in the fullness of time, customers will fine tune those models for their own use cases. So that's fundamentally what powers it. So you have these deep reasons models on top of that. People are like, okay, that's cool, you give me a model, what can I do with it? And they usually start to use it almost like an action. Say at this part of my process where I've gotten all this information from the web or I've pulled all this information from SharePoint and some content, now I'm going to give it over to this model and it's going to behave better than the standard orchestration models, let's say 40 or 45 in OpenAI's case. So it'll kind of reason or think deeper across that information so they start to use it more like an action. And this is very useful if you can think about like you know, inventory optimization or research about a lead and or a company as it comes through a sales development workflow. So this becomes a key step. And I think what a lot of we're hearing about in the organization, you know, in the market is people are abstracting another level and building these apps or research agents which is really packaging up the research models, maybe access to the web or certain tools. And it's allowing people to say I want to use this building block either to generate a report or a response to an RFP doing all the analysis or maybe I'll use that as a building block into a multi agent scenario such as, let's say sales development, where you say I want you to go off and research this later this company so that it can be used by the next step in the process. So fundamentally deep reasoning or research agents or analysis agents are almost a fundamental building block for most processes because that's the thing we as humans do really well, which is reason over a lot of complex information and variables and make decisions downstream based on it.

Are you still running in circles Trying to figure out how to actually grow your business with AI. Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen AI. So yeah, great, great call out there on the Think deeper. Like I think, you know, Microsoft even made a lot of that available for free. So you know I did cover that in episode I think 479. So if you want to know more about that, make sure to go check that out. But, but Ray, like I think one thing, you know, people are going to have questions about or just be curious about right now when we see this thing, you know, deep research or Deep reasoning. Right. Like we think of that certain, you know, brand. Like I think OpenAI's deep research product, one of the like for me personally like I love using it. So is that kind of like what this is? Is this Microsoft's version of it? How is it the same? How is it different?

Yeah, so probably foundationally they're using similar models or as I said, we can make models into interchangeable. But 01 and 03 are the best deep reasoning models in certain scenarios, particularly when we're connecting to your enterprise data. As I said, that app is just the wrapping or the packaging up of how you access that model. What tools, whether it's using code interpreter or ability to create code, search the web to do deeper analysis backed by code, particularly if you're doing financial analysis. The research agent is just a way of packaging all of that up into a simple interface or a natural language. Hey, can you help me with this task? And it goes off and does it? That's basically it. So a very, very similar concept to the, to the research agents or any kind of analysis Agent in the market.

So give us, give us an example. Right, because. Well, first I think the, the Deep Reasoning agent is available now in Copilot Studio. But you know, give us an example like how should someone be maybe using this and what capabilities in this new Deep Reasoning Agent in Copilot Studio, what new capabilities are there by using it that maybe weren't available before it?

Yeah, so I think it really comes down to these use cases. And as I said, we have a number of customers live in production pushing scenarios and the kind of research requirement, particularly in that kind of business development, sales development, use case was became very clear early on around kind of lead scoring, lead qualification, lead versus research. But in other scenarios where it's like engagement management, where requests for information or requests for proposals that come in, there'd be a complex project plan, there'd be requirements from the customer saying you need to meet these conditions, these costs, these timelines, these criteria. So complex number of variables for people to kind of go, how am I going to process this Today the answer or thus far the answer was hand that over to a human to look at all the constraints, all the variables, all the information, and to create maybe an RFP or a proposal at the end of that today this is a great use case where you would give that over to a deep Reasoning agent or deep reasoning model back to it like an orchestration around that and it would intake from, let's say the email so be automatically triggered from an email that says, hey, I'd like a proposal for this product, this many units by this date and you know my pricing agreements. And it would, from that email it would automatically trigger, go through a whole process to generate the rfp. It may or may not have a human in the loop to say, hey, here's the proposal I built. And so you can see how that would dramatically kind of make that process more efficient and ultimately actually help you kind of maybe sell more because you're kind of on top of these, these proposals. So I think that's just one example RFP or sales development as an example. But I think the ones that we typically see is either kind of in the coding space in the report generation, it's really good at generating reports. So when you say hey, I want a report on this and from the code gen, where it's doing financial analysis, where it may be backed by some code that generates on the fly to do this analysis and bring back those results, to maybe take an action on that in a downstream system.

So you know, one, one other kind of question that I had on this is, you know, with this, you know, deep reasoning. And I love that example. It's, you know, taking these things that would normally be multiple, you know, human checkpoints, you know, maybe, hey, does this fit our criteria for a project? Yeah, right? Like going through multiple of these step. How does this change kind of the human role in all of this? Right. Like I know we always talk about like human in the loop, right. Like I like to say, like expert or expertise in the loop. But for those individuals and companies that are going to start leveraging this deep reasoning agent in Copilot Studio, how does this change kind of even their role?

Yeah, I think it's, I think we'll struggle to see any kind of industry, any role, any department, not to be influenced or impacted by this kind of agentic transformation or disruption. So I think we as humans are like, we should learn how to harness the power of these capabilities to be more efficient, more effective in the roles that we're in. And I think we see that across a number of industries, number of use cases, and fundamentally it's going to shift from we as humans having to do as the front line of doing some of this more boring and mundane work that oftentimes we don't want to do, to having that delegated to an AI agent that will maybe just come to me with the research and the pre brief before I jump on the call with the customer as an example or before I do some M and A acquisition or an M and A process where I'm like, I've done the risk analysis, it's been poured over the deal room as an example. So it's, it's maybe either bring into a human or it's augmenting a part of the process that we will be doing today. But it could also be fully autonomous in certain circumstances. But we will, we as humans will be always overseeing these agents handling exceptions if it's unable to proceed or doesn't want to proceed because it thinks it doesn't have enough information or you know, you've codified it to say, don't proceed beyond this point if you think the refund is beyond this point, or if, if there's some sort of risk management that you kind of bake into these agents. So therefore you'll have people in department saying, hey, I'm processing invoices, but the agent is doing 90% of the invoices, maybe 95%, and I'm there to handle the exceptions where it couldn't pull out key information from, from, from the Documents or whatever it may be as an example. So I think, I think we will see a shift to how we can leverage AI more and how we will shift from, you know, individual contributor to kind of managing agents more across our typical processes.

And I'm assuming one other, you know, big difference, right. And let's just pick an easy one to compare to. Right. So if you're using, you know, chat GPTs, you know, deep research, or you're using chatGPTs, you know, 0103, something like that. The big difference with the deep reasoning agent in Copilot Studio is it can access your dynamic data inside Microsoft 365 is number one. Like can it access all your data? Like what, you know, I guess, like what other, you know, tools or apps does it have access to and what does that unlock in terms of capabilities?

Yeah, it's a good point that you make there, Jordan. Because fundamentally we as humans set up these agents and how they operate. Whether it's like what other agents it can talk to, what connectors, what systems, what knowledge sources, we configure it, we build these agents the way we see fit. And it's very similar to how we kind of set up roles within departments. Say you're in the finance department, you have access to these tools and these knowledge sources or these sharepoint sites. So very similarly, we as humans will provision and make these agents with total control over what they can and cannot access. So that's number one. Number two is really, it's like even when we have these agents and we've tested it and we debugged it and we kind of deployed it, we're confident around how reliable it is, we're going to want to oversee it. So that kind of governance, observability and all those kind of security requirements are going to be essential, I would say, because not just when you've got one or two or three agents, but like we're going to have hundreds if not thousands of agents across our business, across departments. They're going to be various agents aligned to one department versus another, all rolling up to, to the business leaders that run, run those teams and run those departments. And I think, you know, I, I would say that enterprise grade kind of connectivity to various systems and tracking all those connections. Similarly, when you bring knowledge into these agents, we shouldn't just have a drag a file from your local desktop. And then by the way, when I share this agent with everyone that got access to everything that's grounded in that agent, we at runtime want to be able to Check does this user have access to this document or this file? So sensitivity labels, the access controls. All of these things I would say is a thing that Microsoft being in the space for so long is differentiated on or kind of is, you know, focuses on around that kind of enterprise knowledge, connectivity and security governance. Obviously all the innovative breakthroughs and the various tools from, you know, kua, operator code, code gen, you know, all of these things we've just even talked about today, which is, you know, Agent flows and deep reasoning, they're essential building block tools. But if you don't kind of secure and kind of have a platform that is, that is reliable, then all the tools don't really matter.

So let's talk a little bit about Agent Flows. I believe that should be rolling out to everyone March 31st. Correct me if I'm wrong on that.

Yeah, I think it's Monday.

Yeah, yeah, there we go. So I, I mean, what is this and, and how does this, you know, change what pot, like what's possible? And we'll be sure to share the, the little video in our newsletter. I think watching that really helps. But maybe just for our podcast audience, just describe Agent Flows and how this changes, really, this agentic workflow.

Yeah, so we've been on a journey over the last number of decades to just automate more across our business. Whether that was kind of scripts, macros in Excel or write programs. And then obviously the last decade we've had a low code automation, so building automations or rpa, so robotic process automation to automate key parts in our business. And that, you know, that has been a hugely successful business, lots of businesses. And at Microsoft we've got a tooling called Power Automate. What we've learned in this agentic revolution was yes, they want lots of agentic reasoning. So this reasoning brain at the top of a process where it looks across the various tools. But there's certain times where you really want deterministic behavior. You want the tool to run from A to B each and every time. And that's where you will leverage automations or you'll leverage connectors into existing systems. And you don't want necessarily too much variability in that. Bringing agent flows and our power automate capabilities natively into Copilot Studio is we're bringing this healthy mix of deterministic outcomes and setting well defined paths through a process that won't change over time. So whether you're generating code or creating a prescriptive workflow, that's where agent flows really comes in. So it's like Two sides to the same coin of how we will do or complete a business process. Some parts will be reasoning or even deep reasoning, deciding what tools to use. And sometimes one of those tools will just be, it's an automation. And that automation, we can, you know, use LLMs or use AI to help create those automations. We can even bring in kind of prompts and reasoning elements into the prescriptive part. But it's a key, key use case that unlocks more kind of control and that's what we're seeing from our customers is that kind of mix between the two. Is, is a, is a, is a, is a good mix.

And I can put myself in the position to some people right now and they're hearing some of these terms, right? Like oh, power automate and deterministic RPA agent flows. But like I go back to what you said earlier, it's like, well, all you really need for all of this is, is natural language, right? Can you just quickly walk people through like, hey, like technically you don't need to be, you know, have a decade of experience in power automate or something like that to take advantage of agent flow.

This, yeah, this is a great point, Jordan. And like it's something where we're seeing this level of abstraction from all the different tooling underneath and it should just be. And even with agent flows, you describe what you want and it will build the flow for you and obviously you verify, you test it and you may iterate back and forth and say no, no, instead of this step, I want you to do this. So natural language we see is the, as the language for generate, creating solutions. And more and more we're going to move that like input up the stack where you don't even decide what tool you want to use. You described it. And the agent itself will decide. I think you're looking for a prompt here, Ray, or maybe we'll create an automation here at this point. Or maybe, you know what, that's cool. We're going to move a mouse around a VM because you talked with some legacy app. So it will abstract more and more away. And I think that's what you'll see in the tooling. It will get to the point where it's like you and I having a conversation here. We'll describe the problem, we'll iterate through it and behind the scenes it's picking the tools, it's either generating code or automations on the fly.

I mean just, just the amount of capabilities and, and you know, opening this up to even non Technical people. I think it's just a really exciting time in generative AI. So, you know, Ray, we've talked about a lot in this conversation, but, you know, you know, from everything that's new with, with agents in Copilot Studio, the deep reasoning agent, the agent flows. Right. But as we wrap up, maybe what is your one most important kind of takeaway or tip on how business leaders can get these to work for them today?

Yeah, I'm glad you asked this one because this is something I usually finish most customer engagements with. It is kind of overwhelming how everything is moving fast. It's like all this technology and I think the market has kind of woken up that this is going to be transformative. There's huge ROI and there's lots of case studies out there and you know, weekly everyone's publicizing how they're saving lots of money being more efficient or, you know, ways to generate revenue using agents. I think that can be overwhelming as well because you're like, I'll build a spreadsheet, 200 use cases. It's got a subtotal to, you know, millions or billions of dollars. And we're going to change. The change that we run our business, first and foremost is you got to get your hands on the tools, right? And I see this across the board. It's like pick a use case, you pick a business process, you break that up into parts and you're going to be like, I'm going to build an agent for this part. I might have a human either side to verify and kind of work that into the process and, and slowly but surely you will automate the whole process or large parts of it by building a series of agents that will chain together. So my advice is just get your hands on the tooling because it really is a kind of a experiential learning of just figuring out how these AI tools work. How do you put guardrails around the tooling that you use and so on. And as we said, it's getting easier and easier with more obstruction. But in the early days, it really requires just hands on experience and a use case focus.

All right, this was a great session, right? We can, we can close class for the day. I, I learned a lot and I know that our audience did too. So, Ray, thank you so much for joining the Everyday AI show to walk us through what's new with Copilot's new agents. We really appreciate it.

Cheers. Thanks, Jordan.

All right, as a reminder, y', all, we covered a lot. If you missed anything, we're going to be sharing a lot of other links and resources to everything that Ray just walked us through. So if you haven't already, please make sure you go sign up for our free daily newsletter@your everydayai.com so thank you for tuning in. Hope to see you back for more Everyday AI. Thanks y'.

All. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.