EP 407: Copilot Studio - Autonomous AI agents for everyday people

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

AI agents are everywhere. Literally. If you follow artificial intelligence, generative AI, large language models, the conversation over the last couple of months has really shifted towards agentic AI. And when I say AI agents are everywhere, I mean literally where I'm at right now. So coming to you live from the Microsoft Ignite Conference. We got a ton of new information on what Microsoft is rolling out to its customers worldwide when it comes to agent and agents. And I'm excited today to talk with a leader at Microsoft in the agents space. So what's going on y'?

All?

Thank you for tuning in. My name's Jordan Wilson and this is Everyday AI. We are your daily livestream podcast and free daily newsletter helping everyday people not just keep up with generative AI, but how we can all use this information and get ahead to grow our companies and to grow our careers. Before we get started, have to quickly give a shout out to our partners at Microsoft Worklab. So why should you listen to the Worklab podcast from Microsoft? Because it's the place to find actionable insights to guide your organization's AI transformation. Tune in to learn how to think more creatively about unlocking the full potential of the technology. That's W O R K L A B no spaces available wherever you get your podcasts. Another place you can go get more than 400 podcasts is our website. So if you are new here, maybe listening on the podcast. Thank you for tuning in. As always, make sure to check out the show notes. Go to your everydayai.com we will be recapping today's conversation. I already know we're going to learn a lot in the next 25ish minutes, so we're going to be recapping it all in our daily newsletter. So make sure you go sign up for that@your everydayai.com also we'll have the normal daily news there. Normally we bring you the AI news live, but because we're at Microsoft Ignite this week, switching it up a little bit different, but we will have all that in the newsletter. All right, enough chitchat. It is time to dive into AI agents and I'm very excited for our guest for today, so please help me welcome to the show. We have Ray Smith joining us who is the VP of AI Agents at Microsoft. Ray, thank you so much for taking time out of your busy Ignite schedule to join the show.

No welcome and glad to be here, Jordan. It's definitely been a busy few days and lots of as you say, agents everywhere.

Oh man, we're going to get into that. But Ray, can you just tell us before we dive in a little too deep, can you tell us like what is your role at Microsoft working as the VP of AI and agents?

Yeah, so I'm actually relatively new to Microsoft here about five years but I've been leading over the last year a project aiming with unlocking kind of autonomous agents. So the early day use cases of agents was, it was great for summarizing text, you know, asking questions, discovering stuff on the Internet. So we postulated over a year ago that we like, we think I will be able to run tasks, we'll be able to orchestrate things and ultimately get worked on because that's really where it gets into the high roi. It's like full, full autonomy. So we kicked started a project which then was released into early access program in May. So it was announced in build. So these autonomous agent capabilities are built and then this week we're launching them into public access or public preview. So yeah, there's been lots of GPUs being used over the last couple of days.

The GPUs are running. Right. So let's look, I mean we'll rewind and I do want to kind of ask you about the, the recent evolution but let's get into the good stuff. Let's get into the news here because at Ignite we heard a lot in Sadia's keynote on Tuesday as well as, you know, sessions throughout the week. But what is new with agents at Microsoft? Because there's so much news.

Yeah, there's, there's so much and I even chatting with colleagues, they're like did you know about that announcement? You know there's so much so I mean foundationally it's a whole new paradigm like AI is going to transform how, how we work and it, it, it's effectively manifesting across a couple of key parts of the products or the AI stack starting with M365 copilot as the, the UI for AI. So the interface or the gateway into all these AI capabilities or agents. Then Copilot Studio announcing obviously the autonomous agents which is kind of where it's the local way you go build these modern day apps. In the AI world when you think of agents, I just think of apps. They're just the modern day apps, they're built a little differently. Then lastly there's the Azure AI foundry announcements all around that platform as a service in the AI world, all those agentic services to be able to support the pro code development to allow developers fine tune models and really go deeper into the agentic architecture. Kind of three key layers of announcements. There was lots of agents or these new apps announced across you know the modern work. So across office like interpreter agents, facilitator agents and meetings. So lots of really exciting use cases where people can go huh, I can see how I'm going to use this every day or here's some really high ROI examples of how my business can benefit from them. So so much to mention in the list. But that's a rough overview.

Yeah, even the like facilitator agent and some of the new agentic capability in teams like I saw that in like the ability to like speak a different language like live in real time like my mind is blown. Like I, I mean maybe can you even talk about some of those you know, not low hanging fruit use cases but for you know, everyday people, maybe for the non technical people how is their kind of day to day work or how could their day to day work really change with some of these more specialized agents rolling out across the 365 platforms?

Yeah, I probably put it into two different camps. One is like boosting productivities for humans on their kind of workflow so bringing all these AI capabilities to drive efficiency or drive better performance. And then there's another camp of autonomous capabilities where it's actually running tasks automatically in the background and maybe coming back to humans. But the interpreter one example you gave is like for, for Star Trek fans out there, the universal translator immediately pops to mind when you're on these meetings. Being able to in real time translate languages is something that everyone can identify with or the meeting facilitator as was announced similarly someone just keeping track of actions time checks and making sure the balance of conversation. So I would say on the productivity across copilot pages, across the various agents across teams and office, very much driving that productivity for humans. You will see some of the announcements on the autonomous agents from team announcements with like Dow Chemical and partners like pets at home McKinsey the autonomous agents that they've built to dramatically transform how they run their business. So whether it's like from you know revenue protection with pets at home, whether it's engagement management from McKinsey or supply chain optimization from Do Chemical, they're actually fundamentally transforming back office processes as well as maybe how they engage customers and partners. So there's so many different use cases I'D say that's one of the challenges in terms of the autonomous agents because it's effectively a platform people can build and these custom tailored solutions for their own businesses. But the ones that I think the average person will identify is whether it's across Outlook teams or the office products where they're like, huh, that's like that I could use tomorrow. So they're probably the more easily identifiable use cases.

Yeah, and those were great use cases. I like. I believe that was shared. The McKinsey one was shared a couple of weeks ago. I think we talked about it on their show. Something like they cut lead time by like 90% or something like bananas like that by using autonomous agents. So. So maybe Ray, can you walk us through for maybe those that aren't familiar? Let's just talk about what the heck is Copilot Studio and what is the difference between an autonomous agent versus these agents that, you know, kind of are available to, you know, people within different apps. Right. We kind of just talked about the facilitator agent or you know, kind of pre built agents in teams, etc. So yeah, walk us through Copilot Studio and autonomous agents.

Yeah, so Copilot Studio fundamentally is where you can go extend kind of M365 copilot capabilities. I bring a customized flavor to the knowledge that's grounding some of the skills and tools or bring in new actions. So it's either to extend M365 copilot or where customers can go build their own custom agents from scratch, tailored to their own business and use cases. And it's all kind of low code. So it's kind of like a rapid way to prototype and build these apps in this new world by just describing it because you're just natural language saying, hey, I want to implement the two way invoice matching process. I'm going to, you know, receive invoices from email and the POs are going to sit in SAP and it's able to then go through, parse all those instructions, pull in the relevant connectors and it'll effectively bring you through this app building experience in a really kind of low code way. The beauty of Copilot Studio we hear from customers is it's that it's this managed, it's managed solution environment, life cycle. So it takes a lot of that going from prototype to get to production. All the challenges around that because a lot of people building early AI apps, they're saying, I'm playing with this endpoint. I've got a cool demo But I don't really know how I'm going to bring it into production. So all that, you know, kind of almost abstracting away some of that complexity around the model management, all the kind of the guardrails and governance, you know, the security when you're bringing in information, making sure you have the right access to that information rather than any data loss prevention and so on. So in short, Copilot Studio is just, it's a low code app building solution to build these agents which honestly is going to see an explosion. It's, you know, we've seen explosions of apps through various technological breakthroughs. You know, needless to say, handsets are mobile. We're going to see the same in this new kind of agentic world.

Yeah. So, you know, one question that I've already been hearing is, you know, some people are like, okay, when do I, you know, go in and use a, you know, as an example, a templated agent? When do I just use, you know, an agent within one of these programs? When do I build, you know, kind of an agent with the, you know, no code, low code builder? Right. So what's your suggestion on, you know, as people start to reimagine their workflow? Right. Because I think that's almost, you almost have to shift. Right. How you think about work in order to really start using agents in the right way. But maybe what is your best advice for people on how to kind of shift that daily workflow and where to go, you know, where do they go to find the right agent to you?

Yeah, and it's, it's so it's a great, great question, Jordan, because it's, you know, the way I kind of tell customers is you get started where you get comfortable in terms of the skill set and what you kind of have access to. So I kind of talked about the three layers we have with M365 Copilot. Anyone who's kind of, you know, an M365 Copilot customer can go in and build their own agent. So a lightweight agen as an information worker and that can be on the kind of the retrieval. So the basic rag Q and A and it can shift into like automating tasks. So I would say that's where people can go to get started with the very simple, the very kind of intuitive natural language. Then when you go into building your own custom app within Copilot Studio, you can shift into more autonomous agents where you can then say, add a trigger. So I want this to run when an email is received. I Want it to trigger on an event in like SAP, Oracle Workday and all these systems and applications that we use across the business. So that kind of shifts more towards people who work in it who are comfortable with all those connections and how these tools are built. And then the third layer is really around. If you're a pro dev and you really want to get access to the raw APIs, experiment with different frameworks, fine tune models, then you'll probably go to Azure AI Foundry. So each of those support an on ramp depending on your skill set or what you feel comfortable with, but they allow you to kind of, they're better together to kind of simulate or bring each of those kind of services and capabilities together. I think the key part is that we hear from customers is they probably sometimes jump in and they're like, hey, we're building from these API or, you know, endpoints and we're using these models and we're fine tuning. And I was like, they just want to start with the use case. What problem will it solve me? And maybe I should start with a rapid prototyping ability, get it very quickly to 80% to go. This is now viable to invest, you know, more time and energy. So it's, it's very easy to get a quick solution. I think where you then spend the time is just debugging on the various scenarios and the examples to make sure it's robust to go into production. So prototyping fast, prioritizing use cases of when and how you can use is probably the biggest learning we see from customers.

Use cases are everything, I think, right? Especially when you're trying to move from ideation to implementation. And one thing I love Ray, about being at these conferences is being able to go around and talk with, you know, smart people like you. But even, you know, in the hub here at Ignite, being able to talk to all the product managers that are helping to build and you know, one thing I love asking is, okay, well, how are you using the product that you build? Right? So you know, there's always the say, you know, you got to eat your own dog food or drink your own champagne, right? So how are you actually using, you know, whether it's, you want to talk about the 365, the copilot studio, the pro dev, right, those three different tiers you laid out. But how are you using Autonomous agents or Copilot Studio in your day to day? All right, I have to take a quick break to shout out our partners from Microsoft Worklab. So why should you listen to the Worklab podcast from Microsoft. It explores the questions business leaders are asking. How can they guide their organization's AI adoption journey? How can AI help them maximize value and create new products and business models? How should they help their teams reskill for this new era of work? And why is it important to be completely transparent about when and how you use AI? Find the answers on Worklab. That's W O R K L A B, no spaces available, wherever you get your podcasts. All right, back to the interview.

Yeah, it's, it's a great one. So I think I would have started on the journey similar to, you know, the listeners, which was, hey, need to write a document or I need to do some research. So that was, I think, the use case that everyone latched on to really fast. It says, hey, it helps me write a document. So, you know, we're in the engineering and product development, so we're constantly writing documents around planning, you know, seeking investment and so on. So that's where most people start. When you then take all of those documents across many teams, then you're probably putting them on a SharePoint and we, you know, have the ability to have a SharePoint agent. So I can just go in and ask a question from all the planning docs across the whole product group and I'm easily able to get dependencies, you know, overlap across teams, ask all those questions, and it's hugely valuable rather than having to chase through various groups and people to get to those answers. So it kind of starts off very much content summarization and creation into kind of retrieval or rag where you're just trying to get access to information, search the web, do research. Then it moves into actions and past completions. Like, the biggest one for me is if I'm double booked in various meetings, I'm like, I just hit following like it's a new pattern. I don't necessarily decline the meeting. I hit following and it stays in my calendar and I'll get the full summary, I'll get all the actions. So I feel like I'm almost in that meeting when there's call outs, when there's actions. So this is the kind of productivity boost we're seeing from AI in terms of some of the autonomous agents. I've seen my team build, you know, weird and wonderful agents that would, you know, when they're doing, you know, book bashes, where they would automatically have an agent that would reach out to the employees, ask for their pizza preferences, order the pizza, and it would arrive and place it via the API just in Time for a kind of a get together or book bash. So I've seen and it remembers what your peep, how many slices you want, what your preference, aggregate it and then it would place the order for the set number of pizzas. So I see lots of interesting use cases, some of them very custom and bespoke. But I think the productivity boost is absolutely real in the roi, I think when it comes to customer examples. So the early access program that we ran since around May, I've had many days with hairs on the back of the neck going Dow Chemical built a whole series of agents around supply chain optimization and their freight. So I think seeing the customer use cases where they come at a problem, they apply it and they, they're able to like achieve like things that even I'm blown away by Daily.

What's speaking of SharePoint, I heard you mention SharePoint in there and I don't know personally I'm, I'm very bullish on these SharePoint agents. Right. Seeing the announcement and seeing some of the information and you know, being able to speak with some of the product managers yesterday on SharePoint and just being able to, you know, quickly, you know, select different folders in your SharePoint or the default SharePoint agent. I mean to me having a grounded agent that it takes like zero technical ability that can change how we work with our files. Can you walk us through real quick, Ray? You know a little bit about those SharePoint agents? I think they're fantastic from what I've seen.

Yeah, I think the agents that we will create from just a click of a button is really going to be unlocked for, you know, pretty much everyone. So for anyone that has like OneDrive or SharePoint and they have folders, I gave the example of our planning documents for example. But it could be like policies, standard operating procedures, it could be like lots of things across the business and you're not necessarily able to connect with the SME. So you know, in every walk of life, oh well, Mary or Jim will be back in on Monday and she'll be able to answer that question for you and you're like, well maybe we should be able to point it to a lot of policies or documentations or sources of information and everyone should have access to that knowledge as quickly as possible. So in SharePoint if there's, you know, a site you just click and say I want to create an agent for this, this you know, file structure and then quickly grounded on that knowledge and that information to be able to ask questions super fast is really, it's a game changer.

Yeah, yeah. I mean just, just the ability to, in a couple of clicks to be able to create a grounded agent. It's mind boggling to me. Like, like you know, podcast audience. I'm just sitting here like chuckling and scratching my head because it's pretty fascinating. But one thing I do want to talk a little bit more about, right, is the data side because I think that especially, you know, smaller companies, medium sized organizations, right, Maybe they've been on the fence the last couple of quarters on, on diving into agents, but maybe they've seen, you know, productivity gains by using other large language models. But those models work with static data and that's not how, you know, autonomous agents and copilot studio agents work. They work with your data in real time. Can you tell people a little bit on kind of the advantages of working with that dynamic data inside of 365 and also, you know, data security because I think that's something that people don't always necessarily understand or get Right. So walk us through the data side of agents.

Yeah, I think data and actions are probably the two of the biggest or most important things that we will add to an agent. So we're giving it knowledge and we're giving it the ability to take actions. But focusing specifically on the data side, the biggest concern customers have is like, will data get leaked? Will people be able to access information that they're not supposed to have access to? And if you just build, you know, using large language models and you just drag a files and the maker then shares that with someone else, they're like, well you didn't have access to those files in the first place. So you're now able to ask questions about documents that you shouldn't have access to. So the beauty of these dynamic sources like SharePoint, not just on the file storage and across office, is that it? Here's all those access policies and information labels at runtime. If you're asking a question that says, hey, you're Ray Smith, you don't have access to these documents, it won't pull those documents or those indexes into the grounding at runtime. So it's making sure that all that protection, the policies, the governance is really adhered to and that's a big deal, not just from files, but also in transactional data. So this is a whole wave that we've done for fabric and power bi over the years is how do we make sure, even for analytical data, transactional data, and then obviously the files themselves within SharePoint that people have access to that information. That's a really important point. Plus you don't want to have to keep changing sources by uploading, changing documents. You just want to point it out a folder and say things may change. And for that information as it changes to be, to, to be current in the, in the agent itself.

You know, maybe this is an obvious question, but I think it's worth, you know, bringing up especially when we talk about data and security because I mean, maybe this is more of a year ago, but I heard so many, you know, companies say like oh yeah, we're not sure about, you know, Copilot or just AI and large language models in general. Yet these same companies are using obviously OneDrive and SharePoint. Right. Is there a difference in how companies data is handled? Right. You don't have to go into the granular, but from a high level is there a big difference? Right. For companies that already have all of their data in the Microsoft cloud, so to speak, is there a difference for once you start using Copilot and agents?

Yeah, I think the, and obviously we support connectors to various systems and tools as well, so. But if it's in the Microsoft ecosystem, you've already kind of invested in those access policies and controls and governance tools that exist across Microsoft. So whether it's purview for viewing your data across what's fabric, so you're bringing those data sources and those governance tools to bear very quickly as you're building these agents so you can kind of have comfort and that the same thing you've invested in those same policies, those same kind of access controls will be followed through. And I think that's a key benefit from customers that they can just, they can build. And we've been on this journey for probably the last decade or so with citizen developers and low code, so power apps and power automate. So the same way of building these kind of solutions and the governance around those is similarly followed through for Copilot Studio. So the same admin center and governance tools to oversee what copilots have been created, what's running, what's it accessing and so on.

Great, great answer. I hope you have a great answer for this one too. Even though this one's a tough one, Ray. But as we start going, you know, and business, businesses at large take this at different, at different paces. But you know, as we start using autonomous agents, right, especially inside Copilot Studio, how does this change the human role, right, with, with these autonomous agents that can go through multiple steps, you know, connect to your Data. Right. Which to me it's like a lot of these are what we would spend a lot of manual time as knowledge workers doing. So how does the human role change as autonomous agents become more and more robust?

Yeah, it's definitely a powerful leap forward in automation, but I think we take a step back. We as humans for the last number of decades in the computer era have always been trying to automate more. Whether it's scripting macros in Excel, building apps and doing software development, we've always been trying to automate more and more. As we've automated more, we as humans have kind of shifted to higher value work and so we've moved to other skill sets. So with kind of AI and agents, and particularly the autonomous agents, there's definitely going to be lots more automation. So how does it differ from rpa? So robotic process automation is that we're now strapping a kind of a cognitive reasoning or brain on the top of those kind of skills, functions, tools or automations and it's able to do a lot more work. So for example, you know, something comes in via email, checking policies, reviewing, assessing the customer criteria before it goes in to executing a transaction that can now be done by AI. What I see the shift for humans is instead of a lot of that kind of frontline, sometimes mundane tasks, humans will shift to then managing these agents that are running these processes. So I would say agents are never going to be perfect. So it's going to have humans in the loop for exception handling, for overseeing, you know, all the run events, maybe for handling some exceptions. If the AI spits it out and says I, I don't think I should proceed here because I don't have enough information. So humans will start to manage more and more of these agents and it's definitely a skill set we need to embrace. You know, it's people would say, hey, two young sons, should they learn how to code? I'm like, yeah, but they also need to learn how to use the AI tools to really get the most out of how they would code, for example. So I would see similar shift on how we've seen any automation allowing us to do more and other higher order tasks. And obviously AI agents is a big unlock for automation and we as humans will shift to managing a lot of agents.

Yeah, I said that like a year or two ago and I think people looked at me like I was crazy. But I said, you know, in the future, in the same way that we just interact with different, you know, softwares on the Internet, right, Like outlook or word, etc. Might the future of work, and I'm not going to ask you to, you know, make a bold claim, but might the future of work in the next couple of years just be knowledge workers really just interacting almost all day with, with different agents? Is that, is that a wild thought or is that maybe where we're headed?

No, I think that's like, at least my personal opinion is that's where we're headed. Like, I think we will all choose a kind of a personal assistant that was heavily personalized for me. So whether that's in my personal life, and there may even be like, bring your own assistant to work that interfaces with your kind of work assistants and then it will tap into, let's say, for example, M365 copilot is this gateway into all these agents across your business. So I, I would definitely see a world where there's hundreds, if not thousands of agents across the business and we as knowledge workers are interfacing via an assistant of some sort. And, you know, even going further, I was like, I think it could even redefine the Internet. Like, we're not necessarily going to go on and type www. We're going to just, hey, talk to our assistant and go, I want to order these shoes. And it will go get that company or brand agent. It will remember my account, the details and what shoes I ordered last time. And it will say, yeah, they have them available in stock. So it'll redefine how we live in our personal lives, but, and also how we work professionally, you know, interacting with these agents. All right, that's my.

No, I love it. I love it because I'm, I'm right there with you. Right. I've always said, like, there's probably going to be more, more agents than, than humans at point, some, some point soon. But, but Ray, we've, we've talked about a lot, right? You, you kind of walked us through this transition. We're going from, from, you know, AI being about productivity to now taking on autonomous capabilities. You walked us through kind of like the three tiers that companies and enterprises can start dabbling in. Agents. But as we wrap up today's conversation, what is maybe the most important takeaway that you want people to know when it comes to co pilot studio autonomous agents, everything that was announced and, and where we're going?

Yeah, I think the fundamental takeaway is this is a fundamental step change in how we work. We've talked about business transformation over the years. This is going to fundamentally transform how we live and work. I think we've talked about that on the POD today. I think my advice I would give people is just bring focus, focus on the use cases and get dabbling with these tools and capabilities to start thinking about how you can adopt them, how you can transform how you work and prioritization and a bit of focus on those use cases. One of the early learnings from the early access program was start small. People would get access to this blank page and they'd say, I want to build an agent that's going to run my whole company or my whole department, or even just an entire process. But it's really a process is made up of many steps. So policy checker, invoice, processor, it may be like a series. So you build these atomic agents to do parts of the process and over time you'll stitch those agents together and it'll then complete the entire process fully autonomously. So focus on the use case, start small to build those skills, build a center of excellence around these capabilities and prototype fast.

So much great information there, Ray. This was a great conversation. Thank you for helping us all make better sense of all of the new capabilities that we got, you know, here both at Microsoft Ignite and everything that's rolling out across Copilot and Copilot Studio. We really appreciate your time and expertise for coming on the show. Thank you so much.

No, thank you.

And hey, as a reminder, y', all, that was a lot. There's so much new, so many new capabilities with both autonomous agents and everything available in Copilot and Copilot Studio. So if you haven't already, please make sure to go to your everydayai.com. sign up for that free daily newsletter where we'll be recapping everything that Ray just shared with us. So thank you for tuning in. Please join us tomorrow and every day for more Everyday AI. Thanks, y'.

All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.