I ❤️ AI Q&A: Building Smarter AI Agents Using Microsoft Foundry & Copilot Studio

Welcome back. Happy Friday. Little delay of game. All right. Oh, I think we are live. Welcome, welcome, welcome. This is our super fun session. I love AI, our community call where we get a chance to, number one, I get to tell you about all the cool things. So today I'm going to focus on Microsoft Build and all the announcements that came out at Build. I will not be talking about all of them. But I will be definitely talking about some of them, the ones I care about the most. I got a chance to fly out to Seattle to be part of Build this year, which means that I was given an opportunity to speak. I was number one. I was in the expert meetup session, which I met so many people I haven't met in so long. It's amazing how small this world is. You all, some of you are just getting into the world of AI, but it's a very small world. And it's cool. You watch people like, you know, some of you, you watch me, like people are movers and shakers and there's people that are moving, doing stuff, new stuff. Some people leave and then come back. I'd met with this, so I did a video, so you'll see it post later today. Um, but I created a video montage of all of the people I met, but there were people that I've, like, I knew 7, 8 years ago that were part of the Microsoft, uh, team that went on to do other things and now they're back at Microsoft. It's really cool. Really cool stuff. Um, welcome in everybody. Great to see you. Um, appreciate it. I know I used to, I do, I used to have this rule that you couldn't have your bot here unless you were here, but people usually proceed, like their bots precede them, so eventually they do show up. So now I'm not kicking people out, but right now I'm like, we got like, I don't know, more note takers than attendees right now. So we'll have to re, uh, recreate that plan. All right, so at Build, Microsoft Build is the conference that allows you to go and see kind of the developer side of Microsoft as opposed to the business application side of Microsoft. But nowadays, those lines are blurring. In the past, it used to be like .NET, Java, Python developers go to Build and everyone else, like builders go to Build. And everyone else goes to Ignite, which is like the business user conference, or a Biz App Summit or something like that. But now because of Copilot, Copilot Studio, AI Foundry, all of it's blended up together. So today I thought I would share with you some of the cool things that I not only discovered that were announced, but some of my favorite models that are finally seeing a little love, a little light of day. And so I want to share with you a couple of those as well. So I'll be sharing my screen today. And walking you through a couple of the announcements that have happened and ways that you can test it out as well. So let me go ahead and start with that. But I had fun. I wasn't planning on being in Seattle this week, but I ended up being there. I got to meet up with a couple of— well, actually a lot of different people that I hadn't seen in a long time. One of them is a member of our community, Barbara Pender. So that was fun. And we, of course, worked together at Accenture years ago. So that was really, really cool to, to see some familiar faces. And reconnect. I mean, that's what these conferences are all about, honestly, is like having a chance to meet up with people. And I mean, all— it was like a family reunion, you know, every time I, I turned one direction, I would see somebody that I'd worked with for a really long time in a different stage of— I mean, one person I had met way back in the day in like my Alexa years, like 10 years ago, uh, and he was now at Microsoft Build. So it's like I said, it's a very small, small, small world. That you are in, even though you might feel everyone's doing AI. There's very few builders, very few people who put their fingers on a keyboard and actually build something. So let me share my screen and give you some really cool announcement-type, announcement-worthy things. Hopefully you can see that. Okay. I'm going to start first by going to— let's see if I already have it open. I don't. I'm going to go to Microsoft. Learn's blog, which I searched for earlier, and go and take a look at their latest posts. So Microsoft Learn is the platform that you can go to to get prepared for certifications on Microsoft, to learn about a topic, and they— it also is the home of product documentation, which is— I didn't know what they were thinking at the time because I was like, documentation and Learn together. I don't know, it didn't work for me at first, but now I get it because of the feature that they just launched. So on the Microsoft Learn blog, there's the latest blog happened on Monday, the first day of Build, and they announced this really cool new capability for you to actually introspect and ask questions not only of the learning content, but maybe more importantly, of the documentation. And so I want to share with you that as well. If you go to this post, you can see it looks great. They announced some of these features in preview, but there's now this capability called Ask Learn. And what that means is that throughout the documentation, I'm gonna just go to the Azure App Service documentation, but throughout the documentation, you're now going to see this really cool button called Ask Learn. And when you click on it, it's gonna take you into, and I've obviously been using it, I've got 3 of 10 responses left, or I guess I have 7 of 10, I've used 3 up. But that's per session. So they keep giving you more as you use it, but it helps them throttle access, which I totally get. But based on me being on this page, there's actually some framing that they offer up. I think it's here. There are 3 different ways that you can ask questions. Like, one is, can you clarify this for me? So if you read something in the documentation and you're like, I don't know what this means, you can ask for clarification, which means they're going to rephrase it through the art of LLM Right? Communication. And then the other one is like defining, like, hey, I hear right here, if I go and look through this, I don't know, I know most of these, but maybe multi-tenant. What does multi-tenant mean? Or more important, I think one of the most cool, the coolest feature is tell me how to do a thing. How do I configure this? How do I, uh, find this package? How do I install this component? How do I test this capability? And so this, when you click on one of these, it'll automatically populate with the prompt guide to assist you in actually asking the question. You can also just click on Summarize Page, and it'll give you a summary of the page that you're on. Now, what you'll notice though is that this page, this Ask Learn agent was created to actually serve all of Learn documentation and learning. So if I go to a different product, let's say Azure AI Services, one of the areas that I was in the expert section on this last week. And I pick on, you know, I go and pick some of these services like Foundry Models. You'll notice I get Ask Learn here, but it's the same one, right? So it remembers my history. But now I can also ask it to summarize this page and it will know contextually where I'm at. It'll give me information on that. And now I can start asking very specific questions on this content. So it's pretty cool. I started getting excited and I was like, While I was at Build, I had access to this, so I was like asking all sorts of questions like, can I get certified in this? And I like asking questions that force context, right? So when you use the words like this, it's like, what do you mean this? Now here's the thing is that notice here it says, I'm sorry, something went wrong. Well, many, any number of things can happen, but this is in beta. And if you were paying attention over here in the blog, it says Try it now on App Service, or you can explore your favorite topic. The favorite topic, not all of these topics have the same capabilities, so sometimes it'll fail, but some of these work. So I, I think it's always fun to like test it out. But here's my invitation to you. If you test it and it doesn't work, please, for the love of heaven, give some feedback. So say this didn't work out. Give it a thumbs down and then tell them more like, oh, uh, incomplete, unclear content. That wasn't really my problem. Inaccurate or irrelevant. I don't know. I'm just gonna say it didn't work. Load any answer. Said there was a problem. I might even copy and paste it if I had more time. Right? And so do them a favor, give them some love, submit some feedback. This is cool stuff. You can define how Ask Learn gets created. It is right now in preview. So this preview button right here that it, it's literally says preview, i.e., might not work, but we get to be people that go in, especially AI leadership people. We get to go in and we use these tools and we provide feedback and we help make it better. It doesn't get better unless we tell them that it's not working. Like, that's literally what preview is all about. Get real users to use it and give real good positive feedback. So, or negative feedback in this case, right? So you can also see my session reset. So I'm now down, oh, maybe I've, yeah, 2 of 10 responses. So it's kind of neat. But let me see here what else. Oh yeah, so it says thanks for your feedback. That's great. I'm just going to type in certifications. See if it knows what the heck I'm talking about. And this is going to be leveraging this bot, this agent's capability to do hybrid search. And so yeah, it's totally wrong. But let's go back. I'm gonna go back and do it. Oopsie. For App Service. So let me just go back to App Service and see if it's better at this. All right, so summary. Now I'm on this page, so I'm gonna say, I'm not gonna say summarize. This. I'm gonna say clarify, give me more information about App Service environment. And now because I've given it the right context and this is one of the showcases, let's hope maybe it'll work. I hope so. Yes, it does. Now here's one of the cool things that I like about this. That's a lot of information. So you could ask it to, um, Can you create a— oopsie, you do need to spell it right— a checklist for this? I don't even know if that's appropriate, but, but now I can ask it to generate it in different modalities that I maybe as a learner, right? So I broke it. That's okay. But it also might be that this wasn't really checklist-based content. But either way, that's not a good error. So guess what I'm going to do? That one's good, actually. Good job. Um, and then I'll fill that out and then I'll say, this didn't work and I wanted a checklist. So it's a great opportunity. Many of you know my experience is I'm always trying to test things out. They're looking for our feedback. They're looking for real people trying to do real things to give them the appropriate instructions. This is called reinforcement learning with human feedback. The model can't be reinforced with positive behavior unless we tell it what it's doing right and what it's doing wrong. So please take the opportunity, go to the LinkedIn or LinkedIn, go into the Microsoft Learn blog, go look at that. Here's the other thing that's cool about the blog post is if you scroll down, you also will have a chance to request early access to, I think, a very cool agent, which is the Ask Learn agent inside your VS Code editor. That's pretty gangster, like being able to be in VS Code and while you're using a package, be like, what is this? How do I do this? Tell me what this means. So it's integrating all the learned content, i.e., documentation, right into a contextual request that you can make to an agent while you're writing the code. I think we'll even get to the point where it even surfaces things proactively. Hey, you're using this package. Would you like to know what this does? That would be awesome. But, but it starts first with at least giving us the ability to integrate the two together. That's available for early access. And I always tell people like early access is where the fun is at. Join, like show up, give feedback. You get to actually be part of building a feature that one day might be really, really popular. I've been on many projects that turned out to be household names. So, and the way it all started was request early access. So go do that. Now let's take a look at something a little bit, maybe more serious, maybe a little bit more technical. I'm gonna show you AI. I'm gonna show you Azure AI Foundry. Now, AI Foundry is basically the development environment for building— oh, let me open this up so you can see that a little bit more clearly. But you can see here, this is the environment, and you can go to ai.azure.com. This is the environment that you can— it's more on the engineering side versus the biz app side, and I'll talk about both. But you can go in and you can actually start leveraging and building models. So you can deploy your own models. I'll show you how to do that in just a second. And there are over— there are thousands, over 10,000 models to choose from. So many of you know I'm a big believer in actually being intentional about your model selection and model evaluation. Like, how do you know the model you're picking is a good model for the purpose that you're building it for? So please take a moment and think about that. The very first step when you go to build an agent is to make sure, number one, first and foremost, that you know what it's supposed to do. Like, an agent— the job of an agent has to be clearly articulated, clearly defined, so that you can then make a bunch of other subsequent, um, requests. And I will now take a moment to say, like, if you're watching this on YouTube or LinkedIn, you can join us live inside of the, uh, Zoom call. If anytime you want, you can ask questions. We also get a chance to kind connect with each other. If you wanna do that, you just go to noelle.ai, join I Love AI community, and you can be part of this. We also have a, a more premium advanced community where you, we are helping businesses get started with AI. We are also doing the Agentic AI for Beginners course. That's a little bit more intense. But if you just wanna be part of this call and have a chance to ask questions and get to know other people that are on the same path as you, go to Noelle, just my name, noelle.ai. And you can join us anytime. We'd love to have you. So now that I'm in Azure Foundry, and actually maybe I'll even take a step further back where Azure Foundry is the environment in which you can leverage Azure AI services. But where do those come from? So that is in the Azure portal. I think I already have here. Yeah, let's go here. So this is the Azure portal. Oh, it's gonna make me sign in. Hopefully I've already been in here, so it'll just let me in without me having to do any MFA stuff. But as I log in to this, you'll see here, this is the Azure portal. Typically, if you have a bunch of resources, this is what it'll look like. You can also— let's see if I go here. You could also go— I think I'm on the homepage and it just gives you a bunch— the ability to create new resources. But I think there's also a dashboard that you can look at. I just start here. This is where I spend most of my time. All the members of the AI Leadership Institute do become a user on my tenant and get access to to playing around with these technologies. We do live learning labs all the time, sometimes once, sometimes twice a month, or we build a class like Agentic AI for Beginners and we make that happen. But you'll notice down here I have a bunch of different resources. But when you start to create a project in AI Foundry, it's going to ask you to create what's called a hub. And you'll see this here, Azure AI Hub. This is like the, the area in which all of your projects will exist. So when you go to Azure AI Foundry for the first time, it's going to ask you where do you want to build your project? And so you'll build a hub and that hub gets represented inside Azure Portal. So these two things are very closely related. Azure Portal is where all the services are created. Azure Foundry is where you consume those services to deploy development assets. So here, if you want to, for example, create an Azure, resource, right? You can click on create resource. There's tons of them. I spend most of my time in this beautiful category, but in order to deploy a model in AI Foundry, you have to start here. Now, granted, you could start it in AI Foundry, but it's gonna send you here to create this, the resource. So this is—

can I ask a question?

Yeah, please do.

What's a foundry? Like, is there something special about that name? Like AI Foundry? Does that mean something?

Yeah, I, I think it, It is a rebranding of what we used to have, uh, in Microsoft called AI Studio. And Foundry is, you know, very similar to like the tactical way of thinking about it. Like it's actually a place where you're going to assemble components together and build solutions.

Okay.

Yeah. So they named it Foundry to move it away from just being a studio where you assemble things together to be, not only are you gonna assemble them, but you're gonna do deployments. You're gonna, You know, kind of think of that like a, you know, an iron, you know, mill or, or like a place where you build, uh, swords, right? Like you're gonna, not only are you gonna melt steel, that's one part, but then you're going to have molds where you put the steel in and then you're gonna have, you know, places where maybe you shine it and like take care of it. So the foundry is like all of these different aspects of building AI solutions.

Yep.

Yep. And, uh, but I always, I mean, people go into the foundry and they're like, okay, now what do I do? And so I wanna make sure you all realize like Foundry is the place you do the building, but in order you have to create kind of the, the, what do they call it? Like the, the base materials, right? Like those core materials have to be created like the Azure AI service and the speech service and the models themselves. So once you have done that, and I just, I'll go into Azure OpenAI just so you can kind of see what that looks like. You'll create a service. This is not like deploying a ChatGPT or a GPT-4.0 model. All this does is create the service that deployments can then be associated with. But this is required. This is how Azure bills you. This is how money is made. This is the transactional component. So it's required in order to build. So let me go over to the Foundry now and show you kind of what this looks like. So here— Yeah.

Sorry. So at some point we'll set up an account and then we'll— That folks who are eligible will connect their accounts to yours? Is that how it works?

Yeah, I actually, you created an account already. Remember you got an account, don@ailleadershipinstitute.com?

Yeah.

Yeah. So that is, that is your, um, Azure portal account. Oh. Yeah. And it allows you to log in to Azure Foundry. And so the services that you use are built in my portal, but you're leveraging them in the Foundry. Okay. Yeah. So absolutely. Okay. So now you'll see Create Smarter Agents with Azure AI Foundry. So you can go in and create an agent. Why don't we just click on that? It'll then ask you, remember I mentioned the hub? So it's gonna ask me what project I want. It's actually gonna build a whole new project for me. So I'm gonna just say Frye Yay Demo. And it's gonna validate that that name can be used, which shockingly it can. And now it's gonna set up this new project for me to do this work.

But all—

in order to build anything, I will need to have the resources set up in Portal to support that development. And many people don't know that and they get stuck. And so I just mention it to those of you who are not familiar, that that relationship is tightly coordinated, that you'll start in Azure Portal to build the services, and then you'll create deployments against those services in Foundry. So while that happens, I'm just going to let— I'm hoping it'll just let me know. I guess I'm stuck here for a minute. Hopefully it won't take too long. It says a few minutes. I had a bunch of projects. You can see here Fortify, Noelle Russell 532. That was just a baseline project that I created. This now becomes the home to, you know, it's just like a project in, in anything, creating a folder. Uh, it is an organizational element for you to put all of the resources of a certain type. In my case, I'll be deploying a retrieval augmented generation model with it. Now, inside our AI Leadership Institute and inside of the I Love AI community, both of those, when you go in, I created a Loom video of me walking through a lab exercise. But what I wanted to share with you is that there's this really cool part of Microsoft that every Friday they do 3 calls where they give you a lab environment in which to do a lab, and they give you a lab guide, and they just open up a Zoom and you just sit on the Zoom and if you get stuck, you can ask for help. I love it. I'm like, I need to do more of these. Super fun. The difference is, is that I don't want to write lab guides, but Microsoft is great at this. And so it's called Cloud Labs. It's launched through Microsoft. They run them every Friday. I am always part of the Lunch and Learn one. So those are the ones at noon EST because that's the hour I have. But they do 3 of them. And today they happen to have focused on building multimodal retrieval augmented generation solutions, or multimodal RAG. And so that is what I'm going to share with you. Although I might cancel this because y'all see how to build a project and go use a project I already have. Oh, of course, it's there. So, oh, success. Look, it was created. I canceled it, but it worked. It was just taking its sweet time. All right. So let me expand this out so you can see all these options. So things that were announced, lots of cool things. You'll see them all in preview. Risk, governance, observability, like a lot of these tools that those of us who are responsible AI leaders and practitioners, these are tools that now make it very practical to do the things that we say, right? So we always tell organizations you're going to want to build governance plans, but how do you activate those plans programmatically? Now we have tools that allow us to do that, mechanisms for actually embedding it into the solution itself. So now I'm not going— I'm going to go into where all of your deployments are. So I'm going to scroll up to do, to do, to do, and scroll down. So more. There we go. My gosh, how do I get to it? I'm trying. I can't. Maybe my screen is maybe too big. Oh, there we go. I couldn't scroll, but it let me. All right, so I'm going to go to models and endpoints. This actually just changed since I recorded it this morning. Let me see here. Crazy, man. They keep changing stuff. So you'll see here, this used to be called— literally this morning I recorded this video and it's already obsolete. But this morning it was called deployments and today it's called like assets. But that's fine. No problem. Fine. So under models and endpoints, this is where all your models are. Now, in order to deploy a model, I'm going to deploy a base model. I can choose from any number of models, which is pretty cool. However, in order to deploy a new model, it's going to ask me, let's say I say I want to deploy an O4 Mini and I hit confirm. The first thing it's going to ask me is where I could— look at here, the connected AI resource. That's the thing we built in portal. So you have to assign it to a resource in order for you to build the model. A lot of people come here and they're like, why can't I build a model? Why can't I deploy a model? Like, did you build the resource? They're like, what do you mean resource? So I think it's useful to know that these two things are very, very closely connected. Now, inside of this, I do want to mention to you this concept of the model catalog. Now, one of the announcements was that Groq 3 is now embedded as part of a mo— in the model catalog. So you can leverage a Groq 3 model in your Agentic workflows. For some of you who like, I'm not a Groq 3 user, but maybe you are. Good news, I, I like choice. So you can see here, um, notice here, this is right after my own heart, find the right model. Um, so there are now model leaderboards, also new, a new announcement that was made that you can go in and find out for quality, for cost, for throughput, right? You can go and browse the different leaderboards. How many of you have heard me talk about HELM, the Holistic Evaluation of Language Models? They were the originators of this leaderboard concept. Those leaderboards were embedded into NVIDIA, and now we're starting to see those same leaderboards here. What that means is that you don't have to pick a model just because it's the first one you see. You should not pick a model because you saw it on Instagram or got a, you know, saw an advertisement for it on LinkedIn. Like, you want to pick a model because it's the right model for the task that you're building. So these types of things, quality, cost, throughput, will give you a sense of how that works. There are other model comparison tools as well. So this is where you can see that models actually will let you— you can see the tradeoffs. If you decide to go with DeepSeq, what does that tradeoff look like? It's a quality tradeoff, right? You're going to get speed and latency will be solved, but the quality of the results will not be solved, which means it doesn't mean that it's bad. It just means you're now going to need to accommodate for the fact that that model might not give great answers from its training data. So you'll need to do more retrieval augmented generation architectures with those types of models. But it's kind of neat. So you could see where they sit on the line of quality. You can also compare, but like I said, there's quality, there's cost, there's performance, reasoning, all the different specific purpose models, right? Reasoning models, coding models. So you can see here, this is across all the models that are in the model catalog accessible to you from Azure. So, if you are not an Azure customer, if you're not deploying models on Azure, this is also available to you through Stanford. Stanford does a platform and technology agnostic model leaderboard. They have published the holistic evaluation of language models as a repo. They even created a Helm Lite that you could download to your machine. So, some of you aren't Microsoft people, and that's cool. Um, and so in that case, it doesn't mean you get out of the responsibility of knowing how a model performs before you adopt it, right? Baby tiger mode. Like, do not just go and randomly adopt a baby tiger into your organization. You should know, like, how likely is it to reason well? How likely is it— if you're building a coding model, you want the best ones available. So it's good to, uh, be aware of how these models work and where they sit in this benchmark. Leaderboard kind of scenario. So, so pretty cool. Okay, so the other thing I wanted to share with you is once you start looking at models, I'm going to— one of the features you'll notice up here is that there's a Foundry agent. This is fancy. Remember I just showed you the Microsoft Learn agent? This is the Foundry agent. It looks pretty similar, right? It allows you to ask for help on how to use the catalog. It allows you to search the catalog and identify, like, use the data that you're looking at in the leaderboard and actually ask it which one has better performance. You don't have to know how to read these diagrams. The agent will read them for you. So pretty cool stuff. I encourage you to just go out there and test it out. And of course, asking it questions like, how do I fine-tune a model? So this is just a question. I could type in whatever I want. So if I wanted to ask a question about Groq 3, I could type that in as well. All right. So, so that's pretty cool. This is in preview, which means it might work. It might not work. Oh, look, it seems to be working pretty good. It's nice. Thank you. It's going to give you the instructions and now you can actually use this as a tutorial. I mean, I'm doing this now for content on YouTube. Like, great, I have a tutorial. I'm going to go and test it out, use an agent to tell me what to do, and then actually test it out, record it, and stick it on YouTube. Super fun for you developer advocates out there that are always looking for demos to do. This will be like dynamic generation of demos that you can, you can build and run. All right, so that's pretty cool. It's pretty like subtle, right? On— it's, it's still subtle even if you weren't on a dark background, but, um, it's kind of small. It's in preview. Like I said, if it doesn't do well, let me go back in there. Notice here, number one, it will tell you where it got it. Sourcing it, grounded. Yes, I love it. But if it doesn't do a good job, I'm gonna actually say this did a good job. But give it feedback, right? These models are trained using reinforcement learning with human feedback. If you don't say that it did good or bad, the people building this model will have no idea whether it's doing good job or a bad job. So again, as a responsible AI leader, our responsibility is to take these preview capabilities and participate. And like you said, if you're not a Microsoft person, if you're not on Microsoft, totally fine. Find ways to be part of the early generation of these models and test them out and give feedback to these models so that they can become better.

It—

there really isn't a scenario where you can be like, this model is awful and I'm complaining about it because you have the ability to actually tell the team exactly what's wrong with it and track your perspective on a roadmap and watch it actually get fixed right in front of you. It could not be— I mean, it's a very, very cool time to be in kind of software, software engineering, AI engineering world. So, so you can see here it sources the information. So I'm now looking at Foundry Agent. Foundry Agent has some capabilities. It sources it. I wanna show you how to build a retrieval augmented agent just like this. And so the way that you will do that, um, is basically by using the AI Foundry to create the, create a, a, a model that allows you to do this. So I'm gonna give you a high-level overview of what this is, but I do have a deep dive video in the community. So if you join the community, you'll see it. But in order for you to build an agent that looks just like the Foundry agent that's sourced, grounded in data, in this case, the documentation. In order for you to do that, there's a few services that are required as part of Microsoft's infrastructure. The first thing you're gonna need, of course, is a model, and we talked about that. So you'll go to the model catalog and pick a model. For me, if I go down to my model deployments, I already have a model, GPT-4o. So that's gonna be the LLM that serves the, like, just like this, when I type in something down here in the describe what you'd like help with, that's the LLM piece. Now when I type in a question like, Can you help me? What did I ask? Oh, yeah. Can you help me fine-tune a model? That now is going to be, how is that question going to get answered? So what we want to do is we don't want the model to go to its training data, the internet data, and get its answer. We want it to be grounded in a very specific dataset. So on our model, what we can do, and I'm just going to click on it here, Right? This is the model. You'll notice here I can open it in the chat playground, which is what I'm gonna do. So you can see the options you have inside this chat playground. Notice here it says add your data. You can actually go in and add your own dataset to ground the model's behavior just like this Foundry agent. When you choose to add a data source, there's a bunch of different ways to do this. You can add a, you can go directly to AI Search if you've already got that implemented, or in new and preview, you can go to Blob Storage, or you can just upload a single or a collection of files directly to the system. I encourage you to use Blob Storage because that allows you not to have to come into this utility to add more files, right? If you open it up to a file directory, you could just add more files or updated documentation to the Blob Storage and the index will get created. So this allows you to, when you create this Blob Storage, you'll notice it's going to now ask you, okay, you're going to build a blob storage of content that's going to ground this model in some sense of truth. Where are those documents and how do you want it to be accessed? So the first thing we have to do, of course, is pick a subscription, which hopefully I have one. I do. And then I have to pick a blob storage, which is a service, a resource we would have created. I have a bunch, so I'm going to pick this one. Then inside that storage, it's basically a big bucket. I have to identify the container I want these files or that these files that I'm gonna use to ground the model, where do they live? So I'm gonna pick this one. I don't actually know what's in there, but, and then, so now, right, I've got my Blob Storage account. I'm telling it where all the files are gonna be located in that Blob Storage. And then most importantly, I'm gonna say, how do I create an index that makes it easy for my LLM to search that content. So I'm going to drop down this list and pull up Azure Search, which I'm going to do this one. Oh, not that one. Let me do this one.

There we go.

Right. So now I'm going to leverage this Azure Search service in order to, to, excuse me, in order to, sorry, in order to actually look at the contents of that directory and serve it up when people ask, ask me for it. In order to do that, I need to create an index. So I'll create an index. I'll just call it index 1. And then it's going to ask me, using AI Search, how do you want— how often do you want to index this folder, right? If you're building something for like a medical practice for them to look at all their SOPs, their SOPs probably don't update very often. So you could say just do it once, or you could say do it daily. Or I don't think SOPs are gonna be an hourly thing, but if I'm, let's say, building an agent that's gonna be on a website that allows people to ask for their order status, I might do that hourly, right? So that every hour they get a new update, a new refresh of that information that, that grounds the respo— the response of that model.

Cool.

And then I'm gonna add vector search because I want it to be able to, um, I can't add it.

Wait, can you, can you explain that again? I'm sorry.

Sure. Explain the indexer?

Uh, yep.

Sure. Yes. So basically what is happening is that we've got blob storage identified. Oh, I know why actually. Um, but we've got like a location, consider it a folder, a folder that's got 100 PDFs in it. And I want that 100 PDFs to be the grounded truth that my model uses when it goes to look for an answer.

Okay.

And so what I'm doing here is I'm pointing it to where that directory of those 100 PDFs are. And then once I do that, now I'm pointing it to a directory, so those files could change out from underneath that, right? I could swap out files, update files, add new files. So what this allows me to do here is say, how often do you want me to index the contents of that folder? Like, how often will the contents of that folder change?

So meaning like if you had new information coming in, then it would check it like, yeah, like let's say it was order data, right?

So every time, you know, SAP takes you to the next step of your order process, it might send a message to this document that says, hey, we're now going from shipped to in delivery or whatever they say in these things, like en route. Yeah. And so now you don't want that to happen daily or only just once because it'll never pick up those new changes.

That's crazy. Okay.

Yeah. Cool, right? I think it's pretty cool.

All right.

So now once you've done that, you've now added this, and again, you could go see the deep dive video where I actually build out the whole thing. But you've now added to your GPT-4o model, which is multimodal, you've added through the Azure Data, you've created an index that now allows you to refine or ground the model in the truth of whatever's in that blob storage account. So something worth trying. Like if you have access to this, and everyone in the AI Leadership Institute definitely does, you can go in and just test this out. Like ground it against one document, a PDF, maybe grab a PDF from Stanford or a medical document online and just create an agent that allows you to, to surface information from that. But I think it's kind of cool because now we're starting to see these agents show up. I, as a business owner, am constantly like, ooh, they did that? Oh, how could I do that in my own company? And now that I just showed you kind of how to do that. Now, this of course is pretty technical. I'm building models, I'm making model selections, I'm using leaderboards. If you're not that technical, there actually is a no-code way to deploy a similar capability. But before I do that, I do wanna share with you some cool things that are happening in the model catalog. So you could see here there's an option when you come in here, it says see all models. So I'm going to click on that. It's gonna take me to the Foundry's model browser. And in the model browser, if I scroll down, look, these are all the models that are available to you to use. We don't want you to just go and randomly pick one. That's why the leaderboards now exist. Find the ones that are good at the problem you're trying to solve. If you don't know, leverage the agent to say, hey, this is the type of problem I'm solving. What's the best model for this? Don't blindly accept it. It's in preview, but it might give you a good sense of like where to look. What I wanna bring your attention to is a brand new launch that occurred. There's a search here, so I'm gonna type in speech. One of my favorites. You know me, I started off at, at Amazon Alexa. That's where my AI journey began. So I'm gonna choose Azure AI Speech. And what I like about these models in the catalog is they actually give you the ability, look at this, they give you a speech playground. So you can test out these models without deploying them, which is kind of nice. So you can go to the Speech Playground and play around with them. But I wanna take your attention to a brand new service that has been— they've been working on this for a long time. It's currently available, GA'd Professional. Oh wait, where is it? Oh, I'm not there yet. Here it is. So Professional Voice is GA'd, meaning you can create a version of your own voice or of an avatar's voice. And it's neural text-to-speech, which means it's very human sounding. What we announced also at Build was Voice Live. Voice Live allows you to add the microphone button to your agents and do real-time voice, also real-time translation in 151 languages. This is cool because it sits as a service inside your cloud infrastructure. So all the things you did to secure your cloud infrastructure now applies to these AI models, meaning you don't have to do an extra layer of security. It's already gonna sit inside your security, ecosystem inside Azure. So that means you've got role-based access control. I mean, all sorts of cool stuff. So I know today it was a little technical, but I thought it'd be cool to share this with you. I'm gonna click on it so you can see what it means.

Yeah, 'cause I'm curious. So does that mean like your agent could speak? Like if you deploy an agent and somebody wants to talk to it?

Yes, 100%. And now you can even combine that with a visual avatar. So now you can have a visual avatar that speaks live in real time. Many times we have to batch feed it a script, right? So it knows what it's going to say ahead of time based on an interaction from a user. Now it's completely live, real-time custom voices to allow your avatars to come to life or just activate a voice-enabled chat. But this is like an entire practice waiting to be built. Now that this has stabilized, it's still in preview. So it's a great time if you are a consultant, or a solution architect, it's a great time to dive in and learn it and use it and build demos for it. And then as it GA's, you'll be ready for clients who wanna implement it. I already have clients that wanna implement this now and I'm like, easy. Like, let's, let's wait till it GA's because it's not perfect when it's in preview. They're still trying to learn all the different ways we'll use it. But the more of us that use it, the better the service will get.

What's a GA?

General availability. Thank you for asking. Sorry. Using all these acronyms.

I'm asking all kinds of questions today.

I know I was raised at IBM, so I like, I just seep acronyms out to the world. But yes, general availability. Yeah. So preview, and then there's, there's different levels of preview. There's like private preview, which most people don't know about because it's private. Then there's public preview where anyone can join, but you do have to have a good use case. And then there's preview preview, which is available to everyone, but it's not ready for production. And then GA, when it actually becomes generally available to for everyone, and it falls under the service level agreements of a production service at Microsoft.

So how do they make these decisions? Is it like, okay, you are a person who has X amount of revenue or whatever, I don't know, like, right? Because you, you get some behind the scenes stuff based on your experience, you've been in the field for a while, but like a person who hasn't, do they request the stuff or just—

Yeah, so in, well, when it's in preview, you could just use it. It's just giving you like a disclaimer, this isn't ready for production yet. So if it fails or whatever, but in the private previews you get invited. So typically it's like MVPs and customers that are invited to participate in the earliest stages, which is the worst it'll ever be. Right. Right. Right.

Right.

Um, but then public preview, anyone can join, but you do have to clarify why you want to use it. And if you don't have a good reason, like, or if you are a bot, like they're not going to let bots like go into this, you know, public preview. So usually it allows you to go in and see, you know, when, for example, if I go and try and use, I think mine might be okay now. But I'm going to, oh, I think I'm good. I'm going to say open in Playground. So I think I already have this. Yeah. So now I'm in the Speech Playground. If you've never been here before, it might, as a matter of fact, you might have seen it on the blog that I showed earlier. It'll give you a Google Form or oopsie, just kidding, an Azure Form. To fill out in order to apply to that team. And, and it's literally humans read that form and go, okay, you look legit, and then they give you access. And I encourage you to fill all those, as many of those as you can find, because it's really cool to be part of the early days of these, these features.

Yeah, because I was thinking, like, especially with all the Google stuff that's rolling out, like v03 and all that, I'd love to figure out how to get, I don't know, some sort of testing environment, or not environment, but you know what I mean, be on the pre-release 100%. I just don't know how.

Yeah, so you could search for public preview, um, but you also could get involved in the GitHub repos. So GitHub repos are also a way to— that you can start contributing, and then they start— they, they will send people— whoever is listening or forking a GitHub repo or starring a GitHub repo, they might send a message and say, hey, we now have a chance for you to test this out. It's like TestFlight but for software. Okay. Yeah. Yeah, great. Alright, awesome. In our last few minutes, I did it again. I did this last time too. So for those of you who can hang out, we would love to connect. This is the point where, I mean, I just thought it would be really cool to show you. Oh yeah, okay, I do have to finish with the last promise I made. Close the loop on no-code solutions. But this is Copilot Studio. I don't want that one. There we go. Microsoft Copilot Studio. And so Copilot Studio is the— oh, all right, I'll sign in again. Hopefully it won't take too long. But this is the platform that allows you to do very similar capabilities, but in a low-code, no-code way. So typically business users or AI engineers that don't want to get caught in all of that nonsense you just saw, right? Creating resources in portal, going to Foundry, deploying a model, like identifying Azure Search, all that stuff. That can get complicated. And so this is a simpler way to deploy agents for Studio and augment the capability of 365 Copilot. So you could see here, I'll just scroll down a little bit. You could see some of the agents that are already prebuilt that you can use or even customize. Now, in the AI Leadership Institute, we did do a live learning lab on how to build a custom agent inside Studio. Maybe we'll do it again now that there's some new capability.. But if I go to see more, you could just see kind of this collection of things like IT help desk, right? Who wouldn't want a, you know, what IT service desk person doesn't want an AI agent that can answer some of the easiest stuff for them, right? Maybe you wanna add the ability for people, if you're an HR person, how do we surface our benefits document? I'll just go into this one to give you a sense. It's gonna build you an agent, but in order for it to surface and become a valuable benefits you know, agent, it's gonna need to know where your benefits documentation is, right? So you'll notice here, down here, it says knowledge. You have to add the knowledge base, which might be your employee handbook, your benefits documentation. Let me go ahead and click on that so you can see what that looks like, right? But it could be a public website, a SharePoint directory, ServiceNow, Salesforce, just depends on what you have access to in your environment. So this is very similar to what we just did grounding our model. But you're going to ground an agent that sits inside a Copilot. And that could be an easier approach or maybe a simpler approach to getting the same functionality if you don't require the customization capabilities of building it inside Foundry. So I just like to mention it to you, give you some fun things to work on this weekend. Oops, that's not what I meant to do. Oh, I'm going to cancel it. Yes, leave.

Goodbye.

So all these agent templates, And then what's new is that these all are individual agents. They used to be called Copilots. But now they're agents, and these agents can be synchronized or choreographed together. So you'll notice over here, there's the concept of flows. So now I can build— maybe I start with the IT help desk agent. And if I do that, it's gonna ask you to configure, right, and connect it to your service desk. In this case, ServiceNow. You can see, right? But I'm gonna cancel that and go back, leave. So you could see here, you know, once you start, you start there, but then there might be 6, 7, 10, 1,000 other things that you want that agent to be able to do. So you could start creating flows, automations with these AI agents that it's kind of like old school automations, but supercharged. We now have the ability to identify what they do, when they do it. They can operate autonomously. They can connect to other systems. They can report back. Audit trails. Like, there's so much more capability that we now get as opposed to just automating the call from one system to another. It's now activating two systems to communicate to each other and provide different levels of output. So there's lots of cool stuff in here. Voice, citizen services. If you happen to be— I created an organization called Small Town AI where we're helping small towns, literally like Main Street USA, implement AI services. And this is an easy— you could just go in and say, hey, I— you're already a Microsoft customer. Let me help you by building in the citizen services, and you can expose that to your citizens and make it easy for them to navigate your super complex website. So it's kind of neat. Kind of neat. Anyway, so I just invite you to take a look at both. Depending on your skill set, you might feel more comfortable starting here, or you could dive right in, take a look at the video that's in the I Love AI community. I created it just for you. It's super fun. And more importantly, join the sessions. I put the link in the I Love AI community, but join the sessions on Friday. They do 3— Microsoft hosts 3 developer sessions. And when you launch it, it actually takes you into a VM with a subscription. So for example, if you're on my VM, I don't have Azure AI Search. Like, it's super expensive to host. So I don't do that because it's a slippery slope and I end up spending tens of thousands of dollars. I'm like, no, no, no. But I would much rather go and use Microsoft's tenant. They give it to you for an hour and you could just play around with it and you can, you get Azure AI Search, you get, you get all of the expensive services and most importantly, they're already set up, which is pretty awesome. So I always like to end our call with just a little chance for us to get to know each other. For those of you who are here on the Zoom, this is our chance. We're gonna, I don't know, it's, I know it's 5 o'clock. Some of you have to jump. But if you could stay, we're gonna just, I'm going to stop the recording. Thank you all for watching. I hope you enjoyed my quick journey into some of the new features that have been launched, some of the new agents that have been launched, and most importantly, how you can build agents of your own. But now we're going to get to know each other. We're just going to open, you know, turn on our cameras, unlock our microphones, and just have a chat together. So thank you all for being here. I'm going to stop the share, stop the screen. If you have any comments, questions, please don't hesitate to join us here, um, in the I Love AI community or send me a DM and I can tell you or answer any questions you might have. Uh, thanks for being here. All right, hold on. I'm gonna stop recording at some point. I gotta figure out where it is. Hold on. Wait. Oh, okay. I found it.

Found it.

Um, there we go.

One second.

Oh my gosh. I'm not in the right place. Wait for it. Oh, there we go. All right, I just don't want all of our like conversations to be live streamed on if we can help it. You know how it is. Um, all right, here we go. Oh, you know what, I have to stop the share first. I was like, I can't see, can't see the options. All right, here we go.