Microsoft And OpenAI Just Demoed The Most Advanced AI Agent Yet

There was a recent demo at Microsoft's Ignite event where they actually showcased a lot more about AI. But this was a kind of agent that's a little bit different. Now, before I show you guys this crazy demo, there was actually a minute where Satya Nadella actually talks about this Azure AI Agent service. Now this is something that I don't think the average person is of course going to use, but it is worthwhile understanding exactly what this is. So, so that you can understand the coming changes in the AI ecosystem and how they actually impact the agents that you might want to build in the future or the ones that you're interacting with.

Right. Our new agent service helps developers build, deploy and scale AI apps that automate business processes. You know, I showed you before how you can use Copilot Studio to build agents just with few clicks. But as developers, you want a code first approach to building an agent, and that's what the agent service really enables. You can build agents that are grounded in data wherever it lives. Public data from the web, enterprise Data in Microsoft 365, SharePoint, or you can leverage even fabrics OneLake to unify your data across all of your clouds. And these agents then can take actions. Right? So you want to be able to give action space to these agents and you can take that, you know, thousand two hundred plus connectors we have in logic apps that we have been using, as you know, in our app services, and you can connect it to the agent runtime.

Okay, so this is where we actually do get to take a look at this demo. And what I want to point out with you guys here is that this demo is actually a lot different than the usual ones because this is where we get to see the AI agent go around and do different things. Like the guy says, it's just not in that contained box and is actually able to truly engage with you as a person.

And by the way, everything I'm going to show you is actually live. Okay. In this demo, I will use GPT4O, our new real time audio model. GPT4O audio, the real Time API and image processing models, all seamlessly integrated into the experience. I'm going to show you in a little bit. Foundry offers over 1800 models for you to choose from if you. It's pretty amazing. Once I've chosen my model, I can use Azure AI agent service, call APIs when needed and perform actions with function calling. In this example, I have a main customer facing agent that is then delegating tasks to other agents to understand what the customer wants what their purchase history is and to recommend products based on multimodal conversation with the customer. These agents can run micro intelligent tasks to provide a cohesive personalized experience. These agents can also be grounded to a variety of data sources, including fabric. It's pretty awesome. Actively. Let me show you how all of these capabilities I just showed can create a hyper personalized customer service experience that solves real world problems in ways that we couldn't even have imagined a year ago. Okay, so I'm going camping and my friend sent me this picture and I want to make sure I have the right gear for a winter camping adventure. So I started a chat. Now notice this is something that's now routine AI interacting with me with chat and images. But I want to. Chat's great and, and we all love chat, but what, what if we can break out of the actual chat box and do a more personalized experience? Because it feels like, you know, AI today it feels like it's all just like inside of your chat box. It feels like there's a, there's a lot of copilots and stuff. But what if we could do a little bit more and maybe make it a little more personal, hopefully with a phone call? Hello? Is somebody there?

Hi, Seth. Yes, I'm here. I see you're getting ready for a winter camping trip. Can I send the concierge page to your browser to provide more details?

You certainly can. Send me to the concierge page, please.

Great.

How cool is this?

Sending you to the concierge page now. You'll find more detailed guidance there for your winter camping trip.

So this is awesome. What do you recommend for this camping trip?

Hey Seth, super exciting that you're gearing up for a camping trip since you've already got the Skyview two person tent. Let's build on that. Here are a few recommendations. First, Mountain Dream Sleeping.

So this is all great. Is there a way that you can write it up for me so that I can see it? Maybe with pictures?

Absolutely.

So right now, what it's doing, it's going to take a couple of seconds to write an entire article for me, just for me. And you're probably wondering, well, hey, it's taking a couple of seconds. When's the last time you wrote a personalized article for someone that called you in 10 seconds?

Seth, I'll put together a personalized recommendation article with all these products and their details, complete with pictures. Give me a moment and it should pop up on your screen shortly.

You already did it. This is amazing. I love everything Here.

Fantastic to hear, Seth. I'm glad you love the recommendations. Enjoy your camping trip. If you need any more help, feel free to reach out. Have an awesome adventure out there.

How about them apples? And remember, like, I'm serious, this was running right here because I wanted to show you something real. This isn't a peek into the future.

Now with that demo. I think that demo was actually really cool because it shows us how these agents are going to be in the future and how they're going to be able to do certain things that allow us to get a much better customer experience. I don't know about you guys, but I've had several different occasions where customer service was just completely poor. But having an agent right there that can do things on my screen, that is going to be immeasurable in terms of the value. Now, do you guys remember how a couple of days ago I made a video talking about OpenAI and how they were working on several related agent research projects? And basically the fact that with these research projects, apparently there is one nearest to completion that is going to be a general purpose tool that executes tasks in a web browser. So I think this is going to be something that is super, super fascinating because as you all know, Agent is something that everyone is really eager to use, especially since we've been teased by Sam Altman stating that this is the next breakthrough. Now, for those of you wondering about which date this is going to actually be available in terms of the advanced AI agents and some kind of AI agents, we do know that OpenAI leadership announced plans to release this tool in January as a research preview and of course through the company's API for developers. So it seems like during the January time we're probably going to get a lot more AI agent demos from potentially a lot of developers and we're going to be able to see what these kinds of tools are able to do when not restricted to OpenAI scope. And the reason that that is really important is because developers take products and essentially build on them in a way that is completely different that the initial creator didn't intend to like. There are a variety of different consumer applications and consumer products out there that the creators of ChatGPT just couldn't have fathomed themselves. So it will be really interesting to see what developers managed to do in January. And also what I'm really excited for is OpenAI's computer agent, because it was also very recently that they announced they were going to be working on an agent called the Operator. So I'm actually wondering what kind of things this agent is going to be able to do. And I'll definitely be experimenting with the CLAUDE computer use to be able to see if I can get as much as I can out of that agent as possible so that when the OpenAI agent rolls around, I'll be able to use that one as much as possible. So with that being said, if you guys.