Everything You Need to Know About Microsoft's New AI Computers

We've entered a world where our computers are now being designed from the ground up specifically for AI. Companies like Microsoft are building computers with special chips in them that serve the sole purpose of running AI. So in this episode, I want to demystify the world of AI powered computers. And to help us understand it all, I've invited Pavan Duvallary from Microsoft to the show. He's the corporate vice president of Windows and Devices and is right at the forefront of these incredible developments. We'll dive into topics like privacy, especially with features like recall, which automatically take screenshots of your computer. We'll talk about how these new AI chips are making AI more accessible and affordable to everyone. And we'll get a glimpse into what PCs might be capable of just five years from now. And trust me, it's moving faster than you think. Also, while I was at Microsoft, I was given the opportunity to ask Satya Nadella, the CEO of Microsoft, just one question. And at the end of this episode, I'm going to share that question and his response to it. So definitely stick around for that. But let's go ahead and get right to it and dive into my conversation with Pavan Davolari.

So the next PC that people might buy, it might have an AI processor in it, the new NPUs?

Yes.

I'm curious what sort of things that somebody that just uses it for, maybe email and Netflix, like, what sort of benefits are they going to get by having an NPU in their computer?

It's a great question. I think neural processing units are going to be, I think, a third processor inside your computer, just like we have CPUs and GPUs. I think NPUs will get added to that mix. And primarily because NPUs will give you a lot more access to running AI efficiently on your device. And our goal very much is to have these AI capabilities become available broadly to both consumers and commercial customers, and really build a platform where developers can build on top of it as well. And I think it'll show up in a couple of different ways. First, your own Windows device experiences. Whether it is something as simple as getting settings to be simpler and easier to use, whether it is how you search for your files and folders in the operating system, or whether it is how apps on top of Windows are built, will all change, I think, going forward. I think at the end of the day, the goal is for those devices to become more simple, more intuitive, become more thoughtful in terms of completing tasks, you know, activities on your behalf. And at the end of the Day, just accelerating. I think what you can find, you know, possible or find ways to make happen on your computer.

Right, Very cool. I know, I know some of the stuff that has been teased has been, you know, obviously NPUs, you can start to run AI locally on the device.

Yes.

And so I imagine, you know, people who are just using it for things like email and stuff like that. I mean, even email should get simpler for you. Right. And so kind of summarization that you can do that goes through the cloud now. Now we can do it locally.

That's correct, Yeah. I think a lot. And you see this with features that we have on copilot PCs. Right now we have this idea of click to do, which gives you a one click moment where we have understanding a screen and context. And when you click right, we open up a variety of tools. Kind of like your email example. Just the ability to write, summarize, understand content, I think will become a lot more pervasive. Certainly with emails you might be offline, you might have encrypted content that you only want to see summarized locally. The fact that those skills can become proactive on your behalf are things I think customers and consumers will see on a broad basis going forward.

Definitely. Can I ask you the difference between. We have CPUs, GPUs, NPU's. For the laymen that don't know the difference between them. Me, I don't really totally know the difference. Can you help me understand the difference between all three of them?

Yeah. I think most of the world has built applications and devices and experiences that utilize the CPU. Over the last couple of decades, GPUs have become really important, especially when it comes for gaming, using high resolution displays, for CAD type workloads where visualization is important. And I think the value of the NPUs essentially for client devices, for laptops and battery powered devices, is to be able to give you the ability to accelerate the ability to run these models and sort of lower the footprint and tax of those models running on your device. And at the end of the day we expect these npus will make it just easier and lower cost for you to have models running on your behalf on a pervasive basis inside the device.

Right, right. And so the npu, it doesn't necessarily mean that you don't need a GPU anymore. Right. So you're going to still have an NPU and a gpu. Correct. Which this is something we were talking about a little bit last night, is.

You might be able to offload the.

Sort of video processing to the GPU and still be able to do things with AI using the NPU.

Bingo. I think NPU's will be a complement to GPUs and CPUs. And I think the reason why NPU's are useful is because they're very efficient, Matt, when it comes to energy efficiency and battery life. And so you can run pretty powerful models, pretty capable models, but you don't need the gigantic footprint of thermals and battery life and heat sinks and all the stuff you'd expect for running a large model. These NPUs really just get efficient in terms of running that model for you. The beauty with that is concurrency. So you can have your apps, all the apps that you know and love today, doing all the things that they do and add new AI capabilities to those things and have those AIs, you know, be offloaded onto the end.

Right. Not bogging down the GPU to do them.

Correct.

And, you know, another thing is, so I got a chance to sort of check out the Applied Science Lab.

It's great to have you.

Yeah. One of the things you guys talked about during, during the tour was that having this NPU really sort of democratizes AI. And I think that's a big concern, right? Is people feel like, well, maybe only AI is going to be for people that have a lot of money and, you know, the haves and haves nots with AI. And it sounds like these NPUs are a little bit less expensive to produce than GPUs. So maybe you can talk into this, sort of like the economics of that a little bit.

NPUs are more purpose built for running AI models and workloads. And by virtue of being more purpose built, they're inherently more efficient by way of the size and cost associated with building those NPU's. So the benefit for us there is we think we can deliver the NPU and the performance nature of the NPU more broadly across devices, across a variety of devices and endpoints. In fact, we ourselves have started to do that with Copilot PCs. Last year, when we introduced those Copilot PCs, we initially targeted a set of customers that were more premium devices and prosumers. And this year now, we're able to offer those same class of NPU capabilities to a much broader footprint of devices, more mainstream price points. And it's very much happening because npus scale better with price because they have the ability to be focused on running AI compute, and then be efficient and performant in that space. And so our idea very much, especially with Windows, is to be Able to bring the breadth of these features and capabilities to the broad base of our Windows consumers globally. And having price, performance be great, have performance per watt be great is important for us. And npus are our vehicle for making that come to life.

Yeah, yeah, I noticed too, one of the things they showed us was that they pointed like a temperature gun, like a fleur kind of thing, at the two computers. And one was running the NPU and it was like 70 degrees, and then the other one it was pointed at, it was like 113 degrees. That's pretty crazy.

Yeah. And I think it points. It speaks to the fact that they're just more efficient. And because they're more efficient, they consume less energy, they generate less heat, they give you a longer battery life, they give all of those attributes a lower price. And so we look at them as a vehicle for then getting Runway and scale with these devices.

Right, right. I want to talk about recall real quick. When I first saw the announcement of Recall was it build last year, that was the first time I thought that was the coolest thing. Basically having the whole history of what was I looking at yesterday? And you can go back and find it. But I know that there was some sort of privacy concerns and things that popped up around it that sort of freaked people out a little bit. So I'm curious what sort of things? Like how has recall evolved since then?

Yeah, it's a great question. And I think we think of recall as one of several places where we think about the capabilities in the operating system. Evolving that capability and feature set, you know, surfaces and manifests itself in a variety of different ways. Recall is one of them. Like we talked about earlier, Search is another great one. For example, click to do is another experience. Camera stacks, audio stacks, paint having, you know, new capabilities, photos being able to relight themselves. So AI is going to show up in the device and in your operating system in a variety of different ways. Recall was a great learning experience for us in terms of understanding our customer needs and expectations. But we have privacy and them feeling like they were in control. And it was a good experience for us to make sure. The development process of Windows allowed us to make sure we were taking advantage of those points of feedback, which is exactly what we did. We had several set of private previews, release previews with customers. We got great feedback through it, and we've now successfully ga'd the product. And the early signals we're seeing so far is there's a set of customers who opt into the device experience. And it really helps Them kind of get into the flow of finding and searching and reliving points in times and really augmenting their memory in a digital context. And we're looking forward to the continued evolution of that feature.

Right, Right. So it's not turned on by default on computers. Right. So like if you get a new AI PC, it's not you have to actually opt in, right?

Yes.

As opposed to opt out. And also you guys aren't sending anything to the cloud. Right. It's staying right on the PC.

You nailed it. A couple of important points. First, it is an opt in experience. And after you opt in, there are a variety of features in the use of the product that are user defined and controlled. So you have the ability to define what you would like your recall experience to be. And then very importantly, the models and the data stay local on the device.

Right. And they're using the new npu.

They're using the npu. The model is running on the npu.

Very cool. So I want to look into the future a little bit too. So five years from now, what do you think we'll be able to do with PCs that we can't do today?

You know, we think about this quite a bit on the Windows team and I feel like we make plans and what's surprising with the plans is the rate at which they are are changing. In some ways it is happening faster than we anticipated. What I think at the end of the day, a core element of the Windows proposition is to make sure we're in the business of empowering our customers and consumers and developers and commercial information workers to be able to do more with their computers, with their PCs and with Windows. I think that will be true more so true five years from now than today by way of actual features and experiences. I think we see a world where Windows makes this evolution to being an agentic OS very much like we talk talked about it. Build with the agentic evolution of the web itself. And I think that evolution of the OS itself will be a platform construct. We ourselves will build a bunch of new experiences where you have models and agents and capabilities running inside Windows in itself. And I think it'll also be a world where developers will be incented to build a bunch of new applications and experiences. Apps that you know and love today will extend themselves with new capabilities either net new apps are going to show up in the ecosystem that use things like model context protocol, for example, to be able to to talk across applications and talk to the OS in ways, quite frankly, we have probably not imagined. Yet that's kind of exciting in itself.

It's funny because I constantly try to make predictions of where I think things are going and I'm like, yeah, that's probably three years out and then it happens three months later.

That's right. It's kind of amazing. Yeah. One real example of that for us is the performance and capability these models are running on the npus. A year ago we were kind of wondering if we would have a billion parameter model run on the edge. And what we were talking about earlier was we just last week had a 14 billion parameter model that has reasoning capability running fully offloader to npus. And so what that means for developers, what that means for the Windows experience, I think super exciting for one, and it is happening at a faster rate than we probably could have imagined.

Yeah, yeah. Is there any sort of misconceptions that you hear around like the AI PCs that you want to sort of lay to rest?

You know, I think the biggest thing is customers just knowing that AI PCs are a full stack experience from the hardware, the device itself. They deliver great fundamentals in terms of battery life and security and performance. And then all of that ladders up to serving a capability or a platform that in turn has great AI experiences. I think is probably the most important things for people to know. And so when you're in your journey of having your next PC, you should expect this device to be just a great device, daily use, I mean, also a durable construct in terms of future experiences that are going to get unlocked, taking advantage of the platform.

Right. So when it comes to AI, right now it feels like we're in this world where like everything is just like super fast. And it feels like companies are sort of motivated to ship things really fast. How does Microsoft see balancing, trying to keep shipping new features and keeping people sort of impressed with the privacy, security, the kinds of concerns people have.

I think you nailed it. I think balance is the key for us. And so in Windows, for us, I think of it in a couple of different vectors for sure. As a team that builds products, we have a variety of mechanisms today for making sure we have active listening systems across our ecosystem. And so we build a lot of these features using release previews in Windows where we get feedback from insiders, we get feedback from the developer community, we get feedback from the industry at large, quite frankly. And so that's one important aspect of our product product development system in Windows in itself that allows us to make sure we're getting rich, robust feedback at the scale of Windows that's one important piece, Matt. The second thing, kind of like with the MCP work that is happening in Windows, it is happening quickly for sure. We are in a world where we are at the rate at which the industry is evolving. In that example, the fact that the Windows team is a part of building new technologies, building these new standards, building protocols, allows us to go at day one, build these capabilities into the base technologies in a way that will serve Windows customers in the long arc of time in itself, I think. And the third one I think is some of this is an ecosystem exercise where we will deliver some of these experiences for sure. And a lot of this is others who are going to build on top of Windows and us getting signals from them on what they are seeing from their customers and making sure we're setting them up for success.

So opportunity and multiple vectors and we have a variety of tools in the toolkit to make sure we're delivering meaningful value at the end of the day.

Very cool. Well, this is my last question. It's sort of a two part question. What's something that excites you about what AI can do today? And what's something that excites you about what we'll be able to do with AI in the near future?

The things that I get excited about with AI today personally is the ability for us to do things like deep research and analyst work on the M365 copilot. It's a capability that is an asynchronous task. It takes a while to kind of run through. It requires a reasonable amount of domain knowledge. It requires understanding of your corporate environment and understanding of your team or your discipline or your department. And I'm very excited with the quality of work that comes out of these high performance agents that are running in the Microsoft copilot environment. The M365 copilot environment. That was a thing that I think a year ago to your point earlier I don't think I would have imagined. It's simply just possible. And now we're getting to a place where they're becoming a part of our collective team's workflow. When we do analysis, when we do reports, when you synthesize feedback, when we make preparations for what future roadmaps are going to look like. So that's the thing that I'm amazed with, quite frankly. And your second question, what's coming down the pipe? What's going to be exciting? I think that's the singular thing I'm excited about is what I consider to be sort of this 10x thing by way of unlock of what is possible on the edge. I think you'll be living in a world where the devices are going to get more performant. We in Windows are spending a lot of time making sure the software tool chains and the runtimes and environments for these models are getting more performant. I'm excited that the models themselves are getting better, like adding reasoning on the edge as an example. And I'm also super grateful that we have a set of class of developers who are building on top of the and so I'm just excited that for years we would invest in how much more experience and value can we get. And Kevin talked about us primarily relying on Moore's Law. That's all we heard. And now I think you have these compounding effects of innovations happening across the entire device edge client computing stack that will just unlock, I think, new things that are possible for customers.

Amazing. Well, thank you so much for spending the time.

Pleasure.

Appreciate it.

Thank you. Thank you for having us. Lovely to be here and build with you.

Yeah, awesome.

Thanks.

Okay, let's be honest. Your AI prompts aren't giving you the results you deserve. But with a little coaching, you can transform from basic prompts to engineering conversations that get you exactly what you want from ChatGPT. That's what this Playbook delivers. Not just random prompts, but a step by step system with the exact techniques top AI professionals use every day. You'll have your own personalized prompt engineering system that delivers consistent results. Get it right now scan the QR code or click the link in the description below. Now let's get back to the show.

All right, I mentioned in the intro that I had the opportunity to ask Satya Nadella just one question. So here's the exact question I asked. If you can design an AI system that would fundamentally change society beyond just answering questions and generating art, what would it look like and what risk and responsibilities come with it? That was the exact question. And here was Satya's response.

I would say the thing that I'm most inspired by was one of the demos I showed even today is in healthcare, because I feel like what touches all of us is this challenge of can we improve care and reduce cost? So if there was one place where I would say this agentic AI has to make a real difference would be take one of the challenges that we have as a society and go at it. And I think we are at the verge of it. Like what Stanford University was able to do by just essentially for something so high stakes like the tumor board meeting and orchestrate all these agents and then ultimately empower the caregivers there. Right? The doctors, the nurses, all the specialists to be able to have a more successful tumor board meeting and then improve. That, to me, is where I think these systems built and then made available can make a huge difference.

Thank you so much for tuning into this episode of the Next Wave podcast. If you haven't already, make sure you subscribe to this show on YouTube or Spotify or wherever you listen to podcasts, and hopefully we'll see you in the next one. Thanks so much for tuning in.

Sam.