AI Agents vs. Human Employees: Where Your Security Fails

As one of the world's leading voices in artificial intelligence strategy, innovation, and leadership, Noelle is a passionate advocate for responsible AI. Her career includes transforming businesses with AI solutions at NPR, Microsoft, IBM, AWS, Amazon Alexa, and she continues to shape how humans and machines interact safely and securely at scale. scale as the founder of the AI Leadership Institute and through her passionate advocacy for responsible AI. There's a lot more that I could say, but, you know, why don't we— why don't we let Noelle introduce herself?

I am merely Noelle, a synthesized avatar here to help with the introduction. Get ready to meet the incredible Noelle Russell, She's not your ordinary technologist. She's a game changer. With her entrepreneurial spirit and expertise in conversational AI, generative AI, and language models, Noelle has revolutionized the tech industry. But here's the best part. Noelle is so much more than her professional achievements. She's a true inspiration for balancing work and family life. As a wife and mother of 6, she embodies the definition of courage and determination. Get ready to be blown away by Noelle's spectacular journey. Join me in welcoming the phenomenal Noelle Russell.

You know, so I do talk about artificial intelligence. So how do you not start with a deepfake of yourself? I mean, and we're at an identity. I just felt this is the place to do it. So that was Nearly Noelle. And how many of you in here have ever used a platform called Hey? HeyGen? Anyone? A few, a few. I mean, it's a thing where you go and you talk for about 2 minutes into your camera and you upload your likeness into an online server. At some point, you do say some words that disclose and basically give away all the permissions to your likeness to this company. And then, so I mean, the fact that there's only 3 of you, I don't know, it's a little surprising, but— I'm not surprised. Just imagine, right? It's— here's the best part. It's free, right? So anyone in the world can upload their likeness. And not only does it take like their face, it also synthesizes their voice. Yay, right? Like, all right. So this is where my family typically sits. We have a term in our household called nerve-sighted. That deepfake of me was trained by an LLM, and that LLM did get a few things right. But one of them specifically, I am the mom of 6 kiddos. And so I do like to start with just a quick recap of who I am before we get into what I do and why I'm here. This is who I am. 6 kiddos. Yeah, my husband's actually here with me. So hi, honey. And I'm really hoping everything's okay at home, but we're not going to ask. So let's just hope for the best. But the reason I am here is actually He was born, he was my firstborn child. He was born with a condition known as Down syndrome. And when he was born, I was a first-time mom, so I didn't know what I was getting myself into. But the doctors seemed to have, they were full of advice, as people often are. And they said, Noelle, this is a great opportunity for you. We actually, we understand what it's like to be a first-time mom and have a kid with Down syndrome. So, you know, special needs like this, we have a couple options. One, we have an institution. There's a federally funded institution that takes care of kids like this, babies like this. And I guess they must have seen the look on my face because they were like, or we also have families that will take these babies and care for them. I mean, starting your mom journey with a kid with this kind of special needs, that's It's a lot. And one thing was true about me then, and it's true about me now. I'm a technologist, and I actually got into the world of tech the first time the world was going to end. You remember? Y2K. That's right. That's right. And we saved the world that night. You're welcome, youngsters. So, and I'm just— if you don't know, spoiler alert, we are about to do it again. So, but that's how I got, you know, my chops. I started Building, I went to the University of Barnes Noble, taught myself everything from TCP/IP to HTML. Today in Barnes Noble, do you know this? There's only a shelf of tech books and they're not even that technical, which really, I'm like, how are people gonna learn? But I learned that way. I immediately went to work for IBM and VMware. I was really interested in infrastructure and scaling back then. I was more worried about utilization of hardware and then The cloud happened a little bit. So VMware got very excited about that, got excited about big data, ended up though at Amazon as a principal cloud architect. And this was in 2013. I get an email from Jeff Bezos and he's like, we're going to build the Star Trek computer and here's the only qualifications to join this team. We need people who are going to work hard, have fun, and make history. And I was like, That's me. So I jumped at the opportunity, and the main reason that was me is because of my dad there. I'm like, Dad, normal face, I'm going to put it on stage. And he's like, this is my normal face. So there you have it, Irving. He's a Marine, he could do whatever he wants. And he raised me on the golden age of science fiction— Asimov, Bradbury, Arthur C. Clarke. Crazy stuff if you have not read it. Considering what I ended up doing in the early days of Amazon Alexa. So I joined that team. One of the— I always like to tell, not every audience understands this, but I'm glad to be able to share this with you. One of the things that happened in the early days of Alexa, so many nonsense things happened, but one of them is that we launched and we did not figure out, well, we inaccurately thought everyone would buy one Alexa. That, and We thought you would put it in your kitchen, and we thought only one person in your house would be the main Alexa user. And immediately, that was not true. As a matter of fact, you can go back and look at videos of people— like, device disambiguation didn't launch until year 2, 1 million devices later. There are YouTube videos of people's Alexas battling it out. Not awesome. I mean, great now in hindsight, but at the time, A bit terrifying. Another thing we did at Alexa in order to find out how people were using it is that we monitored our capability, our failures. We monitored it on Reddit. I wouldn't use Reddit as your product feedback mechanism if I were you. I would encourage you to, I don't know, get that feedback ahead of time, create a team that likes you at least a little bit. But imagine we're like, You know, young data scientists, open weeping occurred in multiple meetings as we read the feedback people would give us. This is garbage. This is like our work. Anyways, very dramatic, very dramatic times. But all of that led me— actually, I spent 4 years at Amazon. I built 4 language models. This is all before Transformers, all before cool models that do some of this work for you. We were hand writing intense. If you want to know more about this, I'm actually doing a deep dive tomorrow. So I can't tell you all my fun stories about the disasters of the first 5 years of Alexa. But just know that if you think there's some nonsense in your work, you're not alone. So we even in productized 500 million home devices, we had lots of crazy things happening. Now I went from there, actually I was recruited in my 4th year at Amazon, recruited to go to Microsoft. Because they had a dream. They wanted to democratize artificial intelligence. And the biggest challenge they had was on figuring out who was using it, when they were using it. And this was very much good old-fashioned. I mean, here's a fun fact. When you use people who build AI, they are data scientists. They're not even really software engineers. So they don't understand really any of it. As a matter of fact, at Amazon and Microsoft, I had I had years. I'm the only engineer. I had years of automated failures. Like, I would build automated tests and they would fail for years because everyone was like, well, that's okay. It's fine. Like, so what if it failed? I'm like, well, if it fails now, it's going to fail in production. And they're like, production? I don't— And this is the world I walked into. So, at Microsoft, I became You might have heard this term, like herding cats. I was like the cat herder, and the cats I had to herd were researchers. I took on AI research projects, and my job was to productize them into real live things, and it was pretty challenging. So what I'm going to do for you is share with you a little bit about that journey. And of course, hopefully I'm going to try and stay as positive and optimistic as possible. through the entire thing. So here's the truth. Actually, Andy talked about it already. Fastest growing identity. They aren't people. I've been on stages and in rooms with the CISO at Microsoft, Anthropic, NVIDIA, OpenAI. Well, they have a different perspective on CISO, but anyway, I've been in these rooms and we've been talking about this for years, what is the future of identity? What's the future of access controls and management? Because these people are building systems and they're not inherently engineers, right? They're data scientists. And so there was a lot of gaps. But today we now know it's not only the fastest growing identity, the people creating those identities are not technologists. They are likely business users, business users who are spooling up little AI systems, not really realizing that they have anything like permissions or an access control list. I mean, they don't even know what those terms mean. This is reality like now. It's the reason one statistic I will not show you on screen, but I will share with you is 95% of AI pilots in 2023 25. 95, yes, are failing. And the reason is not because they're not awesome. They are pretty good. The reason is because the people building them have no idea what they're doing. They don't even consider it software. They're like, no, no, no, no, it's ChatGPT. Right? Like, it's not even in your wheelhouse, IT pros. Like, I don't even know. I'll talk to you about it later when I want to. And so this has become a huge problem. And so I actually do think we are at this moment where you will have an opportunity to save the world again. And for you younger people, you'll do it for the first time. Okay. So this is our— I mean, we— I just saw this presented earlier, so I was super excited to see this, right? Our inheritance chain. Human. You got that. We got that. I multifactored 16 times this morning. So thank you. So that is working and not for the same system, but definitely less than 3. So anyway, and then non-human. This is where we're moving very fast, building systems that are non-human that require their own credentials. But here is the fun part. We are also fast moving into the world where those systems build their own identities, that they are manufacturing identities. I defining privilege, like, by themselves. And, oh, right, when we do this, I lead a lot of executive boards and I lead a lot of executive leadership in building these systems safely and responsibly. Do you know who's not in the room? Are you in that room? Like, have I met any of you? I don't think I have. But I hope to. I hope to. Because the data that we're now seeing, the proof in the pudding is actually proof Proving that we need to have this conversation sooner rather than later. This is the number. As a matter of fact, Andy just showed 109. Just while we were sitting here, the number went up to 144. Yeah, I thought about that on my own. Oh, but one thing that is true is 97% of them have more privilege than they need, and that's just out of sheer oversight. As a matter of fact, sometimes I don't like to use the word ignorance in the presence of the people that are not being aware. But ignorance, right? The root of that is ignore. And that's kind of what they're doing. They're just ignoring the fact that this is software and that we already, as a company, know how to run software. And so what I'm going to share with you today, what is working in enterprises that are building the biggest AI models in the world, is not a net new thing. It is not brand new skills. It is just an admission that what we're doing for humans actually works. We just need to acknowledge the new workforce. So there's quite a bit, thousands. If you talk to Satya Nadella, CEO of Microsoft, or Jensen Huang over at NVIDIA, or Arvind at IBM, they will tell you it's billions because, of course, they win, by the way, in that scenario. But we're definitely going to start to see Thousands, and they don't spin up automatically. They spin up one at a time. So the interesting thing about these agents is they're not in the org chart, though some companies are testing this out. There is no background check. We are hiring these agents into our company to do things, and nobody knows what they're doing until they start doing it. So I have a couple things that I go around and tell people in order to help them think through the massive adoption strategy. So here we go. We've been onboarding agents like employees, auditing them like furniture, when we should be treating them like baby tigers. So I have a baby tiger here on my bestselling book, Scaling Responsible AI. Please go buy 1 or 8 copies. This, it's actually been out for a while, but you should definitely take a look at it. But this is also on the COVID this concept of a baby tiger. And here's why. Because when I go into an organization, everyone's like, my AI is awesome. It's so cool. This is amazing. Kind of like if you adopt a baby tiger, it's adorable. It's cute, cuddly. You just want to be like, I love it. I want to tell my mom. It's so awesome. And no one is asking really important questions or have invited the right people into the room who who will raise their hand and be like, have you thought about a couple of things? So here are the questions I typically ask everyone to think about when you adopt a new AI system into your organization, a new baby tiger. Now, when I was herding those cats at Microsoft, this was my job, right? They were researchers in this beautiful land of let's ask questions and be inquisitive about the world of the model we're building. And my job was to take them and shepherd them into Azure. Like Scott Guthrie, some of you might know, head of Azure. He said, take these models and productize them. That means these people who got into the world of, I love AI, I like researching, I like asking questions. I then had to invite them to learn a new world of, maybe you don't know, but things like deadlines and Reddit-like reviews, customer feedback. Most importantly, SLAs. That's my favorite conversation. Oh my gosh, I love you guys. You know my problems. I talk about this to executives and they're like, hey, hey. I'm like, you just don't know. So that's awesome. But now when you— now I go into these organizations, when it's this small, when you can get your arms around it, this little AI system, there are a few questions you need to ask. Number one, pause. How— I always look at a— have you ever seen a person walking a puppy and the puppy's paws are like gigantic and you're like, should I tell them that that's going to be a 200-pound dog? Like, should I mention it? Do you think they know? Well, that became my superpower during COVID and into 20— up till about 2023, is I was helping organizations figure out these AI models they were building, how big they would be. But that changes in the world of identity, and I'll share with you how. But that's an extremely important question. How big is it going to be? What is its blast radius? The second question, teeth. These babies are born with razor-sharp teeth at birth. What do these AI models eat? Data? Well, only the data you let it eat. But nobody's asking that question. They're just like, arm, arm, arm, arm, just eat everything. And then I will often tell these executives, I'm like, it could eat your soul. You have to be careful with this. But they're not thinking, so I'm always trying I'm trying to just give them an analogy that they can own and understand, right? So this baby tiger, now we know, see how big it's going to be. What's its blast radius? What's the worst that could happen? What is it going to eat? How is it going to eat it? What are the different systems and data that it's going to need access to? We should know that before we build it. Anyway, it's not the conversation I typically get to have, but that's the idea. And then Finally, who's going to take care of this? What zookeeper in their right mind is going to adopt these baby tigers? I don't want to say it's you, but it might be you one day. So somebody has to take care of these. Someone has to understand their guardrails, the moat that you put them in. How do you keep them happy and doing all the things, but also safe, not only for themselves, but for everyone? These are very simple questions. And of course, I didn't want you to go without being able to like take a picture of all of them.

I did.

translate them a little bit, right? Like, how big is the blast radius? What is it going to eat? What does it need authentication rights to? People do not think about this at all at scale. It is a bit of a mess right now. So if you are not sleeping well at night because you were worried that people might not be doing this, yay, because now you know they're definitely not doing this. Okay, so who's going to take care of it? Every app needs an owner. And where will it live? What is the identity, the domain, the network? All things that we would do for a wonderful human. Speaking of, we started launching— I worked on OpenAI when they launched the first iteration of ChatGPT. So during COVID I was massively testing this platform. At the same time, because of my relationship with Microsoft, they were investing $13 billion in OpenAI and they were launching their own ChatGPT-like thing, but specific to code. So this was Codex back in the day. Now, I love that Codex is getting its own little moment, but back then nobody wanted it. So I was like, I'll take it. I'll take the coding one because it was broken at first. But when we started launching our go-to-market strategy to get everyone to start thinking about how they're going to leverage these systems, we started using words like Coworker, teammate, copilot, even friend, therapist. Anyway, so these terms are like the exact opposite of what these systems are in your organization. It's nice if you can get your users to use the system because they think it's their teammate. Great. But from our perspective, it is actually more like a zero-trust contractor. So, let that just sink in. It is a zero-trust contractor, right? Least privilege, full observability, right? Expiring credentials by the second. We're talking about, right, continuous identity? Yes, 100%, but with the least privilege possible. Here's the cool thing. This is 100% possible. It's just reliant on a human to own the problem. You being willing to stand up in that meeting, volunteer to join a meeting you definitely don't want to be in, and say, this is what we have to do. So trust nothing, verify everything. I'm sure you've probably heard the trust but verify. Not in this house, right? Not with AI agents. You're going to trust nothing. No agent gets presumed trust. As a matter of fact, I built a framework that hopefully you'll be able to relate to, and it's based on Microsoft. It has this concept called V-. So, when you're working with Microsoft and you're in a Microsoft Teams, you can tell who's who because the vendors are tagged with the domain V-Microsoft. So, automatically, immediately, you know who these people are. So, and this is not new. This is decades old. Vendors are identified differently, scoped differently, permissioned differently, of course. Because they're vendors. Not bad, right? They do work for us, but they are not us. And therefore, we have to change how we represent them in the network. Makes sense. Well, now we are seeing that propagate into— and I've been talking about this for years, but finally, it showed up this year in the agentic platform, Agent 365. And so Microsoft created this concept of identifying these contractors, these AI systems in your architecture. But I want to take that analogy just a little bit further because there is this concept of contract to perm. You probably have heard of this before, right? You get hired as a contractor. They kind of see if you're like legit and are not going to steal the stapler. And then maybe you get promoted, right? We give you an opportunity. We evaluate you. And then we promote you. That's the ideal solution. I loved contract to perm because I could also choose if I wanted to work there based on the behavior and nonsense of my leadership. So I loved this model. However, in the world of AI, we have what's called— I mentioned it— responsible AI. But when I say that, weird things happen to people, like their eyes gloss over and they start looking around or they pick up their phone. So I don't say that anymore. Now, I go in and I'm like, look, the only thing I care about is that you build systems that are safe and secure. And in that safety and security represents the human element, the oversight. Some of your systems are going to have a human in the loop. That means certain systems, I don't know, NASA, JPL, clean rooms, you know, aviation. There are certain systems where a human must be involved. All the time. And maybe sometimes we get trustworthy with that and we move them from me needing to be in the loop, meaning I am the gate to making a new decision. I could fast forward and be like, okay, not just the gate. I don't want to be the gate anymore. Instead, I would like to just watch. I'm just going to watch you. And so human on the loop means I am still there. I am present, but now it's a Teams channel. And I get notified, hey, this agent made this decision, this agent made this decision, this agent made this decision. The final one is human out of the loop. This is— have you heard of YOLO mode?

Yeah.

Yeah? Oh, okay. So some of you are riding the edge of danger. So YOLO mode is when you tell your AI system, right? How annoying is it when it's like, would you like me to give access? Can you approve this? Will you approve that? Eventually people are like, I approve everything.

Right?

How fun is that? So that is YOLO mode. That is human out of the loop. But here's the good news. There are use cases that this works for. For example, the little robot thing that we sold a lot of, that cleans your house, right? The little iRobot. Now there's lots of them. They roll around. It is possible to train them, by the way, because I know some of you are like, that's garbage. It's sitting in the closet. Like, pull it out and follow the instructions. But as a data scientist, don't blame the machine. But that's like such a deep thread in this whole thing. But the human needs to do its work. But when it rides around, right, the first few times you should be in the loop. You should be like, yeah, don't eat that sock. Oh, I need to be better about not leaving stuff like that. Or, oh, you missed that corner and you train it on going to the corner. Eventually, my goal is to not follow that thing around. My goal is to give it enough data and information that it can do that job without my participation. And honestly, my goal is human out of the loop. I want it to not only clean my floor, but put it in a bin and take it to the curb, like ideally, right? But that's not the world I live in yet, although you can buy one, right? Have you heard of the Neo robot? Yeah, home robots are available for sale right now, $20,000. Right? Sign me— sign you up. You might want one of those. But there are use cases where you don't want to be in the loop, where autonomy is okay. But it definitely is a human decision. You decide on every single agent, thousands of agents, thousands of times per day. There has to be your choice. Am I in the loop? Am I on the loop? Or am I out of the loop? And as soon as you say you're out of the loop, That's a huge amount of responsibility, and honestly, I believe you are shedding that responsibility again unless you did all the work to make sure it's not going to miss the dirt in the corner and eat socks. So there is no perm, contract to perm, when it comes to agents. And I say that because there is no situation where you're like, welcome to the team, you're an employee now. Every agent will always need oversight, and that oversight will have to be defined and owned by someone. So Microsoft created the Entra Agent ID. This is just one example. Lots of platforms have it. I'm an MVP for Microsoft in AI. I was one of the first to get the award 5 years ago during COVID So I like— they treat me nice, so I mention them. But there are lots of options here. But look at these. First-class identity, platform agnostic. But the most important thing they did with this agent ID is they required a sponsor. A human has to be associated. And Jeff Bezos was a big believer in this too. Single-threaded ownership. Not that this is a term I like to say, but one throat to choke. That's terrible. Anyway, have you ever been that person, right, where the person's like tapping their foot and they're like, you're losing us $1 million? Just me then. Okay. But I would, We had production failures and it was very traumatizing because my sponsor was over me being like, fix it. But you need a sponsor. You need a single person who's going to own the outcome, especially as we begin to regulate this industry in the United States. Don't know if you know, but we got a new regulation. Actually, it's an old one that got killed, but it's back, um, that will help us actually build regulatory compliance around these AI systems, it's just starting now, which means we have about 9 months, like a baby, to get our act together and be prepared. So because we have this concept, you might be like, oh my gosh, this seems like a lot of work. I've done this tens of thousands of times. So I built a checklist for you. These are the steps. I also provided a couple of examples so you can see what it looks like. These are the most important things. Know the agents in your architecture. It is possible. It's metadata. You can do that. You could do it today. Governed identities, sponsor, expiration dates. Oh, gold. Scope out what they can do. Least privilege. It's an architecture principle. It is optional today. I will hope that we force that. Stage in, on, out is not an option until you've done on the loop so many times that you can bless that. And then of course, watch. Watch everything, log everything, because one day you're going to look left and look right and find out one of you's being sued or you're suing someone, and you're going to hope that you attract everything that those agents do. By the way, that's happening now. So you're going to find them, badge them, scope them, stage them, watch them.

What?

Like, that's beautiful. Like, let's go, let's do it. But it is a step-by-step process, and it is 100% yours to do. So you get to decide. You— and I literally mean you all in your profession, in your role, or in a role you will elevate to over the next 3 to 5 years. The federal government, fun fact, created a requirement that every single federal agent get a chief AI officer. Every chief AI officer that got that job had no idea what that meant. So you're kind of in that spot now. You are going to manage an agentic workforce before you're ready, but that does, like, doesn't keep you from doing it. It didn't keep those people from doing it. It just means you have a very cool opportunity to work hard, to have fun, and to make history. And I, when I was at Alexa, I jumped at the opportunity. I hope you will too.

Thank you.

So in the last couple minutes, I have actually someone in like one of these 3 sections. If you look at the back of your chair, you have a baby tiger attached to it. So I'm gonna say it again in case you're like sleeping and on your phone playing solitaire. So like look behind and see if there's like a baby tiger. Anyone see one? Oh yeah, cool. Help him out. Help him out. All right, awesome. See, 'cause he wasn't— yeah, cool. Awesome. So because we are talking about robot armies, I brought a robot for you. You can come and get it later, but it's an iLick. It's a little robot pet. So congratulations, may you live long and prosper.

Thank you.

Okay, awesome. And finally, I'm gonna bring out Andy. Andy and I are gonna take a selfie because he did this whole thing where you guys got really excited, and I'm like, we gotta record that. So we're gonna record it so he can have forever this capture of your excitement that he did this. 16 years? Holy Christmas, that's crazy. So, are you ready for this?

Are y'all ready?

Do you need to practice? All right.

Yes, they need to practice.

All right, he's gonna guide you through the practice.

Right, remember how this goes, yeah? How excited are we? Okay, do it again but better.

All right, and we're— y'all gotta keep it going. All right, you ready? We're gonna start on this side.

Go!

Thank you! Thanks, everybody.