From AGENTS.md to Enterprise Deployment

Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Bluesky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

Welcome to another episode of the Practical AI Podcast. This is Daniel Wytanak. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer. How you doing, Chris?

Hey, doing great today. How's it going?

It's, it's going great. And I, and I know this is going to be a great conversation for a few different reasons today. One of those being our guest is one of the speakers at the upcoming Midwest AI Summit, which is going to be amazing. There's going to be a bunch of amazing speakers there. I'm going to do a bit of emceeing, see if I don't mess that up. Would encourage our listeners to check that out, October 15th in Indianapolis. You can get 20% off with PracticalAI20, so check that out. So amazing speaker going to be at the Midwest AI Summit, also a fellow podcaster. which is, is great. Um, one of the hosts of Cloud Foundry Weekly and, uh, a tech marketing, uh, whiz at Broadcom, Tanzu, VMware. So welcome, Nick. Great to have you.

Uh, thanks for having me. Great to be on the show. Yeah.

Excited.

Also excited to see y'all in person in Indianapolis.

Yeah, it's going to be fun. We're representing the good, good representation of the Silicon Prairie here. I like it. So yeah, Nick, I know you're going to be talking— one of the things I thought was cool about your talk at the Midwest AI Summit is you bring in this concept of taking agents.md and shipping that to a production environment where you run an actual agent in an enterprise environment, which seems to be Really fascinating. We recently had on someone from the Agentic AI Foundation who is kind of stewarding the Model Context Protocol and agents.md and those sorts of things. So very relevant kind of carry-on from that conversation. But I'm wondering, as we set that up, what exactly does it mean to be operating in an enterprise environment? So the customers that you're working with, that your kind of day-to-day.

Yeah.

Mm-hmm. What makes an enterprise environment an enterprise environment? What are some of the concerns or characteristics of that type of environment? And I know you're always experimenting. You do a lot of home lab stuff as well. So always have a whole range of experience in working in different sorts of environments. So yeah, if you could just help us understand from a general perspective, what does that mean? Um, when you say kind of enterprise, right?

So yeah, if we think about the enterprise, we think about, you know, a few things. Uh, one, let's just assume you're going to have not internet, uh, limited or no internet access. And, you know, I've been, uh, at VMware Tanzu for about 5 years. Uh, and before that, I was at one of these large enterprises for about 14 years. So when we had vendors come in, it would always be like, they would always assume that we could just go to the internet. And it was always like, well, try again, because that's not going to work here. So that's kind of always like the biggest hurdle where you're going to be regulated and/or, you know, you're not going to— what you can do at home is not going to be what you can do at work, so to speak. So you're going to be highly regulated. You're going to have to deal with, you know, potential like, you know, PCI or SOX compliance or HIPAA or FIPS or, you know, and I could probably go on and on about all the different compliance tiers that you'd have to deal with. So it's just a whole different ballgame where, you know, you could potentially do something that has dramatic consequences. And you have a, you know, highly controlled, highly regulated environment where even some of the customers I work with, they're like, you know, they actually have the real air gap where we're carrying in things, you know, physically into the data center because there is no internet access type of thing. So it's a wild world, but, and it frustrates a lot of, I think, new folks to the enterprise space. But if you've been in it a while, you know, you know, you know how to deal with it and how to handle it type of thing.

And I guess, you know, people have been dealing with these types of environments for quite a while, right? Because there've been enterprises for quite a while. Maybe just give us a sense of like leading up to AI agents, and we'll talk about agents here in a second, but leading up to AI agents, like what were some of the ways that you could I guess, uh, ease, you know, put some ointment on those, those pains of working in that sort of environment. How do you, how would you get an application into that sort of environment or, or have it be enterprise ready, quote unquote? And then we can shift and talk about, you know, what changes with, with agents.

Yeah.

So, I mean, I think so, you know, I currently work for VMware Tanzu and we run a, uh, you know, a platform or a platform as a service. You know, geared for private cloud, whether that be on your, like, on your own bare metal in your data center, or you could deploy it on a private cloud in your own, like, VPC on a hyperscaler, that type of thing. But when we think about that, you know, we want to, in the scope of an enterprise, right, you have all these regulations, controls, et cetera. You want to make that path to production kind of the least resistant and easiest path. And if you make that path to production Like all the checkboxes and everything good to go, that's going to be a highly adopted path. So the whole concept of, you know, Tanzu Platform, and it's actually based on an open source project called Cloud Foundry, which is, I think, I mean, it's, you know, predates Kubernetes and Docker. So, you know, 2011-ish is when it kind of started originally out of VMware and spun out and then became part of an open source project. So it's, it's, it's, it's been around and battle tested and seen a lot of things. where you have that concept of like, I just wanna take my code and send it to the platform, and the platform's gonna know like the best practices for, it's gonna build a container for the developer, so the developer doesn't have to worry about like, you know, securing the container or anything like that. That platform will take care of that. It will handle, you know, ingress, like certs, certificates, all the, you know, load balancers, all of those things just handled, health monitoring, potentially even kind of like sandboxing the apps from each other. Something that, you know, maybe some of these labs may learn from, but— So that the apps can't escape type of thing. And, you know, if it, if the app needs services like a database or messaging middleware, any type of thing, even maybe a large language model, it can what we call bind or kind of connect to a service on the fly. And that's all kind of handled under other underneath the covers. So it's just a few commands like you push, bind, scale your app with just a few simple constructs. And that's how you go to production. And what we see with enterprises, once that path, like once they take this, like this is Tanzu Platform, it's been certified through all these different standards, it's just kind of like an unlock for these enterprises because it's so easy and so simple to use and they don't have to, you know, they don't have to recertify everything. It's just like this framework that they can use to get apps into production. And then the biggest thing is that you want these, the enterprises or the businesses, they don't want their developers, you know, handcrafting a new platform or a new way to deploy apps for every subteam and have like 100 different snowflakes, right? They want the same repeatable pattern across the board so that they can audit it and secure it and have their developers actually spend time writing the business logic versus like fiddling with infrastructure. So that's kind of like the whole premise behind, you know, the whole platform as a service construct and, you know, Tanzu Platform. And as you can imagine—

Oh, go ahead. Yeah. No, no, go ahead, finish up and then I'll follow up. All good.

Okay. As you could imagine, those principles when we're talking about apps could probably play nicely into this agent world that we're living in.

So it sounds like you're drawing a lot of commonality between kind of traditional app development and all of the structure that we're all used to, that we've been doing forever. And kind of now we're into this agentic world. And maybe people have been thinking about that in a different way, but it sounds like you've really kind of said, and correct me if I'm wrong, I'm going to throw something out there, like it's kind of the same thing in the sense of there are differences, but agents and apps should sort of be treated the same way. Is that, am I getting that correctly in terms of how you're structuring that and that maybe people need to look at what they've been doing for years and make that work for agents in the same way. Is that, is that fair?

Um, yeah, I mean, I think it's a, it's a pretty fair assessment. And, uh, we're starting to see that more and more just within our customer base. And, uh, you know, even like if you think about like apps back in the day, like someone would write some app on their laptop and like run it on their laptop, but it's like, we can't just run it, you know, like you gotta get it off your laptop somewhere. And the same thing with agents. You're like, I got this, I've got all these agents running on my laptop. If I shut my lid, everything stops. Like, I, I don't really want that to happen, right? I want them to run 24/7, you know, 365. And do all the work I give them. So, um, yeah, I think that's kind of precisely what we're going with.

So, so I guess then it begs the question, why, why are people kind of throwing out what maybe their intuition, uh, like in certain ways people are saying, oh, we have agents now, we need to do something totally different, which is maybe driven by some things that make an agent not an app, not just a sort of simple application, but also it doesn't necessarily mean, you know, obviously we have to throw out everything we've learned about platform engineering and, and, and how to deploy things. So from your, from your mindset, what, where does the kind of agent equals just another app? Where, where might that fall apart? Or what are the kind of unique characteristics that you're dealing with on the agentic side?

Yeah.

On the, on the deployment pathway that you might not have had to consider before, or at least are considering in a new way?

Well, I mean, I think some of the biggest differences, right, where like you talk about Cloud Foundry and Tanzu Platform, it was kind of based on this premise of a 12-factor application where like storage and state are a little, you know, decoupled cleanly from the actual apps. You could like, you know, I can scale up to 1,000 instances and be fine and you handle session state and things. The traditional harnesses or agents were kind of built originally just to be like, ah, I've got file system access and I can just write a bunch of MD files and that's like my memory and everything's great. And you're like, well, you start to get into a cloud world that things spin up and down and are ephemeral, you're gonna wanna save those MD files somewhere. They're gonna go off into the ether. So that's probably the, I think the biggest challenge when you start talking about how do we take the app deployment methodology, right? And then push that into like an agent running on a kind of a cloud platform per se. It's probably the biggest challenge. And then just kind of getting the fact that There's some valid use cases for like your laptop or whatever, but then also as we're starting to see more and more, like enterprise was, enterprises want agents to be on demand and like usable kind of in a, like say in a CI/CD pipeline or like on, you know, within part of their, you know, e-commerce suite or just normal applications. And you want those to be kind of really close to the apps, right? Because if you're, if you have them way off, you know, even just from a pure networking perspective, if you have the agent or the LLM or whatever, that's like 30 hops away from the microservice that's trying to call it, there's physics that are going to add on to all of that latency and potentially at scale even be problematic. So they kind of want to co-locate the intelligence with the app and the data as close as possible. That's what we've seen too. That's something that I didn't expect really, because some of these responses are kind of long, but it's like what we've seen with our customers at scale, We want to get the LLMs and the agents closer to us and our actual apps using them and the people using them as well. And not to mention, if you're using some of these SaaS providers, they're maybe not the most reliable in terms of what traditional enterprises expect and demand. Um, so you can't just have them, you know, going offline for hours. Uh, 'cause, you know, even, even at my past job, the first, the very first thing I did there was work on a warehouse inventory system. And let's just say it was for a large retailer that, uh, It was a grocery retailer in the US and I was an intern out of college and like the app was older than me at the time, still older than me. And, you know, it was just, it was a bunch of like, you know, C and shell scripts on Unix systems. And if that thing went down, like within 5 minutes, like the warehouse was backing up trucks, like on the interstate. So it's like, you have to like the scale of enterprise versus just, you know, You know, like consumer is a completely different scale. So that's why some of all the things we're trying to make them a little bit more enterprise grade and more used to what these enterprises expect from software.

Some mornings as I'm drinking my morning coffee, I listen to live news updates to figure out what's going on in the world. And it seems like recently everyone is about AI kill switches. But the reality is that there is no one AI kill switch. We've moved from models to agents. Those agents are connected to multiple models. They have an agent harness. They're connected to MCP servers and tools. And those agents are acting within a fleet, delegating to one another and interacting with one another. In that environment, what you don't have is a single kill switch that solves all of your problems. But that doesn't mean that you can't exert your control. You just have to exert that control across various layers of that stack through, of course, component input/output safeguards, but also controls on agents tied to their agent identity, things that track tool misuse or memory poisoning or goal drift, etc. And what we're providing at Prediction Guard, the company that I lead personally, is an AI control plane that you self-host in your own infrastructure that gives you that control plane or control layer for the relevant stack that agents operate on top of. I'd encourage you to check out what we're doing at predictionguard.com/practicalai. Go ahead and book a call with myself and the team to learn more about how you can gain control of the agents operating in your environment and exert zero trust principles in the way that you operate those agents. So check us out at predictionguard.com/practicalai. That's predictionguard.com/practicalai.

So Nick, I would love to dive back. I want to kind of build on what you've, what you've been sharing so far. And, and part of that is that we're all diving into this multi-agentic world, um, where, you know, you know, some, you know, late last year, you know, you'd have an agent and, and we, and, and then, you know, for most people, I think getting into multi-agents, it was probably developers and a develop, you know, in, in the environments that, uh, that the, uh, the AI providers were, were making available and you had multiple agents working on a project. But now we're really seeing, you know, with, with these work products and other things that have come out, we're seeing agents really multiplying across, you know, many different contexts and with a lot of different utility. And, and it's moved out of being just frontier services, and we're now doing that with our hosted agents as well that we're running. And so like this multi-agentic world has just taken off in a very short amount of time. And as you're— I guess I'm asking if you can kind of paint a picture as you start scaling this out in the enterprise and what that looks like. Like, could you describe, you know, and we have analogies, as you've pointed out, with kind of the traditional software development world and the containers and all the, you know, and how you're structuring deployment and stuff. Could you talk a little bit about what that looks like? Kind of paint a picture so that somebody can really get that in their head. Me, first of all, about what that multi-agent world looks like when you were doing all these right things that you need to do in the enterprise.

Yeah.

So, I mean, I think that the first thing that we want to focus on is that the enterprise, you know, needs to get whatever technology approved throughout their system. Uh, and then we, so we are providing a harness, uh, called the, uh, Tanzu Agent Buildpack, and that helps through that process. So our existing customers already have our customer, you know, we're already approved so that we can help provide them those tools. And we just make that process just another app deployment on the, on the platform. So, uh, I probably haven't covered this yet, but I'll explain what a buildpack is, right? So when you, when you—

Yeah, I was about to, I was about to ask you that if you could kind of dive into what the specifics of each of those things does.

Sure. When I would do demos before AI and before, you know, a year or two ago or whatever, we would always take like a Java app and it was like a simple kind of like, you know, music album called Spring Music or whatever. But we take that Java code and be like a JAR file, like compiled Java code. And we're like, all right, now we're going to take this. We have a little simple manifest file that tells it what the, what the name of the app is, how many, how much resources to give it. And we just run a command that called CF push. And that would then just take that, that code, the JAR file and upload it to platform. And the platform would be like, oh, it's, it's a Java app. So I'm going to use the Java buildpack, and the buildpack is basically a kind of a command set to make a best practice container based off that use case. So the Java buildpack knows to do all the JVM memory calculators, do the JDK and certificates and everything that's best practice to run Java in a container on a cloud platform. And that would be great. And then we would attach a database and all these things and show how that scales. Now, we've taken that same concept and then said, hey, we're going to have this agent where we can just CF push an agent. And instead of like, you know, machine-readable code, we have human-readable language, uh, the AgentsMD. So basically you tell what the agent it's going to do, and then you just push it to the platform and it starts up within a minute type of thing. And you can scale it up and push as many agents as you want. And I think the coolest thing that I've seen is that once we're on the platform, then we can run models on the platform. So, you know, obviously a lot of our customers are air-gapped and some of them were, uh, were smart enough to buy GPUs a few years ago. So some of them have lots of GPUs that can run all these local models and have a great experience. Some are wishing that they bought GPUs and hardware as the pricing of all this hardware goes through the roof. But then they can run local models and then tie the agent to that. Or if they don't have, they might have a contract with one of their cloud providers or one of their approved LLMs of choice, and we can register that. But then that kind of gave us a simple, like, here's your out-of-box agent with chat experience. And then, you know, along the way, like, um, we've done a lot of things with MCP. So, you know, the Model Context Protocol, I probably, you guys covered that on the show before, but it's, you know, basically a way for an app to talk to an LLM, like very, very high level. Right. So we, that's become, uh, you know, useful as well as we then give the agent tools via MCP servers. And even at Broadcom, like the only way approved way to use for our developers is to deploy MCP servers. On, on Tanzu Platform so that we have a whole set of MCP servers that have been approved and hosted and they're kind of secured and maintained because they're very similar to applications as well. And then we have a concept of like the MCP gateway service that kind of registers that and secures all those servers. So then we can then extend the gateway to that agent. So then the gateway, those agent now has like secure access to a runtime, you know, battle-tested runtime. It can talk to an LLM that's approved. and get tools via Gateway. And then we're starting to add in like the memory service as well to have a kind of a shared memory service. So we're kind of expanding all of that out where we're seeing all those best practice app lifecycle, go to, you know, see a push, go, here's an agent, here's all the things it wants to do. And then, you know, it can be a security review agent. And like one of the things I showed, I was at a, I think a week or two ago, I can't quite remember. I was in Las Vegas for a conference, VMware Explore, and I had like a 5-phase demo where I walked through this. And maybe I'll show a little bit of this at the Midwest Summit here, but push the agent, bind all these services. But the end goal was like the boss level was like, all right, now we have these agents sitting in there and I can hook them up to a repo with like an event listener or webhook and say, all right, now do issue 5 on my repo, do a security review on this repository. And then it sends it off to the agent. via webhook and like, oh, the Tanzu agent, like it just fires up, gets it, it fires up from the platform, it gets a request and it just immediately pulls down the repo and starts doing a security review and then publishes or pushes up to the issue that I created on GitHub and says, this is a security review. So you could see how that flow would be really useful in enterprise where like maybe on every Git commit, you just have all these agents spinning up on the fly. That could be a security review agent, potentially like a style or code quality review agent, or potentially even like a— I don't know, like maybe a design review agent, all firing up on the fly as things like call. So it's kind of like, and then they all have their, they're all in their own little sandbox and everything's proper, right? Because part of, you know, the platform for a while, it's been like this very secured container runtime where like we have the, you know, they can't talk to each other unless you enable them. So they can't go to the internet unless that's enabled. They can't, you know, escalate otherwise. They only get access to what you give them. So it's kind of that benefit as well. And then obviously they're containerized, so they don't have access to the underlying host system as well. So that, you know, as we've seen, you know, some of these things of uncontrolled agent swarms, I think we have some remedies here that some of these, some of these AI labs could maybe take into account and implement some best practices that's learned throughout the past 2 decades.

The unplanned swarms, the swarms that you don't necessarily want. I actually have a 2-second follow-up. And that is, it's for my benefit. And, you know, we've talked a lot on the show over various episodes about agents, you know, using MCP to get access to different apps and services and tools and stuff like that out there. We haven't really talked about what an MCP gateway is. And I was wondering if you could share that for a moment, just as a detail, because that's kind of the one little new piece of architecture that we've never touched on on the show.

Sure.

Yeah.

So if you think about an MCP gateway as a way to easily control and scale access to MCP servers, especially within, you're in a cloud, I guess, platform. So we may have like 30 or 40 MCP servers that we run, or I don't even know how many it is, right? And then those get bound, those get registered and bound to certain specific MCP gateways. So maybe there's like a, you know, and the MCP gateway is something we can spin up on the fly on the platform. There are many different MCP gateways out there. There's many different AI gateways out there. Some of them are like the same, one and the same. But you basically register MCP servers to this gateway and the gateway kind of controls access, so to speak, and regulates that. So it may say like, you can bind to this gateway and you only get these MCP servers. You only get these MCP tools within it, that type of thing. So it's a way to easily abstract all the different MCP servers that may be in an organization and kind of direct them into a certain, either a certain agent, or we even have a flow where like you can We'll take those MCP gateways that we run that front all of our MCP servers. And then you can say, hey, go give me a way to connect this to my Cursor or my Claude Code or my whatever. And then it just injects it in there. And then your local agent has all the tools that are approved as well for your organization. And then it's just like a simple click and then it just registers. And then like, depending on the MCP server, it helps handle the auth as well. Because some of them are like, well, if it's my If I'm using the GitHub MCP server, I want to pass my end user credentials all the way through to that MCP server because I don't want, you know, I don't want it just to be like, well, here's our generic service account for the entire organization. And every GitHub action is going to be seen by that one account. And you have to, you have to have the individual identity pass through. So you have like the concept of like SSO and sign-in as well for an identity for agents or humans running their agents on their desktop. So that's kind of A quick level view. And then a great way about that too is because it's all flowing through that gateway, we get metrics out of it. So we can see all the tool calls and all the events happening, coming from the agents to the MCP servers and back and forth. So you kind of see what's happening, especially if maybe if there's something unexpected, you're like, well, this one agent made 200,000 tool calls to delete repo. Like, oh, maybe we should alert on that. Like, that's probably not good.

Yeah.

So, and you know, another visibility or metric to, to see what agents are doing and what tool calls are doing as well. So I guess hopefully that made sense because that was, that's kind of how I understand it. Right.

That, yeah, that was great. And there were a couple of things that you mentioned in there I think are additionally worth digging into. And you, you kind of have this, if I'm understanding right, in part of the buildpack that you're working on, part of that centers around kind of pushing an agents.md file. You mentioned like people have these MD files hanging around. You mentioned memory, you mentioned identity. I'm wondering if you could kind of help us understand, like, if I have, if I'm working on my laptop, right, and I'm using a certain agent harness, I may have an MD file that is kind of generic, and then I have files that are created by the agent that it uses that are only corresponding to my single agent. And then I have, you know, memory and then, you know, calls into other things. As you're pushing this like off the laptop now, and I imagine taking that off, like what belongs in like an agents.md file? And what is, uh, like as you were giving that example, right? You may have one agent in the platform that is a security reviewer for. Um, uh, that does security reviews, right? When you spin that agent up, does each instance of that, each session have a unique identity? What is the, what's unique about that session and the data that, that is generated by that session versus maybe the things that are always static? Like, I'm not sure if the agents.md is, is always static. So could you help us understand like, which are the static elements? What's developing over time? How is that?

Mm-hmm.

like state of the agent spread across, uh, you know, the implementation, I guess.

Sure. Yeah. So if you think about it, like in, I guess in, in, in this specific implementation, the, the agent's buildpack is basically like the, the boilerplate code of what you want that agent or app to do. So like I'll do demos and I'll have it be like, you're a pirate. Like, like, like one of my demos is like, you're a, you're a, you're a, uh, uh, uh, Jira, like you're a senior software engineer. You know, you're gonna watch this Jira page, or I forgot the terminology here, but basically watch this Jira queue and look for any inbound tickets. As inbound tickets come in, review them and make sure that they're of good quality. And here's all these different things that you wanna do and don't do any of this. So it's more of a very simple AgentsMD where it's just telling this agent what it should do and how it should act. And then you And you can call that and like, all right, well, like, you know, ticket 500 came in, it was poorly documented and like update that Jira ticket and, you know, whatever. And then you can add fun things like talk like a pirate. So it's like, ah, matey, your, you know, ticket 500 is, uh, it's terrible, right? Like, uh, go back and, you know, update it, that type of thing. So that's kind of where that, the static part of it is. And then like the memory service we have, you can attach it, you can have, you know, you know, potentially memory of a certain project. So, you know, we kind of envision Like each set of teams to have their own multiple set of agents, right? And it could be like, well, you know, Team A's agents have the access to Team A's memory. So as they come up, they know like all of this historical data of like what, like the architecture of the application, the, you know, what's been done before, you know, like the roadmap, that type of thing. So as it spins up and does like a security review, it can make, you know, proper decisions and proper guidance so that, you know, As you kind of build your local agents and kind of give them memory and they become smarter, that memory service, as you have agents spin up and down, they just immediately know what's happened before, know how to proceed.

As you heard at the beginning of this episode, some amazing people like Nick, who's the guest on this episode, are joining us at the Midwest AI Summit, October 15th in Indianapolis, Indiana. Nick is going to come talk about what it looks like to deploy AI agents in real-world environments, and we've got a bunch of other amazing speakers talking about how AI-enabled teams work, how to make decisions about what to say yes to and what to say no to in relation to AI, and much more. Super practical and amazing talks. Plus, there's going to be an AI engineering lounge where you can actually sit down with folks like myself. I'll be there. Get, get review on your AI architecture, your stack, your roadmap, etc., towards the rollout of things like agents or other AI technology. This is a practical, amazing event that you don't want to miss. There'll be great Great food, great people, great networking, and practical value. Do not miss this. It's October 15th in Indianapolis, Indiana, and you can use the code PracticalAI20 to get 20% off registration. So go to MidwestAISummit.com and use the code PracticalAI20 to get 20% off registration.

Nick, before we were going into break, I think I may have cut you off when you were just starting to say something. You want to dive back into that real quick?

No, I think I covered it. Hopefully I answered the question again.

That's no problem.

Okay.

So really, I really appreciate, like, this has been really good for helping me trying to conceptualize how all this fits together. I think I had some understanding, but you've gotten a level of detail. I wanna ask you, as we are looking forward, I have several questions that are really kind of driving my thinking. You've got me thinking creatively based on what you've said.

Okay.

So, as you look at, you know, we've talked about MCP, there are new things coming out, things like, you know, there's the A2A, which is the agent-to-agent protocol, and organizations are starting to look at that, and they already have, their MCP servers maybe, but they're trying to get the structure and kind of bring the sanity around it that you've been talking about, you know, that, that then they kind of go, oh, okay, it's not so different from what we already know. But as, as this is evolving very fast and you have things like these new protocols, do you have any guidance on how to start integrating this constant flow of change in, um, in terms of first on the technology and I'll, I'll hit the other half of that. afterwards been a, you know, as you're trying to bring things in and make it work? And have you guys, have you guys kind of thought about how to structure that?

Yeah, I think it's, I mean, I guess we'll just say take baby steps from, from everything, right? Because especially when you talk about the enterprise scope, um, they can be a lot slower to adopt some of these newer technologies. Uh, so you kind of layer that on and like, you don't have to necessarily adopt all of these things at once. You can maybe just get a simple agent running. And, you know, for most, most places, you know, just getting access to an LLM is still the biggest hurdle, right? A secured, you know, approved. So, and then potentially, you know, going through the, going through the AI council or their committee, right? Like everything must run through the AI council and be approved. If you know, like most, you know, if you're not, if you've never worked in an enterprise, most large enterprises have a council or committee for everything, right?

Yeah, I do.

So I know exactly what you're talking about. So it's like, well, we shall go to the committee and we shall present our idea and we'll wait 6 months and see what has the result been, right? So you just kind of have to work through the councils and the committees and all that fun. But once you finally get something approved, just iterate and you're like, don't try to boil the ocean, make baby steps, you know, and get progress. And then as you'll like, and you'll learn the technologies and then the next step will just kind of be obvious. Like, well, I can make the agent better if I get some MCP servers, get them tools.

All right.

And then we've got to get these tools approved and then add on to that. And then, you know, you know, and some of these guardrails help, you know, of Tanzu Platform help enterprises bring these tools to light because it's not just, you know, all this random stuff downloaded from the internet. Right. Which that can be its own danger nowadays. So having the, you know, having some, some guardrails in place is a good thing. If you've had, you know, if you have experience with some of these platforms or apps running before, take that experience and that knowledge and kind of bring it forward and utilize your existing people and skills, I think, too. I think a lot of times too, I've seen organizations like a new technology will come up and be like a new team will be built and that new team will decree, but there's so much other like really skilled people that could help. And it's always kind of a friction too. So how to try to get the teams to work together too and not just have the new team doing all the new things and The teams that have been there all the while have that, like, that's always another constant flow. So try to break those silos down as well to help, you know, get people up to speed and get through that process.

Having worked in multiple enterprises myself, I think, you know, that last point, I think I've seen that multiple times across different organizations. And, you know, how do you bridge that? I think part of it, you know, you have, you kind of have Every organization has its internal politics. They all have kind of the different structures. It's really common to say new tech, new team there, but then I think people recognize, you know, why? You know, that kind of, you'll see that and sometimes you'll see a reconciliation, but it always seems like a little bit of a struggle when that happens to try to, you know, do you have any thoughts around as you're trying to do that, you know, you may have— A lot of big organizations have, they're very like one business unit, maybe almost like in a different universe from another business unit sometimes.

Mm-hmm.

And so, how do you get that kind of cross-adoption where you're taking advantage of maybe both the new teams that may be there, but also some of the old structured teams, and you're trying to spread the capability across the larger organization in a useful way. Do you have any guidance? You know, I imagine it can't be that different from, you know, the more traditional app dev side, but I think this is a struggle that a lot of orgs are facing right now.

Yeah. Well, I think, yeah. So, I mean, it goes, I think, a lot both ways. So, if you have the new team, right, the new team can, like, the responsibility of the new team is like, Engage every other team because they have a lot of people that can help you out and achieve your goal, right? Because the new team probably most likely has a very stressful and like, you know, like adopt AI everywhere, or before it was like go cloud everywhere, right? Like those goals are like not quite easy to get going in an enterprise so that the new team needs to like, you know, they potentially run, you know, office hours, weekly office hours, reach out to their other teams, You know, maybe before, you know, like a lunch and learn, if you're in the office and have people in the office, that type of thing. And just have like a forum for people to come in and learn and get started and like ask questions and not, you know, like, hey, like I saw this, you know, this post about, you know, this new platform or this new AI thing. How do I get started? Is there any documentation or what can I do to get started? And just kind of have like an open and like, you know, very welcoming place to get started, to bring in the folks. And then if you're on some of the, you know, let's say not new teams or teams that have been around for a while, be engaging and reach out to those teams because a lot of the new people are going to need help and like they, you know, they may be under stress. So it's just breaking that down. And then I think with this iteration, right, it's going to happen a lot faster. And I think people just naturally want to work together because I think I was at VMware Explore and I I did a few talks and I started every talk with like, all right, let's get a show of hands in the room. Who has used one of these tools like at work, like Cursor, Claude Code, whatever? And pretty much the entire room and all of them went up and I was like, well, that's a lot more than I expected. So like the adoption rate, even at, you know, the traditional like, you know, platform or developer engineer, uh, and, you know, or just infrastructure admin at some of these personas is well into that scope. So. The entire enterprise, I think, is picking up on this really fast. And the more collaboration, I think it just will help out. So just office hours, reach out, be proactive. And I guess if you're on, if don't be mindful or be open to change, I just, this is just in general, if what I've seen before and if you're resistant and resist the change and that's gonna kind of get you labeled and kind of put off the side. But if you're You may have your own opinions, but if you present them in a way that kind of is, let's say, maybe open-minded per se, you have a better chance at some of those key learnings that you've dealt with throughout your career to actually be applied in the new way as well.

That makes sense.

I'm curious, I want to circle back for a moment on something you said. As we're talking about kind of spreading across, my mind goes to security. Because I know in my industry, security is a pretty big deal. We do have air gap things and stuff like that. That's what we do in defense. But as you're thinking about that, one of the topics that is a really common topic now is kind of going back a few weeks. We covered it in depth on a previous episode, the whole OpenAI, the swarms got out, Hugging Face. That whole thing, I'll refer people back if they're not familiar with it to our episode recently that covered that. But that is top of mind. It's not only top of mind for security, but it's top of mind for legal, it's top of mind for comms as they're thinking about, what do we do if something like this were to happen in our org, whatever that org is? And can you talk a little bit about, as you've kind of Brought the enterprise structure up to date to deal with agents. And, you know, it's, we're beyond just the traditional software and app world and we're taking some of the same lessons and reapplying them in a new context here. Can you talk a little bit about how that helps on the security side? If, for the folks out there that are really, you know, that's their job is to keep things from really going awry. Can you talk for a moment about keeping things in the box, if you will? Yeah.

Yes, for sure. And then I guess I'll put a disclaimer. These all could be suggestions to help out with. If you really dig into some of the Hugging Face stuff, it's a little scary. So I'm not to say that we're better than all of the AI labs or anything like that. But if you look at some of the things where how that kind of got started is where they started getting out of the sandbox, right? And then they found things that they could get access to. Or, you know, having monitoring and control around what those apps or agents can do is, you know, something that's been in mind for all enterprise apps, right? So like having, you know, kind of guardrails, even like network zones, like, hey, this is a highly regulated zone. It's super locked down. Like even just like traditional firewall would have potentially blocked a lot of this stuff, right? So it's kind of key, like a lot of the core enterprise learnings and kind of maybe take that for like, you know, the old team versus the new team, it's like, well, the guys, you could have just, you know, I don't know, had some basic controls. And like, I think they even talked about like their monitoring wasn't working or they weren't really monitoring it. And like you had all these agent swarms. And then of course, then they got to the Artifactory, which then they found the weak link and the Artifactory had the internet access and somehow they magically, you know, zero-dayed the Artifactory instance multiple times. So not to say that we could prevent everything, but it's just, you take those, these core enterprise technologies that maybe would have helped out a little bit of just, you know, uh, Agent control, like hardened sandbox-type constructs, very locked-down network controls, that would have helped a little bit potentially, you know, just, you know, monitoring what's in the agent, make sure that the agents aren't, you know, somehow injecting tools that they're not supposed to have. I'm not sure if that was an issue or not, but, you know, when you talk about an application, the whole dependency stack is always up to up to grabs. So you want to make sure that the dependency stack, the container, everything is, and the whole OS and the whole platform is trying to patch and reliable as possible. So it's a, I think the key thing is obviously, you know, we had the whole mind-bending effect of that. They somehow made this messaging board on Artifactory and then it got deleted and they recreated it. So there's that factor, which I think everybody's a little like, and that goes into the whole AI is going to kill us thing that happened over the weekend, right? Versus just do some basic Security, basic stuff, right?

Just to clarify that, that was a social media post, less people misinterpret that.

Yeah, that's a question. Yeah. If you're not on AI Twitter or AIX, there's a lot of posts around, you know, somebody left Anthropic and a bunch of Anthropic people started posting around that they're afraid that there's a greater than 10% chance that AI is going to take out humanity over the weekend/Friday. And That stirred up a lot of conversation, let's just say. So that's what I was referring to. Yeah.

Okay. I guess that's useful. I appreciate that. We like to finish off by kind of giving you kind of a freeform question, and that is kind of toward the future as you are doing your job and running your podcast and talking to people, And kind of building up your perspective on where things are going with this. We like to ask guests to go ahead and get crazy and prognosticate just a bit, and we don't hold you to it. But really the way we do this is when you're kind of just letting your mind wander and you're thinking about these things in the back of your mind, going to bed at night, having a glass of wine, whatever it is you do to chill out, Where do you think things are going? Like, what, and you can choose the timeframe that you like, but I really like to think like, what do you think is coming? What's next? Why, you know, what should orgs be having a thought toward for the future, even if it's not, even if we're not there yet today? Any thoughts on what that future looks like as we close out?

Oh boy, it could be many things, but I think from an enterprise perspective, I think we'll see a lot of, Like team change and role change, not even just in enterprise, like the whole, like from the business side, things are going to change a lot, right? Because you're starting to see, and I was going through old episodes of my podcast this year and just some of the old, just the start of this year and there's just now it's like, wow, a lot has changed. And like, can't imagine, like, can you like, you know, imagine a couple of years from now how much change the rate of change is now? It's accelerating.

Proceed.

So it's, it's hard to imagine that, but I, I definitely think from a business perspective, we're gonna see like a radical change in like how, how, you know, development and, you know, product management and everything changes or everything kind of, you know, there's gonna be like new roles get created, uh, you know, orgs are gonna change, I think. And I think, I think for the good too, 'cause I think like there, there's always the fear of a job loss, but I, I find that I have, and I've heard this echoed across many, uh, personal experiences, I am like working harder than I ever have 'cause I have all these agents and they like, they need stuff from me and I'm constantly like, and I'm doing building things that I, Always wanted to build, but just never had the time to. So, um, I feel like we're all going to be busy and it maybe, maybe we need to, you know, not, not get too crazy with the agents, but it's, it's hard for me because I, it's just, it's, uh, it's fascinating. It's just, it's really enjoyable. So doing a lot of things with automation, I can only imagine what these agents are going to be like in the, you know, the next 6 months in terms of, you know, what they can do for you. I think we'll see a lot with voice come, come into play and maybe even a new style of interaction with the computer, uh, as well. So. Um, that will be interesting as well. And then I, obviously, I think, you know, did you see the posts like Hugging Face had little robot things on sale?

Yes.

Yeah.

Yeah. So like the little, little micro duck things or whatever.

I did. I was like, I was like, I told my wife, I was like, maybe we should get one. And she's like, she's like, you have enough toys. You just stop.

Yeah.

Well, I may have ordered a few for the family and I'm like, those things are going to be awesome. So like, like robotics and things like, you know, I, I think the LLM is great and wonderful, but, uh, there's so many more possibilities where we could go that I guess like for humanity, right? As long as we prevent the Terminator scenario, it'll be great, right?

Absolutely. Yeah, great. I appreciate you painting that. That helps a lot. Yeah, the rate of change is just insane now, and I agree with you. I think it's going to get faster and faster. But great way to end the show. A lot of fun. I always love to hear what people say when we're asking about kind of where they think things are going. Thank you for coming on the show today. It was a great conversation. I learned so much. I really appreciate that. And I hope people tune in to Cloud Foundry Weekly to hear you talking more about these topics, and they should definitely go to the Midwest Summit and hear your talk. I was there last year, did a talk. I won't be there this year, so I'll have I'll have to see if I can get ahold of the video of it or something. But anyway, thanks for coming on Practical AI today.

Thanks for having me.

All right, that's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Bluesky. You'll see us posting insights related to the latest AI developments, and we would love for you to Join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.