How Enterprises Are Actually Adopting AI: From the Boardroom to Bare Metal

Hello everyone. I'm Angelina, founder and host of TwoSet AI podcast. Thanks for tuning in. Some quick context if you are new here. TwoSet AI started out as TwoSet, 2 persons talking about AI. I've invited our guest speaker today, Joseph, on the podcast, and I'll share the link later in soon. Over the last year, I've interviewed roughly 100 founders and domain experts on the show. And every one of these conversations opened the door to a bigger world of engineering behind them. And the engineers, you know, the researchers solving the real problems, I think they're the true hero here. And I want to invite them back for this conversation during a workshop. And today I have a special guest, Joseph. He's a— I'll let him introduce himself, but he is— he's the Chief AI Officer at EVT. And he's worked with the top Fortune 100 companies helping them adopting AI. So he has a lot to share about how enterprises are adopting AI. So you'll get a special edition today. And please, a little bit administrative stuff for the workshop today. The talk will be a little bit more conversational and it will last 30 to 40 minutes. And you can ask questions anytime, and I'll moderate questions, and Joseph will, will share his answers. Please stay to the end. We're launching something new for this community today. And one small ask for all of you here, if you find this useful, please hit subscribe to the channel, and we really appreciate your support. Joseph, I'll hand it over to you.

Ah, thank you, Angelina, and thank you everybody for joining us today. As Angelina said, I'm the Chief AI Officer for Enterprise Vision Technologies, and we focus on sort of the global Fortune 250, really from Fortune 1 down to Fortune 250. About half of those are existing clients that we support. And obviously AI has been a big topic, especially over the last few years. In fact, it's one of the main topics in the boardroom, which is an interesting kind of dichotomy between growth and risk. But I'm very grateful to be here today, and it's an exciting time in the world of AI.

Awesome. I have your deck on the stage. Do you want to kick us off? We can start from your deck and I'll time in with my questions.

Yeah, sure. Maybe we go to the 2nd slide and then I can kick it off from there just to resonate with what I was talking about before.

Can you move the slides or do you want me to move the slides?

Let me see if I can—

I think you should have control. Oh wait, yeah, let me know if you have control.

I don't see that I can move the slides.

How about, how about this? Do you want to share your screen instead?

Um, it says present, but I don't have this deck that you have ready to go. But see if you can advance. Can you advance?

Let me do it. Don't worry. Yeah, let me see. This page.

There we go. Yeah, yeah. Perfect, perfect. Yeah, thank you for that, Angelina. So just to kind of kick us off, when we look at large companies and large enterprises, every one of them, even smaller organizations, has sort of the boardroom conversations. I sit on the board of a group called HDF or HDF. which is a large data platform that is like NASA uses and the Department of Energy uses. So I sit on a board, but I also consult with various boards and get invited to talk to sort of boards. And so really, I think the conversation today is going to be about sort of the boardroom to bare metal and really how do we bring AI into production and some of the pitfalls that folks run into and what we're seeing, right? And what's top of mind? So when we look at the boardroom, boards are usually really focused on 2 primary things, right? As you can see here, the risk components, which is how do we protect the business? The board members have fiduciary responsibility to protect the business, right? In addition to growth. So it's real about how can the board support growth? And in that situation, the interesting thing is that AI is really this, the dichotomy is that it sits on both sides and it's one of the most existential threats and it's one of the biggest force multipliers for enterprise organizations today because it will help them build new products and gain efficiencies, but it can also be one of the biggest risks internal to the organization. if AI were to go rogue internally. And because the threat surface has been automated with AI, it is one of the biggest threats, existential threats to these large enterprises as well.

I have some questions for you. Yeah, what are, what are some of the top priorities that these board or executives want to see results from?

Yeah, so when, when we talk about the risk scenario, they really want to know what's the reality of internal risk and external risk. And that's a huge priority. And why do I say that? Because growth is super important to the organization. However, if AI that they dispatch for growth is going to be something that could hack them internally or create a problem internally for their systems, that's very scary to them. In addition, externally, if people are using AI to be able to hack externally, that's a huge challenge to them. They have many priorities from the business perspective. However, if they don't solve this issue with the internal threat of AI, and the external threat of AI, those business initiatives that they're pushing for, for growth, all of those sort of things could come crumbling down on them if they don't deal with this existential threats of internal protection and external defense, if you will, for AI. So that's a major, major problem and challenge and top-level priority in addition to growth. And when we see growth, Growth comes in a couple of forms. A lot of folks are saying, like Amazon is talking about flattening the hiring curve. I think that's probably a nice way of saying it. But a lot of organizations are the priorities on how they can potentially flatten the hiring curve in addition to create new products, right, and create new innovations. So those are key concerns of folks in the boardroom right now with AI.

Yeah, I wanted to ask you that question about workforce management. Does that actually come up as part of the conversation in a boardroom? And I wanted to ask you if board members are discussing about laying off teams or restructuring teams.

I think with AI, it is an easy thing to put folks' minds around because when we say, how can AI help us? I think it's easy for people to point at reduction of resources. However, I think that's more challenging than what— than the reality of it. I think if you look at even like Microsoft, I think there's a lot of anecdotal evidence. I think some organizations have reduced staff. However, if you look at what Microsoft did with their customer service, they saved $500 million in customer service, but they actually ended up spending billions of dollars in order to do that. So in that particular scenario, cost savings initiatives are real, but I really feel like the reality of it is, is what you wanna do is you wanna maximize the resources that you have. And that's why I like using the term of flattening the hiring curve, because if you can enable your people with new technologies, if you can enable those systems that they have in place right now and re-engineer imagine these systems with new technology, then you can grow into the future and hire fewer people to accomplish more, more growth. But it absolutely is a, a, um, a board-level conversation for, um, you know, resources and staffing.

It does sound like, you know, that there will be less jobs, fewer jobs on the market, you know, it's coming out.

I mean, I I think potentially, but I think there are some evidence to the contrary of that. I think if you look at, you know, about 12 years ago, folks were saying that radiology was going to be, you know, nonexistent, right? So that's a use case, but, oh, it's been over, you know, again, I think 12 years since the computer vision models have been doing radiology. And actually we have the highest demand for radiologists that we have ever had. And so it's a little bit like Jevons' paradox, Right, that when the price of something comes down and the accessibility of that thing becomes more accessible, the demand goes up. So I'm optimistic that there will be some jobs that will be impacted, but I think there'll be a lot more opportunities that will be created because I believe people are, once empowered, can really, you know, do good with that empowerment.

Awesome. And we have some questions from the audience. So from one of them is from Kornaztzky. Let me show it on the screen so you can see it as well. So from who do business by AI? AI platforms, startups, consulting companies, independent consulting doing the work, or maybe all internal?

That's a good, good question. So There are many, many startups that have started, many, many products that have started in addition to many platforms. So if you look at like platforms from the hyperscale, and by the way, great question. The hyperscalers, let's say Microsoft, and if you look at AWS and Google, they all provide these harnesses. So in that scenario, your traditional hyperscalers, these are things that you can buy, right? Same way with the model providers. OpenAI and Anthropic are starting to offer more enterprise sort of technologies. And then to your point, there are significant amount of startups. It's actually interesting because I think many of our clients have purchased some of these startups, but now they kind of recognize that they were not necessarily ready to purchase these sort of subscriptions. And they're looking at, hey, we bought all of these various things, now we're not actually using them. So I think there's some contraction that's happening right now in the startup space. There is a lot of consulting opportunities. I think the consulting opportunities are going to actually increase because with the code development like Claude and Codex and things of that nature, people do want to build and they are empowered to build. So that really takes some consulting. prowess to do, but it's more in enabling the users internal to the organization. So I think the consulting companies and independent consultants, I'm very bullish on those entities.

Awesome. I have one more question on this page. Who's invited in the room, in the boardroom, to talk about, to engage in these conversations?

I mean, in our board, it's traditionally the board meetings, right, the traditional board meetings. But then we have, like, larger board meetings that are like, you know, it could be over 100 people are invited to have these topics. And that would be your other C-suites, CTO, CDO, CIO, obviously CFO and things of that nature are part of the normal conversations, depending on, like, who the treasurers are and things of that nature. But the C-level, the C-suite, obviously, the CEO is going to be a part of those things. But dependent on the particular organization, it could be your, your other members of the C-suite and sort of EVPs and things of that nature. But that's not always the case, right? Primarily, though, the high-level C-level C-suite is there. But, but that depends on the organization.

Awesome. And we have a comment about, you know, our optimism about the job market. Less jobs for short-term and long-term for more jobs and more, hopefully.

I can resonate with that. And I kind of alluded to it earlier. It's easy for, let's say, a CFO to get their mind around fewer folks, right? But the reality of it is, I think that those optimizations are a little more elusive and a little more complex than that as we're moving into this age of AI.

Okay, more questions. How important is the realization of AI governance and security, especially after rogue AI agents? And what percentage of AI businesses is, I guess, our origin to be AI governance and security?

Yeah, another great question. Many folks are building agents and MCPs and retrieval augmented generation all over the place. However, the governed frameworks, this is very, very key to my client base. If you look at like Sarbanes-Oxley and SOX, pretty much every client that we deal with has SOX compliance, and pretty much all of them have the additional attestation compliance component. So governance is very, very important for sort of change management and all of those things for governance and being able to have what we call bounded autonomy for AI, which is really sort of the next level of human in the loop and what we call policy-constrained execution. So it's very, very important because when we start giving AI potential access to data, right, We don't want that AI to be able to do things with that data. So there are particular guardrails in place to separate AI from particular types of data and governance and controls so that we make sure that AI systems don't have the ability to go rogue, right? It's not something that they can actually do, like separating keys and giving frameworks to be able to support that. So this is a— a very important thing and I don't think it's going away at all. I think again, this is a key board initiative because folks do not want their AIs doing things that they cannot govern.

Great. Let's move to the next page and then we have more questions I think we can address on the next page.

This one.

Great questions. Thanks.

I love this one because we talk about experimentation and we'll We'll talk a little bit more about experimentation with AI, and because a lot of people are saying, oh, there's a lot of duplication of effort out there and things of that nature, which there likely is some duplication of effort. But well, if we step back and we look at like the enterprise problem, large companies, it is more like instead of just like a sprint, right, or a marathon, it's more like a relay race, meaning that department A finishes some bit of work, some value-added activity, and then passes it on to department 2, who does some value-added activity along the line. And as you can see here, you have the business owners, the builders, sort of the platform components, and sort of that feedback mechanism. The tendency right now, and I think it's been this way for a long time, even more so now with AI and code development, is that people want to see a lot of activity. And that's like, well, I just built this MCP, I just built this agent, I just built this tool, I just built this thing. And that's exciting. However, again, an enterprise being more like a relay race, it's more about how is the baton moving from department to department to department in support of the business. And I think you've got the analogy here, which I really love. Everyone's legs are moving fast, right? But what they should really be measuring is, How is the baton moving throughout the organization in alignment with the business goals?

I really like this analogy of it's a relay race. But what I'm hearing in reality from some industry partners is that this isn't true because what I'm seeing is one of my friends was telling me that their teams are building something and then their manager tells them that, you know, try it out. And multiple teams are probably building very similar things that are very customer to their own team. And in a sense, they're competing with each other and without understanding, like there is no relay because they don't know like, where is this going to go, right? Is this— and you know how it was communicated from the management level to the people who are actually working on a project. You know, it's more like, be creative, do whatever you want. Like, let's increase our productivity and token maxing, right? So there is no efficiency across the board.

Yeah, I think it's interesting. And we, I, we want to make sure that we do encourage innovation and we need people to learn. And because it's exciting and we want people excited about it and there is likely, again, duplication of effort that's happening. However, like what I mentioned in an enterprise, if you're talking about a bank loan, Or you're talking about some research output, or you're talking about some manufacturing output, there's a business outcome at the end of that, right? And in order to optimize that, you have to have your various inputs. And then for regulated industries, there's separations of duty, right? That's why we have Department A, Department B, and Department C. So in that scenario, there is a relay race internal to those individual organizations and components. There's a lot of optimization that can come Locally. However, if you look historically, like W. Edward Deming and systems theory, it really— the optimizations, the larger optimizations only come through restructuring the actual end-to-end processes. And so when I say relay race, I mean, that's what I'm talking about. But I do agree, there are a lot of what we call backyard patios being built out there, and we'll talk more about that. But sort of everybody's running to these backyard patios, and that's the same sort of thing that you talk about. Everybody's legs are moving fast, but the baton is not necessarily moving forward with the organization.

Yeah, I have— we have some questions. How about we address this one first? Joseph, how do you approach enterprises wearing your consulting hat with greenfield opportunity versus orgs with some level of maturity? What are those organizations expecting from someone like you?

Yeah, I mean, greenfield Is awesome, right? Because it's like we're building this new technology that doesn't already— it's not already ingrained. And with AI, you can move, I think, much, much faster with greenfield opportunities, right? It's a clean sheet of paper. We can look at our business outcome. We can look at all the components in between it, the requirements for that. And man, can we really move fast with greenfield opportunities. And it's moving faster, right? Things that were built one quarter ago, we're like, oh my gosh, the tools that we had last quarter are nothing to what we have this quarter, right? So the greenfield opportunities are fantastic. The sort of brownfield, if you will, I think that's where the true power comes because those are the things that are making money today. So it really is the new things, the new exciting things that we're not necessarily sure about, that are going to get people excited. We want to get to those things faster. However, the— let's call it the 80/20 rule. The 80% of things that are already built, let's say in banking, let's say it's FFIEC sort of regulations and the pathway to production, or if it's pharmaceutical and it's GxP regulations and the pathway to production, if we can optimize those things, we know there's a lot of sunk cost in those particular environments. And if we can do that successfully, we know there's a significant amount of optimizations that we can give to the business and give those resources back for the greenfield opportunities.

Another one, are most AI engineer roles moving towards multimodal since people search with both PDF and images, so pure RAG on text seems to be getting outdated?

Well, that's a fantastic question, and I'm glad you asked that because— and I didn't get to share this, but my book that just came out is on video generation and multimodal learning. So 100%, I do believe that going forward that multimodal— yeah, as you can see here, yeah, Diffusion Transformers and Multimodal Learning. The future is not going to be just grammar, right? The future will be multimodal. And when I wrote this book, it's about teaching the AI to see. And spoiler alert, there'll be another book coming out about physical AI. And physical AI is about having AI that can interact with and align with the laws of physics, if you will. And so I definitely agree that multimodal video and audio and things of that nature that are more native to, let's say, the human experience, I do believe that is definitely the future. That's one of the main reasons why I worked with O'Reilly to publish this book.

Awesome. Adding you back to the stage and I think we have more questions.

Great questions, by the way. Fantastic audience.

Rashi, are enterprises starting to feel AI fatigue? So many pilots, so many things going on. It's pretty chaotic.

Yeah, I think there definitely is some fatigue that's coming. I think there's still a lot of excitement. For us, we definitely want to keep people excited, but I think you're touching on something that enterprises are grappling with now. And I think as we were talking about, as Angelina was alluding to, like, when is there going to be this like, okay, everybody pause? I think what I'm seeing is there's going to be a few quarters from now, like everybody's excited, everybody's token maxing, the costs are going to start the bills are going to start coming due. And it's going to be like next quarter, there's probably going to be some guardrails on who is actually making an impact on the business. And then there's going to potentially be some reduction and constriction. And then the next quarter out, there will likely even be more constriction on, hey, these groups are the ones that are actually maintaining business value. And these other organizations are aligning towards, let's say, more status quo. And I do believe there will be some guardrails put in place for the new business users that we see, the business developers that were potentially doing macros and doing like robotic process automation. So those folks will be empowered. The traditional, let's say, IT staff and site reliability engineers will be empowered. And then you will still have sort of your bright stars that are token maxing. But I think there will be some guardrails put in place. And it's a fantastic question. And I do think you're onto something there.

Let's move on to the next page. Okay. Experiments.

Yeah, I think this is exactly what that previous question is talking about, right? Is that everybody's out there experimenting, which I do believe is important. But like what this is talking about, Folks need a way to share work. We call that the foundation for enterprises, the AI skyscraper. To the point of the question before, it really should align towards business value. Does it improve a service? Does it improve an outcome? Does it actually gain efficiency? How challenging is it going to be to bring this thing into production? Because if it's 2 to 3 years out, if it's too big of a bite, That's not necessarily realistic for folks to put into place. So we usually do a foursquare, right, a traditional foursquare of the big bets, right, and things that we can do right now, and then sort of the longer-term things. I would caution, though, some of the things that people think should be further out are actually things that should be done now. And one of those is like what we call a digital nervous system or an enterprise nervous system. like which is based on more knowledge graph and context graph and procedural graph. Those things are less native, I think, to enterprises. I think relational data is more native, but that context graph and procedural graph, those things are actually things that are going to tie the businesses together. But yes, I think this is a great slide here to illustrate what the previous question was about. A lot of experimentation, but how is this actually going to impact the business?

Sorry, I'm pushing you to the next slide.

Let's go.

Yeah, the skyscraper. This is your framework. We want to hear about it.

Yeah, I have a story that I've built over a long period of time and it's like the enterprise does want to build the AI skyscraper or the enterprise, the team members of the enterprise want to be Let's say the, the first folks, so walking on the moon, right? They want to be Neil Armstrong walking on the moon. But the reality of it is, is the skyscraper, in order just to get out of the atmosphere, right, you need all those telemetry things to get out of the atmosphere and get home safely. So enterprises need this foundation for the skyscraper. But what we actually see is we see everybody building backyard patios. And that's what this fatigue is, the one of the things that You know, was mentioned, these little pilots and experiments all over the place, and this is where this deduplication comes into play. But the reality is that when you have all these backyard patios and you try to create the enterprise skyscraper, the reality is that you probably won't even have the skyscraper. What you'll end up with is potentially the leaning tower of AI. And I think, Angelina, you joke about it, you may have the collapsing tower. So what we talk about is sort of the framework and the foundational components, because you need to build that strong foundation. And then with, like, call it the plumbing and the guardrails and all those things so that you can add floors to the skyscraper. Because if you don't have that stable foundation, then again, you're running a lot of risks by just relying on the backyard patios for your enterprise solution.

Question, how big is baggage, uh, how big the baggage is for legacy implementation and shadow IT in enterprises that's pulling back organizations in moving fast? And if that's okay, yeah, go ahead.

Yeah, I think it depends on the organizations. This is definitely here, but I, I would say we deal with this a lot, um, and if you look at, um, one of the things I mentioned, like value stream mapping for like process automation, it's actually exciting now because the tools are much more in your fingertips than they ever were to really understand what the business outcome is and do a value stream map of all of those value-added activities and where value-added activities go to manual effort. So I think before, this would have taken a team of consultants to visualize this, but now we have a lot of tools that can show this. So it's as big a bag as the organization has, right? And the longer the organization has been in existence, Some large enterprises, if you go to Fortune 100, they've been around for a long, long, long time. So to your point, they've got a big boat anchor holding them back. But I do believe that now they can actually reduce that technical debt and with the technology that's out there. I would also say one other thing about shadow IT, and I know there's the term shadow AI. I actually think shadow AI is even worse because Shadow IT, if you had an organization with a million employees, maybe you had, you know, 10% of that organization that was shadow IT, that was swiping credit cards at cloud providers and things of that nature. Now with shadow AI, you got 80 to 90% of the organization that's using shadow AI. So this is a technical debt tsunami that can definitely, you know, constrict Let's move on to the next one.

So what is the foundation?

Yeah, I think this is pretty interesting when we talked about the skyscraper analogy. And it's really, this is just an example of like what we run into in like as technologists, really, we have the business who's planning for the future, We definitely need to have AI supporting these planning initiatives, right, and optimizing the planning. We also need this sort of day 1 and what we call this is really this what they call the pathway to production. So the designs and the plans and then the provisioning sort of elements of that and having AI be able to give you more of a self-service type scenario and give you telemetry and give you guardrails and security, vulnerability analysis being a huge one. And then when we move into the day 2, Operations, same sort of thing, capacity management and protection, and then create this virtuous circle from our day zero planning initiatives for the next wave of planning that we want to do into the next provisioning cycle that we have into the next day 2 cycle. So this is a loop that we think that enterprises should take a look at. If you're just moving all over the place and not thinking about things for the full lifecycle, this is again where, where you're potentially creating more technical debt than you are reducing with your organization.

There's a relevant question. You know, I still don't see a focus on AI security yet. When do you think this will happen?

I mean, we're laser-focused on it every day. I think there's a lot of initiatives. If you look at Palo Alto Networks just recently made some acquisitions, or the firewall companies are making acquisitions and they are moving into AI operations like your security companies. So I think there is a focus on it. I think you won't start seeing these products come to market for another probably 12 to 18 months, but there is a lot of acquisitions happening in the space for secure AI operations. But it's coming. It's just, I think it's nascent right now because of the acquisitions. Great question, by the way.

There's another one that's relevant, you know, the wipe-coded products, right? We've been talking about the pad, the backyard patios, scaling possibilities when these durability demands are up and architecture-wise when agents have full autonomy?

Yeah, I mean, I think vibe coding is a good thing, especially in sort of pre-production so that you can see if your idea is viable and people can figure out the business impact that it could make. However, your traditional CI/CD pipelines, your traditional policies as code, those things can be accentuated with AI. However, taking a Vibe-coded application and moving it to production, there should be a lot of investment in that. Some of the foundational components that we build is really like, hey, here's our center component. You bring all your MCPs, you bring your agents, you have a proven business value. And what we want is a central focus to actually streamline you on your pathway to production so that you're not having to build identity and access management, building all the vulnerability management, building all of the, the sort of code scanning. We want to make those things natural for people who are building these agents. But this is a fantastic point, is that you can't just vibe code your way past Sonar scans. and code quality, right? Those things have to be innate in your pathway to production.

I feel the connotation behind vibe coding has to change because I've heard that a lot of teams are saying starting from January this year, everybody's vibe coding, nobody's writing human code anymore. So it's all vibe coded, but, you know, they have the judgment inside of it.

Yeah, I mean, Angelita, you're exactly right. I think in this slide kind of shows that too. Right? It's like we start gaining some trust with the code. I know when I was first vibe coding, which was when Claude Code first came out, you guys can look at my repo. I think I had the very first Claude Code guide. I haven't kept it very much up to date, but it was like I was looking at every line of code, right? But now it's more like now that you've gained some trust with the AI, you're running through validations and putting guardrails and putting harnesses around the code versus Checking it. And if you look at the slide that we have here, that trust and validation will come over time. And I think to your point, it's a lot less about double-checking every line of code, but making sure that the guardrails and harnesses around it for your code quality and your scanning, those things are, you know, 80% of what you're doing with vibe coding should be that pipeline, that security pipeline and governance pipeline. Whereas the 20% should be the vibe coding part of it.

Yeah. How many loops of day 0 to 1 are acceptable?

Yeah, I think this is an interesting question. I think it really depends on the organization themselves, because you do need to focus on operate. And in fact, I, I, if you're doing like traditional value stream, like optimization, you actually start with the outcome, the operational production outcome, and work your way back. Because it is very, um, common for there to be a lot of toil in planning and a lot of toil in, in the day 0, day 1 loop if you don't really start with the outcome first. And I think that's probably, um, advice I would give. Now, I can't take credit for that. That's more of a W. Edward Deming or Illy Goldratt's The Goal or Gene Kim, right, from The Phoenix Project. It's really starting with your outcome first and then working your way back, because then what you're going to see is there probably is likely a lot of waste in the day 0, day 1 initiatives that don't necessarily need to be there today for your outcome.

Thank you for the question. Let's move on to the next page. Oh, you talked about this one already.

Yeah, yeah, I think this is, um, this is going to become— I mean, we can, we can table this one for now, but I think this is going to become more powerful over time. So, uh, if you're in the audience, pause here, take a screenshot, look at the digital nervous system. This is going to be something that's going to become very important over the next 12 to 18 months.

I'm staring at this page, lots of words, because I, I will— yeah, I will share with everybody sign up for this workshop. I'll share the deck with you so later, and so you have a copy and you don't have to, you know, trying to catch up a screenshot right now.

Yeah, and I, I can talk about it a little bit. Um, maybe we can just Yeah, go ahead. So early days of AI, people were talking about obviously the models. Oh, what model is it going to be? There's still a lot of people talking about which model are you using? Are you using this model or that model? The model's important, but if you look at all of the things around the model for traditional machine learning, in order to bring that model into production, there's a lot of things that have to go around it. Same thing with a large language model. Initially, people were talking about retrieval augmented generation where you have unstructured data and that unstructured data could give you context for the AI. Folks are realizing that that's great for particular document management and things of that nature. But when we're talking about an enterprise nervous system, we're talking about a knowledge graph, right? Graph RAG is one of those things. But we're also talking about context graph on what is important right now? Not just all of our knowledge base, but what is the reality of right now? And if you're talking about a workflow or a business or something that's happening right now, we need the relevant information in context. And then the next part of that is actually our procedural graph or our execution graph. What is the AI actually doing in the background with this context to take work off of my plate so that by the time the decision surface comes to me or comes to an executive, All of these underlying components have been validated. Now we're just asking the human in the loop to validate this decision based on this valid information. So for me, I think that combination of the digital nervous system is taking knowledge, which is the whole body of knowledge we have, distilling that down to context and awareness of what we have right now, and then allowing particular procedures and executions to happen on behalf of particular individuals. And this is something where AI stops just answering questions and starts helping us actually take actions.

I think we have maybe this question is for this slide. Can you elaborate on how this is going to become important in the next 12 to 18 months?

Yeah, I think at least for many companies, like I had mentioned, there's some acquisitions that have happened in like, let's say, the security space. The hyperscalers are, are going to start offering these graph-based systems as part of their harnesses. You'll start seeing these come out, and it's going to be less and less about MCP and, okay, what are your agents doing, and more about how these things can interact together. And so you'll start seeing these offerings like, uh, I think Jira— if you look at Jira, they've already started offering their graph or graph as a service, or ontology, ontology as a service. You're going to start seeing more, uh, of these terms thrown, thrown around. Um, if you think about— there's a company called Palantir, which I'm sure most of your audience has heard about. Their ontologies are more black box. You hire them, they come in, they figure out what your outcomes are, they build this, this ontology that a— that an AI can actually traverse. Well, now this technology is becoming in the hands of of practitioners so people can build their own ontologies, people can build their own knowledge graphs, people can build their own execution engines like LangGraph and things of that nature. A lot of people are using LangGraph, but that full stack that I just described with those ontologies and business rules, those are going to be put in the hands of practitioners. So I think it's just going to become louder and louder because of the capabilities that these things have.

Relevant question. How does this change how we approach AI system design? It's a good question.

It's a great question. I always do think about outcomes. If you think about really a singular outcome, I think what folks are going to be thinking about more is very complex outcomes. If I want to have a business outcome, I talk about this business system. I think about all the support systems and then I think of all the environments of those support systems like sandbox, prod, things of that nature. And I think about how do I go from planning all the way up to running this production system. With these things that we are talking about, we can really go to that production component and think of that as a main, like, business outcome, and then all the sub-outcomes required for that thing. And we can build all those sub-outcomes and have AI traverse and give us what-if scenarios. We can take natural language business rules, we can overlay that on with what they call first-order logic, which the AI can reason about. So we can help the AI— have the AI help us make these decisions and calculate them mathematically versus trying to look through like retrieval augmented generation. So I think the approach to system design will be more about what is the complex outcome, what are all the sub-outcomes, and what are all the supporting knowledge context and system processes that happen to create this business outcome.

Awesome. Great questions.

Great questions. Yeah. Wow.

Nick, I don't think we can go through the deck within the hour now. Oh, no, no.

It's the— yeah, the next energy. Thanks, everybody, for the fantastic questions.

Yep.

Yeah, I think— yeah, this was just an example. of what we were talking about. I wanted to give kind of a concrete example from sort of request end-to-end validation, right? Sort of a standard environment. But in this particular scenario, when we talk about those graphs and those systems, we talked about like the procedural graphs. And for enterprises, like I mentioned, if I'm dealing with a bank and a bank is trying to get a loan or something like that, That's an FFIEC kind of guidelines. There's like 6 separations of duty required for these activities to happen. However, there's a ton of work that has to happen within each one of these areas. The AI can do these things and help you again make those decision surfaces. Now, this is like request and validation. Think about patch management or vulnerability management, which is one of the key things we talked about. You've got the AI out there that's an automated attack surface, but now you have to create a system that's looking at vulnerabilities, that's looking at what's exploited, that's looking at your landscape of systems and help you determine what things should be patched now based on your business rules. Right. This is something that the AI can do. It can align with your policies. It can test these patches in sandbox. It can slowly create what we call bounded autonomy. Patching things in sandbox, elevating them up to the non-prod environments, testing them there, moving up to pre-production or DR. And then once you get to production, obviously this should be with your CAB and your human in the loop. But by the time it gets there, these tests have already been validated potentially a dozen times before you make a critical change. That's what this, this, this slide is meant to show. is that AI can actually do all these verification loops for you.

Great. You want to talk about this one?

I think you asked me a question, that's why I included this about like, how does training impact organizations? I think that we should be training constantly. I think that enterprises should be Holding lots of training. And the system we developed is called Levels. And we spent a lot of time researching how to help people change, because change is the hardest thing for people, right? Is thing to do is to change, right? So we looked at just giving training classes. We looked at doing quizzes. We looked at doing vibathons. And we really settled on taking like the hackathon concept, but really turning that into what we call the Levels program. And that's what we're talking about here. We want to have safe spaces for people to test AI safely. We want to have safe spaces for people to learn, but we want to be able to bring people together and have them solve business challenges together using the new technology. Because what we actually find is that the biggest impacts in business is when you can get multiple departments together thinking about solving business challenges. I can give some examples of that, but this is more about taking leaders to think about a new way of engineering and engineering discipline. And that's what this is talking about. Like, the levels in Vibeathon concept is, is once you get these teams together that can sort of vibe code and be creative about these things, they can solve these major challenges. challenges that enterprises are facing. So this is a very, very important slide.

It really sounds like a top-down approach. It's like the waterfall approach from engineering, but you know, you still need the senior management to really understand cross-department, how they coordinate, how they can add value at a higher level to be able to distill this down to the frontline engineers. Does it feel that way?

I'll give you an example, and I'll— and I can't— I have to take the names out to protect the innocent. But in one of these events that we held, there was a business team, and then there was a technical team that ended up joining forces. And it's a team that was— had, let's say, the— let's say it's a power company, theoretically a power company, and there was a team that was producing the power Right? So they had access to the backend systems of where power was going. And then there was a business office that had access to all of the accounts, right? But these teams, every time they tried to work together, a large consultant would come in and say, it's a quarter million dollars for us to determine this pilot and to join these datasets together. Well, in this Vibathon training, after each team was trained on, on sort of how to do coding, how to do it, in a principled way, how to come up with your ideas. They joined forces and they were able to join these datasets together in a very unique way. And they were able to identify $25 million worth of power that had gone unaccounted for a quarter, a quarter, right? And that's from power that was being produced and accounts that didn't exist to where the power was going. Right. And that, that was a huge change. But if those 2 departments, the people who were producing the power versus the people who were, um, had the billing office, those people hadn't come together, imagine what that, um, you know, what that had done. They would have been continuing to, um, charge the ratepayer for that power that was out there, right? Now they can recoup that. Now that impacts everybody that's in that community. So these are, these are big, big numbers that you can impact a large organization with. That's just one example.

So, so if that, if that didn't happen, it could be that those 2 teams might be vibe coding within their own department and they won't, they wouldn't go get anywhere because the communication, the power, the connection never happened. So my question is, does it mean that to leverage AI's power within an Enterprise, with a larger organization, the best is leadership has to rethink how we structure the team, how we redefine roles to facilitate that?

Depends on the organization, but I think I can just give some history here. If you look at, and I mentioned like Deming a few times, and there's, you guys on the audience can look that up, W. Edwards Deming, and there was a gentleman by the name of of Russ Akoff. And then there was Eli Goldratt, who wrote the book The Goal, and Gene Kim. All these are based on the same sort of concept of a value stream. Historically, let's call it Sloan Business School, this is silos, right? So you have a lot of silos and a lot of structure, right? The silos and the structure, depending on how far apart that silos and structure are from the work, this is the amount of inefficiency that you could potentially have, right? So that value stream optimization that you're talking about, where the work is actually getting done, they say, Deming said that mathematically, if you optimize locally, you can only actually impact the business by about 5%. But when you actually can align towards the outcome, Deming called it the aim of the system, that's where you can actually, the business can gain the other 95% of optimizations because you're looking at the problem from a completely different perspective. Instead of optimizing for my team, which may be great for us, right, Angelina, it's great for us, but everybody else before us and after us, if we don't understand what their needs are towards the aim or the goal of the business outcome, it's very hard for us to really optimize to support the end-to-end business outcome. So absolutely, I think these are common practices, but they're much harder to do in large organizations, right? Because you lose sight of who's downstream of you. And you lose sight of who the folks that are before you in the relay race.

Who should be responsible for these outcomes?

I mean, you can start with the business leaders and you can also start with the architecture teams. I think the architecture teams of these organizations are the ones that are trying to structure all these things and structure these things together. But really, the top business leaders should be understanding that this is a huge opportunity for folks to be able to gain these optimizations. And one of the interesting things, again, like we talked about the double-edged sword, were recent conversations with like the security teams, like the CISO. A lot of conversations I've had with people was like, well, why would you want to use AI to optimize this particular process? And then, you know, I have a conversation with the CISO and I say, well, you want to have AI optimizing this process because the attackers are optimizing the attack surface. And if you don't optimize this underlying process at the speed of this, the attack surface, right, the defense at the attack surface, well, you're going to be vulnerable. And that goes back to the boardroom. There's an existential threat to the business. That's what the board members have. It's the same applying that to your optimizations, right? So I think to your point, it should come from the top table sort of down. But now I think it's laser-focused on security. And if you can do it for security, well, then you should be able to do it for your business use cases as well.

This slide is about who does what.

Yeah, I mean, I think this was just, again, that same workflow and really understanding how this would work and how how AI could assist with policies, but I think this is probably one that we can share out as a handout.

Fantastic. Measure progress. How do we measure progress? I have that question too.

Yeah, I think it's interesting because it's going to be like when and how, right? Right now, I think a lot of folks, let's say on the street quarterly, are thinking about Like, let's say stock price or, you know, headcount or these various things, right? I think it's really about how you can actually move the business forward, right? And those sort of times that it takes to support your customer, customer service, quality customer service, the speed at which you can defend your business, right? Like, if you can identify the change processes and move those change processes through much faster, From request to actually complete— completion. I think those are very good, simple metrics. Like when a request comes in, how much is rework and how much does it take to actually support that request? That's going to show you the efficiency of your business, right? You want to be able to optimize for that. And then again, on the security and risk part, when a vulnerability comes down, how quickly can we remediate that through the enterprise? That makes it so that we are moving at the speed of the defense, at the speed of the offense that's coming at us. I think those are 2 metrics that I think I would recommend that enterprises focus on.

It sounds to me that for CISOs, there's no opting out. Like, you have to start acting now. Otherwise, the attackers are already, you know, they're taking actions.

Well, and again, we talked about it earlier. This is a board-level Fiduciary responsibility is to protect the business. AI is an existential threat. So if you don't solve this problem, it doesn't matter how exciting you are for the shiny new, you know, tool that you're, you know, putting out there. If bad actors can compromise that, then the reputational damage, the damage to the business, that far outweighs the marginal gain that you have for a potential new product.

I have a related question. So, so if this is the case, how do we measure team or team member performance?

Yeah, I mean, that's an interesting one. I think right now you just want to be a part of these conversations. You want to be learning. You want to be getting excited about this. I mean, let's think about this for a minute. This is the most exciting time in all of technology. Like, there's technologists listening, listening to this. I think we'd talked about this on our last conversation. If you have an idea, a dream, you know, something that you want to accomplish, there is no time in existence that it has been easier for you with the tools that we have to accomplish that thing. So I think right now, as a leader, you should empower and be excited for your people. And then as people who are operating, you should be excited that now this is within the grasp of you to accomplish these things. that may have been out of your grasp before. You can make a much bigger impact on your team. So I think first, it's just about the excitement from the leadership for empowerment. And the second part is really aligning towards those business outcomes and how you can support those business outcomes in your teams and across teams, like what we talked about before.

If I feel, if I'm siloed, right, I only know, like, everything about within my team, then it's going to be harder for me to understand what is really the outcome that the organization is driving towards. And what happens is this, you know, my manager, assuming I have a very creative manager and have all kinds of ideas, and then he or she might throw those ideas at me and I'm so excited in executing all of them, but then I'm doing busywork. It doesn't mean that I have a high performance, right? And as an individual contributor, sometimes I think it's very hard for me to distinguish, like, which project should I work on? I'm excited to work on all of them. But after 100 of those little projects, I realized I'm just doing busywork. You know what I mean?

Yeah, I think it's something that we— that I don't have a great answer for other than to say, at the manager level, to your point, there's a concept from Toyota Production Systems, it's called niwawashi, where teams and groups share what they're doing across departments. Like when initiatives come up. I think as managers and leaders, I think that's a great practice, because you can at least see what other folks are doing. And then I think that would give, empower your team members to be able to come up with ideas that could solve more, more broader challenges, to your point, versus just staying laser-focused on, on what's in front of you. The challenge is, is like with anything creative, creative components have to ask, Like deterministic folks, like usually for permission for things, right? It's like somebody's creative for an idea. Well, they've got to come to the CFO to make sure it makes business sense. So there's still going to be that sort of dichotomy. But with, with Anima Washi, you can figure out where those business problems are when people are sharing these across teams. You can really figure out something that can make a bigger impact, and then your individual contributors can contribute to a much larger opportunity.

If you think about it, the roles for managers become similar maybe to an investor, so because they have to have the eyes of the investor to know what comes, what can come out as a better outcome, isn't it? Because you're placing more bets now.

I think it's interesting because I do with this technology, I mean, I think everybody who's listening here, but let's say there's this new emerging like business developer that's coming out, right? And so business people are becoming developers, right? And so I think to your point, it's crossing all aspects of managers and leaders. They're becoming technical and very, very technical and very empowered to be able to accomplish these things. And I think there will be a shift in leadership over time towards the folks who can gravitate to take advantage of the tools that they have. I think historically it's been more about just people management, which is fantastic. You really need to have people skills. However, you need to know the tools that can empower your people to accomplish these skills too. So I think there's going to be more of a reliance on leaders who are not just people or financial. There also need to be technical leaders for all of those business leaders.

Okay. Joseph, in the interest of time, let's answer some of the questions. How— yeah, I think this is a relevant one. So what are types of projects we should be building for relevant skills? That's a great one.

Yeah, I think, um, I think anything that optimizes a process that you have would be great, right? And then really look at the process itself and deconstruct that, deconstruct your outcome and pull back that outcome to what you're— the tools that you have and the individual pieces of those things so that you can really procedural determine, hey, here's an outcome, here's the 5 things required for that outcome. I'm going to build a tool that's going to address not just the outcome, but is going to be able to prove step by step by step by step, each step along the way that these things can be validated and these things can be what we call policy-constrained execution, right? That each individual step and the handoff can be validated because if If you're not doing that, then you're just vibe coding. And if you want to vibe code, cool, just do something fun. But you're really going to have the next layer of regimentation is going to be, let's look at each one of these. Let's deconstruct these steps towards the outcome. So I would say anything that you can do with that, definitely do it.

Where do product designers, developers, product PMs' responsibilities lie now?

Man, I'm so excited about this one too, because again, the book that we had talked about, I think I shared before, my kids are creative and they're not necessarily the biggest fans of AI. Let's just put it that way, right? They're very creative. But what I shared with them, and you talk about product design, creative people can create like beautiful designs of things and it's the user experience that's super, super important. And I really hope that there's more of a focus on this design. Because that outcome, the convenience of it, the, you know, creating, the creation of things and the experience is super important. And then that creativity comes all the way through from the developers and the product managers. So I hope that there's a virtuous circle that comes and more of appreciation for the design, for the experience, as all of us should be able to create, put our creative input into the products that we're designing.

developing.

Awesome.

Last question.

Yeah, I mean, that's an interesting thought. I don't know if manipulate's the right word. But, but a great question. Definitely learning the tools and the landscape of tooling and keeping up with it. is going to be important. I shared some tidbits, some intuition on things that will make AI more powerful and capable for you, and that is a little bit of the backend knowledge. But I do think there is a finite amount of tools that we actually need to actually help you get that AI agent capability and the systems, right? The foundation for the skyscraper. Once you've got that, Then you should actually be able to move much faster with your business outcomes.

Awesome. For the rest of the slides, do you want to touch upon anything? Or I think this is it.

I think that's pretty much it. Unless you have some questions. I mean, wow, what a fantastic audience, Angelina. Fantastic questions.

I know. It's a, you know, we didn't expect that. But, but if you have any questions, feel free to reach out to Joseph on LinkedIn and check out his book. I'll share his book link in the chat and also in the YouTube description. If you come back, you're welcome to check it out. So the announcement I have today is very, it's very early stage, but I want to share that we're today we're launching Pain Points. Pain Points is TwoSet AI's curation of industry pain point boards. And Joseph is going to be my partner on this. Here's how it's going to work. So companies and domain experts can post a real problem that slows them down. And then we welcome all builders, all our audience to come pick one and ship a solution. And the best solution will get a stage. We'll feature you on our workshop and we can make an intro between you fixing the problem and the team actually behind the scenes. with the problem. So this is a very, very early stage, but we will, we will keep working on it and make these opportunities available for everybody in the future. So definitely.

So very excited about this. So yeah, everybody come with your pain points and come with your solutions. And very excited to partner with Angelina on this and your community, Angelina.

Yeah, thank you so much. And also, We have more workshops coming up next week, not next week, in 2 weeks. We will have Sentience founding energy engineer coming back. He was sick last time, so he couldn't make it, but he will talk about how do you build a digital twin with many voices. And if, you know, connect with me and connect with Joseph. Here's our LinkedIn. And thanks to our folks who stayed with us all the way till the end. We'll see you next time. Bye. Thank you, everyone. Okay.