Hybrid Intelligence - Integrating human and artificial intelligence for Strategic Advantage

Hi, everyone. Welcome to today's Resilience Roundtable. We'll give it just 30 seconds or a minute here for the digital door to finish swinging wildly open as everybody joins the webinar. We'll go ahead and get started shortly. As we get everybody situated, feel free to go ahead and use the chat to go ahead and introduce yourselves. We're intending this to be a highly interactive conversation, so I'd love to hear from each of you all. You know where you're coming in from. Go ahead and, and utilize the, the chat for that as we start getting going. Give it another 10, 15 seconds here as we get going. All right, looks like we've plateaued on the intros coming in or on the digital door coming in, so go ahead and get us going here again. This is the the Resilience Roundtable series that here at the center for Sustainable Business, we're hosting on large societal level, impactful conversations that we really aren't seeing being discussed in a, in a civil way elsewhere. So we intended for today to be highly interactive, a peer to peer learning session, community of practice. So while you're going to see a few of us up here on stage, as it were, we intend for this to be more of a fireside chat and conversations with you all. So looking forward to that. Go ahead and start introducing folks here for today's conversation around hybrid intelligence. Right, so that's that integration of artificial and human intelligence. First, we've got Dave joining us as recovering former chief data officer of the band that used to be known as the US Department of Defense. Dave, how are you doing today?

Hey, I'm doing real well, Chris. Thanks for having me.

No, no worries. And we'll excuse any of the little kittens that decide to join the webinar in process. Thank you for taking them in. Also with us today we've got Albrecht from Accenture. Albrecht, how are you doing?

I'm doing great. How about yourself, Chris?

Doing all right. I appreciate you guys joining. We would normally have a third co host with us, Ravi, from our strategy practice and complex projects at the school. Unfortunately, he's in such high demand, he wasn't able to join us today, but we'll share the recording with him. And again, just to tee off for today's conversation, we know that there are a lot of conversations about AI that are happening out there concurrently. Right. There's conversations around security leaks and your data getting out there and your business model becoming part of the training data. There are conversations around energy use and water use. There are conversations around how rapidly people are changing their pricing structures or the collapse of software as a service. Perspectively, we're going to end up trying to spend our time more focused on what are the whole new worlds of ways of doing business. What are, what are those conversations that aren't happening elsewhere? Right. You'll probably hear some contradictory points, some hot takes here in the conversation. Do please continue to use the chat in there. Just a technical point. Do use the chat for sharing with each other ideas, resources, as the conversation goes along, as an invitation to you all in the audience. We intend for this to be a conversation with each other. If you all do have questions, know that we will be looking to that Q and A function to tee up those questions. So with that, I'll go ahead. I think, Albert, we're at a good spot where let's actually start talking about the substance of the conversation. So I'll hand it over for you to set the stage because I know, Albert, you've talked before about this blending this merging of human and artificial intelligence and what that looks like in working space. So what are your thoughts? What are you seeing?

No thanks, Chris. It's interesting. I think we're seeing one of the most pivotal revolutions in the past century. We've had the industrial revolution that brought steam and power and the computer revolution that automated a whole bunch of mundane tasks like spreadsheets and such, and then the Internet and the smartphone thing and that brought all the world's information to the palm of your hand. But this next revolution that we're seeing right now, where four and a half billion people are on the Internet, eight billion people have, you know, smartphones or mobile phone subscriptions. And now you add in gen to that. I think we're living in a whole new world right this next decade. We're going to see this digital world and technology and the physical world converge together. You know, it's kind of like when atoms and bits come together. And what's interesting is it's a. It's an era of new human potential where technology becomes more human, like humans adopt technology. And I think there's going to be a radical shift in productivity, how people work, how people are creative. And it's not a good thing or a bad thing. It's just. It's just change. And change is hard. And you know, we look at companies that they don't necessarily need that super special expert with the advanced degree and training you can get somebody and they can go and get the information they need. They have a partner with these Gen AI Tools to now help augment them, help give them the skills to be able to do things. This is a game changer. Like, people now have more skills. They can be more productive, they can be more creative. They've got superpowers. Right? But. But there's a dark side to this, too. I mean, what about artists? What about musicians, authors? Our daughter's a graphic designer. Are computers going to replace them? So, you know, I'm an author. I ask AI to outline a book about spies and espionage and lost treasure. And it wrote a pretty good book. And I'm a musician. I ask AI to write me a song about a, you know, wistful country song about a lowly consultant on the road every week, missing my cat. It did a pretty good job at that, too. And these are rhetorical questions and things we're going to touch on in the next 45 minutes. But where's the humanity in my book and in my song? How do these AI generated songs and books bring the human experience and perspective to bear? How do we put guardrails around all this? How do we deal with bias and learning models and deep fakes? Are we rewriting history with changing all the digital pictures out there? Are businesses ready? Are humans ready? Is the world ready? I think this is one of the most exciting times, certainly in my lifetime, certainly in the last century. But I also think it's one of the scariest, too, as to where the world's going to go. So let me stop there for a minute, Dave, I know you and I have worked together for a long time, and maybe you can give a quick introduction to yourself because you've got just an incredible background with what you're doing now with DOD as a Marine, and you've got a very interesting perspective on the world. What are your thoughts?

And, Dave, you might be on mute.

It's like that's the first it's ever happened since COVID Albert, I appreciate you and Chris and the team, you know, allowing me to take part today. Obviously, I've worked closely with you over the years and appreciated your counsel. When I showed up at the Pentagon to be the first Chief Data Officer to write the data strategy, to write the data decrees, to try and set conditions so the department could begin recognizing the full power of data driven decision making and data driven software, AI as it's called, I'd say I probably have a little bit more of a aggressive look on the application of it. While the humanity is something that I do pay attention to and do care about, I also understand that there's people who do evil in this world, whether it's at a federal, a municipal or a municipal level. And what I want to do is get data driven software in their hands so they can be more proficient and efficient at their operations in the name of good. And we can make sure that everybody has an opportunity to have an equal playing field and live safely and grow their businesses and grow their arts and their sports and do that in a manner that a free society should be allowed to. And I think this technology is a game changer when applied against those, whether it's security or whether, you know, government functions at whatever level to, you know, go ahead and find an advantage and disadvantage, you know, those who aren't taking advantage of it over.

No, I think that's, I think it's really insightful. You know, one of the things that we're seeing, you know, at Accenture and just in the industry today is how rapidly companies are adopting AI models, agentic models, and trying to differentiate. And we've talked a lot about, you go back and you look at the Fortune 500 from back in 2000 and half those companies are gone. So it's all about being resilient, it's all about being nimble and agile and being able to change rapidly. And I think AI is one of those areas that just as you said, if you're not adopting it and you're not adopting it fast, you're going to be behind. And I think this is changing industries. And I was wondering, you know, the work that you're doing with, with the government, with, with Palantir, with industry, what are you seeing out there as sort of some of those new game changing solutions technologies out in the industry?

Yeah, I think, I think, you know, as, as we've talked for many years, you know, the organizations, whether it's on the commercial sector or the government side, who began organizing themselves, bringing in a modern data management platform like a Palantir to get organized and be prepared, really were the first to benefit from this wave of generative AI that crashed ashore a few years ago. If you were ready for it, you were ready for it. And your ability to engineer new workflows across all aspects of the business or the domain that you're jumping into is pretty profound. And I'd say that three year advantage, if you're just getting started now, the best time to plant a Tree was what, 20 years ago or today? You better get moving today because those who have adopted this technology, it's going to be hard to catch them. And it's only going to get harder as they begin to recognize how to ensemble not individual workflows, but start to bring those together. As we used to talk about from the battle space to the boardroom, how you fight with this technology and also how you run the Pentagon or how you run the United States government with it. When you bring both of those together, you get that proficiency and efficiency gained that really allows you to start at a different place than you've ever gone in the past. I think in the department I'm talking about things where we used to use similar technologies or the lack of them in workflows for I'll just say a kinetic action on the ground somewhere in the globe. And it would take a day at a minimum, if not more than one. We reduced that down to hours, if not seconds now. And while I can't go into some of the methods that you're using those, what I can say is this technology is keeping the Red Sea open for commercial traffic. This technology is allowing everybody in the Middle east to maybe find a moment of pause and start to enter a era of peace, fingers crossed, that we've never seen in the aor. That's really on the, the, the, the national security side. When you start talking about industry, it's, it's as simple as, you know, an organization, I'll say that, that moves goods across the planet, you know, every day at a volume that's hard to even comprehend. You know, I'll say equal to we're matching the U.S. postal Service and this organization, when they understand that they're about to have a hurricane crash into the Gulf coast, will need to move operations and adjust those. And there's so many secondary and tertiary moves associated with reallocating assets and moving them that a human would have to use Excel spreadsheets and all kinds of thousands of different areas and just frankly the gray space in between your ears to get it done. Whereas a machine now can do all that mathematically for you and predict where you could and should move these operations based on near real time, what that data is presenting itself and the impact that it's going to have so that you can both maintain continuity of ops and improve efficiency and delivery of your material that ranges from, who knows, you know, a baseball glove that your son needs for Sunday, prescription medicines that your mom might need so that she could stay alive or, you know, we'll, we'll, we'll say, you know, chips to put in some of our high end weapon systems that protect America. I think the only limiting factor of this Technology giving you an advantage is you hitting the gas pedal and the I believe button and just going for it. Getting that campaign of learning started inside your business, your agency, your department, your activity, and just going for it and adjusting and continuing to follow that campaign of learning as your workforce starts to unlock the power of it in their own domain specifics.

Love that. I love the excitement, enthusiasm for the space. Just adding on to that. Some other industry examples that we're seeing out there, and it's funny, they're across the board, right? And I think that the. We have so much data today, we have so much computing power today. We have the ability to do these models and generate these AI algorithms. What I think we're lacking is, is ideas and ideation. Like, where can I apply this in my business to actually make a difference? Because, you know, you look at, you know, you mentioned a little bit on, you know, operations and preventive maintenance. I mean, companies are now, whether it's a water distribution company or an electric company, they're staging, you know, spare parts, they're staging extra pipes, they're staging trucks, expecting not even a storm coming in, but expecting, you know, a water main is going to fail in the next six months. So they start staging materials. They're realizing that that may happen and that they're going to be ready for that. You look at hr, you know, there's been a lot of talk about, like, bias on humans and such, but what we're finding is that in HR for, like, hiring, these tools are now able to do much more inclusive hiring, to take the human bias out of the hiring process. Right. And as much as we say we're not biased, we all are. We have stereotypes, we have biases. But these AI tools can now challenge your thinking and say, why are you picking this person? Because I have this other one that actually may be better. We're looking at the legal industry where it's taking contracts and not just summarizing them and putting them in English. But what I find interesting is these tools are now pointing out areas of negotiation. So what are points that we should be thinking about? I mean, we talked about in the aerospace and defense industry, the value of data. You have a missile from one of the companies. There's all this internal data that's captured that oftentimes goes back to the manufacturer who owns that data. Should we be looking at that? We can now skim through tens of thousands of contracts like that and find things like that. So, you know, all industries now are, are looking at value creation coming out of Here. And, you know, I think what everybody on this call should be doing is looking introspectively at your job and your industry and saying, what could we do differently? And, and you know what, what could be a game changer in this.

Yeah, no, Albert, I doubling down on that point. You know, I, I, I, I regularly joke about, you know, people, you know, going the way of dinosaurs if they're not embracing this technology. But it's, it's not the developers of the technology that understand how it's going to be used. It's the domain experts. It's people who have to do something over and over and over again at their desk that now, if they can automate it with some of this technology, if you can give the large language models in an environment that orchestrates the data and the tools that they need, you know, you can really just take some of those mundane tasks that people waste 30, 40% of their day on every day, automate that right, and allow the human to move up, you know, a place where they can start to look at other areas of the business, other areas of the operations. They could say, hey, if it can do that, can it do this now? And I think once you start stacking all of those kind of new operating concepts on top of each other, you know, you start to realize the shape of a new doctrinal way of running an organization or a new way that a business can start generating better outcomes that they're trying to achieve. It's when you get it in the hands of the operators. I've got a guy at U.S. central Command right now who I was texting with last night continues to absolutely blow my mind. Was a major, now a lieutenant colonel, Joined the Army late in life. Never had his hands, you know, on a command line. But when we gave him the Palantir platform, when we rationalized the data across classification levels and then we integrated some of these large language models and taught them how to do low or no code, you know, building. He's completely transformed how the entire fires office operates at US Central Command, and that's being adopted by the rest of the Department of Defense because he had an idea and a little bit of intellectual curiosity to say, hey, let me, let me just try this. Right? And when it started working out, he just kept going at it and kept going at it and got better and better. One of the kind of, I'd say, leaders in kind of the modern AI movement spent a lot of time with me when I was the SOCOM Chief Data Officer, Director of AI, writing the AI strategy, and as I was standing that office up, didn't exist before. I thought there were all of these very technical archetypes that I needed to worry about bringing into the organization. And he waved his hand at those. He was like, these aren't like industry heard you, academia heard you. It's important. They're going to build these, I think, know, University of Pittsburgh, probably building some of them right now. Right. And those are the experts that'll be behind the scenes. But the real power of the technology is when you get it in a domain expert's hands, because the engineers know how to build the technology. They don't know how to apply it in the organizational construct for the outcome that that business or again, that organization's trying to conduct. And so if you get it in one and you get that one proselytizer, you're going to be amazed how fast everybody else in that organization or business are wondering how Albrecht Powell is making all of those gains when they know they're smarter than Albrecht is. Right. And they're going to see Albrecht using a calculator for the first time instead of an abacus. And guess what they're going to start doing? Adopting an AB or adopting a calculator. Right. That's the era that we're in with this technology right now, in my humble opinion.

Now, go ahead. I was just going to say we see AI touching these job functions and the pundits out there say, well, AI is going to take all of our jobs away. And I look at the computer age when it came in, in the 70s and 80s and it did a lot of the back office accounting stuff, right? Yet somehow we still have accountants and finance people, right? So one of the things that we're seeing is as much as this is technology, there is a human side to this and companies need to think about how this is impacting their employees. If I spend all day writing Python code and that's all I do, there's a risk there, right? Because now I have tools that can do that. So as an individual, I think we all need to be thinking about maybe I need to change my game a little bit, because how do I now become the person that's overseeing 10 bots writing Python code? Still need my skills and my value, but how do I leverage that to either do my job five times more efficiently or oversee others? We're seeing changes in universities, right? I'm University of Pittsburgh, I'm an adjunct faculty there. And it's a big debate in the classroom. Should we be using chat GPT and there's some professors that say, nope, you're not allowed. Others fully embrace it and there's no right or wrong answer. Well, actually I think there is. I think if you're going to, you know, a world class business school, frankly, you need to figure out how to use these tools, right? But it's, how do you use it in the right way and how do we teach differently? Like, you know, sending everybody home to write a term paper doesn't fly anymore. Right. So how do we, how do we do that in the classroom? How do we do more experiential learning? Right. So I think companies have to change the way they think about the functions that they do and how to integrate these tools in, to be much more efficient and effective. Are you seeing the same?

Yeah, I'd say, you know, one, from, from the school perspective, I would just say if a professor says no, we're not going to use this technology, they're actually hamstringing their students because when they hit industry or they hit, you know, government, whatever they're going to do when they leave the education bubble, they're going to get slapped in the face with, with this technology. They are going to run into organizations that have adopted it. So get them learning how to use it every day across everything that they're doing because then they'll come out with, with the ability to see new creative ways to apply it when they, you know, get their first job after college, so to speak. So industry's not waiting on the professors like it's time to get moving. One, two, I love your point. On, on, it's changing the way that education, you know, occurs and how you measure whether people are becoming experts due to the education or not. I'll use an example of my 12 and 14 year old who now have summatives, right? And these summatives are verbal. When they go to class, they sit down with the teacher and they do a math project, you know, on the whiteboard, digital whiteboard. More often than not anymore, they, they discuss, you know, the history lesson that they had and they're graded on their ability to recall the details and relay the concepts that they're supposed to be learning. I think it's really getting us back to humans being human because we have these other, other now artificial intelligence technologies that allow us to establish a baseline that's different. And the teachers have to get more involved, not less involved because they have to evaluate whether the students are making progress or not. And that human to human bond that that creates, right, that's something that goes back to what you said about music and arts and sports. I don't fear this technology in those areas either. Frankly, I'm afforded an opportunity because I work for an individual who lives in Pittsburgh, Thomas Tall U.S. innovative technologies, to watch how all of those industries are taking this technology not to necessarily make a prettier piece of art, not to necessarily make a better piece of music, but to acknowledge the individuals who are creating those things, the professional athlete who's performing at a high level, to keep them at that high level, to understand how to unlock their true potential, you know, whether it's on a field, whether it's on a canvas, or whether it's in a saxophone. Over.

All right, well, Dave and all right, be remiss interrupting the chemistry. One thing to maybe add even a little bit more spice to the conversation so far. David, we talked before a little bit around your experience around responsible AI fallacy. We led already in the conversation, like, hey, what are the responsible ethical ways to be using this tool? How do we go about doing it? There are some other ends of that spectrum of that conversation. So I'd love again to help make this a little bit more of a catalyzing civil conversation that's not happening elsewhere. What's your experience to share on that front?

Yeah, and I know this gets a little edgy and I get a little black and white with it just to spur the thought, maybe. But the reality is I hear the term responsible AI and I run from the door. I, I hear people talk about responsible AI and I think they've probably never tried to build a system to give you the proficiency and efficiency of operations you're looking for. Because from my perspective, the engineers that are building this technology should be focused on precise AI, on performant AI and when we start burdening and governing and putting regulations on top of something that is just coming out of the gate. I mean, we're just getting started with technology. And here's an example that I've had in the past. You know, we were going to push, we were going to subsidize a pretty powerful capability down from the Office of Secretary of defense into the 11 combatant commands in the Joint Staff. These are our four star headquarters that run operations in the CENTCOM AOR and the SOUTHCOM AOR, an Indo Pacific, you know, AOR. But, but as we were, we were figuring out how to get the technology to them, how to get a data professional into those organizations, because they didn't have a data lead yet. I had another organization that was the AI organization trying to push a budget proposal to put 10 responsible AI professionals, AI ethicists, AI philosophers into these organizations so that they could do it right. I was like, guys, we haven't even like integrated a technology down there yet. We don't even have a data lead and you're already trying to create a cottage industry that's going to take resources that are scarce away from actual development, away from workflow integration and you're going to start talking about management and governance. I think that's the complete opposite of what we need to do. I've seen too many times, I'll say on the federal side where we try and regulate before operate and I think we're danger close on this one to stepping into a place where we're talking about AI philosophers and ethicists instead of more engineers, more developers getting next to domain experts and actually integrating this technology, learning what works and what doesn't, and then identifying the policies that we need to either adjust, reinforce or delete because they're applicable as we move into the future. Over.

Cooler just to before we start transitioning here, because this has gotten a lot of good chat going, a lot of the questions going, I want to start shifting here to integrating the audience in here. But just to give you a chance to respond. Any reactions, thoughts to how to drive innovation thoughtfully, especially for folks that are coming in with zero experience but while also being able to integrate and hit like large public sector, large enterprise level needs of like, hey, we need to figure this stuff out.

Yeah, I, Dave, I completely agree with you on, you know, the, the fallacy on responsible AI but. I'm going to put a but into it. I think, I think we need to be mindful. This is one of the biggest game changers that's out there and we have to move fast. We can't hamstrung ourselves too much. But I do think every organization does need to put some thought around this. I think they need to have some guidelines around. If you're building for example, an LLM, what data are you training it with? Right. I don't necessarily look at that as being heavy regulation or such, but knowing that is the data that I'm putting into this, is it potentially prone to bias or just putting tools in that says, hey, let's take a look at the outcomes and make sure that it meets ethical standards. And I don't view that as much different than the ethics that we use for running our business today or the core values that a company has. I think we just need to, we Just need to be aware of it. And I think that any company that's heavily going into AI needs to @ least be thinking about it and have some guiding principles. I agree with you. Hiring 10 responsible AI leads before we even launch is probably not where the value proposition is going to be. Hiring one at, you know, at dod, for example, to help provide some guidance across them, that probably makes sense. Just my thought.

Over. Yeah, no, I agree. I mean, I think there's definitely a balance. I just tend to. And again, as early as we are in this journey with this new technology, I hate when people start to discuss limiting its applicability because. And you brought it up, Albert. HR as an example. Everybody's, oh, hr, you know, bias and like, wait, wait, wait, wait, wait. If you're not using a data driven technology, how are you making sure mathematically that, that those human biases aren't actually more prevalent than what you're doing? You know, I, I, I, I, for years, as you can imagine, have had to confront the individuals that it's unethical to use artificial intelligence in the conduct of combat operations, in lethal strikes. And what I try and remind people that we're actually deploying our men and women into these combat zones to defend America, to defend our partners and allies, to defeat evil. And if I have a technology that can help them do their jobs more proficiently, that can help them get home safely to their families without a loss of life or limb, then I think it's unethical not to use this technology and we should be going faster, not trying to figure out what, you know, the regulation should be before we continue slamming this into the organization, learning from it. But you know, I mean, I am talking about organizations and I'll say societies in America and with our partners, allies, who are grounded in at least a Western belief system, not autocracy. And I think if you're on the wrong side of that, you could definitely do evil and do harm. But those same organizations argued not to develop nuclear weapons as well. Right. And they have them and we have them. So ours needs to be better. Ours doesn't get better by.

Overregulating. So I think this is at a point where again, we've stirred the pot a bit. There's a good dialogue going on in the chat. I'll take a moment just to encourage folks to go ahead and again, if you haven't used the Q and A feature, throw questions that you would like for us to be surfacing up here and address on stage. Amongst we did get a couple of questions in advance from folks. We'll try and prioritize a few of those and popcorn back and forth between the live questions and the pre reg questions while folks get those going in there. Just 30 seconds for that. I'll share some stuff in advance if you've been enjoying these conversations so far. If you would like to be participating in more of these types of opportunities, there are three related ones of note that I would flag for folks. 1 Next Tuesday we have the forum, our fall forum coming up and one of those conversations is going to be around the responsible AI for customer satisfaction, right? So how to tee that part and throw that particularly interesting given what we were just talking about. We also have our Q4 executive cohorts of our Reset program coming up. So if you're looking for ways to gain experience putting these strategies and techniques to use, actually using them, think about how to build up those skills that coming up in December 4th and then our Q1 instance for our energy Transition workshop, which came off of our last Resilience roundtable, given the interest there, that one will be coming up in the last three February, last three Fridays of February. For that nexus of AI and energy, I've given enough time for folks to get going here in the chat. So we'll take one from our pre questions and then we'll popcorn back and forth between those and the ones that are coming in from the chat. So Dave Albrecht, how how does AI change organizational strategy? You guys have talked a little bit about, like how do you, how do you do the learning journey? How do you get going? How do you try things out? What does it mean.

Organizationally? I mean, I can start again. This is where the human piece comes into here. You've got to have a company, an organization, an ethos of growth and advancement. This is new, this is exciting, this is scary. Whatever adjective you want to use, the organization has got to get behind this and say this is important. We need to be teaching all of our employees, all of our practitioners about AI. It is here. It's part of our daily lives. And companies that are starting to implement they need to help educate the workforce on what these tools are doing and frankly help shape that it is a strategic asset to the business. Because otherwise people might get scared about it. They might, you know, oh my God, I'm going to lose my job. Whatever. No, we've got to think about how this is going to augment what we do and how we're going to do our business better, faster, more efficiently than our, our, our competition. The other change I think organizationally is, I think we're going to shift from, I'm going to call it predictive planning. Right. Where we do a five year plan and that's what we're targeting. We still need that. Right. But AI is now allowing us to do much more adaptive, much more agile, reinvent like every quarter as opposed to every year or every three years. And we've got to change our organizational thinking to be able to adopt to that and to be able to change and reinvent more quickly. And I think we've got the tools to do that. It's now shifting the mindset to do.

That.

Yeah. Your thoughts.

Dave? Yeah, no, I, I think you're spot on, Albrook. You know, as I kind of said, starting small but recognizing that project that, that you're going to start has to drive an outcome in an organization that's willing to accept it. You know, I mean, I, I'd say even just from a strategy perspective, hey, where do you put your cdo, your cda, you know, who does that individual report to you? And I have had many conversations about this. And the worst, the worst organizations, pardon me for saying it, are the ones that bury it under a cio. Because when it's under a cio, it's not focused on a business outcome. Right. If it's reporting to a CEO, a CFO or a coo, Right. It is, because that's what those individuals wake up every morning and every day thinking about. Not, hey, is it good enough? You know, are all the blinky lights being monitored and are we cyber safe and all the things. No, this is, this is a technology that allows you to drive new business operations, generate new business ideas and pick up on outcomes as you start stacking all of those small wins together. You know, I think you see that business transformation that you were discussing, you know, in particular, and so I think you're, you're spot on in how this works. It starts small, it starts with a pilot, but be ready to go because once somebody realizes the power of it, you know, more, more, more is usually the answer. And that requires a lot of different things. Sometimes it's going to require you to make significant investments in your infrastructure. That's one of the things that I'm very passionate about. And, and it's not that I don't like CIOs, you know, the CIOs that I've known, I just think somebody needs to hold them accountable. And when you start talking about, does the business have the infrastructure that it needs tomorrow to be AI Future proof. I don't think you can ask a CIO to answer that. I think you ask somebody who's driven on delivering outcomes if they have the compute, the transport, the energy, the cooling, all of the components that allow this brain to run at scale and ready for what's going to come tomorrow.

Over. So we're getting lots of conversation here in the chat and take Steve's points that at the end of the day we're talking about humans, right? Corporations aren't humans, AIs human. A lot of the behavior change that we're talking about has to happen through the leaders, whether that's a formal leadership role or an individual leading in their team in driving that change. So one thing that I think a few folks in the live chat have been asking about, both Juan and Gary are thinking around. Albert and Dave, you both were talking just around organizations generally, right? How should they be thinking about starting that learning campaign and iterating and moving it out if we were to start thinking around Public resources are limited, private resources are limited. Are there particular spaces or particular topics or particular things like this Seems like the strategically most relevant space for folks to focus limited resources on. If nothing else, just people's time. Where should those be focused for those greatest strategic.

Returns? You want to start on this one.

Dave? Yeah, I mean I really think that that, that comes down to a conversation with the senior most leaders. I mean I'll start out, you know, challenging the notion that corporations aren't human. I think they're very human. I work in them and they're staffed with people that I enjoy spending time with. And I think in any organization you find like minded individuals, you know, finding their way into that. And so it's impossible for me to separate a human from a corporation, an ethos associated with that, or like I said, a department and agency and activity. I think inside that, as Albrecht has mentioned and as we've learned and experienced, we're seeing generational change occur in the workforce itself. We're seeing more senior leaders who have grown up accustomed to having what, what we now call AI. But we didn't, you know, five years ago, 10 years ago, you know, that, that, that they live their daily lives, you know, in, in a world where this tells them how to operate most effectively. And every application in here is using data to some extent to help them identify a risk, an accelerant, you know, a new method of doing something. So I think you need to find those courageous leaders inside an organization who are ready to help you change the entire thing. I Know, that's what I did when I found a couple generals who were ready to do something better. We got the technology in their hands. They showed their enthusiasm for using this. They showed their ability to operate faster and better with this technology. There were a lot of people watching what they were doing. And when they understood that it was safe, that they could start trying new things, that they could start leveraging this technology to do something in a new manner. And even if they failed, it was okay. Because what we learned about the infrastructure that was in place, the data quality associated with it, all of those things that just trying to put this technology into play allowed us to uncover, actually allowed us to go backwards and start policing up some of those gaps and lacks that we had. I don't think you can over plan and then go. I think you just have to go and then continue to just understand this will always be a messy game. That's why there's a place for data professionals, you know, into the future, because data is messy. You know, I remember when we started Project Maven in the Department of Defense, one of the things that we went into Microsoft and they told us, they're like, if you thought you had a data problem before AI, just wait until after. And we kind of laughed and we're like, haha. And now like, oh man, like, how prophetic was that? And so I just think you got to get going, you got to start small, you got to be ready to scale fast. Find a courageous leader that wants to uncover gaps in their formation and that will empower you to go clean those up as they continue to take the next Ridgeline.

Over. Sure. And any particular factors or anything in there that you might hone on.

Or. Well, just one thing I'd add to what Dave just said. The concept of companies having an innovation lab or an innovation garage around AI, I think is so important a place that they can test, that they can try, that they can play with this technology and try out new ideas and new use cases, I think is so incredibly important. You know, with respect to industries, we're seeing benefit across all industries. Right. There's some, you know, you look at, you know, financial services at call centers. There's a lot that's being applied there to help, you know, drive. I think companies need to sit and not be worried about is my industry ripe for this? They need to look at their blocking and tackling functions that others are applying this technology to and making sure that frankly they're keeping pace. Right. Or they're going to fall behind. But again, it's the innovation side where companies have got to think bigger and broader and bolder about what could we do with these to change Again, not just our company, but our whole industry. Those are the companies that are really going to succeed. Last point, Dave, thank you for bringing up the foundational data piece. Everybody's focused on AI and AI experts. And what we're seeing now is the folks that do data governance, data management, data quality, they're suddenly now a whole lot more important because if the foundational data is not good, the AI stuff just does not work. So we need both and we need both at.

Scale.

Yeah. All right, so there also be remiss in there. I think a few folks have been talking around, you know, cyber security again like the, the risks of this. Right. So again Dave, if everyone's going full on this, right, how, how to do that intentionally, how to maintain, how to be defensive around that. And I know Albert, you've also shared around the, some of the dark side of like oh great, let me use this to experiment with my, and like building on my business model. And now my business model became part of the training data that's publicly available out there as well. So from a responsible defensiveness around, just like other people are going to be using AI, how, how to use it intentionally. Any, any experience that you guys have to share.

There. I think we're woefully underinvested in technologies to monitor the performance of models in real time. In real time and report back when we see slippage as to why so that you can roll two which is a new model, or roll three, which is a new model and continue operating as you do the forensics for why we'll say Model 1 began to be non performant. If I were an investor and I do work for a couple of VCs to be completely, you know, clear, I would be putting additional resources into that because we do as we charge into leveraging this type of technology in, in more and more high consequence situations, our ability to say it is performing as it's supposed to, or wait, it's not. Let's, let's, let's identify that at the point of not performing and inject on top of it the new model to continue performing. As we go back and understand did we have a data, you know, flow issue, did we have a bad data issue, did somebody poison our data set? And you know, there was a, you know, kind of trojan horse, you know, in our midst that we weren't aware of that was uncovered in time and space that was Opportunistic, you know, to a business or nation state criminal actor. I think doubling down in building businesses around independent monitoring of the performance of models and live workflows is something that I haven't seen industry take on enough yet. There's a couple out there that have started but they're struggling because they're not getting the purchase orders. Probably because we're paying responsible AI people. But that's a different conversation. We already covered. This is low blow. I know, Albrecht, I'm sorry, but I think that's an area that we could continue to improve.

On.

Over. Totally agree. And we're seeing the same thing across industries. Chris, you brought up the comment about working on a business plan and seeing it out and training data. There's also the personal side of this. I mean we're all trained to protect our credit card numbers, our Social Security numbers, our logins, our passwords. We're not trained on how to do and use these AI tools just as humans. Right. So going out into ChatGPT just in the open ether and using it on the web, that's one thing. Having a subscription and turning off the access, feeding stuff up to training data makes a big difference. Right. So again, some of this is we've got to do more to educate, you know, our workforces on how to use these tools, how to use them. I don't want to use the word responsibly, but smartly and, and, and use them to optimal.

Benefit. Yeah, and that's I think a lot of what the conversation has been around. Right. I'd say that aligns with one of the other questions that came in in advance as far as thinking about ethical use, thinking about like hey, how do you do this in ways that are helpful? And actually, full disclosure, Dave did not actually we did not brief on any of the model methodology, but I've seen a few folks chatting in here about the model behind me on the screen. So if anybody has been thinking about resilient organizational architecture, how do you identify your organization's core values, what your vision is? How do you identify unmet needs in marketplace? How do you stand up experiments and turn those into entrepreneurial ventures? Happy to chat with folks about that. We'll drop links to the resilient organizational architecture diagnostic in the chat and send it as follow up. But maybe before we do getting to wrap up here, any last thought just around how to like with any other conversation, is there anything different about thinking about our norms, our ethics as we approach this technology versus any other different technology? Do we need to stand up a new governing body, do we need to have new different councils or new different ways or do existing ways of. Hey, if you're thinking about engaging your customers ethically, you're probably already capturing this. If you're thinking about the ways that you're integrating for your stakeholders, if you're a municipality or a federal agency, as long as you're engaging in ethics baked in there, this would naturally come up as that. Or does AI, given the novelty of it and the newness of it, does it need its own separate, hey, let's make sure we're doing this ethically space in an.

Organization. You know, I'll jump in and I'll just talk about my experience at Palantir right now. You know, one of the hidden gems we have and people can find him on the web and you can listen to some of his conversations and podcasts, YouTubes, whatever is an individual named Courtney Bowman who runs our privacy and civil liberties, you know, organization. Even a company like Palantir who does business, you know, unabashedly in support of our service men and women, our intel community, but also, you know, Tampa General, you know, Institute of Health in the uk, I mean globally, both sides of the national security and commercial sectors, we rely on Courtney and his office to ensure that we do have a North Star, that we do, you know, move in a way and build in a manner that does respect the individuals privacy and civil liberties. We won't enter into an agreement with somebody who doesn't match our company's, you know, core ethos again unabashedly, even if we could make a lot of money doing that. And, and I've, I've watched the organization deliberate and I can tell you that, that when Courtney weighs in, Courtney weighs in and, and that's where you know, a decision is brought to our, our CEO, who's very opinionated and allows us to be. And so I think inside organizations, as long as you're kind of working for an organization that, that is righteous. I'll say that, that you believe in their ethos. I, I think, you know, the, the, the pieces of the organization that have to be in place to either be publicly traded company and you name the sector or to let the market vote for, you know, what, what businesses they want to, you know, work with. I think, I think market, the, the capitalist market, you know, votes with its pocketbook for winners and people who are aligned to what those western values are. And, and I think again I'll go back to over regulating early setting up governance and management bodies before you even really started Unlocking the full potential of this technology makes people worried it's not safe to follow their intellectual curiosity to begin integrating some of these technologies that will allow them to be more proficient and efficient in their operations.

Over. Spot on. I have nothing to add to what you.

Said. I was going to say. Yeah. Recognizing time. Just last parting thoughts from either of you gentlemen. A quick closing thought of something that we haven't covered yet. We're about to leave. A conversation about AI and how to integrate it effectively in the workforce. A topic that we haven't been discussing elsewhere or something that's like, hey, make sure the folks are checking this out and any last.

Thoughts? Hey, Albert, can I go first to leave you the last.

Word.

Play? Sure.

Thing. Yeah. You know, I just say like, it's, it's. We think about this in these profound manners. We think about it, you know, is AI going to launch a nuclear weapon or not? You know, but, but what I'd say is it's actually helping generationally start in a higher order. You know, my son and his buddy were over here two weekends ago and, and they're like, you know, my 12 year old, you know, dad, we want to start a lemonade business and we've decided our launch is going to be on Halloween in the golf cart around the neighborhood. And this is what we need and this is what. And I looked at them both, I was like, where did you guys get this? They've got an amazing business plan laid out. They've got their trajectory for this is what we need to generate so that we can get canning. And once we get a canning machine, then we need this much scratch square foot. And this is how we have to do it. Like these are 12 year olds thinking about this. And it's because they were iterating on Grok, right? They were going to Grok and they were iterating and by the way, then they came in and asked for my chat GPT, you know, login, because they wanted to compare results from one large language model to another to see if somebody would give them a better idea or if one of the models, you know, was telling them this part's wrong. And so like trust the kids, trust the new generation that's coming in. They are starting at a complete different level because the tools we're giving them are allowing them to unlock their true potential, right? And their creative application. While they were building that, one of them was over on, on Gemini building like logos, you know, for what, what each flavor was going to be and how they were going to do this and coming up with marketing plans. So like I'll stop there. I just like we have a new generation who understands when something's confabulating when it's not. Give them access to these tools so they can unlock its future in their own.

Over.

Cool. Alright, closing thought from you before wrap us up here on.

1.2.0. It is so hard to go after you Dave because you always have such incredible insights. Three points. One, I need to go back to your, your your launching nuclear missiles. Agreed that there's some questions there but an interesting topic that we always talk about is AI in in the military and you know there's something called the CWIZ system close in weapons support on ships where if a like a missile is launched it comes over the horizon, they have like 2 sec 2 or 3 seconds to be able to respond to that. We have weapon systems that can shoot that down autonomously. We want that there's a need and a use for all of these that saves lives, that is benefit. There's areas that we need to be careful. Second, your example of what your kids are doing is staggering. That is the definition. That is the litmus test that I think we all should be using as to how and why AI is so important and so prevalent for all of us, for our industries, for our companies. And let's all take back to our respective companies that example of a couple of kids that are creating a new business model and leveraging these tools. And if we all took that and applied that to whatever it is we do on a daily basis, we make a huge.

Impact. Exactly. All right, well it's been robust conversation again for folks who would like to join more of these kinds of conversations. We do have a few coming up. We'll go ahead and share out here on the QR codes and you'll have links in the email that goes out as well. But this way at least for those who need via recording and would love for folks to join us either next week at the forum to continue the conversation to get more into the skills on that in our Q4 executive cohort or to dig into this as the other was a lot of conversation around energy today and how to do this and how to think about this intentionally. We'll be make sure to tackle into that conversation in more depth in our Q1 strategic disruption workshop. It is top of the hour. Want to be respectful of everybody's time. Thank you all. We did not get to all the questions. I will endeavor to follow back up them but make sure keep the conversation going again. Trying to have these civil conversations that aren't happening elsewhere and just advance the. The edge of knowledge in ways that are going to be impactful and help everyone. Thank you, everybody. Albrecht, Dave, been a pleasure. Thank you to the team. Ravi. Thank you, ntia, and we will see you all next time. Take care.

Y'. All. Thanks.