AI Proficiency: From Users to Builders

Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Bluesky to stay up to date with episode drops, behind-the-scenes content, and AI You can learn more at practicalai.fm. Now, onto the show. Welcome to another episode of the Practical AI Podcast. This is Daniel Wightnack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer. How you doing, Chris? I'm doing good today.

How's it going?

It's going great. I'm continually impressed by just the amazing AI work that's happening around, not only on the coast, but in kind of the, I guess, the heartland, the middle of the country where a lot of actually the large enterprises of our country are located. And we have a guest related to that today, but I do want to remind folks, we're also involved and a sponsor of the Midwest Midwest AI Summit, which is coming up October 15th in Indianapolis. Really cool event, Chris. You were there, you saw what was going on. There's tables where you can sit down with actual AI practitioners and, you know, rather than just hear a bunch of talks, you can actually get feedback on what you're doing, suggestions, design, et cetera, and hear great talks. So recommend people check it out, Midwest AI Summit. You can use the code PracticalAI20 for 20% off. But we have an amazing AI practitioner from close by my area, more towards Cincinnati. Mike Lewis was on a previous episode with us. We got a ton of great feedback on that episode. He's Chief AI Architect at Tier 1 Performance. Welcome, Mike.

Thank you, Daniel. It's great to be here again.

Yeah, like I say, we got— I had multiple people comment on just the utility and insights that they got out of our previous discussion. And just for context, because I think it's so impressive what you're doing and what you're involved with, could you just set a little bit of context for kind of the types of projects that you work on? Like, give an example of kind of some of the types of companies that you work with in relation to AI initiatives?

Yeah, sure. So, Tier 1 Performance is an end-to-end organizational performance and transformation What does that mean? We kind of help some of the— most of the largest organizations in the world rethink transformation. I would add in though that we have a non-traditional approach and a multi-decade track record of success there. So the types of clients we'd work with day to day would be like Google, Eli Lilly, Takeda, Air Force, I mean, just most of the Fortune 100s. And we see everything related to transformation. So obviously AI would just represent a sliver of the types of challenges we encounter as an organization day to day. But actually that sliver is my whole life.

Interesting.

So I don't really see all the other stuff. I kind of get a glimpse into it every now and then because more and more AI touches everything. But we are sort of a transformation partner. And so you can just imagine how the phone starts ringing when, you know, a massive disruptive technology is birthed into our lives, whether we want it or not, right?

Yeah. And a part of that I think people are struggling with on the transformation side is there's just so many sources of information. There's so much noise around like what you should be paying attention to, what people are actually doing, what You know, people are saying they're doing, what is hitting the news. Like there's all sorts of a range of things, and obviously a lot of that can be noise. And one of the things I've appreciated in interacting with you over time is that ability to kind of pick out some of that signal from the noise and develop some kind of structural ways of thinking about AI and transformation. In an actual real-world organizational environment, in your kind of day-to-day, obviously you're hearing a lot of things, both from customers that you're working with, the news cycle, the things from, you know, Anthropic, OpenAI, Hugging Face, whoever that's coming out with things. What does your process look like, or how do you feel like your rhythms day-to-day help you kind of isolate some of that signal, maybe kind of distill down some of what you should be paying attention to? Any suggestions? I think it's something on all of our minds, certainly on my mind.

Yeah. Well, maybe I'm very good at this because I am just like the weirdo oddball guy who stumbled into this at the perfect moment. I mean, Daniel, you know this about me and your audience might remember I was a portrait painter for almost 20 years. I mean, from when I graduated college till 2016, I painted portraits and I owned a fine arts company and apprenticed a bunch of artists. And I only got into this because I had heard about the DALL-E model from OpenAI and applied to become one of the commercial arts beta testers. And I lucked into that. I got an email one night at dinner and I was like, oh wow, I just got access to this thing no one had heard of at the time. And so, and I knew a little bit of like Python and coding or whatever. So enough to like fumble around and get things installed. And I felt in love with language models. I mean, almost instantly, I just couldn't believe all the new things you could make computers do. And, you know, this is before we really thought about them as chatbots or whatever. In my mind, it was like something you plug into code to just manage edge case weirdness to, you know, pick a path and make your app keep working. And, you know, context models were minuscule back there. I'm running things locally on my gaming computer, you know? And then medical records companies start calling and I build this book of business and Tier 1 acquires my company. Here's the only reason that's worth unpacking is to say, like, I don't have this muddy, long history in the industry where I had opinions about everything or like a solid way of thinking about it. It was all just fun toys for me. And then I think the way I've made it through business my whole life, I've owned maybe a dozen businesses, all small, but is I've always just, I've just had a firm rule is like, I do not concern myself with things outside of my sphere of influence. So, you know, there may be this big thing going on at the edge of the AI sphere, but the reality is like, if I, if I don't have influence or if it doesn't impact like the way I'm interacting with my clients, if it's not, then I just don't, I just ignore it. These things are not going away. Like I know for sure, whatever this is, it will be here probably forever. And, and in, in my experience, I'm a much better worker. I mean, I can fool almost anyone into thinking I'm a smart guy when I'm using AI and I'm aligning context to like whatever my work project is. I mean, it just works so much better, uh, using these tools. And I just have that conviction. That's probably true for, for most people. So, so Daniel, my trick is really only focusing on like what, what directly impacts like the things going on in my day-to-day. And, and at tier 1, what that is, is Workforce enablement, performance, you know, basically not worrying so much about tools or whatever, but just outcomes and performance.

Yeah, that makes a lot of sense. And one of those things, you know, last time we talked, I looked up the date, it was like 2024, right? Yeah, sometime in 2024, obviously a lot has has changed. It's impacting all of us in different ways. At that time, we had a discussion about what's, I think, doable, scalable, et cetera. You've, I know, spent a bit more time recently thinking about this concept of AI proficiency. And I know myself, obviously, if you're doing AI things, you're searching them, you get targeted online. I'm all this all the time seeing these like ads for whatever it is, Harvard School, like what the engineering or what becoming AI proficient in your business or whatever the course or thing is. And so there's a lot of people trying to understand this because of, like you say, the wide-reaching impact across the workforce. So next week, Our company's doing an AI accelerator with one of our customers. I know that there's going to be a whole range just from discussions of AI proficiency, but general, other than thinking, well, I know that there's that range, right? It's another step to go and say, well, in light of that range of proficiencies, What do I need to change for different proficiencies? How do I advance people through proficiencies? Do they even need to advance through different levels of AI proficiency? What is enough for some people? What is too much, right? So I don't know if you want to help us launch into this topic, maybe just starting with a little bit of definition of what we mean by proficiency would help us just so that we're all talking about the The same thing.

If you don't mind, Daniel, I'd like to even go a step further back and just give you a sense for how and why I even started thinking about this. So if, if inside of our, if at Tier 1, we have, you know, most of the largest, we and 3 or 4 other companies are helping the largest companies in the world navigate this disruption. And we also see the same story playing out inside of every company. Like, I mean, I could, I could tell you 95% of the companies, I could walk through the timeline and you'd say, yep. I saw that at fill in the blank, fill in the blank, fill in the blank. They've made all the same mistakes. That, I, it's kind of watching that happen, sort of helpless at the periphery, you know, as a consultant, you, you don't have a lot of control, but, you know, and you don't have perfect ideas either, you know, so everyone's just sort of watching this thing unfold and like, oh, I really thought that was gonna work, but that didn't stick. And, and the big thing that I focused on early on was developing tools that help people learn and get good with AI. So I, I developed our, you know, solution archetypes framework where we kind of like tried to pin down. We talked about that on the last episode. And then I built our Superuser Habits Index, which is basically, it's like, hey, these are the 24 habits we see superusers displaying. And, and, and then I built a coaching tool that people could use. And, and, and in my mind it was like, I'm gonna put all this together and turn people into superusers. And guess what? It just kind of didn't really happen. For some people it did, but I couldn't help but feel like Most of them, it was gonna happen anyway. And at the same time, I'm watching, I'm at the same time, I'm sort of reading all the Substack articles on it, watching things unfold at companies, looking at my work history, you know, thinking through the people, looking at the statistics we create or, you know, stuff in the literature. And then I think the breakthrough moment for me, I was invited to a bachelor party with some young guys who used to work with me. I, you know, never again. But, I brought 200 pages of research citations and synthesis. So this is not 200 pages of research, it's 200 pages of like the name of a paper, a URL, and maybe a sentence about why it might be relevant to my work just to read during that week.

During the bachelor party.

I thought I was probably going to need to pull away a lot. These are all younger guys. And I did. I read it all. And there were just patterns in there that answered a lot of questions for me. And the breakthrough moment with all of that just sort of happening and within like 30 to 60 days, that stew forming, I read a Substack article. It was Peter Yang. I'm kind of skimming my screen here for the details on it, but basically it was called Your New Job Is to Onboard AI Agents: How AI-Native Companies Actually Operate. And And they just had this way of thinking through the L0 through L3. So I didn't invent that. I read it in that subset, L0 through L3 classification. So L0, L1, L2, and L3. And the L0s are the people who aren't using it. And the L3s are the ones who are, you know, just making a difference across the entire enterprise. And we all know those. And then they labeled L1 and, and L2 though, like knocked me off my chair almost when I, when they talked about the concept of a non-technical builder. And, um, they just gave a name to a thing that I, I just thought, wow, this is, this is something we need to really focus on is like the idea of a non-technical builder. We're all focusing on converting L0s, people who are disengaged, and we'll unpack what that is. I think we need to, 'cause there's so much important stuff there. And they're trying to get people from L0 to L1, and L1 is what everyone is, what everyone knows. It's the user. It's the one who's like, I couldn't live without this. Like, and they kind of feel like it ends here. And then that leap, you know, the big jump from like, okay, someone who like uses AI to wait a minute, no, this is a non-technical builder. This is a person building durable solutions that emulate the company's work in an uncanny way. That jump is actually the one that now I sort of run around evangelizing, like, this is actually where you need to focus your energy. But so, so Daniel, I just, I guess I unpacked all that to say, like, for me, I had the, like, the stew, I guess, is the way to say it, of like, you know, the, the guy who like didn't have a dog in the fight, built a company on accident. It was bought by tier 1. Now I'm here, I'm, I'm inside of all these companies trying to help them make sense of this thing. And then, you know, all these external inputs sort of just clashed. And, and, uh, so now we, this is, this is the framework we, we push and advertise and help ask people to think through.

If you've been listening to the show over the past few months, you realize how transformative agentic AI is. Whether it's Claude Code or Hermes Agent or your own custom software that you're deploying for operational efficiencies or as a product into the market, these systems are transformative and it's where the industry is going. But agents have a lot of risk associated with them They have agency. They take action within your environment, within your infrastructure, within your systems. And that's why security teams and governance teams view it as risky to deploy these agents within high-impact environments in industries like manufacturing, logistics, financial services, in the public sector. And that's why I've personally been spending my time working on a product called Prediction Guard with a great team of AI engineers. Prediction Guard is an AI control plane that you host within your own environment, whether that's on-prem, hybrid, or in your cloud VPC, even air-gapped scenarios. You can set custom AI policies or those that are aligned with NIST and OWASP standards, and those are then enforced automatically across every handshake Across every agent, whether that's being powered by your own self-hosted models or models in your cloud environment like those from Azure or AWS Bedrock. And all of that telemetry goes back to your monitoring and observability system. I would love for you all to check out what we're doing, schedule a demo and a call with our team to find out more at prediction.ai. Like I say, schedule a call today. We're happy to show you how this works and why we think it is so beneficial to the market. People already have this deployed in production, powering transformative agentic systems across a number of industries. So check us out at predictionguard.com/practicalai. That's predictionguard.com/practicalai. Awesome.

So, Mike, wanted to ask you a question. As you are thinking about how you get the employees of one of your client companies moving from the L0s up to the L3s, I want to throw another variable in, and that is the attitudes that people have, which are very varied in terms of like, there's always people who are excited. As you mentioned before, there are people that will naturally want to go to that L1, L2, and up to L3, but there are also people that are going to push back hard on that. And so when you're kind of coming in and you're trying to start your process of helping them achieve that, how do you bring along the L0s that don't really want to move, or even the ones that have done a little bit, but they're also pushing back on that? Maybe they fear for their jobs or something like that. So there's that kind of L0s and maybe some L1s. How do you get over that process as you're trying to bring everybody into this to kind of level up through those? Any thoughts? Can you kind of throw that in as you're talking about your process?

Yeah. I love that you're asking me about L0s because the reality is we don't try to move in L0. Here's the thing. The article I mentioned, the Substack article, I disagreed with 98% of what I read in that article. I just latched onto the concept of an L2, a non-technical builder. That name resonated with me. The L0s in that article, at least at Ramp, the way I interpreted what I read, they were saying that, hey, if you're an L0 at Ramp, of course it's a software company, you know, if you're an L0 at Ramp, that's grounds for dismissal. You have until this date for our system to assess you as an L1. And man, that just broke my heart because it's kind of like, okay, wait a minute.

Terrifying.

Let's, let's back up. Let's back up. Name another technology that we would say that's true, but can you imagine you just go company-wide, hey, by the way, everyone, we want you to geek out about Smartsheets and be great at Smartsheets by the end of this, you know, year, or, or, you know, AWS, or, you know, like just, it's ridiculous. This is just, it's another technology like that. Yet I think a lot of executives are kind of scrambling and panicking and they're saying, hey, guess what? We're gonna be AI enabled, we're gonna be AI activated. Everyone in our company needs to speak AI by X date, or there's not a place for you here. And so first of all, you remember I read 200 pages of science and a lot of it was like, it, it was just about learning. It wasn't even about AI. I mean, and, and, and so, um, threat framing actually does slow down adoption. So, you know, if you put, if you back someone into a corner and say, learn this or there's not room for you here. They will not learn it as well as if you came to them and said, let's figure this out together if it interests you. And I will acknowledge that there are some positions where you absolutely need an AI-enabled individual. It's true. Like, you know, and so there, there are cases where an L0 is not going to work in that role. But I think we're going to realize in 10 years, looking back at all this, that it's just not true in every role. And I think the one that hit me the hardest is my wife, probably the smartest person I know in the world. My wife, smartest person I know. And she has no use for AI in her job. She's a nurse. And she kind of just, I know what she does. She pokes people in the arm with needles and, you know, some other things and there's medical records. Maybe someday it'll be useful to her. It's just really not right now. It would be in the way. And I started doing research on the concept of L0, people who are disengaged. And I realized there are multiple buckets. So when we talk about an L0, it's not a thing. So an L0 is not just someone who doesn't use AI. I found 5 major categories. So one is the performative user. They've been doing this. They've been nodding their head in all the trainings, the meetings. They've logged into the thing. They've submitted a chat and they've gone about their job. Like, you know, like they're not really getting, and And hey, they're still getting their job done. There's the disinterested. I think the moment when this hit me the hardest was when I was looking at all the research and I saw a chart that showed the adoption rates of personal computers in the office space back when PCs were first released and AI. And there's almost the exact same adoption curve over the same period of time. It looks like it's happening at the same rate. And when I saw that, I thought, How silly is it like to think back to that? And, and, and like, would no one is in a panic because the people in their company haven't figured out personal computers now? Like it took care of itself because everyone wound up with one. And the same thing is happening with AI. Everyone went home and, and just created a ChatGPT account or whatever. It's free. This is not true with your CRM. This is not true with Smartsheet. Like everyone didn't run home and create a Smartsheet account because, you know, like they just gotta have one. And so, you know, but the disinterested are still detached there. Okay, here's another bucket, the too busy. I could give you 10 names at tier 1. We almost don't want them running around trying to figure out AI because they are just so critical in their role. They're so good at what they do. It's just like, it's like a, it's a don't break it or don't touch it. It ain't broke type role. And, you know, who cares if they're using AI or not? And I think that is also really hard for the people who are, are here in communication. Use AI or you're outta here. And then they can look around and see the people who like, very often it's some of the highest performers in the company who don't really care about it because they've, they're in a rhythm. Um, so those I kind of push past, but the next 2, these are still buckets in L0. These are people who aren't using it. Really matter. And there's something you can learn from these people. And I think when you say like, hey, you're gonna be doing this, you're gonna be using AI at this company, they will just zip their lips and they will not, they will nod their head and, you know, and they'll say yes, you know. But here's the thing. First of all, we have the job fearful. And Chris, I think you mentioned this, you might've mentioned this earlier, the job fearful. What do we do? What do we do about the job fearful? First of all, very often these people don't really understand how the tool works, and they're under the impression that if they use AI to do their work, it's going to learn how to do what they do and replace them. And that's almost never true. I mean, just if you kind of know how these things work, I mean, you know, maybe self-driving cars. I could think of a few examples of like, The AI is actually learning from watching us, but my vote doesn't really impact whether or not it's gonna replace that profession someday, you know? Um, and so I think with the job fearful, there's an opportunity to help them see like, hey, you might be in the middle of a self-fulfilling prophecy here. Like, if you continue to push back against this, you might lose your job, but it won't be because the AI replaced you, it'll be because Joe, who's willing to use AI, will replace you, you know? So, so that's one way to address an L0, but another one, And this is actually 6 out of 10 people that we talk to, and I've seen this number in research hover around 60% in more than one place, are the quality disappointed. This is where I think every executive's ears should turn on. They should lean forward in their seat and they should realize, do not dismiss these people. If they are resistant to AI, there could be some very valid reasons why. And I met a person, uh, at a, at a company who Um, her complaint was, it's not good enough for what I have to produce for my work. And she kind of showed me and she's just like, humans do this better. And it was a, it was a marketing role. It was copy generation. It was some other stuff. And she was just right. And it may not stay true forever, but no one had ever actually just, no AI expert had ever just sat and listened to her. And I, I walked away from that conversation with just a brand new perspective on how much value there is to mine from the group of people who say, It's not ready to do this work yet, or either it's not, or they need better tools or better training. But you can't really do that until you listen first. And so, and I also don't think about it like a ladder. I don't think an L1 is better than an L0, and I don't think an L2 is better than an L1 or an L— actually, if any, it, I don't see it that way. I just see it as, as kind of a division, um, within an organization for how different ways of thinking about the toolkit. Yeah.

And maybe one follow-up question on that. I'm wondering, you mentioned maybe part of the stumbling block here is the quality disillusion or however you put it, you know, getting into these tools. I sometimes, I also wonder about the way in which, like, I would put myself into the category of the AI people. Like, yeah, I'm building a product, right? How much of this is Us providing AI in a way that is just another thing that they have to add, like a different tool that they have to add in their workflow versus something that actually works in their workflow. And what I mean by that is, you know, if I'm, to give an example, you know, we're working with an organization now, they have teams of translators, we have essentially a team of digital agents. That helps translate with it, with an agent harness, and they use existing tools. They go through the, you know, drafting and quality checking and post-editing. They do these cycles, but we, we don't actually say, hey, go to this different interface that you've never used and do things the AI way. The, the sort of agents work and then stuff pops up in the translation management system that they already use, right? And so what they see is, yes, there's change, right? But it's actually like allowing them to do the thing that they want to do with the tools that they're using in a way that actually they want to do that more. Now, that may not always be possible. Like, there may need to be a shift of like how people, you know, the interfaces that people use to do that. But I don't know. Do you have any experience with that? Like how much of this is we're asking people to use yet another tool and they've already got tools that they like, whether they're AI or not. Right. And how much of it is the actual AI output? I don't know if you have any sense of that.

No, I hear, I think I hear the sort of the spirit of the question. And so this is where I land, you know, when it comes to You know, how do we, who do we train to do what and what do we give them and how do we, you know?

Yeah.

And honestly, at this point, I just feel like that is part of the noise to me in all the conversations. The main thing it all traces back to is executives are complaining, we're spending too much on all of these licenses. We don't even understand what we're buying. I can't measure, I mean, yeah, I see this or that anecdotal piece of data, but I can't measure like the real impact. Like what is the value of this investment compared to the cost? And so we just say like, hey, whoa, let's zoom way out. Let's zoom way, way, way, way, way out. And let's think about where we should actually focus. I mean, if we've already said, okay, L0, we understand some people just really aren't going to be using it maybe ever, but definitely for a while. Okay. Let's not even think about them right now. We got the L1s. This is kind of who you're talking about, Daniel. It's, you know, it's the people who like, Believe it or not, it's the people who would say, you could pry it from my cold dead hands. I could not live without this. You'd think that must be like an L3 or whatever. No, no, actually it's just people using it, but we can look at that and we don't really understand the value there. I mean, they, they don't want to lose it, but they're just doing their email faster or they're interpreting technical documentation. But the company can't look at that and say, there's $4 million we saved or whatever. It's just so So we say that's noise too. That's going to happen. It's going to take care of itself. I wouldn't focus too much on that. When we talk about L2s, so at Tier 1, we have a, we have a process for assessing and identifying quality L2 candidates. Now, Daniel, I'm going to work my way backwards to your actual question from here. So the idea is with an L2, this is a person who you might think, oh, great. How do you assess them? So they have some aptitude for AI? No, we really don't even kind of start with any sort of technical aptitude assessment. What we care most about is. Company DNA. Do they understand the work in an uncanny way to the, to the company? Do they know which cell of a spreadsheet to like fight over? Do they like, do they know how to deliver the work in a way that the company wants it delivered? Is their work style uncanny to what we want? Like, that is, is absolutely necessary before we screen anyone for L2 training candidacy. And here's the reason why. Right now, the wrong people are in the driver's seat. It's the, we call 'em the AI excited. The AI excited are the ones who are getting all the attention in the company. They're running down the hall screaming words like nano banana and Mythos class models, and everyone's just like, well, they must be the, they must be the person, you know, who should be building things. When in reality, what we see, Daniel, is these people who maybe by definition are often distractible, um, and not very focused on the work. Don't know exactly how to align an agent to work in a way that is absolutely uncanny to what the smartest and best and most qualified SMEs inside of that function would call good or accurate or what we want done. And so we say we wanna start with, it doesn't need to be the SME necessarily, but it needs to be someone who, who in the Venn diagram, like, Can replicate the work the company needs done in the way they do it that honors the brand and has the aptitude to learn these models, the interest. And though, Daniel, we only want one of those on each team. We find that it is not, you, it is not a 1 1 2 thing. Uh, too many chefs in the kitchen. If you have 2 L2s on a team, they are not necessarily as good as just an L2, a few L1s, some L0s. And we wanna make sure every L2, every non-technical builder has an L3 within arm's reach so that they can double-check the work. The L2 can make it uncanny, but the L3 can make it scalable, durable, make a thing that won't break, um, or doesn't break laws or, you know, violate, you know, governance policies. And Daniel, this is like your whole world, right?

Mm-hmm.

Um, but so this is kind of what we tell, and the other nice thing about thinking about it this way is if you could see like, well, we really only want one L2. Inside of each, you know, like high-performing team. That also means we don't have to think about as many expensive licenses, and we can focus training on pre-qualified candidates. So if we're just letting the L1s happen naturally, we're saying, okay, L0s will deal with them someday, but what can we learn from them now? And L2s are really the ones building the machines that we want to work in a way that honors Honors the work. And think about it, how many tools have you used? It's like, who built this? Where'd this come from? Like, you know, these things tend to last for years or decades, and it better work right, you know? Anyways, particularly when you're building something that works at scale.

If you're listening to the Practical AI podcast, I'm guessing that you value practicality, not just the hype around AI, which is why I think you should check out the Midwest AI Summit. This is an amazing event. I'm going to be there this year. It's happening October 15th in Indianapolis. This is an event like no other I've been to. There's actually an AI engineering lounge where for free you can go up and get expert advice from practitioners and get feedback on your architecture, your design, your agentic harness, whatever you're looking at. That you can get feedback on in real time in between amazing speakers that are on the main stage. So don't miss this event. Again, October 15th in Indianapolis, and you can use the code PracticalAI20 for 20% off. So go to MidwestAIsummit.com and grab your ticket today. Use code PracticalAI20 for 20% off. Midwest AI Summit.

So Mike, you got me pretty interested and I'm really thinking about this kind of L2 process that you're describing. And I have sort of a, I wanna rephrase a little bit in my own words and kind of finish with a question from there. And that is, it feels from what you're discussing, what you're describing there, that that L2 kind of has the knowledge of the value that the company is producing locked in their heads. And so they may not be the L1 that's running down the hallway screaming, hey, AI is cool, we should do it. But they're the ones that are fundamentally and historically bringing that core value into the products and services the company's trying to produce. And that's kind of how it sounded to me. And whether or not they're into AI, they have that. So it seems like you're trying to get kind of the L2s to be able to best use these capabilities to enhance their ability to drive value creation in the company. Is that a good way to interpret that? And, you know, depending on what your answer is, can you kind of give me a course correction or can you kind of, you know, kind of go down that path and explain it more? Because I'm, I am pretty keen on that idea.

Yeah. Great. So, you know, an L2 is a good one when the SMEs in the area and don't complain about whatever the AI tool they created, you know, like whatever its output is, you know. You know that they are a good L2 candidate when they can align an agent so that, you know, when it creates outputs, it feels familiar to the type of work we do and doesn't require, you know, a whole bunch of babysitting and handholding. But also to back up a little bit, if you think about what that actually means to an organization, because, and in a minute I'll give you an example of a recent, a project I'm on right now that I think would help you see the value of this instantly. A good L2 is also not only building tools that can sort of address like just repeatable work at scale, they're also Converting tacit knowledge to documented process. Because once an AI model is aligned, now you have documentation. Maybe it's in code, but it is documented. And so that's a big problem in like every industry right now. It's like, oh, you know, the aging workforce and, you know, what are we gonna do? We got don't get hit by bus guy over here. And if, you know, if that's me, if we lose them, you know, the plant shuts You know, kind of thing. And so I think in my mind, like, that's how you know you have an L2 there that you don't have to worry about the tool they're building and whether or not it's going to work well. So, but here's the other piece of it. And really, like, we executives lean forward in their chair at this part of the conversation because when we say, you know, if you're wondering, you know, about the value of this investment, Um, let's, let's stop thinking about this as AI work. And really, you know, like the work of yesterday is going to look like the work of tomorrow. Believe it or not, the more things change, the more they stay the same. And I know it feels like everything is shifting beneath our feet, but the reality is we will be doing the same things before tomorrow. Whenever I talk to AI teams, it feels like what they want to build is agents, what they want to build is tools. But if you actually go look inside of the work at companies, They don't, they're not thinking about AI. They're thinking about this particular problem in front of them. So with one of my clients, uh, let's just say one of the top 4, um, pharma companies in the world, they were sitting on a stack of, uh, I can't remember how many thousand documents they needed converted to look from, to look like this, to look like this. And, um, I, I can't, probably can't say too much more about that project, but you know, it was a $4 million job. It was a $4 million job. Uh, they, they knew exactly how much it would cost to convert each one and it requires SMEs. And, uh, an L2 took a look at it and said, this feels like a Claude skill. Had the Claude skill built within 3 hours. Um, we drug one of their documents onto that skill, you know, out the other end squirted the almost exactly what we were hoping these things would look like at the end of the process. You know, of course everyone's jaw hit their desk like, wait a minute, you're telling me that was gonna be $4 million? We, it needs to be done. And now our Claude, which we've already paid for, you know, you just drag 'em on and wait, you can also run 1,000 concurrently and like this whole thing could be done in hours. Like, wait, what? That is measurable. You know? And so we kind of push people to thinking toward like, you know, you're not going to find out the value of this investment based on how many tokens people are spending or, you know, like you are going to find it when you start intentionally forming teams with a couple of strategic people in there who know how to spot an AI opportunity and just make the headache vanish. We call it doing their laundry, you know? And so, like, this is where the real money is and this is where the real savings is. And Daniel, I know you know about this because I've heard you kind of almost complain about it. Like, you know, it's like, this stuff's not fun, but it's like, it's real work. Chris, is that helpful?

It is. No, that helps frame it very well for me. I appreciate that.

Yeah. And I would be curious of your take on this, Mike. I think you're the right person to ask for a critique on some examples that I've been using even personally, but I've been trying to use this example of, you know how in like leadership retreats, they always used to show like the F1 pit stop and how it advanced from like 20 seconds to 2 seconds. Right. And how that happened was like everybody knows their job in the pit lane, right? And one guy's job is just to move the tire from here to there, and that's all they do. Like, that's their full responsibility. And it almost seems like in some of these use cases that I'm seeing, at least those kind of very targeted jobs or outcomes, right? But like you're talking about this document from here to there, right? That needs to be done. And That is maybe a good candidate for, however you frame it, a use of AI, an agent to take care of it, however you word that. Like, that's a thing. And it kind of, in my mind, it then, it doesn't remove the human, but it actually kind of elevates in some cases the dignity of the human from being The person is like, all you're going to do every day is move the tire from here to there or to do this task. And certainly I recognize there is legitimate, jobs will shift, right? But, and people will struggle around that. But it seems like now kind of the human, you're not framing your team of humans as, hey, don't go, don't get out of your lane. Just move the tire from here to there. To actually being able to leverage AI in these individual tasks and you kind of coming into the orchestrator mode or the team principal or the strategist mode and actually thinking about outcomes. Well, what is the outcome I want here? And what are the individual things associated with it? Many of those which can be individual tasks that AI accomplishes. I don't know if that rings true at all. I've been trying to think about, yeah, how People are very used to thinking now, I think, or many people are very used to thinking of the one-to-one interaction between a human and AI tool, let's say, or a chat interface, and maybe less familiar with this, or familiar with this way of thinking of maybe one human is actually aided by multiple different instantiations or manifestations of AI that help them work towards outcomes. But I don't know, any critique on that rambling example of kind of the F1 pit stop and some of these tasks? What are the tasks in your pit box that need to be accomplished and what of those are good candidates for AI? How do I think about managing this set of— AI workers or tasks or whatever you, however you might frame it.

Yeah. One of the things that popped in my head while I was listening to you, Daniel, which I always enjoy doing, is, yeah, I'm very often thrown into rooms with people who live on a spectrum of attitudes. And one of the big ones is just like all the fear wrapped up in this. And I kind of always open with, hey, guess what? If you're worried about losing your job to, I think There's thunder in the background. I don't know if you can hear it.

Yeah, it's definitely thunderstormy in the Midwest right now, for sure.

It's giving the answer extra drama right there.

Yeah, that's the fear part. No, really, you know, I, when I go in a room and I know there are people who, who are dwelling on that, hung up on that, I, I mean, you know, I just open with, guess what? You are losing your job. You like, your job is going to change. And that's true for all of us always, whether or not AI was invented or not. I don't know how many careers I've had, and I think the average person has 7. And even if you stuck with one, your job would change year over year in time. And if someone is uncomfortable with the concept of change, guess what? You're not cut out for the workforce. I mean, like, the reality is, the reality is like, if my daughter was scared there's a monster under her bed, I wouldn't go and say there's no monster under your bed. I'd say you're stronger than a monster. Like, and monsters aren't real, you know, like, you know, and it's kind of like that in this job. like, hey, adapt or die, and I don't think you're going to die. And maybe some of this, it might sound a little heartless, but it comes out of the research is that disruption is always, it turns out in hindsight, it was not as hard to adapt to as you thought it was going to be, and you adapt more quickly than you thought you could. And so this is just, documented truth. And so, but it never feels that way. I think I'm just the weird bird who really likes the idea of like uprooting my entire career and just trying something new. You know, like I just have done that so many times and I almost enjoy it. So, but then I think the rest of what I heard you talk about, Daniel. Yeah, I addressed that, but Remind me, like, what else? What was the other thing?

Yeah, I think it's this idea of one element of the change, I think, is, is understanding that maybe your, your position, which might be consumed by this task right now, might not be just come into work and do this task, but it, it sort of elevates more to an outcome kind of creative orchestration mode where you're actually orchestrating work that needs to be accomplished, like you say, the same functions, the same roles, like the same outcomes need to be accomplished in your business, right? But yeah, it does take a different level of thinking to think about, well, how do I orchestrate these things to get the outcomes versus how do I do this task? Because I know how to do the task. I know how to get the task done, right?

Yeah, I hear you. Maybe part of why I struggle with just rapid firing an answer to this is my mind darts back and forth from the hundreds of work environments and situations I've had to kind of inspect and solution for literally every—

Let me frame it for a moment, because I think I see where Dan's going, and I think you are the right person to answer this. And that would be, if you go back to that analogy of the F1 pit stop that he was describing, And the nature of the jobs that everyone had there. And you're kind of looking forward. And so, you know, we're kind of, what is our expectation of the future? And, you know, at least I think one of the things that probably the 3 of us would agree on at some level is the fact that the human that's in that process, it may have these AI agents that are able to take over some of those very specific jobs along the way. And you're almost becoming the pit stop manager or orchestrator overall. So you're still doing the pit stop because that's the value of what your organization's doing, but the nature of the humans may change to better do it, to do it faster and to grow rather than be kicked out of the, you know, rather than loss of job. It's a change of job that's actually more valuable in a lot of ways. Could you talk a little bit about what that future might look like? for those individual employees that are adjusting their pit stop job, but maybe moving a little higher in the abstraction layer to owning that whole process where they are valuable as humans and yet they're taking full advantage of the agents. Any thoughts, like as you take companies through the process, how you would do that and what your expectation of the future will be on how that progresses?

I don't think you're gonna like my answer.

It's all good.

I decided to stop worrying about it. I don't even think about it anymore. Like, and maybe, maybe it's just because I don't fully understand the question. I think I do. I spent probably 2 years inside of all of these organizations trying to like lead and organize and, you know, and implement enablement and activation campaigns. And in the end, it just kind of felt like I was just watching water run down a river. It was going to go down the direction it was going to go. The river is It's just winding the way it winds. And I think I'm always just searching for like, where can I actually make an impact? Where can I actually activate change in an organization that is measurable and something that we're all glad we did after we're done doing it? And, you know, when I come in and, you know, and we try to just, you know, really analyze and think through, you know, like, well, what's the impact of this gonna be after we do that? And, you know, what's it But in the end, like what, what it's going to be is what it's going to be. And people tend to adapt to these tools very quickly once they've used them once or twice and seen the result. And so in my mind, it's kind of like, I'm going to do everything I can to get them to do that. But beyond that, if I can't get them to, I don't care. What I care about is, can I find that L2? Can we get them equipped and activated and synergizing? There's a better word than that. Effectively cooperating and contributing inside of their team in a way that, like, I wanna know the difference before and after. And so it's not just that, you know, maybe the pit crew thing was a distraction for me, 'cause I was focused on like, well, it's faster now. And it drives me nuts when people say, I want things faster.

Sure.

At Tier 1, we say like, faster's not always better, especially when It's a, it's, it's a system with a whole bunch of parts and you've just created whiplash and angst for everything around that fast piece. You know, like nothing else is ready to go that fast. So that's not necessarily better. Cheaper. Sometimes that doesn't even matter. Believe it or not, a lot of the companies, that's, that's not like the main thing we're talking about. Like, but new emergent types of work, like that is interesting. So what can we do now that human minds just really couldn't do well before and equipping these language models inside of like really clever harnesses. So again, it's kind of like Daniel's world, you know, like, whoa, new kinds of work. And so what's that mean for, you know, the product or this, you know, department or function or whatever it is? I mean, so I guess I put that whole concept just right outside the periphery of what I can control. And so therefore not something I worry about and I'm sort of done thinking about.

Yeah, that makes sense. I actually do like that answer. I did too. I think there are a lot of people and maybe this is, I don't know, a little bit too personal, but certainly like a mode that I think all of us as humans get in is not being present in the moment and thinking about Of course, the past, dwelling on that, but also the future and all the stories that we tell in our head about how things might go, right? Which won't— there may be elements of that that are true. And it's not, of course, I don't mean that we shouldn't innovate and look at new ways of doing things. I just mean we often dwell on eventualities in the future that will never take place. And there's dynamics that are happening that we need to dig into and think about how they matter in the moment, which I really like that perspective. As we close out here, Mike, after kind of working across these different levels of proficiency, and I know also you're thinking about structured ways of thinking about the actual things that can be done by AI, the outcomes, that sort of thing. What's on your mind today as you're going into your next meetings as like the kind of challenges or the things maybe like you had done this work to parse through all of these, this research and work on AI proficiencies, right? What's that next kind of, or is there a next kind of area in your mind where you're like, I don't totally have a grip around this bit of it yet, but I'm really curious to dig in more and see if I can parse through some of it. Anything that comes to your mind as we close out here?

Yeah. Yeah, I think the thing that's been most exciting to me as like an idea or like a rallying cry when it comes to the rank and file inside of organizations or with leaders who look at me and say like, what's, you know, like, where is their energy? Where is their momentum? Like what, you know, one idea that I've kind of had lately and I've seen pay off over and over and over is the reality inside of most of these massive organizations. It's mostly, I'm talking about big business now. Um, is that everyone's kind of got a toolkit, uh, in place. Like most, if you're, you know, uh, Eddie Punch Clock, Procter Gamble, you probably have access to 1, 2, or 3 AI tools and they don't work well together and they don't necessarily work well. And it is very easy to try to use them and just see it doesn't work, you know, super well for the thing that you need to get done and give up on it. And I think like the thing I'm realizing is that there is massive, massive value in the people who are willing to lean against the brick wall and push and push and push. And you would, believe it or not, you'll start to find ways to use these tools that are hugely transformative in spite of the fact that they're so burdened by necessary governance and rules and restrictions. And why can't we turn that connector on? And why can't, you know, But, you know, when Claude tells you something's not possible, you know, ask it to get creative and, you know, and, or, you know, think through, you know, alternate approaches. I think creative solutioning is gonna be really, really valuable over the next 2 years, 3, 4, 5 years before, you know, all the wrinkles are ironed out and who knows, you know, Terminators or whatever, whatever comes after that. But I think, yeah, the thing, Daniel, that I've been preaching lately with my trusted colleagues and with different leaders and organizations is, do your best to encourage that type of thinking and behavior of, okay, I know they aren't perfect. Figure out what you can get done though.

Yeah, that's, I think, a great rallying That's a beautiful cry to end with. Appreciate you joining us again, Mike. We look forward to having you, having you on again to learn from you in hopefully not too long again. But yeah, appreciate you taking time. It's been a great conversation.

Thank you.

That's all we have for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Bluesky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show. Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening.

Bye.

That's all for now, but you'll hear from us again next week.