Ep 800: Celebrating our 800th Episode: 8 AI Truths, 10 Smart AI Moves and 10 Questions You Must ask

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Well, we made it to 800 episodes and guess what? We're still here and so is artificial intelligence. Turns out this AI thing really matters. So much so, we're still doing the Everyday AI podcast 3 and a half years after we started. But here's the other realities. No, there is no AI bubble. Yes, AI is smarter than all of us. And no, AI has not taken 50% of white-collar jobs, regardless of what some CEO that's trying to drum up IPO hype tries to sell you. At Everyday AI, I've spent the last 799 episodes trying to break down complex theories into digestible daily takeaways with my random rumblings sprinkled in, obviously. So for today's 800th episode, we're going to continue the no-nonsense trend as we give you 8 uncomfortable AI truths, 10 moves smart leaders are making, and 10 questions every leader must ask. ask. See, it's 8 times 10 times 10. I'm not going to give you 800 random AI facts or something like that, like I used to do when we were at episode 100. So, and also sometimes these hundred-ish episodes, right? 500, 600, 700. Sometimes they go kind of long. I'm going to try my hardest to make this one a shorter value-packed episode. So let's get straight into it. And if you are new here, well, welcome. Hello. Welcome to Everyday AI. My name's Jordan Wilson. We do this thing, well, every day. It's your daily unedited, unscripted livestream podcast and free daily newsletter to help everyday business leaders not just keep up with what's happening in the world of AI, but how we can make sense of it to grow our companies and careers. So starts here, but make sure to go to our website at youreverydayai.com. We're gonna be recapping the highlights from this show as well as all of the other AI news that's important that you need to know to get ahead. All right, let's get into it. And if you did miss our 700th episode, we're taking a kind of similar take. But if you listen to this episode and you're like, wow, this was really helpful, I think you'll like our 700th as well. Although some things might be a little bit out of date now, even though it's only 100 episodes ago. That's how quickly AI moves. But you might want to go listen to it. So we went over 7 ways AI is reshaping how we work, 10 AI workflows that actually deliver ROI, and 10 AI skills every professional needs in 2026. But let's start off this round with our 8 uncomfortable AI truths. So number 1, single model AI strategies are now continuity risks. And I mean, the timing of this is obvious, right? We saw Anthropic roll out a very capable, uh, and much hyped model in their Fable-5, which is of the Mythos-5 family, right? It's Mythos-5 without guardrails and I saw a lot of chatter online, you know, companies saying like, oh, you know, we're goodbye ChatGPT, goodbye Gemini, goodbye Copilot, and trying to move all of their operations inside this new model because it is a step change. Or I could say it was or kind of is because it's currently no one in the world can use it. So that's why this is a single model strategy is a risk because we saw the, US exports control action on Fable 5. So for companies that rushed to try to move all of their operations into Fable 5 from Anthropic, number one, okay, Scroogey McScrooge Duck writing blank checks. How can you afford that? It's a very expensive model. But number two, I think that shows where everything is headed right now because you have to be thinking across multiple models. And I don't know if this kind of US government export action could be a common thing. The only thing with the new president's new executive order is a 30-day review period. So this kind of export control action was not part of that initial executive order. But we've seen that Anthropic and the US government have kind of had some ongoing beef for the last couple of months. So this isn't shade at Anthropic. This is just the reality of if you can trust your entire organization's AI strategy on a single model that the federal government literally at 5:00 PM on a Friday can click nope and then the entire world loses access. Because if one model breaks your workflow, I mean, procurement misses business continuity planning, right? I did a training for a company that was using Claude models and they were trying to set up all of these complex workflows. And one thing I talked about them, after the session I did is what's your backup plan? You have to be building these things modularly because if you spend so much time, uh, you, you know, especially weaving some core part of your business's operations into a single model, you could be screwed come Friday at 5:01 PM when said model is done and gone, whether that lasts for a couple of days, a couple of weeks, or indefinitely. All right, so that is the uncomfortable AI truth number 1. Truth number 2, the moat has moved from models to operating layers, and that kind of goes with uncomfortable truth number 1. And we actually did have a Start Here series, uh, on this one. Uh, I believe it was, I'm trying to do my math. Uh, I think it was yesterday, right? When we talked about AI super apps kind of being the, uh, the new wave or the new trend on where work happens. But I mean, Codex, you can't overlook that with now 5 million weekly users. So yes, in Codex at least, you are still using GPT models, but you are using models, multiple of them in the same conversation with a unified memory, being able to use a browser, a file browser, an internet browser, being able to control your computer, being able to use your browser, all of those things. That's why I think the models are becoming less and less important as are capabilities that we get through large language models become more and more agentic, right? As they start using browsers and using computers and using terminals and using files in the same way that humans do, I do think that there is maybe less of an overall emphasis away from a single model. And that's why I do think when we talk about moats, when we're, you know, which I think for, you know, business leaders that are working on the front-end AI strategy, before you, you know, say, hey, where for this department, we're gonna go kick off a, a 1-year program with this big AI company. You have to understand what their moat is, right? Like, are they developing a harness that is gonna be able to fully use any of these tools? Is it maybe better to look at a more open harness and then you can plug and play different frontier models at will? So I think that's a really important thing. You know, talking about things like plugins that bundle all of these different skills and apps together,, from Codex, I think is another important thing. Also, as we talk about context connectors, all of those things, all of these things on the outside, the second layer, if you think of the model as the base layer, all of the things surrounding them, I think are gonna become more and more important. I'll probably do a, maybe my final Start Here series. And if you've missed that, uh, you know, make sure you go to starthereseries.com. Uh, go listen to that. It's a, you know, an ongoing series. I think we're gonna cap it at about 30 episodes cuz that's probably enough, but I think it's been great for, you know, beginners to AI leaders to tackle all of these ongoing changes. But I do think that there's maybe gonna become a certain capability gap, not where humans are, you know, not using the full capabilities of models, but actually where we, well, for most of our day-to-day work, at least as it is today, our work will not require 95% of what these models are capable of. You know, we kind of saw this with Microsoft looking at DeepSeek, as a potential, you know, maybe cost alternative, uh, to running the GPT and Claude models. And at first I'm like, well, that doesn't make sense, at least now. But it might make sense in, uh, 6 to 9 months because I think at least until the work that knowledge workers are required to do changes drastically, right? For the most part, I think in 6 to 9 months, you know, some of the more basic, I'm not saying, you know, Deepseek, right? I'm not saying that model, but I do think some of the more basic models, or even a company's second or third most powerful model, uh, maybe for a period of a couple of years might be sufficient enough to do the majority of all this work. Which is why I think, you know, yes, when we were talking about models' capabilities from, you know, 2024 and 2025, the model really mattered. But I think it matters less and less today than it did. Truth number 3, prompting is losing to workflow design. All right. And if you haven't noticed this, again, this goes along with the push toward the super app and the harnessing and all of those things. But, you know, even Google, right? Google is pushing agents, scheduled tasks, subagents, orchestration, right? If you are still spending a majority of your time or your team is still spending the majority of your time prompting large language models versus reading and acting on their outputs, You are behind. Let me just say that very bluntly, right? If you are still going in and typing things and, you know, your team has a prompt library, it's not like that anymore, right? We have skills. That's a universal language that all large language models speak. For the most part, they all have, uh, scheduled tasks or automations. So for the most part, humans should be working more on the front end, you know, making sure that, uh, your context stays up to date, making sure that, you you know, you are making sure whether you're working with markdown files, skills across your organization, making sure those stay up to date with your most up to date context. But then from there, you should really be spending, right? Humans should be spending more of their time taste making, right? If you have a project that you would normally go in and prompt a large language model for, right? And that project happens every Monday afternoon. Instead of you prompting every single day at Monday afternoon, well, you should be looking at the deliverables that 5 different agents delivered and saying which one is best. And then Tuesday after that deliverable is due, you pour in with new updated context and make sure that, hey, next, next Monday, you know, those 5, uh, you know, reports that 5 different agents are gonna take, uh, a swing at, they actually have more, better context and directions. Truth number 4, static business artifacts are becoming organizational debt. And I, I didn't say have become on this one. Okay? I'm saying they are becoming. Let me give you the quickest example. Again, did in a recent episode on this actually when we talked about Codex sites. So make sure to go back and listen to that one. But as the harnesses become more popular, right? As things like Codex, Cursor, right? Cloud Desktop, I think has some catching up to do. We'll see, you know, if Google continues to invest in the Anti-Gravity 2.0 app or not, right? But as these harnesses become, number one, more capable, but more team-friendly. I think the concept of, you know, emailing files back and forth or even dropping, you know, static files in a shared drive, I think it's pretty soon gonna become archaic. And I think our artifacts first need to become AI-native. Yes, we need to figure out what that means for history and versioning and all of those things. But, you know, I think Codex sites, is maybe the first iteration, maybe it's not the final or the best, right? But I think that's a good example of the difference between how a static business artifact can actually become organizational debt. And I didn't think it at first, but I thought back throughout the, the, the course of my career, how much time that I spent either communicating about different file versions, trying to find old versions, um, you know, corrupted files, right? All these things, just tackling, communicating, finding, organizing around different files, sharing them. You know, you're waiting for approval on all these files because, oh, it turns out they had V4 and they actually needed V4 final, right? All these things. I think we have to look at what does an AI-native deliverable look like? And maybe it's a codec site, maybe it's a, you know, Claude live artifact. I don't know. Uh, but that is truth number 4 is we have to start thinking about AI native deliverables. Truth number 5, AI can create more work if leaders don't remove old work. All right, let me tell you what that means. AI often just adds more and more, right? If you're looking at text, walls of text, that doesn't necessarily mean that your organization is winning back time. Right? When we talk about ROI, if all you're doing is continually experimenting with different AI, with different large language models, or if your employees, if your team members are just banking their save time, which I would argue the majority of companies that saw, you know, AI gains in theory, but not on the books in 2024 and 2025. Yeah, that's just your, you know, employees having AI do all the work for them, doing the same amount of work that they were doing pre-AI, and then saying, well, look, right? But on the flip side of that, AI, if you don't properly, uh, hold it in, can actually create way more work if you don't redesign workflows to be AI native. So if your job description hasn't changed, if your SOPs haven't changed, especially since 20— late 2024, early 2025, when models became more agentic, You have to do so because all you're technically doing, uh, is you are just doing a disservice actually, now that I think about it a little bit more, right? If you are still producing artifacts for clients, industry reports, whatever the, the novel work is that you're creating business value within your organization, if you're still doing it the old way, let's just say the 2022 way with 2026 tools. Yeah. That's bad, right? Yes, you are still creating more work, but one of the reasons you're creating more work is because you are creating all of these processes that in theory are not gonna be transferable once your company, your organization, or your artifact, uh, becomes AI native. Or with whatever your— whether it's for a client, an internal team member, et cetera, right? What about when their demands change? Change. And I think the consumer demand will begin to change, right? We're getting oversloppified with all this AI slop, but I think that demands are going to change. So if you are just trying to use as much AI as possible on old processes, old job descriptions, uh, companies and, and organizations that are set up in a pre-AI way, I think ultimately you are just setting yourself up for failure, even if it seems like success in the short term. So real ROI starts when the AI actually goes in there and removes steps, removes meetings, and reworks what you're actually working on. Truth 6: Longer AI task completion will reshape jobs. All right, so good example here. Anthropic says autonomous task length has doubled roughly every 4 months. And as an example, Anthropic said that Claude authored over 80% of Anthropic's merge code in 2026. So what does this mean? I'm not sure yet, right? This is one of those things that still kind of keeps me up at night. And I think we, it's, it's gonna take even people on the edge of AI, it's gonna take them a while to figure out what does the future job look like, right? Not like, oh, what happens when we have, you know, embodied AI and human, you know, human AI robots doing all these things. Even before that, Right? What happens when autonomous AI harnesses are doing all of our normal deliverables that we were doing 3 years ago? What happens then? But I think that managers must start to redesign work around delegation, supervision, review, and exceptions. Right? We have to start redesigning old work processes because as these models can work, whether we're talking about in loops, in managed and controlled loops or just working truly autonomously toward a goal, right? As they start doing things that would normally take many, many hours or many, many days, right? We don't know how jobs are going to look. And I'm not talking about job disruption, I'm not talking about job creation, I'm talking about the jobs that will still be here, which I think is the majority of jobs. How are they gonna look differently?, but as you have these, uh, models now that can run and do these long tasks and they can work for 8, 10, 12, 15, 18 hours on a single type of project that would normally take a team. What does that mean for the future of jobs? I don't know yet, but it will reshape jobs. Truth 7: Open weight models are becoming resilience insurance. Right, so I already talked as an example about the recent Microsoft news that said that they're looking at Deepseek as a potential rollback or different offering, right? So they're looking at fine-tuning a version of Deepseek and maybe using it internally, offering it up to customers within Copilot. We'll see. Right now it's just reporting that's still shaking itself out. But the reality is open source and open weight models will soon become at least a resiliency insurance, right? This is something I've even started to experiment with my, with on my own. I will say this though, open weight models, they're great if you think that you can slap, right? A general open weight model and, you, you know, have it run, you know, locally on your machine. Like, oh, why would I ever, ever, you know, use Fable-5 or why would I ever use GPT-5.6? I can have an open weight model running on my computer. No, that makes you look foolish. FYI, I'm just saying that. In that case, open weight models will still, you know, open weight or open source models will still be multiple years behind in terms of what a consumer can go and run on hardware. So unless you're spending $30,000 on hardware, it is much more economically feasible, and just responsible for you to be paying $200 a month, then just use the Frontier model, right? But open-weight models are becoming a great backup plan for specific use cases. So as an example, and this one's kind of recent as well, GLM-5 2 Max is now technically the number 1 model in the world for front-end coding because the only model that's ahead of it, Fable-5, is not technically available. So it is the number 1. available model. Think of that, an open source model. All right, again, you can't really run this locally unless you have a cluster of supercomputers. But again, thinking about how this shapes the future of work and how this impacts decisions that your company makes. I mean, you have to look at what Microsoft is doing there. If Microsoft is thinking that, hey, we might be able to, whether it's internally or for Copilot customers, If we put enough human effort into this, uh, we could fine-tune an open-source model, um, and run it at production for even half of internal or external purposes. That's big. And I think that business leaders need to start looking at that. For most companies, it's not there yet, but you have to understand that open fallbacks matter because vendor changes are gonna happen. Policy, prices, access. I think through the rest of 2026, we'll see what shakes out between the Anthropic and White House. But you have to start thinking of these things, and you need to start routing workloads by sensitivity, volume, compliance, cost, and quality threshold. Truth number 8: agents make AI risk operational, not theoretical. All right. And I talked about this a little bit more in our 2026 AI prediction and roadmap series, but as the default, As the default models become more agentic, they're getting read/write actions. Let's even take out the fact of running these, you know, desktop super apps that can access every single file on your computer. They can read, write, use the terminal, all those things. Even look inside something like ChatGPT. Even look inside something like, I don't know, Grok, Perplexity Computer. Even the quote unquote non-technical AI chatbots are getting more agentic power, right? Very small example. ChatGPT can send emails now. All right. Doesn't sound like a big thing, but ChatGPT has always had the ability to draft emails and to read your emails and to pull in all this context. But just as of a week ago, it can actually send an email. Do you know how much risk there is when you give any AI model the ability to send an email? Right? There's great promise, but there's also a great downside, um, if you don't have the right expert-driven loops. Because every agent needs an owner, limits, approval, logs, and rollback plan. All right, so those are our 8 uncomfortable truths. Now let's get into the 10 AI moves that smart AI teams are making. And yeah, we're gonna pick up the pace here. All right, move number 1 is they build portable AI stacks before disruption. Hits. So I'm talking about primary, backup, cheap, private, and manual fallback paths. You— these all have to be documented, right? What happens when AI works? You have to be able to answer that first. And then what happens when that AI that works fails? All right. So smart teams are making those moves already. People always are, are so quick to say, once you find a win and you're like, oh my gosh, This is gonna save us 40% of staffing costs, so we're just gonna go ahead and cut 10% of staff, call it a win, right? No. Use those humans and start building these portable AI stacks because disruption will hit, whether it's internal, external, uh, from a policy standpoint, whatever. You need to be, uh, kind of creating these redundancies. Move number 2, they map workflows before buying more tools. All right. I think so many organizations that I talk to, they find one success and then they roll that success out across different teams, right? Which is great. That's a great way to go and, and do things. But then what happens is then they just say, okay, then what's the next good tool that we could do this same process, right? Instead of replicating that workflow mapping across the entire organization, instead Right? And I get it. It's hard because companies always wanna be chasing the shiniest, latest, greatest AI tool and model. Let me, let me be very direct here. Even if you, I'd say if your organization, unless you have a team of 100 people whose only job it is, is AI deployment in your organization, which is no one, they have no outside responsibilities or deliverables, unless you have 100 people that that's their only job. Mm-hmm. You and your company will get much more, much more value if you literally stop. Don't try GPT-5.6, don't try Fable 5.1 or whenever it comes out, don't try Gemini 3.5 Pro. You will find more business value if you literally stop trying the next best model and instead look at your next or your last big win and then use the findings from that and map that out to everything within your organization. That's very hard to do. So instead, normally you just go through this concept of we're just gonna map, uh, or we're just gonna try to replicate our big win. And oh, by the time we, you know, rolled out that one big win across all organizations, there's a new model. So we're just gonna go chase that. It's hard not to, but you are gonna get much more, uh, ROI, um, by just mapping entire workflows before you just jump onto the next big model. Move 3, smart teams are building role-specific AI packs. All right. Luckily, Anthropic was really ahead of the game with these skills. And surprisingly, it did take OpenAI a little bit longer than I thought it would to kind of jump, I won't say on the bandwagon, but to follow suit. skills are extremely powerful, right? This kind of takes away a lot of the grunt work of prompting, right? So skills, I mean, you could make the argument skills are just a bunch of text prompts stacked together, right? And they sometimes do some technical things behind the scenes, right? But this is the basics of, well, prompt engineering, right? The skills are essentially this prepackaged markdown files that tell a large language model in a very specific way what to do, what not to do, but around different skill sets. So as an example, there's great skill packs for different type of marketing roles, right? So if you're a marketer, you should probably at least start with a marketing skill even before you go in and prompt something or customize a workflow to work for how you need it. Uh, but this is huge. And, you know, skills are obviously transferable across different systems. Uh, Codex just rolled out a bunch of kind of, you know, plugins, I guess is what they're saying, because their version is more using skills plus apps, which is great, right? But every organization needs to be experimenting with skills, and you need to find a good skill that's already prebuilt for your organization. There's great open— Open resources for skills because you can plug and play them into anywhere. Uh, right. You need to start with a skill that is highly rated, vet it across your organization, test it in a sandbox way, make sure it works, customize it, and go on to the next. All right. Move number 4, smart teams are automating recurring reports and briefings. My gosh, if you are still having 20 people in this boring 3-hour meeting every single week and someone's still typing up to-dos because No one has access to the AI note-taker. Oh, or can't use it for this meeting. And then you're preparing a deck for the meeting and a deck after the meeting. My gosh, no, stop. You have to start automating all of those things because number one, the majority of the people in those meetings don't wanna be in there anyways. Right? The, the majority of people, uh, reading a, a 30-page slide deck, they only care about one. Little slide anyways, right? That's the reality. I think you have to start pulling in all of these things that are, uh, spending— that you are spending too much human capital on in a non-AI native way. And you have to start blowing up those old processes. All right. Move number 5. Already kind of talked about this, but it's worth talking about again. Converting static files into living artifacts. So, Sorry, spreadsheets, PDF, PowerPoints, all these things. You have to, and, and a good one too is skill, skill libraries, right? Your company, your organization needs to have living AI-native documents. Old static files are going to slowly die. Probably not by this year, right? It'll probably still be another year or two before they actually die. And until we see the rise, right? Right? Even look in the AI community how popular things like markdown files are now, right? We are going to get, whether it's something like Codex Sites, I don't know what it is, but you have to start converting your static files right now into living artifacts. Move number 6, smart teams are testing agents in read-only mode before giving them control. My gosh, I can't tell you the amount of times I've seen this., right? Whether it's, uh, you know, hearing about it on, on, you know, conversations I don't have here on, on, uh, on the show, uh, reading about it in news articles. The process of sandboxing an agent and putting an agent into production is pivotal, right? I think a lot of teams think, okay, well, let's sandbox the agent. Let's test it against the guardrails, make sure it's safe, make sure it doesn't drift. You know, they go through, they check their boxes, right? That makes the compliance team happy. And they're like, look, we're, we're super responsible, right? And then they do this, you know, this is usually those, uh, those companies that I think are a little too AI trigger happy. And then all of a sudden they start setting these agents out and they're like, oh, don't worry, Bill from IT. He's our human in the loop. You know what? I know I talk about this a lot, but the world would be a lot better place if no one ever came up with this human in the loop thing, because it's absolutely garbage. Because, and I, I tell you, 90% of the time, the human that's in the loop, number one, is the wrong person. Number two, they're not driving or steering. They just look in there when it's, it's, it's more of. They're a crash scene investigator instead of an FAA flight operator, right? Most of the time, a human in the loop is only quote unquote involved when something goes wrong. So you have to test agents in read-only mode before giving them control, because as the models driving these agents become more sophisticated, so too does your process of the human or humans you have overseeing them. AI moves too fast to follow, but you're expected to keep up. Otherwise, your career or company might lag behind while AI-native competitors leap ahead. But you don't have 10 hours a day to understand it all. That's what I do for you. But after 700+ episodes of Everyday AI, the most common questions I get is, where do I start? That's why we created the Start Here series, an ongoing podcast series of more than a dozen episodes you can listen to in order. It covers the AI basics for beginners and sharpens the skills of AI champions pushing their companies forward. In the ongoing series, we explain complex trends in simple language that you can turn into action. There's 3 ways to jump in. Number 1, go scroll back to the first one in episode 691. Number 2, tap the link in your show notes at any time for the Start Here series, or you can just go to starthereseries.com, which also gives you free access to our inner circle community where you can connect with other business leaders doing the same. The Start Here series will slow down the pace of AI so you can get ahead. Move 7. Smart AI teams are making. They route work by value, risk, and cost. So we had an episode recently in the Start Here series talking about going from token maxing to token efficiency, and this part's huge. You should probably not— if I'm being honest, you should probably not be using Fable 5 for 99% of your tasks. You shouldn't be using GPT-5 5x high for 99% of your tasks, right? It's crazy the amount of people that are just using the best model for everything. A blanket. That's not the way to do it. Right? I think most companies take these, uh, I won't say they're habits, but maybe best practices. Because for the most part, in 2023 through 2025, we were all living in this subsidized, uh, AI nirvana. Right? Where it's like, oh my gosh, I can just use the best model for anything regardless. All right? But times are changing. Times are changing. Token maxing has led to token efficiency, right? We saw all these stories, all these huge organizations, you know, hit their AI spend limit or their token budget within 3 or 4 months, and now they have to change. And the way you change that is by routing the type of work to the proper model or models that it needs. That's why I think that model routers, mixture of model setups, I think there's gonna be some great AI startups. I've already seen a few launch in the last few months that do exactly this, right? So you are not wasting, as an example, a billion tokens a day. It's not crazy for, you know, I very easily go through 500 million to a billion tokens a day sometimes. Me, small business owner, right? Luckily, I'm still heavily relying on the subsidized, you know, uh, OpenAI $200, uh, ChatGPT Pro plan. But subsidies are all going to eventually fall or diminish. So you have to start making these calls. Appropriately. And, you know, similarly, you do have these now very capable models like GLM-5.2, the KIMI models, right? I'm not saying use them for every task, but instead of using, you know, one model for 50 different tasks within your organization, you might be using, you know, 5 different models for 8 different tasks or or something like that, or 15 different models for 3 different tasks each. So you have to start routing them by value, risk, and cost now, because if not, you are going to be in for a rude awakening. Because as an example, right, if Anthropic's like, yeah, you know, here's enough Fable 5, we'll see if the June 22nd date gets extended or not. But, you know, it's, it's very likely Anthropic might go down this path of, releasing these models just to all subscribers. People are gonna be reworking their workflows. Then they get pulled and you have to start paying API costs. So, you know, $200 becomes $14,000, give or take. So you can't keep doing that. You have to work smarter. Move number 8 I'm seeing smart teams make: measure accepted output, not AI activity. Lines of code is not return on investment. All right? Tokens burned doesn't mean a thing. Are your deliverables getting better? Are they moving the needle? Are your blog posts bringing in more human visitors? Right? All of these things that I think we rushed to start using AI for. Oh my gosh, look at Claw Design's so great now, I can go. Yeah. Get out 5x the number of proposals. Okay. Number 1, did you? And number 2, are you bringing in 5x the amount of business or are you just forcing AI slop down people's throats who are already being avalanched upon with just, you know, more and more content? Just because we can create more and more text, more and more photos, more and more videos, more and more audio doesn't mean that we should be. I think human taste really matters here. Uh, so you have to look at accepted outputs, right? Which of those fi— you know, to pull from my random example, random rambling from earlier, you know, when my agent came back and gave me 5 versions of a certain report, which one was number 1, accepted by the expert driving that agent loop? And number 2, which one ultimately won over my peers, won over the internal stakeholders, won over the external stakeholders as well? Which one got us more business? So you have to start measuring those things, not activity. Activity is not movement. Move number 9, they red team the workflow, not just the model. Okay? Yes. Large enterprises need to have red teams. You absolutely have to. Right? We saw this with the, you know, even though we don't have the full story yet, but we saw this with the Fable 5 rollout. Essentially, at least according to the White House and the federal government, you know, they said Anthropic shipped a model that wasn't safe. So what happens if you put that into production and for whatever reason, one of your customers, one of your clients who maybe relies on your services or your outputs that you use somehow stumbles upon a piece of code that wasn't safe and you're like, oh, well, this model did it. Well, okay. Yes. We, assume that labs like Anthropic and Microsoft and OpenAI and Google have the best red teamers in the world, right? It's those engineers that go in and make sure a new model is safe and they try to break it internally before releasing it. That's kind of like what red team is. That's red teaming for dummies 101, right? But you have to have an internal team that does that as well. Not just the model, but the harness. So in the same way, you know, if, if, if you say, hey, we're gonna roll Codex out to 10,000 people, we're gonna roll, roll out Claude Code, we're gonna roll out Cursor, right? I think before when you would roll these things out, it was usually developers using them to write code. But as we start producing non-technical work documents, as that becomes the norm, how are you red teaming that internally? That's what you have to do. Agent risk includes the inputs, the tools, permissions, actions, and the integrations. So you team— your teams must test prompt injection, what happens when you use bad data, tool misuse, and also runaway costs. And move 10, smart teams are capturing decision reasoning in systems they own. All right, models are rented, but company reasoning history can become durable memory. I've been talking about this for years. Right. One of the most important things your companies can do is to start collecting first company reasoning data, right? FCRD. I've used the acronym once or twice before. So the simplest way to think about this, the old school transformer models needed structured data, right? It gobbles up everything on the internet. Models that reason models that think that's today's models. Yes, they're still good. Yes, they still need structured data, good data, your company's good data. It also needs how your company thinks and how your company makes decisions because the smartest thinking model in the world is only gonna be as helpful as the nuanced information that you give it. What do I mean by that? In the same way that you would teach and invest in a human over years to understand why you made this decision. Hey, here's why I rejected this proposal and here's why I crossed off this line item. Those things aren't always found in a spreadsheet. That's not always structured data, right? These are the things that smart teams are capturing, um, putting them in documents and training models to work through. All right, so now let's get to 10 questions every leader must ask. All right, question number 1: What breaks if our primary model disappears tomorrow? Yeah, maybe a little recency bias in putting this all together, but I think the whole situation with the Anthropic Fable-5 and the US government really sheds a much-needed light on this AI race that has become a frenzy? What happens if that model is gone? You have to be able to identify the workflows, the customers, employees, and systems that are dependent on that one model. And what happens if it goes away? All right. I'm not gonna answer these questions. These questions are for you and your team. Play this episode for them. Share this episode with them. You need to be talking about these. Question 2: Are we solving a task, workflow, or operating model problem? Because tasks need prompts, workflows need systems, and operating models need redesign. So if you just have an AI summarizing meetings, that's a lot different from updating projects and owners. Because you need to be not just solving the task, but the bigger picture as well. Question 3, what proprietary context makes our AI better? And this goes back to that first company, Reasoning Data. Generic models just produce, produce generic outputs. And it's the same thing, even though these models are becoming smarter and smarter. If all you're doing is feeding your data to a world's smartest model in the correct way, that's not gonna help. I mean, it's gonna help a little bit, but that's not gonna be what ultimately separates you because your competitors are gonna be doing the exact same thing. They're gonna be putting their best structured data with the best reasoning models or the best harness, right? If competitors can match your outputs instantly, then the model was never your moat. Question number 4, what can AI read, write, change, or send? You need to know all of those things. And there are very few— yeah, let me say this honestly— there are very few AI leaders out there that are in charge of AI rollout across their organization that can accurately answer these questions. And it's not their fault. It's because this space moves too quickly. So unless you have a team of people reading every single changelog, Right? Or unless your whole team listens and reads the, you know, if you listen to our newsletter every day, if you, or sorry, if you listen to the podcast every day, read the newsletter, you're probably ahead of most teams. But still, even at that point, you have to know every single model, every single harness, every single connector, which one has read-write access, which one requires approval, what permissions, right? Which permissions are needed, you know, when using which AI super app, what can do what. You have to know those things. Question 5, who owns every AI workflow? What one person? Asian crash is going to happen at your organization, in your industry, your competitors, et cetera. You have to know now. Number 1, you have to map the workflows, but number 2, you have to know who owns each workflow. And that person has to be able to answer those questions like question 4, what can AI read, write, change, or send? And you have to know who Who owns that workflow? Question number 6, what is the cost ceiling? The ceiling is important. We saw some stories, we don't know if they're actually true or not. There's one story that came out that one company, quote unquote, accidentally ran up their monthly AI bill to $500 million because they didn't put a spend cap on Claude. Was that true? I don't know. But what is the cost ceiling? You have to have those measures in place. Not just how do we monitor what the cost is, but what cost should you be paying, right? Should you be paying, uh, I don't know, a million dollars a month to summarize emails? Maybe. I don't know if, if, if your organization, if you have hundreds of lawyers reading very important emails, maybe. Right. But you have to know what the cost ceiling is because Agentic AI can spend money invisibly. While appearing to be productive, right? That's why you need to look at things like artificial analysis's cost per intelligence, right? Because a lot of times people just look at a single benchmark and be like, oh my gosh, this Fable-5 thing is great. Well, it, what if it costs 3 times as much, uh, as something like GPT-5.5? If you get the same exact output, but it costs 3 times as much. What's your cost ceiling? Question 7, what counts as a useful output? My gosh, I've seen people just really redefine and twist what a certain end artifact should be because an AI created it. Generated content means nothing until the business accepts it, uses it, and you can prove an ROI on it. So you need to say, what counts as a useful output? Where is your AI spend going? , and is it producing an actual useful output? Question 8, what happens when the AI is wrong? Okay. Not when it goes out, but what happens when it's wrong? Do you have the right expert-driven loop that can identify when an AI output is wrong? Hallucinations, yeah, they are not really a problem anymore, but not everyone is using the right model, so they obviously still exist. If you do the context engineering thing correctly, if you have expert-driven loops, If you're using the right model for the right purpose and you've gone through the guardrailing and the scoping and all those, let me just say it. Hallucinations are extremely rare if you do all those things, but most companies do not do those things. So you have to ask the question, what happens when the AI is wrong? Question number 9. What evidence could we show 6 months laters? 6 months later. Yeah. Leaders, you need records of models, data. Prompts, actions, approvals, and changes. Some, right, at some enterprise levels, like if you're using ChatGPT Enterprise, Claude Enterprise, right, Copilot, they do have some useful, um, metrics, I guess, right? I like Microsoft's approach with their Entra ID, uh, and, and a lot of the other companies are starting to adopt something similar as well. But every single piece of value that is created via an augmented relationship between a human and AI, it needs to be observable, it needs to be traceable, it needs to be auditable. All right? You need to be able to go back 6 months later and say, oh my gosh, we ended up winning this huge RFP, right? And we had this, you know, RFP agent cooking it all up. Let's go back. How do we do that? Who set that up? What models were we using? What was the workflow design process and why did we get wildly different results from all these other RFPs when we were using the same process? You have to not only be able to go back and say, hey, this worked, let's go back and, uh, make sure that we apply some of our successes across the board. But you have to also say, why did it work in some scenarios and not others? And then the last question, question 10. What should we stop doing if this works? And this is the big one. This is the thing about shifting human agency. And maybe I'll— this is good to end this episode, episode 800, with a bigger question. Something maybe thought-provoking because it's something I'm thinking about all the time. What happens in 6 months? What happens in 9 months? What happens in 1 year when these agents are even more and more capable? Go back and look at where we were a year ago and look at where we are now. It's scary. There is no wall. It is a ramp going straight up. If capabilities continue, if the harnesses continue, if these agents that run on your local machine 24/7 and can produce artifacts in the same way that humans can and that they are economically as valuable or more valuable than expert humans, if all these things continue, What should we stop doing when this continues to work? Should we stop spending time as humans on the things that we've spent the majority of our career on? Should we stop? Let's just say, let's just say you love creating PowerPoint presentations, right? Let's just use that as a small example. You love getting the team in the room and, you know, uh, brainstorming and, you know, pulling in all this, all these documents and creating a banger presentation and you present it to the client and they love it. What's, That's your job, right? If that's your job and you love it, but it's shown that an agent is better, what do you stop doing? I don't have the answer, but maybe on episode 900 we will. But that's a wrap for today, celebrating our 800th episode. 8 uncomfortable truths, 10 moves smart teams are making, and 10 questions every AI leader must ask. I hope this was helpful. You know what? I'll say one thing that is helpful for me, hearing from all of you. So if you didn't make it this far, 50 minutes into this episode, reach out. Let me know what would you like to see in episode 801? What would you like to see down the road? I know we normally on Wednesdays do our AI Working Wednesday. So thanks for, you know, letting us throw a little curveball in the rotation. We'll be back next Wednesday. with our normal Working Wednesday series. But thank you, whether you've been around for 1 episode or all 800. I hope we can keep going to 900, 1,000, and beyond. So if this was helpful, do me a favor, tell someone about it. I can only make it to 900 and 1,000 and more by giving you unbiased, straight AI info when you tell people. So, uh, please, number 1, subscribe to the podcast if you haven't already on Spotify or Apple Podcasts. Then make sure you go to youreverydayai.com, sign up for the free daily newsletter. Do me a favor, drop me a line today, drop me an email. I would love to hear, you know, whether if something's been helpful along the way, how long have you been listening? What would you like to see us do differently in the next 800 episodes? I work for you. So thank you for tuning in. Hope to see you back tomorrow and every day for more Everyday AI. Thanks, y'all. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.