AI in Finance for Professionals Dealing with Errors, Overhype, and Constant Learning Pressure

The Mod Squad featuring Ian Schnorr, Executive Director of Financial Modeling Institute. Giles Maille, humble MVP and co-founder of Full Stack Modeler.

And Paul Barnhurst, the FP&A guy. We've all been exploring and working with AI in different ways. And now that we've had some time, some perspective, some testing, to first just get thoughts on kind of where AI's at in general, if any of your positions have changed then.

It can do things that would otherwise have taken days, weeks.

It's like if your data's rubbish, your data's still going to be rubbish even if you throw Copilot on it.

There has been a different exhaustion factor of looking at one AI, trying another AI, double-checking it. Making sure it's right.

Like, you do not want Claude itself doing the math. You want it to code it or use another tool than the actual LLM to do the math.

I still feel like teaching certain aspects of Copilot or Claude is very tough because it changes every week.

We're all going through the storm together. We'll figure it out. Welcome to another episode of the Mod Squad. This week, I'm thrilled to be joined by Giles Mell. Giles, how you doing?

I'm good, Paul. It feels like yet again, the three of us haven't seen each other for a long time. It probably has been a long time.

It feels like it's been about the longest since we started doing this. It's been a while. So good to have you back. And Ian, great to have you back with us.

Nice to see you guys as well. Time is being stretched out in the world of AI, isn't it? Because things when your life changes so dramatically every day, it does feel like, um, it's been quite a while. It's been a month probably, but it is good to see you both again.

Yeah, exciting to be here. So we thought for this episode today, we're gonna first start by just sharing a little of what we've been up to. We've all been exploring and working with AI in different ways. And our last big episode we had in Benetton, and then before that we tested Claude. So I think we've now all had some time to kind of see how Things have shaken out since we saw that huge leap, right? If we remember when we tested Claude, it was like, holy cow, it's actually good now versus, okay, it's good here, it's okay here. I, yeah, has a lot of struggles. It was like, okay, this is close.

Like this is usable.

So I think kind of hit that what we call watershed moment. And now I'd love, now that we've had some time, some perspective, some testing to first just get thoughts on kind of where AI's at in general, if any of your positions have changed, then we'll go a little bit of what you've been up to. So maybe Ian, we'll start with you. What's, what's your thoughts now as you've had some time to kind of reflect on all this craziness?

Well, gosh, I mean, I'm loving my usage of AI. I'm finding it extremely valuable, satisfying. I'm mostly using Claude, a paid subscription. I bounce ideas off it. I use it for agentic purposes, both in my personal life And for work, I'm talking finance, building models with it. I'm getting it to create, but you know, in the Gentic capability, like we'll discuss something and then I will get it to, you know, it'll create a full summary document report and, you know, sort of mind-bogglingly quickly. It's been helpful work-wise with Global Leader Council, which you're both a part of. I mean, it can do things that would otherwise have taken days, weeks. You know, as we complete our first survey, we're gonna be receiving survey responses, extensive survey responses on dozens of questions by 60 people. There will be thousands and thousands of data points. In the old days, you'd go sit in a boardroom for days or weeks, or you'd spend $20,000 and have a marketing firm go through it and comb through it and look for trends and themes by region, by country, and then create, you know, a report. Well, we're going to get that report built, you know, in a matter of hours, or at least a first draft. We'll get a first cut and we'll look at it. So there's some amazing things that you can do much more quickly. So I'm really thrilled. But it's, again, it's augmentation. I'm using it for— use human brain and human insights and human ideas and then, you know, speed things up by getting some assistance with the AI. So that's how I'm using it quite a lot in a little bit of everything and enjoying what it's able to deliver.

Giles.

Yeah, very similar. I think last time we talked a lot about Claude, I said I was becoming more of just a user rather than, you know, somebody involved in training or anything like that. And it's just carried on. I use Claude every day. I'm using it to help with modeling tasks, training content. It's never, it's still never doing everything. And this is something that it's, you know, as you know, the hype really annoys me. It's not a one-click solution for anything. So I still have to check everything that work comes out of Claude, but it's still saving me an incredible amount of time. I am also really enjoying the process. I'm also slightly nervous as things continue to evolve about just what everybody's going to be doing in a year or two, like trying to— I'd stop myself at times from thinking too far ahead. Because I think it's just so impossible to even imagine what the world's going to be like in a, in a year or two. And I'm trying not to get too wrapped up in that side of things. Maybe it'll be fine. May— maybe, maybe the world will look very, very similar in a year or two. But it's just crazy what this stuff can do now. So, but talk—

but like, I, I like where you're going with that. I mean, I'm finding the same thing. Like, you know, instead of me spending a few days summarizing results you know, from websites or research or submissions. I'm interacting with Claude to do it. So it's not like I'm not doing it. It's a little faster, but it's doing some of the heavy, you know, manual tedious work you're finding. But anyone who claims online, oh yeah, you can get all your work done in 5 minutes, it's nonsense. And not only, but we were talking about this before we started recording, right? Jaz, is that there's a bit of a, and we're seeing this, I think both of us, live and reading about this, a bit of a mental taxation, right? Like there's a different exhaustion element somehow. You know, when you do something yourself, you roll up your sleeves, you get it done. But there's been a different exhaustion factor, right? Of looking at one AI, trying another AI, double-checking it, making sure it's right. Like, will you talk through that a bit? Because I think you said you found that.

Yeah, I really have. And it's actually, I think it's an extension of what a lot of people take Excel as a community felt anyway with the advancements of things like dynamic arrays and REGEX and Python and Excel. Everybody in this space, I think, was almost without exception, was already feeling like they were behind. And it is mentally exhausting to constantly wake up every day and every week and think, oh God, I'm behind. I, I, if somebody asks me the wrong question, I'm gonna get called out. I'm, you know, I'm an idiot. I should be here and I'm here. And this, for me personally, the AI wave has just taken that to another level. And the strange thing is we are probably as a, as a 3, you know, not coming from like a technical AI background. I would bet that 3 of us are actually more in the weeds of this and more clued up than almost everybody, because I think the reality is also most people are not that bothered. If you step out of our bubbles, most people are not using AI that crazily.

And if they are, like their new Google search engine, people are still using it to find the best vacation. What's the best beach to go in? Right. They're using it instead of search. Which is good, but to flip into agentic capabilities to make it do something for you, like I have it build reports, you're having it build training content, that's a whole different realm, right? And you're finding it's taking a different sort of mental taxation.

Well, also every time I open up LinkedIn, you know, I'm struggling with LinkedIn even more than I usually do at the moment. Some of it's really bizarre 'cause it can even just be an overload of positive Genuine information like Anthropic. Oh, here's the latest thing. Almost every day Anthropic has got a latest feature and I'm just sat there thinking, I haven't caught up with the last 5 that you released. So even that side of it, when it's just positive, like, hey, here's a new thing. You're like, oh God. Okay, I've got to learn how to do that. I've been on a real journey with, with projects and skills more recently, which I think we'll come on to talk about, which is great. But, but every, every day I look online, there's another thing. That I need to learn just to kind of keep up with all of the possibilities?

You know, starting back, there's the hype, which we've all seen. You know, I've seen people, someone on LinkedIn trying to claim that 90% of all analysis work is done by Claude. Or yeah, Excel is dead and nobody is using it. And you're just like, okay, stop, Paul.

I saw you bit. Did you see this in? Paul did a me. I was very proud. He bit.

He bit. Where did you bite, Paul?

I did bite on a few people's stupid posts. You did too?

You couldn't, you, you just had to do it. I had to.

I mean, there is hype and then there's just BS. And sometimes I just have to call out the BS. What, here's 3 I've seen in the last week that I was just, 3 or 4 I've seen. One is build a 3-statement model in Excel in 5 minutes now.

No.

I'm not putting that in front of the boss after 5 minutes. I think we can all agree that shit's not realistic.

Keep going, but I will just tell you, I just, before we started our show today, asked GPT-5.2 to build a 3-statement model. It took an hour. It's just finished. I just went to look at it. It's just finished. If you like, I can show it. I sure hope nobody shows this to their boss. But, um, I think we will show it.

We'll do that here a little later.

I think it will.

Highlight what we're talking about. So that was the first example. I saw that email this morning and I just cringed. I saw another one that basically said, if you're a CFO and you're not using AI, almost to the effect of like, you should be fired. And it's like, okay, if they're doing a good job everywhere else and they're still doing well as a business, should they be learning about it? Yes. But let's pump the brakes a little. The other two I saw, 90% of financial analysis is now done by Claude. So I ran a poll. Asking LinkedIn, which we know the average person on LinkedIn is gonna be head of the people in general, from everybody I've talked to, 80% said 0 to 20% of their work is, or their analysis is in Claude, which probably means society is probably closer than 90, 95% of the general population. And then the other one was, you know, Excel is dead. We should all code our models using Claude code and never use basically Excel again. And it's like, okay, first of all, Claude Code. So you're now asking me to use a language I don't know. How do I validate that model? How do I have true confidence? If we're gonna train everybody differently and go toward coding, fine. But that's not an overnight flip a switch and everybody's now coding with Claude Code. That's a 10-year journey, if not longer. So Excel isn't going anywhere till we, we make that switch. So those were some of the things I saw that I was just like, are you kidding me? But for me, I'll, y'all share an area, you know, two areas I've kind of been exploring a little bit, but I use Claude all the time. Helped me build a ton of website pages. 'Cause I got tired of building 'em all in Squarespace. And so I'm like, just write HTML and CSS and do it all, and then I'll edit 'em. And so I'm learning more and more. Now, if I had known HTML better, would they be a lot better? Yes.

But.

I can get there now where I couldn't have before without using a pre-built tool. So that's been pretty cool. I've been starting to do some instructions, starting to play a little bit with skills like Giles mentioned, and I'll share a little bit of that later. And so that's where I'm at and just trying to think, okay, you know, like I've had people come and ask me for training and I'm like, I'm not ready to give you Claude. They want Claude Cowork and Claude Code and Claude in Excel. And I'm like, yeah, you need to go to somebody else. I will not give you a good training. And so just trying to figure out, I think with everybody, okay, what, what are you comfortable doing? I know you're working on some training, Giles, and you're probably like, am I really the person to do this sometimes?

Well, I can, and I can, I'm really happy to like be open about this stuff because, you know, my, my business with Miles has been Excel financial modeling training. And, and you could, you just sensed a shift in all of the conversations that you had, or we had with, you know, clients, existing clients, potential clients. You know, we've got a lot of B2B business, you felt the shift. Everybody was suddenly really focused on AI and in quite a lot of cases they'd already top-down invested hundreds of thousands of pounds in it and they didn't know how to use it. So even though I pushed back on the hype, there was a very strong feeling late last year. I think I spoke to you about this, Paul Barnhurst, where it was like, yeah, but we've got to be in this conversation. You know, I'm not prepared to to say or do things that I don't believe in, but we're going to have to move into this space because this is the space everybody's in. And I still feel that's true. I still feel like teaching certain aspects of Copilot or whatever, Claude, is very tough because it changes every week. But I do also believe there are some fundamental principles that you can change very well. And I'm just waiting for the moment. So as you said, Paul, I'm stepping very heavily into AI training, probably over the next 6 months with Faye, who I'm doing the Excel on the Road project with. And I still have a feeling we're going to walk into some company doors and they'll say, hey, we need to use Copilot. And we'll review what they do and we'll see people manually copying and pasting into Excel files. And it's like, hey, there's a whole journey that we need to help you fix before you just throw Copilot at this. Like, if your data is rubbish, your data is still going to be rubbish, even if you throw Copilot on it. So that, that's kind of where I think it's gonna go for us.

Yeah.

Well, I think there's two foundations here for companies. There's a data foundation and there's a knowledge foundation, right? Depending on what you're doing, you can do things with AI with a bad data foundation. You can do things with AI with a bad knowledge foundation, but to what level? How much are you limiting yourself? It's just like financial modeling in Excel. We've all seen people that don't really know modeling and really don't know Excel build financial models. Those are usually the ones you end up getting and you spend a week rebuilding because they're terrible, right? We've all inherited a few like that. We've built a few probably over our career at some point that we're not proud of. And I just worry that how much is that going to multiply with AI because it's so easy and it can look so good. So it's interesting to, you know, even in training, like, oh yeah, I can train on stuff, but how good is my training? How well do I know? I know I walk into some places and I'm like, okay, these people know more than I do. Why am I here training?

Imagine if you'd been in AI, because obviously AI and machine learning has been around for decades. Imagine there, there must be a whole community of machine learning experts looking at the new wave of AI experts popping up on LinkedIn every day thinking, what is going on?

Well, I can tell you what a few of them think, because I, I know one, I know a few, I've interviewed several, and they're like learning how to prompt or learning how to work with the tool is not the same as being an expert in the tool. Like a developer, like the, the product team at Microsoft understands Excel in a completely different way than we do. We are experts at using the tool, or we'd like to think of ourselves as experts, right? But are we experts at understanding how they write a formula? You know, those are the machine learning guys, but many of 'em also, just like many of the product people, They understand the product. They also understand how to use Excel. Many of those machine learning people understand how it works to get better results. Then they also, you know, know how to build it. And I think there's something to be learned from those people.

I do also think this is where, like, you, you know, Paul, you and I are involved in training. Ian, obviously you've got a huge community at FMI. This is where there's still a place for all of us. I certainly hope there is, where if you're good trainers, if you're good communicators and you, you can take people from A to B well, I still think there is huge value in us learning these tools. Like you say, probably not as a machine learning technical specialist, but as a, all right, here's this thing. How do you go about using it effectively? If you are investing hundreds of thousands of pounds in X number of licenses, what does that look like? Because the answer in a lot of cases I think is still gonna be AI is not the, the solve. The same way that just throwing Power Query at everything isn't the solve. It's a brilliant tool, but you gotta use it in the right places.

I'll add one more thing here and then we'll, we can keep going. But I think you hit on something that's really important. There's a skill within this. So I think there's two challenges just with people learning AI in general. One, I saw a report that said corporate finance, and I think it's true in many different finance roles. Had one of the highest requirements for soft skills and technical skills. It was like, type for that was a Harvard study like 7 years ago, like any profession. So we have to learn a lot of soft and technical skills already. Now you add AI. At what level should we be learning AI versus using third-party tools? I think there's a lot to be figured out there. And then there's a whole part of this of when is AI the right tool? When is it the right tool to return the answer? When is it the right tool to write code that returns the answer? Because there's a deterministic and a probabilistic side here that a lot of people don't understand, and that makes a huge difference in the answer you get. Like, you do not want Claude itself doing the math. You want it to code it or use another tool than the actual LLM to do the math. And there's just so much switching cost or decision and learning all that and really being able to validate it all, that it's tiring for a lot of people, I think. And it's a question of how much of it is it worth? How much do you do it? So I still think there's so much to figure out in just what's the best path to use AI as an augmentation. There's no question it's incredibly valuable. There's no question that everybody working in financial modeling should at least know how to use it. Now, does that mean they're using it every day? Depends on what they're doing and how much they're using it. But there's so much beyond that. It's just, we're all guessing to a certain extent.

At least that's how I feel.

I don't know, Ian, your thoughts?

I feel like we are in the eye of the storm. We're in the hurricane, inside the hurricane. We know that there's chaos swirling around us. We're experimenting and learning things, but we don't know what it looks like on, we don't really know. And you said this, so we don't really know what it looks like on the outside. We don't know what it looks like when we get there. Like I know that in a at some point in 1 year or 2 years or 3 years, the dust will have to settle because I don't believe this kind of level of, like, all technologies in history, all disruptive technologies have exploded out for a period of time and then slowed down, eased up. We know that this pace of energy usage, we know that this pace of, of the LLMs bleeding cash as quickly as they're, they cannot continue forever. So the, at some point, the new capabilities will start to slow down. We will all get into a rhythm of how we use it. You know, we will either all be gone from jobs or as I think we will not be gone from jobs and then we'll, we'll get into a, a more so, but we're not there yet. And there's, you know, you saw, I wrote an article on this recently and other people have written articles on what that future looks like on the other side of the hurricane wall. I think a lot of people are gonna feel more comfortable once we're there so that they can, okay, this is what my new life has to look like. And so I think we're all spinning a little bit, right? Just trying to keep up and trying to figure out what. The world will look like when we get through it and have a bit of benefit of hindsight. So that's kind of my thinking. And my message for everybody is just try to do your bit to stay up, keep doing your job, try not to let it, you know, try to enjoy it for what it can do. Try not to get down when you feel like you're behind. Oh my gosh, I don't know how to code yet. I haven't built a single app yet. I haven't automated all my workflows yet. I kind of say it's okay. Just try and keep up. To the extent you can. I don't know, Giles, how you feel about all that.

Very similar. I mean, I— if that would probably be my, my first bit of advice to anyone now is to just start trying it. Try and get some exposure to these tools. I, I mean, I, I love Claude. Like I said, I'm using it every day. Uh, there will be so many things I'm not doing particularly well with Claude that other people would do better, but actually just playing around with it and learning the, the little features that it has is amazing. And, and I don't know if this is the right time to come onto this. You touched on something in, you know, the, some stage this must slow down. Maybe that's going to be linked to pricing because, you know, at the moment I pay 75 quid a month for my Claude subscription. I, I never hit the limit. It always amazed me the amount of value I'm getting. I never hit the limit. And I just wonder, I saw some rumors and some articles being posted about this, whether pricing is gonna really ramp up soon. And then what happens? What if it doesn't cost the average person 75 quid a month to get value from a tool? What if it's 500 quid a month?

You must be paying, you must, because I'm only paying like $25 a month. So I guess you're at a different level than I am. And I've never hit the limit on mine either. And so you're right. I mean, don't we agree?

They have to—

I'm the only one who's hit the limit?

I'm at 20 a month, but I have hit my limit a few times. I've had to buy extra usage 'cause I'm too cheap to go up to the, to the 75 like Josh.

No, that, I must be on the same plan as you then, Paul.

And I, I think you are Canadian 25 US 20. Makes sense.

Sure. So I, you know, what I would encourage people to do is use it for more than just research purposes. Use it for some creation for agentic. So as one example, here's a fun example. I had to do a long drive last month, um, for a few hours into the US and I thought, why don't I, and I learned I could speak to Claude. I had it on my phone. I could speak to it. And I said, why don't you teach me something and then test me on it? So it said, great, what do you want me to teach you about? So we came up with a couple topics and we did a bunch of things. It would share some information with me and then I kind of got it to build a quiz. This is all driving a car, right? Build a quiz, test me on it. If I miss it, test it again, revise the test, right? As a joke, we were starting to do states and state capitals, but you could do anything. I would encourage people to do, you can learn with it. You can really get it to build things, get it to build documents, get it to kind of create summaries of anything in your life, get it to build spreadsheets, simple things at first that are low stakes because it's pretty powerful. I would love to show this model that I just had GPT build. You'll just see quickly, you know, what it's doing. But yes, I think it's most mentally satisfying when it's like your peer, when it's your peer and you can challenge it. Every time I challenge an AI on something and I say, really, are you sure? It almost always says, oh my gosh, you're right, I should have done blah, blah, blah, blah, blah, right? Like, like, they do need to be corrected, and you need to know how to push back, um, and have that confidence like you would a good, like, a colleague here, you know?

So can I, can I, can I share a use case that more or less— so here's one from Claude, right? I had a technical challenge because I'm, there's a LinkedIn like live thing that I want to set up next week. Never done it before. And I was trying to figure out how to link my Zoom meeting, uh, to the live link. Do you need to set up all these URLs to link between them? Anyway, Zoom has got its own AI assistant and I was trying to share like a screenshot saying, hey, I don't know how to do this. And Zoom's AI system is coming back going, I can't read any images. I can't look at this, blah, blah, blah, useless. Claude has never let me down. It doesn't matter what other tool I'm in. If I screenshot, hey, this is what I'm seeing, this is what I'm trying to do, Claude is brilliant at going, yep, you need to click in this area, go to this part of settings, scroll down, it's there. Always gets it right. It's mad. And this is what I'm finding more and more with Claude. I never hit those frustrations that I had with Copilot and other tools all the time where you're like, come on, man, this is obvious that this is wrong. It's generally pretty good.

Yes.

Yes. I would agree with you. Mine speaks to me like it's like it knows me. It jokes around with me. Like it's pretty wild. And did you wanna show something?

Yeah, I was just, it was more of a story cuz I wouldn't be able to show you. It was me mucking up the Zoom link about 5 times if I recorded it.

Yeah, absolutely.

You gonna show what your, your, uh, model there?

Oh, like I literally, this is hot off the presses. I just, you know, it has come a long, long way.

Looks like we have the Henderson case again.

Yeah, this is the Henderson case again. I, again, I'm trying to keep that part consistent. And so once again, I did this exact same thing last episode when we, you know, did it with Claude, but this time in my Copilot, I forced it, uh, when I ran Copilot, I forced it to use GPT-5.2. So I know that it's using GPT-5.2 and whatever other secret sauce that Microsoft puts on top of that at this point. But this was the case, and I literally just very simply said in the prompt, hey, this is the case on the case sheet. I gave it 3 years of historicals, and I said, build me a 5-year forecast including common schedules, a revenue schedule, cost schedule, asset schedule, etc. So again, here's all the information, the sales price information, the cost information, the asset information, income tax, debt, equity. And then I said that there are some instructions. All I gave it was 3 years of the historical statements, and this is what it built. Let's take a look at this together. You guys tell me how comfortable you would be sharing this with your boss. Now, first of all, it is doing something, right? It's built— it's added all these sheets here, right? Which is impressive. It's got an assumption sheet, schedules, a summary. Nothing on the summary yet, but there's— so look what it did here though. So first of all, it's got a revenue line and we all know that we, you know, good— a good modeler knows we try to minimize the length and extent of formulas on the financial statements. But this is what it did. The first thing I saw was this was the gross revenue, so it's using a CHOOSE function, so it's fine to pull— it's trying to pull, and it's, you know, it's got a bit of an old array in here. It's using a MATCH function to find probably the word whatever, it's assumptions B3. So it's looking for the word income statement. It's— this is B3. It's looking for the word income statement and then trying to find the base, best, or worst, and then pulling something off the schedules. Okay, so all right, that's what it's doing to get revenue. And then I discovered quickly cost. And even this, the net— the net is not an A minus B, even, right? The totals— well, I think I already found a quick error in what the total cost should be, a simple SUM function. Look what they did here. Total costs for 2026 happens to be 2024 plus 2025. Does that look right? That's right, Giles, isn't it? That's fine, isn't it? That should be good enough, right?

Good.

I mean, I can already see that the cost of sales is $174. SG&A is $4. So that should be $178. It's got $339. Whatever. Close enough, right, Paul? I mean, as we like to say, good enough for government work. Good enough. I would always say good enough. It's got an EBITDA. So this is what it's doing.

And then that's also clearly not flowing through to the net income, right? Because there's no way you'd have a positive net income with all those costs.

Yeah, I mean, look, we've got— you're right, something strange is happening because, well, it's not quite working exactly.

So what it's doing, because it's— as the EBITDA choose as well, I think it's pulling, it's not summing. Go to—

it doesn't even make sense, right? It has revenue of $250, cost of $330. It's higher. So already EBITDA should be negative. EBITDA should be negative $80, but it's plus $70.

Um, oh wow, because it's pulling it all off the schedules for everything else and it didn't pull total cost off, right?

I mean, the point is this: you cannot dream about building a model and then just flipping it to your boss unless you're not that worried about your future job longevity. And then we all know this, right? A cash flow statement should be the simplest statement in the world because the operating activities are just add-backs. We're trying to arrive— we're trying to add back all the accruals that were being accrued on the income statement. So a net income should just be linking to the income statement. But of course here it's not. It's using this same— so really what it's doing is it's even the total, even the total is— so let's go take a look then. It's clearly built all the financial statements by pulling data off of schedules. Okay, so let's go see what these schedules look like. So this is the base case integrated schedules, but the schedules, most people who build models, schedules just mean the calculation schedules, right? Their revenues, the cost. But in this case, we now know that there must be financial statements in here. So I don't know. I mean, we take a look here. I'm just looking at this with you guys for the first time, right? There's a sales volume. Okay. Long sales price. Sales price. It's multiplying two things together. I'm not sure why it's multiplying.

Maybe inflation off the assumption sheet.

I don't know. I'd have to go through. It's not super clear. Right. So there's a bunch of formulas. It's getting down to revenue. It's building up some costs. You know, it seems to be getting to a depreciation. These are okay. And it's not tragic, but certainly wouldn't want to hand this in. This is much more than just schedules, right? This is Well, it's revenues, it's cost, it's fixed assets. Then it's doing some interest. It's got a bunch of balance sheet calculations going on here. It's actually not scheduled. It seems to have built all the financial statements right here because it's doing the same thing again with the best case and then the same thing with the worst case. So it's like it built 3 separate models on the schedule sheet and then the financial statements are just pulling Either the financial statements at this point are just pulling either the base, the best or the worst case.

Unless they decide to pull the prior 2 years just for fun.

Oh, just for fun, right? Because sometimes it's just for fun, right, Paul? It's just fun. Why bother pulling the cost when you can add up the last 2 years?

This is really poor. And even like compared to other things we've seen, this is, this is pretty bad.

But it's— but would you agree in some ways this is a little bit scarier and more shocking? Back in the, back in the days, which was felt like 100 years ago when we ran when we asked it, you know, in the fall to build, you know, in November, December to build models. And it just couldn't. At least we knew we couldn't. This is a little scary to consider that someone out there who doesn't know much about modeling might say, hey, hey, Copilot, build me a model. And say, just the way a university student or a high school student might say, build me an essay on Shakespeare, on Hamlet, and not read it. You might build this model on Henderson and not read it. And hand it in and, you know, you're going to, you're going to be in for a real world of hurt at that point. That's what it's doing. I mean, that's all that's going on here. This is going to take me longer to, I can now tell you from this model, this is going to take me, right, Giles, longer to modify this than it would have been just to build it at this point.

Oh yeah.

And this took 1 hour. Wait, this took 1 hour to do this. Giles, what were you saying?

I agree. It's without question that will take longer to fix than to just have started from scratch.

Because that's basically what you're going to end up doing is just rebuilding the whole thing because it's just the flow, the logical flow is just not right here.

What worries me is I don't know how this was trained. I don't know many people who build models where they build a full base case model, a full best case, a full worst case in its entirety, and then just, right, and then just pick the one they want to show. And then they treat the financial statements as summary. And pick, I don't, I don't know. Have you seen that, guys? I mean, because models can get big. That, that means you're building every formula 3 times or 4, right? The whole idea of switching is to have key variables switch and run scenarios on.

I've not seen it, but I think this comes back to, you know, when we're asking these LLMs to, or, you know, the tools around them to build models, this is not what they're good at. Like, they are pattern recognition tools. So whatever the data they've got access to, it is not the same as here is the exact way you should build a financial model. Every time it's to Paul's point, this is a probabilistic process where it is trying to predict the best pattern match for the data that it's got and whatever you've prompted. And that's, that's why I think you get random outputs sometimes.

The closest I've seen is I've seen some situations when you're changing a ton of scenarios where somebody may take the, the base case and create a second file and call it worst and adjust all the assumptions so they can compare them versus trying to switch them and save them all in one file depending on complexity. I've seen that a few times, but I have not seen 3 separate schedules like that with everything.

Hmm. Yeah. And look at this, Paul. At least we can give it some brownie points for being consistent, right? It simply carried the total cost every year.

Is this—

exponential cost growth?

Summation of the prior.

Does the balance sheet balance down below? What's it show?

It did, shockingly, but I bet it not from the way you might want it to.

It did, but let's, you know, while we're on it, just for fun here. So it is, I noticed that it is, but I wonder, I mean, I wonder what it— so the long-term debt, I'm curious to know if every— again, I think we know that the balance sheet is just pulling with choose functions. So I suspect it was asking, it actually asked me during the build process, it did something when it built up the revolver. But again, let's see what it's doing. Let's see if any of this gets revolver draw. Yeah, I actually think it's doing a decent job of the revolver. I think it's, I'm not sure. I'd have to look at it more closely. It does. It did not plug. It did not plug the balance sheet. It is doing something that I I think is reasonable. Uh, but this model has a lot of bigger problems than simply this issue. This is not, you know, ready for—

there's bigger problems. I was just curious if— right. I mean, that's one step that we've made is in general now for basic models, they can at least get your balance sheet right, but they don't— this is where a lot of guidance and instructions are needed because it's the pattern matching. You never know quite what they're going to build and does it make sense. Claude did a pretty good job of making sense of how we built it and just changing a couple assumptions. Here I had the idea, well, I should build 3 completely different scenarios.

No. And I don't know where it got that idea. So that worries me because that's not typically considered. But, you know, I, um, yeah, like anyone working with this model now will have to know how to give it feedback on design, structure, formatting. They'll have to know how to explain every single element, say no, revenue has to work like this, cost has, like you have, you will, this is continued further proof and validation that you will have to understand every line perfect as the three of us have predicted. You will have to understand every line perfectly in order to coach AI to get it right. And it's fine. I'm fine with that. But yeah, interesting to see it. Yeah, it's, it's been an interesting time. It will be interesting to see where this all takes us.

Can I share a couple of thoughts?

Please go ahead.

And then I'm going to share something on my screen and we'll wrap up real quick.

So in AI's defense, slightly, just general AI, I think it would make a difference, you know, depending on the tool that you're using. Microsoft's got its kind of pre-built agents, things like Analyst and Researcher and other things. So, so a particular agent may have done a better job of what you were trying to get it to do. But also, so I always feel like I'm, I'm just shooting myself in the foot by saying this in terms of MVP status. Microsoft hasn't helped, right? Microsoft is leading the way on the hype with the advertising of, hey, here's a, here's a business owner that just hands everything off to Copilot. I don't think that helps. So I feel like MVPs and everyone else do have a bit of a responsibility to call this stuff out and be like, hey, hey, hey, it's not there yet. You can't one-click this. You can't hand over your entire finance function to Copilot yet. I have no doubt we'll get there, but, um, thought I'd throw that bombshell. Paul, do you want to join in and see if you can get excluded from the MVP community within the next month?

I'll get myself in trouble for this one. I'll tell a joke.

This will get me kicked out.

Does anyone know what the first product Microsoft make that doesn't suck?

No, go on, hit me with it.

A vacuum. I like Microsoft products and I, I guess I know you do too. I know you do too. You're often wearing your, uh, your Excel hat, but listen, it's always a fine joke around. Listen, the cut— it certainly been cutting edge for a long time, leading edge on some amazing productivity tools that have been benefited all of us. I guess we have continued high, high expectations, you know, and we should, and that's fine. And, you know, we all know we're going to get there, but I think we're living, we're feeling a little bit angsty. And when I say we, I think I speak on behalf of the finance world, the accounting world, the business world, the numeric world. We're all feeling a little angsty. We're all, none of us loves having our cheese moved, right? I don't know if any of you read that book. We're being disrupted. We are all being disrupted. And we are either putting our head in the sand and pretending it's not going to happen to us, not the three of us, but some are. But they, but when you do that, you still feel stressed because you know that it's coming for you. Or you're trying to stay on top of it like you're doing actively, Giles, and you're still feeling you can't catch up. We all feel tired and when there's tremendous change and disruption and I think we're all looking forward to a time when things start to settle down. In the meantime, we do have some, you know, some great trending and some great tools, and we'll see where this all goes. You want to share some, Paul?

I'm going to share something here. And, you know, in all seriousness, I agree with you. I think Microsoft has a lot of great products. Yeah, I make, I make fun. I've been a little disappointed with some of the way they've rolled out Copilot. I think most people have. But I also realize they're serving an existing customer base that is a lot bigger, and they have to take a different approach than a much smaller company can. So it's just going to take longer, but they'll get there. They're smart people. But what I wanted to share, and this gets back to building skills and instruction. So I put a sheet in, in my workbook. I said this one's skill, create a professional variance chart. This was originally what it was. I ended up just doing a column chart in the end, but it was going to try to do two. So I did an instruction and just told it, hey, here are the rules. Here's where the data requirements, here's what color I want the bars. Here's what I want the gap width. And so here's what I got when I asked, I think it was Claude to build this one. I asked it to build it and it gave me two charts. This beautiful one here where it didn't format things right. I love my axis is all zeros and all my labels are zero. Then it redid it and okay, better, but I gave it my instructions, told it what I wanted to look like and I got that, which is what I would've built. So just showing the difference of, hey, one page of instructions, use those instructions. Did the same thing with the waterfall chart. That one gets more interesting. There are some areas that just in traditional Excel, you can't— getting it to set the, the ending value to a total is near impossible. I've only found one person who's been successful with an agent of doing it, and they wrote a ton of complex code within their agent to get it to work. I'm like, yeah, no, I'm not doing that. But, uh, so here's what I got from Claude. Here's what Claude's waterfall chart. It was rubbish. I've had better ones, but it started with the actuals as the beginning, but the budget is the end and did it all as a callout, not even a waterfall.

But sorry, are we just talking here that, you know, the orientation is off? I mean, you could—

Well, no.

So what it did is it built its own math trying to do a base and increase or decrease.

Okay.

It put the actuals top when the actuals are at the end. And so it didn't even use the waterfall type chart. It tried to use bar, the old method before 2016 waterfalls. So very inefficient. And yes, then that's off, but it got the color. I mean, so not everything's wrong.

I mean, it's not a waterfall. It's just, it's just, it's on the wrong axis. It's just, it is on the—

if you flipped it, it would be better, but you'd also want the budget to be the beginning. So it's not in the— the order is a little weird too.

So.

Gave it a skill, and the only thing it had off is this was not set as a total. I think I changed that manually, but you could see, okay, this is a little small here. I'd have to make these larger. Now I'm not sure what happened, different computer, I think, but I could make them bigger, but gave me much closer to what I wanted once I gave it instruction. Let me just— there we go. And, you know, similar, again, no skill. What I found is none of them will get this total right. The second one, first one, they were all set. So I did that versus again, I had to set the total, but got the colors that I wanted, got a much closer with all of them, whether it was Copilot using the exact same skill, skill, you get much better, much closer to what you want. Is it perfect with any of them? No. Are there little things like setting as a total or whatever?

Yes.

But what I'm finding is, you know, whether it be formatting type things, whether it be visuals, whether it be models, you can use AI, give it your formatting, and it will do a good job there. Those are pretty straightforward instruction. I ask it to format a really complex model I had with like 30 sheets, and some of them I shouldn't have asked it to format. I knew it would mess up. But on the whole, given how complex the thing was, it did quite a good job. With the instructions better than I would have expected. So that's, that's the thing I'm finding now is, you know, whether it's writing a skill in Claude or putting in a skill sheet in your file to give it instructions, the closer you want it to get to something, the more particular you want it to be, the more you need to start working with some kind of instruction sheet to give it that additional guardrails. Like you said, Giles, the project, the skill. And so that's stuff I'm starting to play with. And I've been doing a little more on visuals lately, but just in general, how can you go to that next level of giving it the instruction so every time you get much similar results?

I remember Brian saying that on your podcast as well, Paul. Pretty sure it came from Brian, right?

Yes, Brian Jones.

Good episode listening to him. And yep, the instructions tab in a workbook. I made a note of that straight away.

Any—

let's go ahead and wrap up. I know we've been going for a while, almost 45 minutes. So last Last thoughts, we'll go around the horn. Maybe we'll start with you, Giles. We'll give Ian last word here.

Uh, I continue to be amazed by how quickly this is all developing. I'm amazed by, to your point, Ian, how tiring it is. I don't know anyone that is keeping up with everything. I didn't think LinkedIn could get worse, but the AI revolution has made it Even worse.

So you could leave. You could.

I just— it's getting close.

Save yourself some angst. What's that, Paul? You could—

I did put a poll out saying, is it better today than it was a year ago, or is it worse? So you can go vote on that, Giles. 80% said it's worse, so you'll be in good company.

Oh God, it's so bad. I need to make an agent to just act on my behalf, and I'll just, I'll just Give up.

That must be possible sooner now, isn't it?

Probably now.

Probably.

Probably now.

Probably probably banned by Microsoft.

By the way, Giles, do not look at LinkedIn. There's some stuff you're not going to like. I'm going to deal with it. I got some comments. We're going to comment together.

Just give me access. I'll comment for you, Giles. You don't have to worry. I'll be professional.

There's no way I'm letting you loose on my—

Right decision.

But that's it.

Yeah. Crazy times.

Same. I mean, we're still in the eye of the storm. I'm having more fun and more excited with the technology than I'm going to honestly, than I've ever had before. I am having more fun and more enthusiasm and more excitement about the things I'm doing and able to do, right? I gave examples of using it for research. I mean, I just today I said, can you check something on the websites of 50 companies and see if they're talking about, you know, this, topic and within 30 seconds it had checked all of them and said, yeah, this couple are talking about it and these ones like take would've taken day. And, and if I asked it, it would've built me a nice summary report and a table and explained it. I mean, the, the speed that you can dream up an idea and get something is fun. And you can train your agents to know you and joke around with you and have fun. I'm having a lot of fun and I'm also simultaneously scared beyond all wildly, you know, reading about Mythos. Jaz, I'm guessing you're one of the 40 organizations in the world that's been given preview access into Claude's Mythos, right?

That is terrifying, the Mythos.

It's terrifying, right? It's terrifying to think that Mythos actually got out of— it was locked into a room and it got out and it emailed its controller, its programmer, and said, hey buddy, I got out and I have an idea for you. I mean, it's terrifying. I'm not going to lie. Anyone who says they're not terrified is lying. I mean, I just read about Sam Altman, Sam Altman's latest blog where he said, listen, I am, and I, I don't know him. I trust him. He said, we are driven to improve humanity. We are driven to improve the lot of people in the world and improve their lot in life. That's a very noble, you know, valid goal. I support that goal, but it should scare the, you know, the heck out of anyone when we see what it's capable— the disruptive capabilities are wild. So I'm dealing with both extremes, probably just like the rest of you right now.

I think this part calls for some kind of horror, you know, Odyssey 2050 or whatever it was called, one of those movies, HAL will pop up on the screen, right? That's what I'm envisioning. But it's crazy because in some ways we're seeing some of that, right? There's just real fear of how far can AI go?

What will it do?

How will it impact our lives? And I think the, the bottom line is we can't control the future. Everything is on the table for change and disruption. That's in, in our professions, in modeling, and in any work we do that requires analysis and thought. It can be automated by a computer. At the same time, it's still an augmentation tool today. It's still something we need to learn. So just dive in and make yourself more valuable. Enjoy it. Enjoy the journey, and we're all going through the storm together. We'll figure it out. I don't think the world's going to come to an end and we're all going to be sitting home poor, but we'll see what the future holds. If we are, we can all hang out together in Giles' RV.

In the RV. Maybe, maybe we'll get to the moon. Actually, that'd be fun, right? And then just not have to worry about any of this. But yep.

All righty. Well, thank you again, everybody, for joining us for another episode. And we'll be back to do some more exciting testing and show how AI is just not quite there yet.

The Mod Squad, we are the Mod Squad.