How Excel AI Agents Like Rosie Work for FP&A Tasks but Fail at Building Models with Giles and Ian

Foreign the Mod Squad, featuring Ian Schnorr, Executive director of Financial Modeling Institute. Giles, male, humble MVP and co founder.

Of Full Stack Modeler, and Paul Barnhurst.

The FP and a guy. What's interesting is it doesn't show you all the thinking right. I kind of like that on a.

Quick glance, has got all seven levels right. Quite impressive to me is all of the levels are formula driven.

This is using Excel's column function to automate that, which is very nice.

It works in both Excel and Google.

So I asked it to build a DCF for me at the bottom by using a weighted average cost of capital of 10% and a terminal growth rate of 2%. That's all I gave it and I wanted to see what it did. And it actually did a quite a good job on one mistake which I had to fix. It was using the wrong.

I'm pretty impressed with it. I think it's done well, especially on the esports cases.

Instead of me giving it the prompt since it came out and said, hey, do you want that? Let's see what happens if I just say, hey, go ahead and do what you said you're going to do. Welcome to another episode of the Financial Modeler's Corner. Again this week we have our special series featuring the Mod Squad that includes Giles. Mel. Giles, welcome back.

Pleasure to be here again, Paul. Nice to see you. Who, me or Paul?

You.

Giles had a pretty major announcement, so. And a pretty major release. Yes.

Oh, the music video. Yeah, yeah, yeah.

I think we'll cover that in the news because we have a new section about the news and that's important because Excel had a big birthday, I heard, so. Welcome back, Giles and Ken, welcome back.

Thanks, Paul. Great to be back for another episode here.

Yeah, really excited to have you. And you know, as always, my name is Paul Barnhurst and I get to co host with these two amazing modelers. I almost said something else, but I can. I stopped myself.

I don't want to know what you were going to say.

We're here again to test tools, have a little fun, share our views, and at the end of the day, we're really hopeful this is helpful and it's one of the reasons we've added before we get into what tool we're going to test and then the actual testing is we're adding a section each week. It was called, you know, the Mod Squad. The latest news, this space is moving extremely fast, but just from the last testing we did to today, there's been a lot of changes. But before we get into kind of the AI news. Excel had a birthday September 30, it turned 40 and there was this big reveal, a video that was released and I heard it was quite popular. Giles, tell us a little bit about this video you did.

We doubled down. I did a music video a year ago when I got made MVP for the first time. And this time we went even bigger. So, yeah, to celebrate Exile's 40th, we hired a thousand seater theater for two days, got Deb Ashby back, roped Mark Proctor into it, and somehow roped Leila Guerani into flying over from Vienna and about 25 dancers, a children's choir, God knows what else. It was bonkers. But yeah, everybody seemed to like it. That was the only thing I wanted. I just want people to have a good time.

If you haven't seen it, check out Giles Mail's LinkedIn page and take a look. It is absolutely worth your time and well watch, you'll get a great laugh and enjoy it. So I highly encourage everyone to take a quick look.

Thank you very much.

Fully agree. It was very well done. It was a lot of fun. It's amazing to think Excel is 40.

It's older than all of us at this point, isn't it? No, maybe not, Maybe not.

It's older than one of us.

Maybe close. All right, all right, Paul, let's go.

So, other news. This last week, Excel released in beta. An Excel agent, and I think that's a really big announcement. I'll share a few thoughts on that and then we'll go around the horn, so to speak, here and get everybody's take. I've done some testing, I've seen a lot of videos. I think it's very similar to many of these tools. We test in areas that will be stronger, in areas that will be weaker. But there is one big drawback against all the other tools that will change with time. By the time this released, it could change with how quick things move is it's currently only available in web. And Giles, when's the last time you built a full model in the cloud version?

I've never built a model on the web version of Excel. Never. I can't use it.

So, yeah, what's your take? Do you ever use the cloud version?

No, I don't. And I, you know, in my prior life, when I was a trainer before, you know, heading up fmi, when I was a trainer and ran a training company for 20 years, and I trained thousands and thousands and thousands of bankers, you know, all over the world. I Never worked with anyone who, who was using it. So I'm not a big fan of most powerful tools like that in a cloud version where you're really keyboard intensive. Certainly Excel I have not. So I'm not a big fan of it. And yes, for now it only works for now this new agent only works in the cloud version of Excel. I assume that it, I have no reason to say this, but Paul, maybe you do. I assume it will become available in the, in the, you know, desktop version of Excel at some point.

That's the general assumption. Time will tell. I mean usually they start with the cloud and then they release it to desktop. So I'm with both of you. I've never built a model in the cloud. I mean, I've used it for a few little things, mostly when I need to download my forms because it opens them online and it's just a quick look at something, not building anything, consumption, great building.

No, there's a great purpose. There are a lot of great reasons to using spreadsheets in the cloud. Collaboration is better, et cetera. So great. But for building truly hardcore rigorous spreadsheet tools, I'm not aware of, I'm not aware of anyone that's largely using Excel in the cloud as opposed to a desktop version. And the other issue is, as you said, I mean it, this new agent tool looks really powerful and does, I would think at this point, I would think that it must threaten the numerous companies that have come up with their own add ins to Excel. It looks like Excel is going to fill that gap and meet or possibly exceed what these existing financial modeling AI tools are doing. Time will tell. I don't know if either of you have a thought on that point.

I totally agree, totally agree. It feels like, I was going to say it just feels like. It's interesting, isn't it? Because what we're doing, our whole focus for this podcast series was on the tools and we're kind of one full episode, full review down and Microsoft has come out with the agent and I think it's great, it's a step in the right direction. I think everybody's just gotta be objective about what it can and can't do and how you use it. But yeah, I love the idea that it could all be managed by Microsoft. Maybe we will be building full scale financial models with Microsoft's AI tool in the future.

Paul, what do you, what do you think about the. What do you think?

I'm a little more measured. I think it, it definitely could be a big issue for a Lot of these tools, but I also think there is some room for some of them in that they're very much specialized. This agent is still going to be broader.

Yep.

It's not like it's just doing modeling. So is it going to do that? Those extra little things that push it over the top and make a modeler want to use it and how long will that take? It's not the same priority. So I think there could be room for some of them. But do I think they're nervous? They better be. Do I think if Microsoft wants to. If it's a primary focus to say we want our agent to be the best in the world in modeling, it will be these other tools. Right. Because it has access to. They've announced Anthropic is Now available for OpenAI. They have ChatGPT. They have the resources. They have access to any of the code to do whatever they want. Right. Once you integrate something and you decide you want to do it, it doesn't have to be on parity. I mean, it has to do most of the things they do. But if it makes other things easier because of the way they can integrate, yeah, that's hard to compete with. So if this is a real. If modeling is their focus, it's a matter of time.

And I would also say. And how you feel about. I would also say there's. We are in an era where there's a lot of hype still and no one really knows exactly what the future looks like. You know, I continue to be of two minds. I love the direction things are going. I'm amazed at the technology. I'm amazed at the speed and some of the benefits that will accrue as a result of these tools. And yet I also remain firmly entrenched in my view that with modeling specifically, which means thinking and understanding and analyzing, that there will always. That the best and the brightest will always balance usage of tools with deep knowledge and deep insights themselves. Which means I don't foresee a day when you push a button, get a tool and show up at a board meeting that. That day. I don't know if that day is ever happening. You'll, you know, just be wildly embarrassed if you show up at a board meeting and you haven't understood every single thing going on in your model. So you have to understand it and know how to challenge it, and you have to know how to push back on these tools and use it as your partner, just like. And that takes time, just like you always used to. And I don't know if the two of you have a thought on that.

So similarly, I think we're all of the same mindset, which is a great start for the podcast, but I'm excited for the future. I'm just really anti hype and I think people need to be really careful with the language.

Tell me you don't like cheat sheets, Giles.

I'm anti hype. I just get annoyed every now and then with loose language. When you hear people saying, oh, you can, you can click a button and what you used to do, in four hours, you can go get coffee and it's done. So that's, that's, it'll do something, it'll build something and it might say income statement, but it's not at the level where you can just go, oh, we don't need an analyst to do that now. But I'm sure it will get there. But just, just don't, don't hype it up because then when people use the tools, they're going to go, oh, this, this is not what I thought it was. This is not what I've been reading about.

But that, the commentary, when, when there's commentary around, take a six hour task and get it down to six minutes. There will never be a day when you can learn. I mean, that six hours was not just clicking away in Excel. That six hours was your thinking process and your design and understanding how you're going to frame and shape and form the data. So taking that down to six minutes is some of the build and that can be great, but you're still going to need a lot of time to kind of get yourself caught up and up to speed so you can be an intelligent leader on a team. That's not going down to six minutes. And so I think there's some misconception over, you know, how it's going to, you know, impact people's jobs.

But anyway, Paul and I think one thing less, you know, people think we're like, hey, the agent's just going to take over. There's no space for these tools. We still think it's important to test them. We still think this testing will allow us to see where the strengths are because I guarantee you not every tool is going to respond the same. We'll test agent in this group before we're done to see how the Excel agent responds. So with that, why don't we jump into this week's tool? Really excited. It's Rosie AI. I think all of us had opportunity to chat with their CEO Dennis, and I'm going to Start by just sharing a little bit about the company. We'll look at their website similar to what we've done before and then we'll jump into testing and we're going to do the same order we did last time for testing. We're going to start with our humble mvp. Then the bearded guy, then the smart one of the group will close out for us.

It is not true, Paul. There's only one MVP in this group.

Well, there's only one humble MVP.

There's only 100. Oh, that's right. I forgot you were an MVP as I forgot that. Paul, you don't talk about it as much.

Why?

Why can't you talk about it all the time, Paul?

Because you do more than enough for both of us.

Giles, you can be in the next music video. You'll be all right?

If I can, then I'll be famous. All right, so let's jump into Rosie AI. So what's interesting is Rosie has now launched. It's out there. It's one of the few many are planning on doing it. It works in both Excel and Google. So I think that's a good step. There are definitely people that's really helpful.

Right. You got that full cloud.

And Google, if you want a cloud spreadsheet, it's the strongest cloud spreadsheet out there, most history and so forth. So launch date, it's one of the older tools. It's been around for two years now.

Hilarious. We're in an industry where an old player is two years old.

I know, but when you think of AI, that's a lifetime.

Absolutely.

It's focuses modeling in Excel. It's really looking at finance and consultant professionals. So I think ideal testing. I really like what it's done in its pricing. I love that it has a student version. You know, $10 a month. I'd love to see that even lower. But I like that they're hey, trying to accommodate everybody. The individual is at a very reasonable price point at $20. And then hey, if you want a team, contact us.

Yeah, right.

I appreciate the pricing. Any thoughts so far, Giles?

The same thought just again with the spectrum of pricing that we're seeing. 10 and $20 a month for, for a proper license is fantastic. I think it is interesting that they're on Google Sheets and Excel. Yeah. I'm not a Google Sheets user other than just to manage things with my own team, but I'm sure that's a good bonus.

I quite like Google Sheets for that. It's funny, I almost think them as different sorts of tools. I mean there are certain tasks that I naturally gravitate to Google sheets like collaborating with our team building collective spreadsheets to analyze, project, whatever we're doing that of course we use sheets for. I think we've all. I've met Dennis, the founder of Google, Rosie, a great guy. I really admire what they're doing. I'm rooting for them and for all of them. I'll be honest, I am rooting for every one of these because I think there's, it's amazing to see what's happening. I'm rooting for them all and I really hope that, that there's a space for them. It's not going to be easy. Right. But as we've, just as we, as we seem to think but, but I, I, I, I root for all of their success. So. Yeah. Did you anything you want to finish on the, on the summary, Paul?

I think I'll mention is they use GPT.

Yeah.

They're, they're married to one company, they have multiple different models. So I think that's interesting. You know, we'll say what they, you know, what they say makes them unique. And we'll get to kind of test this. Rosie combines deep semantic indexing with native integration. You know, some of that work they've done on, on their own to really allow you to accomplish more. So we'll get in and test at.

I think we discovered in our last episode that, that having the choice sometimes, you know, I think we all find in life having too much choice is never always a good thing. Right. Just, you know, just show me one paint color, you know, if red and it's perfect. Right. I mean having too many options for anything can be overwhelming. I think we discovered when we had a number of choices with the last tool. Sounds like a good idea. But I think we found that that was.

There were definitely some challenges in that. Right? For sure.

So now we're going to see the opposite. Now we're going to see the opposite. A tool with only one. And we'll see how we feel about.

Switch model if it's going to use any different GPT models. So we'll see a tool that hey, choice is kind of gone, so to speak and we'll see how it performs.

But even that's interesting because I remember last episode, Ian, you were saying like God, it would be so much better if the tool made the choice for you. So to some degree, albeit within one area it's doing that which is interesting.

That's what Rosie's going to do you're saying? Yeah, yeah, yeah, yeah, yeah, it will but it still stays within. It still stays within one family. Correct? That's what you were saying, Paul?

Yeah, yeah. It stay. It's all within open AI. So we'll take a quick peek at their websites, similar to what we did before. You know, very, very plain, not in a bad way, very simple website.

Right.

You come right in a blog, data security contact us help center.

I love it.

You got Rosie, you got the options, you got a video, you got the. Everybody has their testimonials. Here are some of the companies that are using it.

It's not bad companies, Tesla.

No, it's not. And what I wonder is I Ikea. I'm wondering if some of these are individuals testing as small departments. I doubt those entire companies have integrated the tool across their finance team. But if they have more power to him, he's doing awesome, right? So hey, Rosie gives you spreadsheet powers. Catch errors before they catch you. Modeling made magical. Now here we discussed this a few times. Rosie can build a new model from scratch or iterate on existing ones. Just ask Rosie and it's done. Little, little hype, but I get it. They're selling. Less struggle, more answers. Why you'll love Rosie. No formula error, save hours, work across sheets. Excel for everyone. Smart data analyst works with your data and then you can watch a video of it in action. And it has pricing so pretty similar to last one. Anything you guys want to say as I scrolled through that.

Yeah, no, no more formula errors is a bold claim. Did you say it said that?

Yeah, it said eliminate hash, ref hash, NA and other common errors with AI that understands what you want.

Okay, okay, I see what they mean. I thought it was making the claim that by using it there will no longer be any errors. It's saying it can find hash errors and remove them for you. Gotcha.

Yeah. So you can eliminate those errors from your models and. All right, so why don't we go ahead and jump into the actual testing here.

So this is the first of the two esports cases. This is the simpler one for quick memory. It's seven levels that I'm going to get it to do starting from simple things like pulling apart string from a text. And it gets more complicated as you go through counting characters in a text string, referencing a map somewhere else in terms of the rosy interface. Again, this is an add in. So you've got a button on the home Ribbon tab, you've got a very nice friendly note that comes up saying like hello this is. I'm Rosie, your AI spreadsheet assistant. I haven't clicked anything and it's already basically given me a summary saying, this is what this spreadsheet does, in my opinion. Here's what I can do. So very nice. Where would you like to start? I'm going to type in that same prompt again, saying, look, I want you to analyze this, solve all the questions. So what have we got? We've got the, the A. A more concise summary of what has happened here, really just says like, here's the levels, here's what the area is, here's what I've done as a solution. But I have to say so Rosie, on a quick glance, has got all seven levels right. And I think what's quite impressive to me is all of the levels are formula driven.

Yeah. Can we see one?

Yeah, yeah. So level two, so trace light again, this was one area, I think, where it wasn't quite as efficient. It is referencing this text string. It is doing the text split, taking the two largest values and summing them together. There's no hard coded value, do you remember?

And I remember on the last one. And we were impressed. First of all, this is very impressive. What I remember we were very impressed is it was able to read all of the values. And of course what you're asking people to do in this question, right, is to. You've got a big giant string of text in the numbers area. You're asking them to pluck out the individual numbers and then what were you.

Asking you to add on the top? The height, the largest.

And it was very clever. The last tool was very clever, but it managed it because it's a computer. It typed in the dead numbers within your formula and it knew the largest and smallest. Right. Not something a human could learn from or a human could do themselves. So we were impressed, but recognized. So this one is literally doing it exactly as you would expect a lot.

Of the Esports players would do. It is the same for level three. You got to pull out the column.

Paul, were you adding something there? I mean, I'm impressed.

One thing I was going to add, I think one of the key differences here, and we'll see this with different tools, is the last tool you saw, a lot of times it said it was doing it in Python and then kind of converting it. I don't. We don't see any mention of that here with Rosie. So I think there's a little bit one different model, but different approach on the back end of how it's solving These and that's why I say there's room for a few tools. I don't think agent necessarily just takes everything because right. There's different approaches and so there's going to be different strengths and weaknesses across the board. And I think we'll see that throughout all this testing.

Sure. So I'll run through this quickly, but for me, every level has been answered with good formulas that are linking to cells rather than putting hard coded numbers in. So this is using indirect and column to get the column number. Counting the number of times this character is in the string. This is exactly what we would do. Len minus len with the substitute of the character for nothing. Level 5, which tracelike didn't interpret correctly. This has done correctly. Level six, you've got indirect of the map. Level seven, indirect with this shift for above and below.

Oh, you can see the OP with an offset with a simple if. Really? Yeah, really clean formula, really strong.

And again, you know, I'm going to keep, you know, coming back to this point. I love it and I love the fact that it can do this. And I also believe that someone looking at this needs to understand what it has done because someone is on your team is going to ask you, what does that offset? What is an indirect? Right. Indirect is one of the most feared and least used functions in all of Excel and you need to understand it if it's using it. And as we're seeing, all of these AI tools so far have at times an inclination to hallucinate, get things wrong. So you do need to be able to double check it and explain it. So I love that it can do it, save you time and you need to know exactly what it's doing. I would strongly encourage anyone, never, never hand in something like this without double checking to make sure it's working. Would you guys agree with that?

I'm going to push on to the second one. I mean, the other thing that I would just mention again is, as we said last week was my prompt wasn't saying too much in the way of like specific. Here, here, here's every bit of information. So I'm impressed that it can interpret what the challenges are and then do the right work and use the right formulas. I think that's really impressive. So, okay, we are on to the what. What I think is a significantly harder case with the humble MVP case. Let me find the same.

So who is the humble MVP? I mean, we have two MVPs on the call. Z, Me. Or is that you?

That's the humble.

I just, I just I don't like to talk about myself.

If you have to ask, Paul, if you really have to ask, then you're not. You're not in with the program.

So I know I need to get in with the crowd. All righty. So we have the Microsoft Excel UK Championship case running and we'll be back when it's finished. It's all about our humble mvp. So stay tuned to see how Rosie does.

Okay, so we're back and it's interesting because I, I tested Rosie on the. These cases before the recording started and actually it did. It did this second case in full. It did take a while this time. It did levels one to three and then it just seemed to die. It said there has been an error and it stopped. I've restarted it. It has managed to do levels one to four and it's just still churning.

Over the do level four.

It has done level four as well. And I mean that. So this, this is a hard level. The. This is difficult. There are a lot of things in play here about characteristics of my mood and the coats and everything. That's not easy to come up with. It's definitely still kind of working through five to seven. It did do this when I tested it before, but we, we paused for 15 minutes. I think that's too long, but I would still say that's quite impressive.

Very impressive.

Coming up with very complex, which is not necessarily a good thing, but it's coming up with very complex single cell solutions.

Well, look at that. Like it's using let and it's naming everything. You could. Someone could go through and trace that. That's complex though, right. Momo transpose is number search index and a let. Did I miss anything? What has seven. Oh, short by in a sequence in a row. So what is there about 12 different formulas in that thing?

There's a lot, but. But it is doing it formula based. This is more of a sort of programming approach rather than shoving hard coded numbers everywhere. It is leaning supportive information. If I F2 that it is leaning. I didn't tell it to do this necessarily exactly like this, but it's done a pretty good modeling job of those levels 1 to 4. So I would say 4 out of 7 just ran out of time. Maybe it would have got to the others in time. We don't know.

I mean it got there before once. So I mean overall impressive. Right. I think that the result is it.

Does a really good job here.

Yes. I'm going to move straight onto the third one and again, I'm not going to go through the whole subject again of what we're doing. This was kind of raw, raw trial balance data was essentially analyze the raw data and produce a sort of comprehensive summary with some visuals in there.

And this is the one that had the two sheets where you'd done your summary sheet. And if I remember right, Trace Light referenced that a lot, not a lot on the balance sheet. We would have liked to see a little more work on the transactional, so it'll be interesting to see where they focus.

Definitely. The prompt specifically says look at the trial balance data, which is this one, and use that.

We are back about seven and a half minutes of work. I think there's a couple promptings in there. Giles, why don't you kind of walk us through how it did and what you're, what you're showing us, what your take is here.

Yeah, it's a bit of a mixed bag, I guess, a little bit like trace light. So, so it has actually produced a summary that's formula driven, it's using good functions, it's looking at the source data. So that's good. I mean, there's a few, you know, hard coded things like revenue and you could have looked at the headers here, but that's not bad. It's giving me the outcome that I kind of wanted, which is to show that net income drops negative and then kind of comes back at the end. So that's the right sort of insight that you're looking for. So if you look at stuff like this, like it's, it's, that's okay. It's not beautiful, but it's done an okay job. I would say the graph looks a bit horrendous.

And why do you say okay? I mean, the, the analys in the columns looks bang on. Or is it missing? Is, Are you concerned about, Are you concerned about anything in columns A through H? Is it missing. Is it missing anything? Is it.

You know, no, I, I think that's my point. So, so in terms of like a, A, a professional report, the good bit is. I think it's actually done the technical modeling pretty well.

Yeah.

Would you show this as a final report page? No, I think it would need work done to it. But it's, but like it's, it's okay.

It's a great, it's a great start, right? I mean, it feels like it's a great start.

Well, it's, it's an improvement on what we saw last week because I think the approach to getting the data and looking at the source data Using the right functions is good.

But one thing that's interesting I think was better last time in my take is remember it gave us the product breakout. Here we don't get a product breakout. I think it's summary is a little better in that it took it all off the trial balance. I like that it gave us operating expenses. The last one missed that. So I would like to take some from both of them, you know, if I'm doing my final work. And that's what's interesting is there are some areas on the whole I think this one did a little better job on this task but there were some things I liked from the first one.

I think that's fair. I think it's done. I think it's done better. It's not perfect. We probably aren't expecting perfect but again, pretty good use of functions and logic to get like a pretty future proofed answer. I think.

Before we move on to my testing, any last takeaways from what you've tested for Rosie?

I mean I think it's done well. It's, I mean especially on the esports cases I think they got full marks on the first one. When I trialed it before on the harder case it actually got full marks as well. But it just again that's an interesting talking point. Same prompt, same challenge. It just broke after 3, 4 levels a second time.

Part of it could also just be how busy AI is and the app versus Rosie. It could just be some of the, you know, what's the volume hitting it.

I'm, I'm pretty impressed. I think it's done a good job.

It's only getting better. It's going to have to, you know, continue to stay a step ahead of of Microsoft's built in tools. Right. But, but it, this is pretty, pretty mind blowing that it can do this. But again like, like take, take it with. Right. It's not a, it's not replacement yet. Anybody. It's a great start. It's getting some great stuff done and it's impressive what it's able to do on the things you've tested it for. Cool.

I think that wraps up my bit.

Well, let's go ahead and move on to some FP and A use cases. We'll be able to test two or three here. We're going to test, we tested this last time, we'll be testing it again. This is a price volume mix where it's looking at margins and saying how much did the price impact it, how much did the volume impact it and then how Much did the mix of the individual products impact it? It's Paul called Paul's Getting Spicy. It has three different products. We won't get into the details about them, but there's one called BBB or bb, PMP and boss. And we can see we have budget and actuals. So what we're going to ask it to do is we're going to ask it to fill in the price right here for these three products. We'll start with that prompt. We're going to go ahead and copy it in. Basically I said, hey, look in this workbook we have a price, volume mix. I want you to give me the answers in these cells. So let's see what it does here. You can see it already did some work. It says this workbook walks through a PVM topics. I can help with variance calculations, result visualization, summary. Where would you like to start? I'll give it my prompt. All right, we're back. It only took a about a minute to do the first one and then another minute to do the next two. I didn't ask it to do volume or mix. It said, hey, do you want me to answer those as well? And I said yes. So let's walk through it on price. Did it perfectly. You can see it took the gross margin from the actual, compared it to the gross margin of the budget and the units and said here's the difference in price. So we're $30 better on price on our, well, gross margin per unit, which is, you know, we're kind of calling our price part of this pvm and that's really BB was better. This one was worse. This one was slightly better. Makes sense. All good volume. It didn't. Didn't quite get that right. So what we wanted for volume is we want actual volume in total minus budget volume times average budget price. It took the individual prices, but it didn't look like it did it quite the way it was supposed to. And so it came up with negative 250. The actual. And we can drop the formula should have been 125 so we could ask it to redo mix. If you look at average budget price is 125 and our volume was 100 better. So 100 times 125 should have been our 125 here it gave us negative 250. So it did something wrong on that one and then it got the mix right. So pretty good. Not bad. One issue. I'm just going to ask it. Please calculate the volume in total using row 29 for the formula. And we'll see if it gets it right.

You're looking for it to correct.

Yeah, I'm seeing if it will correct it.

Because you didn't tell that you didn't tell it that you think it made a mistake. So we'll see.

It is saying I'm adjusting the volume formula. So we'll see. It says I realized. And then what's interesting is it doesn't show you all the thinking. Right. You see almost everything with trace light. And I kind of like that because sometimes it felt a little overwhelming. Again, it gets back to which one.

Transparency, it's first. And you like, just to be clear.

I like this a little better of seeing the thinking. Sometimes I want to see step by step. It'd be nice. But it just, I don't know if there's a way to compress it or, you know, go back and see the detail behind it. You're not saying you like, don't see everything.

Yeah, you're like that. You're seeing dots indicating that it's thinking, but you're not, you're not seeing huge amounts of paragraphs explaining it. You're saying, correct.

Yeah, I don't want the huge paragraphs to come back, but I would love to be able to audit and go see them. And I don't know if that's, you know, there, There we go. It got it right now.

Right.

It did the, the right formula we wanted. Now it's 220. It matches here.

Well, listen, again, it continues to validate, you know, my notion that you still need to know what the heck you're doing. I mean, you need to know how to calculate these things. If you didn't, that was a pretty impressive formula it had in there before, which you knew was incorrect. But, you know, I, again, I continue to be worried about, about junior professionals running, using AI tools to fill to, to, to get answers and not really having the background or ability to check and audit and make sure it's being done correctly. Right. And if you handed in the prior one, I don't think there's going to be a lot of sympathy from your boss. Right. Paul, when you say, yeah, but I did it and you know, the AI tool got it wrong, I don't think there's going to be a lot of sympathy with that answer, is there?

And definitely, if you think to your point and you know, you still need to know what you're looking at and what you're doing. So, so maybe, I mean, what are the options here? If you know what you're doing, maybe you would do something through Power Query. You'd have most of the logic already set up versus you go to this sort of half finished template where you're going to choose to click an AI tool. I think people that know what they're doing are probably still going to default to the way that they operate and they can trust. I don't know, it's interesting.

I just think we're in for an interesting time. And like you said, Jasmine, if you do use a tool like this, you have to, you have to be able to validate whether it's correct, right?

And then when she did it was one check, one prompt and then it's right. So that's impressive, I guess.

I mean, the reality is, here's the thing. The reality is spreadsheets are still one of the few open source software applications, right? The expectation. This is why I'm always concerned when people use overly complex, you know, formulas within Excel. When you use, when you use a piece of purchased software, off the shelf software, there's no expectation that you're going to, you know, audit and check the code or that's fine. But the whole notion of using a spreadsheet for anything implies that it's open source and you need to be able to understand, check, demonstrate, present, show everything going on in it. And I worry that some people will move away or think that that's no longer the case when I think it might be more important than ever to know how to do that.

All right, so let's do the next one. This is the deferred revenue schedule. So last time we gave us some billing data where the customers were prepaying for multiple years and we said, hey, build the waterfall schedule of how we're going to recognize the revenue. So here's what it analyzed. Let's just take a quick look, open the file. And on my we said topics I can help with automating calculations deferred revenue schedules. It already said I could do it. So instead of me giving it the prompt, since it came up and said, hey, do you want that? Let's see what happens if I just say, hey, go ahead and do what you said you're going to do. We're back and we're looking at the deferred revenue schedule. I did ask it to format some of the dates, but I was editing one of the cells so I don't think it got it all done. It told me what it wanted to do and it told me to exit the formula mode and it would do it. But let's walk through it anyway, we could see what that looks like. But if you remember, you know, one of the others we did, and I've seen this with other tools, often we see them doing some hard coding for dates. But here it definitely referenced it. Right. And so that's good. Now here we'll just quickly convert this to dates so we can kind of see that the dates look right. We've seen others where the dates didn't quite come through. Right. That all looks good. We'll do this. Since I didn't give it the chance to format it all and it all looks. Looks right. What is weird, and you know, you and I were talking about this, Giles, as we were watching it, as we were waiting is the formula it wrote here. And so let me, you know, C4 divide. So C4 is the total amount. It's dividing it by the number of months. And so it's taking, you know, months from the beginning to the end. And so instead of using date diff or just taking the dates between each other, dividing months, it's taking the year times 12 plus the month. So like 20, 25 times 12 plus the month. So, you know, some crazy big number minus the other. It works. It's not wrong, but it's just. It's a very not intuitive way for a modeler. Right. If I handled this to an early stage modeler, they'd look at those formulas and go, what is it doing? I mean, we did for a while, right? Neither of us just said, oh, here's what it's doing. It took us a minute versus a traditional one would be like, oh, that's easy. We know what it's doing. The checks all look right. You know, one thing it didn't do is it didn't give us a total at the bottom. The other one did. So the. The other felt like there were some areas that was a little cleaner, but here it was better with the formulas, which I would prefer. Right. I can clean. Clean the other up. So I do think it did a really good job here. I like it. I mean, this is something. Within a couple minutes, I have a presentation ready. I can dump in more companies and it can do the work.

Yeah, I think that's pretty good.

Yeah. So solid. I want to try one last one here real quick. I have one more here I want to test. And anyone who's worked in FP&A has built many a waterfall bridge. So it's analyzed the data. It's already asked me, hey, do you want me to create a waterfall step? By step visualization set up. I'm going to copy my prompt and give it, you know, the opportunity to see if it builds a waterfall chart. Now, you know, as we're doing this, what's interesting is different tools definitely take different approaches. Like, you know, we tested Trace Light previously. It does all its charts in Python. Some will write it against will do it in using the objects in Excel to create a chart. So let's see how Rosie does that. I'm curious to see, you know, kind of the approach it takes on this task. It didn't complete the waterfall chart. First I copied my prompt that I wanted it to use, then it hung up on that prompt. So I closed the file, reopened it and asked it to build the waterfall it wanted. Based on the setup, it said, hey, there's two ways I can do it. I said, yeah, I want a native waterfall chart. I don't want a placeholder chart. I want you to build the real thing. Thought for a few minutes, you can see here. Native waterfall, please. Never built it. This is an area where I think charts and something. There's some real opportunity because let's take a look at this. If I'm an analyst and I want to build this chart, I'm just going to come up here, insert recommended charts. It's the second one. I have a waterfall now. Am I going to clean that up? Of course I am. So then I would come in and you know, say, hey, I need to get rid of the grid lines, change the colors, probably get rid of this, all those things. But all right, put a chart title. Gonna take me, what, two, three minutes at max to get this here looking like what I want. So obviously that's an area where it didn't do well. Not sure the reasons. I've tested this on several different tools and I've never been able to get it all the way there. Even here. It got one thing wrong that I'd have to fix, right? It didn't put this, this should be a total. So I want that there. So now they're both totals. So it really shows the start and the end. So obviously an opportunity where we'd love to see some improvement there. I've played with multiple different tools different times and I've never been able to get the chart quite perfect. I've got some to build it. I'm not sure what happened there, but disappointment is what I'd say at this point and something we can do on our own pretty easy. We're going to get in, set up to do some financial Modeling examples. And we'll come back here in a moment. But we're through the first two sections so far and before we jump into ins, Giles, any thoughts so far after seeing, you know what, six tests?

It's mixed, isn't it? I think there's some really impressive things and then there's some things that it just doesn't seem quite as capable or, you know, such a good fit to create the solution for. So, yeah, I'm still impressed. I'm still blown away overall by what these tools can do, but they're not.

Perfect to in for some financial modeling use cases here.

All right, thanks, Paul. Fantastic. We're going to get into some financial modeling and I have brought my favorite modeling helper, Benny. Benny the modeling dog. Right, buddy? Are we going to do some modeling together here? He loves financial modeling and he's really pretty quick on the keyboard as well. But yeah, we've got some great tests we're going to run here. I'm gonna have to put him down. He was barking so I picked him up. But I'll put you down, buddy. Okay, you can say hi to everyone and I hope that settles him down. All right, let's do this. Let's dive in. We're going to do some similar things that we tried on the prior tool. And you can see here I've got the standard one of our fmis Henderson model. This is the exact type of model that our AFM candidates have to build from scratch on the exam. So again, if you haven't didn't see the last one, this model includes a cover sheet, an executive summary sheet. It's got an assumption sheet, it has the scenarios and then the full. This is a vertical. This model is built using sort of a vertical orientation style. And it's got all the pieces of the model, the revenues, the costs, the financial statements are underneath it. The income statement, cash flow statement and balance sheet, and then followed by all the supporting schedules, the depreciation schedule, tax, working capital, et cetera. Now I'm going to start my testing by looking at an existing fully built model. There are some problems in this model and I'm going to start by asking it to help me understand what's going on and if it can figure out the answer to these problems. And then we will get into seeing if it can build something like this from scratch. So then what we'll do is we'll turn to another Excel file here. This is just the historical three years. This is just the historical income statement, cash flow statement and balance sheet. And we'll see if it can transform this. This is what, this is what any modeler has to start with, right? You guys would know, any modeler who's building a model has to start with just the historical statements and build it up and we'll see what Rosie can do on that front. So let's dive in here then. Let me open up Rosie. And the first thing I'd like to do is check for some, for some issues. So I'm going to ask Rosie a couple of questions, including this. I always like to do some checks so it realizes, by the way, I didn't it when it turned on, when I turned on Rosie. It recognizes that it's a comprehensive financial model for hinders manufacturing even that's impressive with detailed assumptions, scenarios. It's telling me what's in here and so it's even suggesting how it can help me. So that's pretty cool, actually. But let me just start by asking number one, are there any hidden sheets in this file? I mean, it's not one thing I'd like to know. It doesn't take very long to look for them. I did that in the last test. But it's thinking and it says it did this pretty quickly a few minutes ago. I just want to know, are there any hidden sheets I should be aware of? And it says, yes. Look how quickly that thought. It thought for seven seconds. It's impressive. It's impressed with itself, I think, right? It's seconds. It says, yeah, check me out.

I only took seven seconds.

Look at that. I know it's, it's flexing right now, but I'm impressed. It knows that there's a sheet called hidden and Clandestine. It's a very hidden sheet which I talked about last time. Would you like me to unhide either of them? Let me, let me see that. It's, it's offering, it's offering to. Should I ask nicely also?

Yeah, yeah, definitely, always. Goodbye.

Ask nicely and say, can you please unhide them? And I'm saying yes, please. Let's see if it, it can take action quickly. What do you guys think? Will it be able to do that?

Good enough. There's a study they did where there are two different things I've seen kind of conflicting. One says you generally get better results if you'll say please and thank you.

Oh, come on.

Really like a human for conversations. And another that said you got worse result. You get better results if you're mean to it.

Well, it's not my instinct.

That was really interesting where you basically need to tell it like, just shut up. Don't give me all this extra fluff and give me what I want. It was really interesting. When you're very direct, almost curt with it, sometimes you'll get better results. And then someone else said your conversations tend to be better. You're willing to go further when you say, yes, please, and thank you because you treat it more like a conversation with a human. And you're willing to go back and forth with the tool.

Isn't interesting. Well, I mean, it's. It's seemingly was not, you know, being polite. Being polite has not exactly won the day here for me. Has. I don't know where it's at.

Be a jerk. Say you haven't unhidden them. Get to work. I want to see these files.

How would I say yes? I said yes. How about that? Is that rude enough, Paul? Is that aggressive?

I think you could do better. I guess for a Canadian, that's not bad.

That's fine. Let's try it. That's pretty aggressive.

I mean, us Americans are aggressive.

Let me. Let me try that. Let's see it just. And it didn't really tell me anything. It did not unhide them. And it's clarifying. And so it. Got it. Do you want me to unhide both hidden sheets or just one of them? Now you want me to. I'll say both exclamation marks and see what it does on this front. And let's see if it does it, because again, it very.

Yeah, I would have probably said something like, are you stupid? I said both.

Oh, really? Oh, that's. That's aggressive. That's aggressive. I don't know how I didn't want it to get annoyed at me so quickly, but it's. It's creating a plan. It is creating a. Now if I had to unhide, I wouldn't. I wouldn't need a plan to unhide a sheet anyway. I'm just kidding. It. It's creating a plan.

Be super easy, right? Anyone would get the first one. The second one you have the very hidden. You'd have to know to look in vba. The first one anybody would get with it.

Well, it knew, but. But somehow it knew by looking in the VBA that there was a very hidden sheet here. So now it's. It's. It is aware that it is there and it's. It's. Okay, well, maybe we'll stop this, obviously, and move on. What do you think? Because I think so.

It's doing longer than one would expect.

Well, and Also, it's taking significantly longer than. Right. Than it would take. Correct.

That's what I mean when I say than we would expect. Right. This is a fairly simple.

There we go. It now did it. Right. It's now done. It. It took for a minute. It says it took 55 seconds plus the previous. So it took a couple of minutes and it managed to unhide both sheets. Perfect. Okay. So it did that. So I'm, I'm. I'm pleased it. You know, it would have been faster to do that manually, but that's okay. It is. It clearly was able to figure that out. That's great to see. Let me ask it now. I have, I have. Let me try this one. I want to try this. You know another thing that I always like to look for. If there's any white values. Are there any white values on this sheet? Let's ask it that question. This one. It responded pretty quickly last time as well. And let's. I'm going to see if it gives me the same answer because it would be nice to know. And it says I'm trying to figure out what the user means by white values. So it somehow it didn't understand what I meant. It says it's planning a color detection strategy here. It's actually thinking. Right? It's thinking. Do you mean cells with white font? Yeah, I'll say. Yeah, yeah. Yes. Are there cells with white font? Maybe it didn't like what I asked.

And it asked is it okay to add a temporary audit font colors tab with the results. Let's see what it does.

We didn't know. I just exploring. I need to check the model again.

This is something that as a person we could do. It's taking a lot of thinking. So is it a right. This gets back to. Is it. Is there a way we could have been more, you know, detailed in the prompt? Did it get it to do it do it faster or is it just sometimes AI has to think through it.

A lot to do it. Maybe. But I think this is a pretty clear question, don't you? I don't. I don't have any other. I don't. I can't think of a simpler way.

I mean, I think you could say simple way instead of saying white value, say there are some font. There is white font. Please identify what cells the white font is in.

Right, Right.

That's what I mean by you could potentially get a different result.

Trying identify the cells that have.

That contain white font on this sheet.

Please identify the cells that contain white font color. I Mean, let's just see what that does. I'm not sure. Anyway, we're going to stop this because.

Yeah, stop that one.

It's obviously not, you know, it's doing something. Previously when I ran this before we started I ran this test, it actually said to me that it was not able to check for colors. So I don't know if that's always the answer that I was going to get. It's. It's not told us that on this particular test, but it's still. It was still struggling a little bit about that. Now I want to look for. The next thing I want to show is there are a couple. There are a couple rows in here that have problems, including this row where there is a hard coded value at the end of the formula. So what I want to ask is this, I want to ask is. Want to ask it. Is there. Are there any. Are there any inconsistent formulas on this sheet? And let's see what it comes back with on with regards to that question because this one it did pretty well the previous time I asked I was looking for inconsistent formulas. And so it's doing something here to check the file. And yeah, it seems to be in row 13. And row 13 we know is a is an important row that has inconsistencies. So it's already discovered something in row 13. And it was pretty quick last time at. But I think it's helpful. But I think it's also. I mean in this particular. It's not bad for people to see what it's doing. Right. I mean it took. It took a little while to. It is. Yeah. So it does know. Short answer is yes. There are several spots on model where formulas differ across the horizontal series. Most look intentional, but they are inconsistent relative to neighbors. Notable examples. So in row seven, row seven is the years. Okay. It's pulling up something there. The pricing in the one of them.

You format it differently, you have the actuals and then you don't have it. Right, Right.

That's pretty good pricing row. This is the problem row. It says M and O contain constants. So that's great. I'm really happy about that. It figured that out. It discovered that this cell has a constant, as does this one. That's excellent. There's another row though that I didn't show you yet that has a problem. I'm going to show you. It's a classic modeling error. Sales volume operating rate set up J20J.

Oh, it's the back calculation of the volume.

Yeah. Noticing you calculated the first that's Fine. So that one I should probably just put in a box. This is a historical. So it realizes that the historical is a different formula than the forecast. That's okay. And then several first year. But there is one that it did not revenue roll up J20. So these are all deliberate. But there is one thing that's a classic error that is. And that's here. This is a very common modeling error is bad links. And this happens especially when people use their mouse. So you can see here on the income statement the revenue has to pull from the revenue schedule. And what. Of course this one is linking correctly. It's linking to cell K26. I'll show you. That's correct. That's the right link. But sometimes. Especially when people use their mouse or they're not copying across. This one is K26. And then you know I might have changed or fixed something. And this one is also linking to K instead of L. Right. I might have done this one manually and then tried to link this one up. And if I was using my mouse and I'm not oriented to where I am. It's not hard to. It happens all the time. It's not hard to link on the wrong column. So now I have two years that are linked to the same one and then the rest of them are all off. This should be M. It's L et cetera. I ask it because previously. Are there any other inconsistent.

We could also ask it analyze row 8 since we know 83 if it doesn't get it here and just see if it picks it up. If you basically point it to it.

Yeah. It actually solved it last time on its own when I did this. So I'll see if it finds it.

It was in prompts. Isn't it that. That's part of what's so wild about. Because we know it's generative AI. Right. You never get the exact same thing twice.

Right. And so I'm. So I know. So here's the thing. I know. Sorry. Giles. You say something?

No.

I was just gonna say it's interesting that we've all. We've all. We all keep having that experience. We try something beforehand just to see how it runs. And then the same prompt gives you different answers. And sometimes you just can't do it. I think that's the more surprising thing. Getting slightly different answers. I would expect with everything I've learned about LLMs. But sometimes where it just goes. It just breaks and doesn't. Doesn't give you anything.

There's a lot in here. You know, I see more places where the form. Yes. I see more places where the formulas differ across row. It thinks it looks appropriate. Like it deliberate. I'm looking. I thought I saw something for row 83, first year. It's talking about the seed versus rolling calcs. No.

Yeah.

It hasn't seemed to recognize. So we'll leave that one at that. Did I have an issue where it's. Yeah. Where these two are the exact same. Let me fix this one, though. This is k. But I guess the point is simply asking it, and we saw this last week as well, simply asking it to look for errors or inconsistent formulas might find something, but you're not necessarily guaranteed that it's found everything unnecessarily. And so. So you still need to be able to check things. This balance sheet also does not balance for the same reason I showed last time. It does not balance because the change in working capital is the wrong way.

Formula is wrong. Right.

Formula of change in working capital is wrong. The change in working capital formula is going a classic modeling error. It's going the wrong way. It's taking current year minus last year, whereas it should be the opposite. And I'm just going to ask it. So again, if I'm a young, if I'm a new young analyst and I'm staring and my boss says, ian, we just received this client model and the balance sheet's not balanced. Go find it. You know, I'll ask my tool here to say, why doesn't the balance sheet balance? And it's my starting point, Right. It's my first go to option say, why doesn't the balance sheet balance? And. And I hope that it can find something. It's inspecting, it's checking. You know, what it seems to do is it shows us some of its thinking and then takes it away. Right? And then it comes back to the dancing dots here, which is fine. I'm analyzing. It's doing something. I mean, listen, I recognize this is a very difficult ask. It has to understand model construction, model design. It has to understand how every formula needs to work to be able to determine that that one formula on change in working capital is wrong. I recognize that this requires an enormous amount of knowledge and insight to pinpoint why this is off. But it's. Let's pause and, and we'll come back in a couple minutes and we'll see where it gets. All right, we are back. It only went for another 70 seconds. Just over a minute. And it says the short answer. The balance sheet is out of balance in the forecast columns and it, and it confirms that through the check. So it knows it's looking at some checks. Again, this is very, it would be very difficult to do, which requires a fair bit of knowledge. It seems to think that what's driving it is cash versus financing. And there's some explanation. There is the mismatch grows. So it realizes not just rounding working capital lines don't look consistent. But it's not quite getting to the answer. It is. So it's giving me some ideas and suggestions. It's not. Of course, of course the only reason it's out of balance is because I'll do go to the bottom here because a classic, classic modeling error. The working capital should have been previous year minus the current year. Right. The change in working capital represents the change in cash as a result of working capital changes. The sign was the wrong way. Happens to people all the time. And simply by fixing that, my balance sheet is now fully balanced and working properly. So it was one small issue that, you know, a strong modeler would be able to detect fairly quickly. So that wraps up that, you know, part of the test. I want to move into a build example now. Did you want to.

Yeah, I want to ask one thing. I thought it was interesting, you know, at the bottom, would you like me to create the audit sheet to show it and then fix the driver logic? You know, typically by setting the Revolver as the cache plug to a target minimum cache are simply hard plug class to force the check row to zero. Interesting that it gives kind of those options.

And it's, and it's really. And it's. Yeah, I'm glad you pointed it out. So right. One, one option it wants to do is an audit sheet. This seems like it's going to do a lot of work as the cash and my suspicion is it would have forced cash. Like it would have plugged something on the balance sheet. Maybe, I'm not sure. And then this one says, you want me to do that thing I'm never allowed to do? The thing that you should never ever do in a model. Should I just simply hard plug cache?

It's like, it's like, like this is not recommended.

Not recommended. It's like a little, a little like that little devil on your shoulder that comes at two in the morning when you're like, just plug it, just plug it. Just go to bed, plug it. So it's, it's giving me that option I probably shouldn't take it. But again, as a, as a financial modeler, I know that I'm not supposed to take, you know, I worry about, I worry about people without modeling skills trying to do this and not knowing what direction to go and saying yes, but okay, well that, that is, that is that, why don't we move over then And I'm going to actually going to clean it up here. Why don't we just go back to the beginning and I'll start a new conversation. A new conversation. And I'm actually going to go into the other file here. This is just the three years of historical financial statements. This is exactly how most modelers start their models is with three years of historical data. And then you have to build in sheets for assumptions, sheets for scenarios. And this is exactly what people have to do, you know, on the AFM exam, right. To prove that they can do it. And it just hones your thinking. So let me take a prompt here. Let me just see the starting point. Let me just see. I've said here, just like build a five year forecast model on the model sheet. This is the model sheet for the years 2025 to 2029. And I've said build separate schedules, build separate schedules for revenues, cost, depreciation, income tax, working capital, debt and equity. So for, so I obviously I had to even know enough to say do that. Right. I'm trying to give it, you know, you have to know enough to know that you need schedules. And then I've literally said make reasonable assumptions because I don't care what it uses for growth rates or margins or days. I just want to see that it can understand the design and construction. So let me run this. This will take a few minutes. So I'm going to click go. Why don't we, why don't we stop this Paul, and come back in a few minutes once we've got something to report back on.

All right, well we'll go ahead and pause and see how it does. We'll be back. All right, we're back. It ran for about six minutes and it really struggled on this task. But it did ask us a few questions. We'll try a little bit more for here. Why don't we first kind of maybe end, just show a little bit. What are your first thoughts? It feels like it really struggled here.

Yeah, it did. It only took six minutes as you said. But you know, I deliberately asked it, as you recall, I was pretty open ended and said just build a five year forecast model and try to build schedules for each of the key components of a model. And I just said make reasonable assumptions. So it's come back Yeah, I mean this is obviously needs work and it's asking me a couple more questions. I'm going to go back and run some more. I'm going to try and clean it up. But it's kind of broken up the income statement a little bit. It is added a bunch of rows, it's got some stuff off to the sides. Put in five year forecast, sort of worked on the income statement, the cash flow statement sort of. And balance sheets are not complete. There's nothing underneath it yet. So why don't I come in.

Maybe we can ask it to put the assumptions on a separate sheet and then complete everything.

So why don't I say please put all the assumptions on a different. She then finished forecasting the financial statements. Okay, let's come on back in a minute Paul then and see what it, what it does at this point.

Okay.

All right, we're back. The bottom line is. And I'll let in speak to it a little bit. You found the model, struggled to build the model. Right. Kind of your take there in on that part.

Yeah. And again we're not totally surprised. I mean it's, it's, it's, it's done some impressive things but what we're seeing is, you know, having more guardrails, more direction, having more formulas to fill in. Tools we've looked at so far, you know, have, are doing some impressive work. Have a bit of an easier time. It was struggling to build a full three statement model with little guidance. I kind of said do what makes sense, pick reasonable assumptions. And it really was not creating anything that, that we would find usable. And again, I don't, I wouldn't have cared if it used silly assumptions. I still just, I mean one of us could build, could have some pretty basic assumptions and build a five year forecast pretty quickly, mechanically, logically understanding how the numbers talk to each other. Right. Understanding the discipline of the build approach that to build it, link it in, get the pieces connected and then you can refine your assumptions later. So it was struggling with that and not generating anything that was worth showing. But again that's not a huge surprise and I think the company even talked about that. Is that right? That, that's, yeah.

I, I mean I, I remember speaking to Dennis a while ago. I had a really long.

And Dennis was, Dennis is the founder, right?

Just the founder. Yeah. So, so he really open and, and honest and very approachable with, with all of us. I think, I think he was very open with the fact that it wasn't like ready as a financial modeling tool and he wasn't claiming it was going to do full financial models. He was actually very keen to get kind of models from me when I spoke to him to kind of help the tool learn. So I'm not massively surprised that it's not really built as a three statement model.

And what I appreciate is he admits that I've had others that like, it's just not ready to build a full model. And I think this demonstrates so far they're not ready to build work ready models.

But we're early, early days. It's doing some great work. Early days. There's nothing to suggest it won't but as I said, even, even if and when it does, you have to understand everything it's doing. You're going to need to check it. I do not believe there's a day anytime soon where your managing director is calling up models into AI. I mean there's still someone is going to have to be responsible for building models and if you can use AI to help you, fantastic. I'm a big fan of that. And then being able to check them, double check them. AI is actually going to be great at helping you check them. So there will be a. Use it as your partner. You know what I did though do is because it struggled with the model, but I asked it to add it, I asked it to go to the completed model because it seems to have more success with, you know, when there's already structure in place. So I asked it to build a DCF for me at the bottom by using a weighted average cost of capital of 10% and a terminal growth rate of 2%. That's all I gave it and I wanted to see what it did. And it actually did a quite a good job on this. I mean it's not quite client ready, but it made one mistake which I had to fix. It was using the wrong cell for the tax rate. So it grabbed the wrong one and I said, I think you use the wrong cell for the tax rate. And then it figured it out. I like it. Look how friendly it is, Paul. It said, good. That's because I'm being nice to it. It said, good catch. You're right.

Yeah, you're rubbing that part in that I'm not nice.

Absolutely. So it did though correctly. Now it's all, it's built it all into one cell. Now as a modeler, I need to understand what it's doing. It's taking the, it's taking the ebit, it's multiplying it by one month as a Tax rate, it's adding back depreciation and the working capital and taking out the capex. And then it has properly calculated a discount factor for the appropriate number of years. Automated. I like that a lot. And in most people's discounting exponent to discount years, they would have a hard one here and in this one they would have, you know, a hard two or they would have a row above that had the 1, 2, 3, 4, 5, which is fine. This is using Excel's column function to automate that, which is very nice. But all it's doing, this entire series of columns is just getting to the exponent, the discount years, the 1, 2, 3, 4, 5. It's doing it correctly. And then it's calculating the present value of the cash flows. I like that. Of the five years in the forecast period, it calculated the terminal value by taking. I'm not always a huge fan of this approach, but I didn't give it more direction and I think if I did, it would have done whatever I wanted. It took the fifth year's cash flow, it pushed it out by one year, it grew it forward a year into the future, and then it used a perpetuity to discount it and, and take the present value, which gets it back one year and then it further brought it back five additional years and then the enterprise value is adding it all up. So there's certainly some tweaks, but that's an impressive piece of work. I'd love it to ask it to do a levered cash flow for me to get to the equity value and compare that against the equity value that arrives using the unlevered approach. So there's a lot of things I would push it forward on. But we'll do that another time. For now, this is something I think it did quite well at. Much better than its ability to build up a full model from scratch. So I think that's good for now then.

Paul.

All right, so I think that's wrap we've gone through and we've tested nine or 10 different use cases here. We've done some modeling, some FPA. There is some charting in there. Just analyzing data and I'll give my kind of final thoughts and then we'll go around the horn area of a final words. I think it's an impressive tool. I think it did some things really well. I think it struggled in some other areas. You know, some other tools I've seen have done better in things. But what is clear to me is if you know what you're doing, there are areas that can do things and make your life easier. There are other areas where it's not quite ready yet, but I thought on especially the Excel cases, it did fabulous thought, did really good. Right. When it has that structure and the way it gave those formulas, you could learn a lot from. So those are my takes in your final take.

Same thing. I mean, it's. It's still. Again, I still do give them tremendous credit. All these tools. Incredible to think that. That you can use English language prompts and get something to build a spreadsheet for you. But again, my strong recommendation for anyone. Well, I would highly encourage everyone to start playing around and testing and these things and getting comfortable. The future analyst will need to know how to use AI in their work. But my strong view today, and I don't think this is going to change anytime soon, is we need to view them as our partners. It's not a competitor to you. It is your partner that can hopefully assist you and help you. And if you think of it that way, I think there's great things in store for you and for all these tools.

I think I've got two thoughts. I think if you think of the job of reviewing a financial model, I use a tool called Oak all the time to help me review models. I could see myself using a Rosie or another tool. Or maybe it will be, you know, the agent in Excel to do some additional work, like a partner.

We're not leaving until we get a second out of you.

Jazz, it's late in the uk. We're forcing you to stay up.

What was my other point?

You had an auditing tool you use. You said you loved your auditing tool.

I think it's gone. Whatever the point was, it's gone.

Take one of mine. It's fine. I don't mind. You can copy.

I guess I'm still where I was before. I think there's some really nice, strong signs, and there's no part of me at the moment that is just ready to go. Right. This is going to replace what we do. But I think we've been. That was our gut feeling before we started this. And everything we're seeing is kind of cementing that view for me.

Yeah, it's a tool, right? It's a tool. It's a powerful tool. Like Excel was a tool 40 years ago when it came onto the scene. Like calculators were a tool, you know? And I believe that, at least for the foreseeable future, I think, like you guys, those you guys at the top will use it, supplement their work.

Paul, and I think that's a great point to end on. Look, it's a tool. It's a tool. If you're not using in your tool belt, shame on you. You should be finding ways to make yourself shame. I'm publicly shaming you if you're not finding things to be more effective. So there you go. See, the American in me is coming out, and I'm way too nice for that. And Giles, like it's late at night. I don't care what they say. So, on that note, we're going to call this a wrap and we'll be back again for another round of testing. So if you have questions, you're curious about what we did, you'd want to reach out to us. Please do. We'd love to hear from you. We'd love to know what you think. And thanks for joining us.