How TabAI stacks up as an Excel AI Agent for Financial Modeling Pros, with Ian and Giles

Foreign.

The Mod Squad featuring Ian Schnorr, Executive director of Financial Modeling Institute Giles Mail, humble MVP and co founder of.

Full Stack Modeler and Paul Barnhurst, the.

FP and a guy the biggest thing to me we could probably ask at the prompt, but notice it's hard coding the dates.

He's done better than any other tool so far, so it's interesting. Even to get levels one to five is massively impressive for a long time.

People are still going to want to check and make sure that these tools are actually generating the right answer because we've seen a lot of instances where they do not. You're going to save time for sure. It's doing an awesome job.

You'd validate, maybe make a few changes, but this is the closest I've seen to presentation ready.

It's. Yeah, I agree.

The other tools did take a few minutes to do this particular step. It's doing it perfect.

It's using things like indirect and column to get the column number. That's probably one of the cleaner approaches I've seen, but there's a couple of weird ones. Okay, so level two.

Welcome to another episode of the Mod Squad on Financial Modeler's Corner. Once again, I have with me my two distinguished co hosts. We have the Humble mvv. You wanna take a minute to introduce yourself?

Hello, Giles Mail. I'm co founder of Full Stack Modeler and today I have come almost color coordinated with my other guest. That's all I'll say.

Rub it in that I didn't get the memo. Thanks, Giles. All right, why don't we go ahead and introduce in.

Yeah, thanks, Paul. I am also rarely in red, but I was pleased that Giles did get the memo. I'm just back from the massive AFP conference in Boston, which was great and I was able to deliver a talk there and I'm coming back with my wicked Smart Boston T shirt that I've decided to sport for today's podcast. I am the executive director of Financial Modeling Institute, the world's only financial modeling accreditation body. Lifetime in modeling and excited to be with the two of you testing the AI financial modeling tools.

Thank you for those introductions and for those who don't know me, I'm the FPA guy lucky enough to host this show talk all about FP&A. And we're super excited. Today we're going to be testing another tool. We're also going to talk a little bit about, you know, kind of the industry and what we're seeing briefly before we jump into talking about tab AI I think an interesting place to start. I'd love to get your guys perspective. We also have a note this week that Tab AI made the decision not to sell to retail. All existing customers continue to use it, but they're just going to go to Enterprise. So maybe I'll start with you in what was your thoughts when you saw that announcement?

You know, in the last episode we've talked about the fact that a couple tools have already decided to shut down. We know the funding environment has changed a lot once Microsoft announced the addition of Agent into Copilot, which is doing a very good job, is certainly as good a job as the as any of the tools we've tested so far. To me this is symptomatic of a full, fast changing, explosive new industry that's moving in real time. And they've obviously felt that it's going to be difficult to compete in the BDSC space and you know, choosing their battles and moving after Enterprise. So I don't, you know, I think it's an interesting angle and we'll see how they do going forward. That's my take.

Giles, your thoughts?

Yeah, it feels potentially logical when you consider the kind of marketplace. I mean, I run a training business so we've got B2C and B2B. If you bring on a team in a large company, it's a very impactful step. So I kind of get it. And as an mvp, I know Paul, you are as well. We get to see what's going on with Microsoft. Obviously it's all under NDA in terms of detail, but Microsoft aren't slowing down. So what we talked about before with Agent Mode, where it is now, it's just going to keep getting better. Uh, so yeah, really interesting time.

I agree, it's an interesting time and I thought the Tab AI announcement was interesting. I also see, you know, Trace Light is still very much committed. I saw they were looking for a chief of staff this week. So you know, many of these tools, you know, they have funding, they're continuing, but they're trying to figure out what's the right path. Right. What's the product fit, how do I compete when we all know Excel's an 800 pound gorilla and nobody wants to go head to head directly against Excel.

Yeah.

And you know Paul, you made an interest comment before we started, which is that Copilot is a paid subscription. Right. So you're already, I mean if you want to move into AI tools in office, you're, you are likely going to pay to subscribe to Copilot and now that includes Agent. So if you did not have to pay for Copilot, you might feel inclined to try something else. But. But since you have to pay for co pilot anyway to get it, are you going to pay for a second AI tool as well? And I don't know what you guys think, but that, you know, it's an interesting dynamic. I don't know what your thoughts are.

Well, I mean, unless it's superior, right. It has to have real benefit. I'm happy to pay for a second tool if it gives me enough savings. But the question is if, you know, Microsoft is really focused on modeling which they appear to be with Agent, among other things, can another tool really reach that level? Where am I going to pay an extra 50 bucks? I think that's the question that are whatever the, you know, 30 bucks, a hundred bots, whatever it might be, that's the question you have to ask. And I'm right now not convinced from what we've seen that most people that answer is going to be yes, we.

Asked question right at the beginning of the series genuinely between us saying like it would be really interesting to understand what these tools add on top of just a link to an LLM. But now Agent in Excel is Copilot's going to be linked to Claude as well as ChatGPT and probably other LLMs in the future. If the only thing these third party tools do is link to an LLM, I think they're in real trouble. But we don't know, do we? We still don't know.

Yeah, we know they do some things, but it's how much and what we're seeing doesn't tell us it's enough yet. I think what we've tested so far also, you know, Claude just released Claude for Excel. So you know, you can do a lot of these things just straight in Claude with your $20 subscription. So it'll be really interesting to see. It's why I say you're going to have to have a key differentiator to survive. So let's go ahead and take a look at their website. I think we're going to skip their document with the recent announcement. It just changed some of the pricing and things. So I think we'll just kind of go through their website. So, you know, here we can see transform your team's Excel workflow, see your spreadsheet in a new light. Excel at any task, you know, kind of similar to the others. Although I think it's interesting they focused on workflow, you know, not modeling, not be superhuman in Excel. I thought that was an interesting choice of words. I don't know if you guys have any thoughts, but. Little different angle than what we've seen from the others.

Yeah, there's nothing that feels overly hypey, which obviously, you know, I'm very sensitive to, but yeah, it looks, it looks reasonable.

Admittedly as a finance person and a corporate finance person, I'm not 100% sure that I know what they mean by that. I mean I, you know, when, when someone says build better models, I understand it. I mean, I think I can infer what they mean by transform your team's Excel workflows, but I, you know, I think all teams work. Is it obvious to the two of you exactly what that means or.

It makes me feel like they're thinking more about data and, and what you might use in a one in nowadays, in Microsoft world workflows, I would be thinking of things like fabric and where is data going through, you know, a fabric environment, Power, bi, Excel, Python. I don't know that, but that's what something like workflows makes.

The other thing I think of when I think of workflows is, you know, FP and a. What are the monthly tasks I'm doing in Excel? The variance commentaries, the PVMs, the whatever, the reports. And so how is it changing and simplifying those workflows?

Yes, I suspect those are the things they're talking about and I guess, you know, we'll learn more.

All right, so let's just go through a little. I think it's interesting. You know, this speaks to the. It says try CAB AI for free. You have to contact them, right? It's the enterprise. Hey, if you're an individual customer, it's kind of like, yeah, you're not our. So they're definitely taking that approach of gating it. Now here's. I thought it was interesting. You can read the whole story. We know Lance Rubin is a well respected modeler in the industry. You know, has done a lot of great work and you know his take, you know, he's clearly a big fan of what they're doing. CAB AI is like giving every finance professional an Iron man suit. It supercharged our services and transformed how we deliver value to clients. Any thoughts on kind of that?

Just that I want to see. Like, let's see. Because if it does that, you know, a statement like that makes me feel like we would expect something maybe better than we've seen so far.

Agreed. The other thing I like here, this gets to the Workflows showing inputs. Hey, we connect with Xero QuickBooks and we want to add these other data cleaning automation. This really speaks to the whole. You can see right here your client serving workflow supercharged. I think that's where they're advertising a little different than the other tools.

Yeah, no, I like and I like it. Listen, it feels like it is in that image is great. That's much more visually helpful at explaining what they mean by workflows. I always think of project management when I hear about workflows in some of the my world. But this is clearly helpful to just visualize with and it makes sense. It helps explain why they're focusing on the enterprise market that is needing to, you know, connect with. So anyway curious to.

It's interesting though because that makes me feel like they're. They're actually quite vertical in the accounting space if they. The highlighted inputs are just yes, zero.

And customers are focused there. And what I like is what they've highlighted. I think this is where they want to play in some of these other places. It almost feels like hey, here's our strengths, here's where we're at today. But here's a map and I could be wrong, but here's a map of where we want to be because right. They say PDF images, web search, they mention all these but you don't see them connected. So I don't know if it's. That's what we want to be able to do. A little confusing but I think that what they're trying to say is here's where we're at and here's kind of where we're going. We'll run through the rest of it. Data dump to structured Excel. Yeah. Formula recreation PDF to Excel with citation.

Okay.

You know, formula builds and web searches, workbook revision and template fills review and board ready presentation. I think it's interesting they're mentioning board ready presentation but they're just an Excel tool. I don't see PowerPoint here. So that's a little interesting. But nothing else really. I mean calc. I think this is an interesting one. We'll see what you guys think about this. But calculate your roi. So what they're claiming okay, financial reporting. So let's take financial modeling to have model reviews. Hourly cost takes you 30 hours. They're saying you're going to save $2,250 an hour cost and deliver faster, more accurate work to clients saying it's going to cost you 250 with tab 4500 without buying it I'm not.

No.

I struggle with very like specific numerical measures like this. It's like, okay, how did you come up with these numbers? Uh, so, yeah, that I'm not gonna assume.

So it says with tab it's 15 hours. So it's 2002 50. Okay, so they're saying it's gonna be a 50% savings. I just. Okay, maybe, maybe not. Interesting.

Well, so, so, okay, take that example, review time. If it's saying it's gonna halve the cost of review time, what you're saying is literally that using this tool, you would half the the time taken to review financial models professionally.

Yep, that's what it's saying. Because it's going from 4500, it's saving you 2250, which is half the cost. So let's, let's just run through a few of these. I think this is really interesting. It's a bold. They're saying 50% here. They're saying almost 80% on data ingestion. 70, 80, about 65. 70% here. You go from 50 to 15 hours. So 70% savings on data cleaning, workbook updates, 50% format and reporting. 50%. I'd love to see the math of how they came to that conclusion that I struggle with. Telling me you're going to save me that much with AI, I just haven't seen it yet. Next, we have a brief video from the founders of Tab AI. They were kind enough to share with us a minute video talking about the tool and thanking us for doing the testing. It's a great little video to give you an idea of Tab AI. They've done a lot of great work. And then we'll be on to the testing, which I think you'll be pleasantly surprised with. Hi, I'm Iris. When I see an Excel workbook, I.

Don'T just see rows and columns.

I see endless nights and cancel plans. That's why we built Type AI to.

Make your Excel work easy and fast.

Here's how people are using it today.

Hey, Andrew, here from Type AI. Here's how we're helpful for our FP and A friends. Firstly, we're an add in right to Excel so you can download the ser and it's not another spreadsheet app. You can use Excel Desktop for this. Three examples of how we're helpful. One, creating a tab on sensitivity analysis, plotting stack bar charts.

Right.

We natively read your sheet, write on your sheet. We can search your web. You can make plots, make charts. Everything can be done in our sidebar here. Secondly, Data massaging. No one likes to do this. Just bring your own data. Ask Tab AI to add a column to standardize supplier names. Right. Tab AI gets it done in one shot. Lastly, variance analysis. Just type it in the sidebar and Tab AI can do all of it on its own. And you might say, hey, I don't like the format here. Right. I don't like how it's structured. TABI has a memory feature. You can come in, type out whatever feature you want, turn it on and off depending on the workflow you're in. So we'd love for you to try this out. Bring on your sheets. If not, we have a starter kit as well for you to try out with real world sample financial betas. We're very thankful for our friends Giles, Ian and Paul here for being independent arbiters and testers. We hope you appreciate our love and sweat into Terry AI. Thank you.

So are we ready? Should we jump into testing? You ready to go, Giles? I'm ready. All right, well, let's. Let's put it to the test. I'm excited to see Cab AI. I mean, I've definitely heard some good things and I think we'll be good to jump in here.

We're going to get moving. So the background is I've got like a hundred pounds worth of credits for this for assuming that is the equivalent of sort of token credits. I think I have logged in appropriately or successfully. This is the easier case for esports.

This is.

Looks like I've got a. Oh, okay. There you go. It has. Let me sign in. There we go. So as a reminder, we have a esports case. Five bonus questions, seven levels. I'm not really looking at the bonuses.

Before we jump into the case, I'm curious, what are the things across the top? Like what's the auto. The new. The agent.

Yeah, fantastic question. Don't know. I'm assuming that is new prompt.

Yeah, it's a new prompt. So auto. It looks like auto select the model versus select it yourself, which I appreciate. Then let having default be auto versus.

Asking me to decide look at how many models there are. My goodness. But yeah, gosh.

So it's great.

We talk. I like the auto. But do you think there's a part of you that would be still be curious and say it's chosen one. I wonder if it chose the best one. Maybe I should just try.

Yeah, there will be some people for sure. I would be a little interested. Right?

I mean there's like 10 tools, isn't there?

Well, there's I think there's five tools because you have OpenAI, you have Gemini.

There's about. There's 10 plus options there.

I was saying Owen and whatever GLM is, I don't recognize. The bottom two are either. Are you familiar with the rest? I recognize. But those bottom two, are you familiar with those?

No, I'm not.

No.

Interesting. And then click on Agent. What's the agent versus ask? Like what's it say when you hover over it? Okay, so it will directly. And then ask is like best. If you click on ask, you get to approve each change before it does it.

Oh, okay. Well, should I go with. Oh, let's just do. Let me just stick with Agent. I think that's fine.

Let's try it. It's clear they've given a little more thought here than we've seen with other tools of, hey, you could automate or you can kind of do it yourself. The agent asks, others have different. Like the audit mode and trace light and things. But I, I do think they've done a pretty good job there with that setup.

So that was literally, what, 30 seconds? And it's answered level one. It's done it correctly. We've got.

You know which tool. Do you know which. Which tool it used?

Yeah, that's what it's asking. Does it tell us what model it used? If you scroll up, do we know?

Good question. I'm not sure it. Oh, it doesn't.

What if you click on that at symbol does that.

Yeah, I'm not now when you.

Because when you click on. When you click on Agent again and it says auto, but it doesn't somehow in that download it says auto, but it's not telling you which one it's actually used.

I would like it to tell me which one it used. You could save it as a workflow. I mean, I wonder if you ask it, can you tell me what model you used?

Oh, good question. Wow, that's smart, Paul. A good idea.

So it looks like it chose. I'm assuming it chose Claude for that one or that's its default to answer questions. I don't know for sure. But yeah, now that level one is confirmed. So it looked like it used Claude 35 sonnet. Okay, cool.

Okay, so I'm going to get it to solve the remaining levels. We might want to pause again. In theory, this should take a bit longer. And then we'll come back.

Alrighty. So we'll pause and see how long it takes. All right, we're back. That took what, about eight, nine minutes There I think you had it reprompted in the middle. So why don't you talk to how it's done? Let's walk through some of this.

Yeah. So I think it kind of stalled somewhere after level five and I had to stop the thinking and say, hey, can you restart? And it's fine. So it's got all of the answers right. And actually, for the most part, I would say it's done a very good job. So we've seen in prior episodes, it's using things like indirecting column to get the column number Len and substitute. It's used. These are the solutions that we would expect from, you know, esports players. For the latter levels with the map, indirect is correct. It's done a really simple ish approach to level seven, which is an offset with an indirect. That's probably one of the cleaner approaches I've seen. Or you're using two volatile functions. But it's fine. But there's a couple of weird ones. Okay, so level two, which was take the text string that includes a bunch of numbers, find the two largest and add them together. Genuinely, we were all just saying, I don't think any of us have seen this function, but it's used filter xml. It's getting to the right answer. I've just genuinely never seen that before in my life.

I mean, I think the three of us could put our heads together. We need a bit of time to, you know, dissect that, just to understand what it's doing.

Yeah, but it's right. So a weird one. I'm not going to knock it too much, but unusual. And then on level five, I think it was where you've got to find the number of times, you know, any of these numbers appears and essentially add them up. It did a really odd thing where it's. Rather than just substituting out the number of times the number occurs, it substituted the number and the semicolon next to it. So every find is actually two characters that it's taking out. So then it has to divide the answer of this by two to get back to the number of characters that it found and then multiply it by the number. So it got the right answer in a bit of a weird convoluted way, but 100% score wise.

Yeah. Let's see about the bonus. I'm curious if they get the color.

Yeah, you are.

Because nobody's got that one yet.

So I think they did. Where's my answer sheet?

I'm pretty sure that's the right. They just changed it to 4:30 or 4. So they're close.

This was my right answers and these were. So he's got a couple of them wrong but yeah, close.

What did they, what did they put in there? What they just put the number in.

Or did they hard coded it so I don't know.

Oh, interesting. But the closest I've seen yet at least they were in the ballpark.

That is the closest I've seen too. So okay. But you know case one and I'm trying to move this at pace, you know so of course the answers 100% couple of weird options.

A hundred percent a couple missed a couple bonus. I think the others all missed that one bonus. So similar to others you did miss one other bonus. So I think slightly lower than Rosie.

But very similar and just out of interest because I think this is interesting, I'll bring up the dashboard I've got on my website interface with them. So we got £100 credit for this. That's cost us £2 40. So for so far and we've made 36 API calls.

How do you feel about that? How do you feel about the fact that you've got this? Because we haven't seen this on another tool, have we? How do you feel that you've got this credit? Is this going to encourage you to use it or are you going to be the type person that is going to use it carefully because you're watching your credits go down. Is an interesting psychological relative to just having a monthly subscription, right?

I don't know think they do have some monthly subscriptions. I know they gave us a usage so I think they allow both.

Got it.

And there definitely is a psychological to if I have credit, right? If you see how much you're using every time you're gonna be like well how much is it for another model? Because I saw an article recently the guys who do all this which is one of the tools we've talked about testing he showed credit usage and optimization and hey if I use chat GPT every single time versus this tool, here's how much it costs, here's the accuracy and tested like 10 or 12 and I think if you didn't optimize your credit usage it was like 50 times higher the cost. It was really interesting which way but.

Credits were more were way more expensive.

Yeah it was way more expensive if you're not optimizing which model you're using for which because if you use a lightweight model for a simple task it could be 10, 15, 20 times less expensive than using a complex model because it's going to do a lot more calls and think a lot longer. So it's really interesting, you know, they're now starting to show those type of things of, you know, as a business, and I know that's not what we're focused on, but how do you optimize that? How do you incentivize behavior? Because if you're. If you don't use usage, you're using a monthly. And someone's always using the most complex model. Hence why I think that you want to select the model to start with so you can try to optimize your cost as well as accuracy, because. Right. You got to consider both.

Yeah.

Yeah.

Interesting. By the way, I just hit go on Model 2. I've started my stopwatch, but I. I'm just letting it run. Obviously, we can pause at any point on this one. I just get it to get stuck into all seven levels.

Yeah.

And this is for anyone who hasn't joined us before. This is a case Dancing fur coat all about Giles or humble mvp. It's a much more difficult case than the first one, of course. We'll see how naturally. Because it's about a guy who's an awesome modeler and Excel competitor.

Great pictures in here, though. Look at this. Who's this wonderful Jesus, like, character is stunning.

That's going to even touching that, you know. You know, the tool is going to suck up all your credits for that. Like, it's so beautiful. It's just going to charge you a fortune.

Yeah. We'll see how much this one cost. All right, we'll go ahead and that might be a good spot to pause. We'll let this run for a few minutes and we'll be back. We're back. It's run for about 15 minutes. How did it do, Giles?

I think it's done better than any other tool so far. So it's. It's interesting. It's struggling on level six. Remember, this is seven levels. But every tool has struggled pretty significantly. And actually it has so far got levels one to five completely correct. And there are some tricky bearing in mind. There's a lot of information in this model. I didn't direct it. Remember, we've got all of this closet information on one slide tab. We've got dancers and formations. I didn't say, like, here's where this stuff is.

It had to interpret everything.

It had to interpret all of that. So even to get levels one to five is massively impressive, I would say. And again, this is where it gets interesting. If you look at some of these formulas, they're not pretty. It's definitely choosing kind of single cell solutions. And again, maybe I could have helped by prompting that to say, you know.

Pause here for a second. I think it's interesting. Why is it doing time 0 times 0 times. So is it turning on and off somewhere times two?

Yeah. So this was a hard level. Level four, depending on my mood. There are certain characteristic sc.

Oh, that's right, I remember. Okay, so those are the.

So that's okay. Look at this. This is the dancer ecstasy scores based on their proximity to me on the stage.

You're a let user. Why is it using a let there?

Well, because it's chosen a single cell solution. There are lots of elements to what you would have to work out in this. So I haven't taken the time to look at everything it's doing, but essentially this looks like it's working out positions of me versus dancers. And then off the back of the kind of steps, spaces between us is doing that calculation for the ecstasy value. It's clever. I mean, it's clever. We've left it running just to see if it did get any further, but I think the answer is really impressive. Nothing's got this perfectly right yet. I have a feeling if we left this for half an hour, it might get there. But we, you know, we've only got so much time, so. Yeah, 100% of what it tried and it hasn't solved six or seven yet.

Yeah, it had six wrong and it's trying to figure out why it just hasn't been able to get there.

I'm impressed. I don't know, we were saying just before we restarted the user interface, the information you're getting back, I mean, compare this to agent. I think, you know, Excel's agent has got a long way to go in this instance. The information you're getting back about what it's doing at all times is fantastic. And you've got this checklist as well that one of the other tools had. You know, it comes up with a task list and then it's just taking them off one by one, which is great.

No, I think it does have a clean interface. Fairly useful. I mean, I think it even allowed, I can't remember, but earlier it may allowed you to drill down in some cases, but it feels like as the right level of balance for the most part. Yeah, I agree.

I'm impressed. I mean, it continues to make me believe that again, you're. I think there's, there's gonna be A long time. Where for a long time people are still gonna want to check and make sure that these tools are actually generating the right answer because we've seen a lot of instances where they do not. So I mean, you're gonna save time for sure. It's doing an awesome job. But I sure hope you spend the time checking it and double checking it and you might have to learn a few new Excel skills in the, in the process. Right. That are less common, let more obscure to ensure that you know what it's doing and why it's working. Right. That it will always work.

How are you going to test this?

Watching a YouTube video on how to build a filter XML. What are you guys doing?

I don't know. Huh. What were you saying, Giles?

Well, just to this, that exact point. I mean, if it comes out with this answer, obviously we've got the benefit of green ticks here, which you don't have in the real world. If you don't know modern Excel, if you don't know what a let is and what variables are, how are you going to debug this formula?

Well, my point. Yeah, my point is exactly. So you're going to have to figure that out. You're going to have to learn let functions and watch a bunch of videos and understand what variables mean. So it solved this problem. You know, if we talk about real world issue, it has solved, you know, sort of a problem in call it five minutes. But you might spend a couple hours learning and understanding what it did, which will elevate your own skill set. Right. To make sure that you can confirm to your boss that it's right as opposed to building it a simpler multi cell solution which would be. Take longer for you to build, but you'll know what you're doing and you know. Right. So there, so there is trade off. Undoubtedly.

All right. Am I handing over to you, Paul?

You had one more, didn't you? The data to analyze the trial balance.

Yeah. Do we want to do that as.

Well or we go ahead and do.

That one real quick?

Okay, I'll try and be quick on this one. So let's bring up round three. I'm not going to go into too much detail about what, what I'm asking it to do. Again, if you've been watching this series, you'll know, but essentially there is raw trial balance data on a tab and I'm just asking it to produce an output, you know, something with KPIs and potentially some visuals. There's some interesting stories I think to tell in this data where the net income I think it is, goes negative halfway through the year and then pops back up at the end. That's it. Should we pause and come back?

Yeah, let's go ahead and pause it. Give it a few minutes to run. All right, we're back. Giles, take it away.

Yeah, so I think again it's doing a good job and I mean I have literally just re prompted it to say can you just format these numbers? It may have been getting there anyway, but obviously, you know, in the interest.

Of time, because it had been running for quite a while, we did cut it a little short. But let's see what we got.

But it's done. I so this is the first one I think out of any tool again that's built good charts. Like these are actually charts that as an analyst you would probably turn to, you know, what's the revenue trend, the EBITDA trend over time? There's a graph there that has got those lines or bar charts together. So revenue, gross profit, EBITDA it's continuing to go. So we've got like key ratios and metrics being created at the bottom here. I'm really impressed again with this tool so far.

I think it's the most complete analysis we've seen. I mean doing all the ratios and everything. Now obviously we need to check, make sure everything's right but on surface it all appears kind of right. Numbers, right?

I think so. I mean certainly the high level number that I look for is it showing.

Formulas, like if you look at some of them. Can we quickly trace. Yeah, yeah, we could trace it all.

Formulas. So but that, that line, that EBITDA line that goes negative. Here we go. So it's not. Yeah, this is the thing we, we haven't let it finish but it's now formatting these.

Oh, it's doing that now. It's still finishing. Wow. Okay. Really nice.

So let me just zoom out a bit. I'll just expand this out.

It's doing some, got the currency right, did pounds for you.

Wonder if it's going to take off these decimal places for me. But it's, yeah, I think what it's getting towards, you know, if I expand that out a bit, maybe it would have done this for me. But this is getting to the stage where you would actually have a pretty comprehensive dashboard. Yeah, I think that's great.

You'd validate, maybe make a few changes. But this is the closest I've seen to presentation. Ready?

Yep. Yeah, I agree. I think the column widths would, would be Fine. There you go.

So, yeah, yeah, I think it would get there, but we can just. I mean, that's all you're doing is adjusting column width. I call that a win.

Oh, totally. And there's. It's. It put two decimal places on for every number, which is not ideal, but I would say that that is a pretty good job.

Absolutely.

All right, well, why don't we go ahead and switch. I'll start my screen now and we'll run through just a few. So we can leave some time for in here as well.

I'll let this carry on running, by the way. So if we have two minutes right at the end and it's finished, we can always see what it looks like at the end.

Yep. All right, so the example I have that we started with each time is the price variance mix. So I said before we have budget and actuals, we're looking at, you know, gross margin, volume and mix for three different products. Having it explain it. So what we're going to kick off here is we're going to start with asking it to do just the price and see if it gets that right again. I've done agent mode similar to what you did versus asking, and we'll see what it comes out with. So we're back. And we did the pvm. So we could see here first formula. It took the actual minus the budget times the actual units, give you the. The price variance. Then it did the volume, then it did the total mix variance. And so what you're going to see is, you can see up here, I have the total variances. It's 220. This is 220. So that works out. If we look at, you know, kind of the individual ones, it looks like because this one's all three together, it all looks right. Let me just pull the actual answer here for a second and validate it all. But what was interesting is it didn't put anything in. And then it asked me if I wanted something and it answered all three, even though the prompt I gave it was just the first one. The other ones haven't done that. So I thought it was kind of interesting. And here's the. Let me look at the actual answers here for a minute. All right, so, yeah, so it looks like it got everything right. Interesting.

Okay, good stuff.

All right, so let me now open the deferred revenue one. I'm going to close these and let's open that file. Actually, let me just open it over here since I have it up. Okay, so we've done this one before we'll go ahead and do the deferred revenue schedule here, we'll give it just one minute to pull up. Let me get my prompt. Go ahead and get started and let's see what it does.

Do you want to read the prompt again or as it's working?

Yeah, I'll go through the prompt. So basically, let me just kind of explain the data. And what we're doing here is we're doing a deferred revenue schedule. So if we take customer one, you see the first invoice, customer name, there's a contract start and end date, how many months, the contract amount. So there's 24,000 roughly, I think it's 999amonth to spread over the 24 months as the product name and address. So what I've asked it to do is on a separate sheet, take this and create a deferred revenue schedule. That schedule should include the invoice, the customer name, how much revenue I should, you know, recognize each month and a check to make sure I recognized all the revenue told it it should be over four year period from January 1, 2026 all the way through the end of 2029. So we can see now it's labeled a sheet called deferred revenue schedule. And, and it's thinking. So we can see it's put in the dates and we're going to go ahead and pause it and we'll come back in a minute. It'll take it a few minutes and see how it does. But it looks like so far the work it's doing is similar to others so far. Alrighty, so we're back. It finished its analysis, you know, it's formatted it. You can see kind of giving me all the instructions. It went through its seven tasks. The schedule is ready for use for monthly revenue recognition. So if we look here, you know, one thing it didn't do is column width come in and fix that. But the dates look right. Number formatting. I'm not a huge fan of showing the zeros, but I'm sure I could ask it to correct that. But the biggest thing to me is again, and you know, we could probably ask it to prompt, but notice it's hard coding the dates.

Don't love that because it can do it instantly, but that becomes unusable. And see this is the thing I actually believe as we see these things go more and more, what I believe is going to happen is these are going to be our partners. I believe that it's going to build tools that people will take over. I think it'll do a great job. You know, they'll do a great job building a foundation, a tool, but someone is going to want to play with it on their own and extend it. And I think by doing it. And we saw some of the tools do that. Admittedly, this tool did not. Giles, for your. It didn't stick in hard dead numbers like some of the other tools did, which is great on tab AI, but I think that we're going to have situations where people will take a formula built, know it's working and then copy it down without realizing that it's done something that makes it not modular, not dynamic. Anyway, that's my thought.

So I'm going to ask it to make it dynamic and just see how it does real quick. And then we'll move on to ends examples. I know, Giles, you're going to bring those up. You know, the other thing that's interesting is the other ones all did an absolute value and said, hey, as long as it's within a penny. Because sometimes you get rounding errors. They didn't do that here. I like that a little better to have that, you know, kind of, hey, within a penny. Tell me it's okay because here many of them gave conditional formatting. This just said, hey, here's all your zeros. And so now I gotta filter. Is it really all zero? I love when there's just a quick check, but that's a minor thing. I don't see totals at the bottom. They checked it, but some of the others did that. So I mean, it's good. But on this task, I think it's. It's on par with some of the others. Some actually did the date formula properly. See, now I asked it right. That's what I would expect is, you know, J1 less than or equal to D4 or, you know, E4. So if it's in that date range, then recognize the revenue. So it fixed it once I asked it to use a dynamic formula. But you would expect it to do that. I would. You know, I'd want it to do that. So easy fix. Took a minute. Did a good job on the whole. But this hard coding is a concern to me. All right, so why don't I turn it over to you, Giles, give you back the screen here and.

Yeah, give me back the screen and just for 30 seconds. I'll just give you a quick update on that trial balance analysis tasks. Actually, I was going to call it out for getting something completely wrong because I. It's so. It's created this budget versus actuals analysis. Now I didn't even spot that I did have some high level budget variance numbers against each trial balance line. So I was going to accuse it of hallucinating and making up numbers which to a degree I think it may have done. But also it's kind of picked up that I did have some budget variance percentages in there.

I wonder if it used those percentages like the average of them or. It's hard to tell.

Yeah. So I think that may have been what it's done. In which case it's done a fantastic job. Here's an example where you've got to be really careful. Albeit it hasn't quite finished, it's built out a really nice little balance sheet summary. But if you look at these formulas here and I see this in so many of these tools, total liabilities and equity formula is just linking to the wrong cell. It's a cell short each time. So it's got zero there. It's just like it just like a human accidentally has linked to the wrong cell. So it's not perfect. However, an incredible job. Ian, I don't want to take up any more of your time. So we're doing it.

So I, I don't have. As you know, I wasn't able to install the tool because they, they may have switched as we know, to their. So Giles is going to be my partner right in this instance and we're going to, we're going to do this one together. Now typically I would be getting it to test an existing model and then asking it to build its own. Do you want to start this way? You want to start by asking it.

To build a. I think I don't know about you. I think that's like the most interesting first check in your area. It's like can it even get close to building a model like Agent did?

So Giles, why don't you take this prompt that I've shared with you and toss it away. You know, I've been asking them to build a five year forecast model. I've been asking it to make reasonable assumptions. And so for revenues cost appreciation really just see if it. To see if it understands the architecture, the design, the structural requirements and how it goes. So you've pasted that in and we'll see. Now in, you know, the other tools did take a few minutes to do this particular step. I'll help you build a comprehensive. It's doing it perfect. It says it can see the historical data. Now I'll go ahead and build A comprehensive with separate schedules. Let me create a. Oh, it's building a to do list. I like that. I like that. It's got a to do list. Right. To do list. Good modelers.

Starting with the assumption section. That's a good. Reasonable.

Absolutely. I like that.

Do we want to come back? Is it worth pausing? Because I'm guessing.

Yeah, let's pause and we'll let us know how long this takes and. But as we've asked it to build a pretty blue sky thinking five year forecast model, let's see what it does.

All right, we're back. It's been running what, about 18 minutes now, Giles? Something like that?

Well, not. I think it's like 21, 22 minutes.

So how's it doing so far? Let's talk for a minute and then we'll let it finish.

Yes, I think we've all decided, even though it's been quite a while, we feel like we need to try and give this a bit more time because it seems to be doing quite a good job in the sense that underneath the three statements of actuals that we had, it is building out the underlying kind of assumptions and drivers for the forecast. I think we also, we don't particularly like the fact it's got the forecast years kind of along the same within the same columns as the actuals. Really pedantic, but also not pedantic because it's just important. And then it is creating each of these schedules. Now again, there always seem to be like one line or one row or column issues a bit. So it's was misreferencing something here on the growth rate. And because it was doing that now the formatting's a bit off. The growth rate is here and then the actual kind of value is here in percentage formats, which is a bit weird, but it's going through each of these schedules. You'll see lots of div zeros and it's finding its own errors. And what we're getting towards is on the right hand side of the actuals. We do have five years of forecast being set up. There are some gaps here, but in principle the revenue line looks reasonable. The kind of freight and warehousing net cost line looks reasonable. It's not doing something right for cost of sales and operating expenses. Essentially this should be on this line because that's actually ebitda. So not perfect. But I think we all agreed it's close if it can fix its own errors. So we want to see what happens.

That's the question. I think Excel agent You know, unless this fixes its errors was a little more clean than what we're seeing so far. But we're going to give it another 5, 10 minutes here. We really want to finish this. Then we'll be back and kind of wrap up the episode and share how.

It did so well, I guess.

I guess the point is none of the other tools other than Agent even seemed to be getting close, did they? So they didn't warrant the extra time. But this feels close.

A couple of them stopped early, didn't get there. We're really not getting there. This one is. Well, I'm curious, of course, to see, you know, some of them. The other ones were also iterating. Iterating and improving upon themselves and then kind of stop. This one's still working. So I want to know what it feels comfortable with as a deliverable to us, because I've, you know, I've learned that it might look, you know, might look messy or sloppy, but it might keep working at it, and I want to know what it delivers. So we'll. We'll come back.

All right, so we're back. We've been going a good 30 minutes now on this one, Giles.

Good 30 minutes. And you know what? It's. It's really interesting because I think had we not experienced what we did with Agent last episode, we'd probably be sat here going like, wow, this is. This is miles ahead of anything we've seen. Agent did far better in less time. It's getting there, but there are some massive holes all the way through this at the moment, like to the point where there's just. This is not a finished or complete model. I mean, look. Look at this. It is. My caveat is it's still fixing things.

Yeah. It's still thinking. But it's mess you've got here.

Look, you've got a term, you know, kind of mini corkscrew of sorts for term debt beginning, debt, amortization closing, balance. But it's just the wrong links and you've got, like, the signage switches for the amortization from the actuals to the Forecast is negative 25 there, then it's positive. So that, I mean, that's just wrong.

If you look at 20, 23, I've never purported to be the most, you know, the fastest on my arithmetic. But how is it that you could start the year with $250 million of debt, pay off 25 and end with 25?

The answer is because it's going through the wrong column. And this is just a little Bit.

Next month you're 275, then 50, then 250, even though you're paying off 25 every month.

Right?

Yeah. So it's a column out for its references. But again, this is. I think Tab AI has done brilliantly across the board today. When you get into this financial modeling territory, one mistake like that, the model is useless.

A couple of my thoughts as well. And this seems consistent with some of the other tools. They seem to be better when they have very specific guidance and instruction on populating and answering questions and filling in cells. When you give it a set of historical statements and say, just go for it. Build something, you know, unsurprisingly, they've struggled more. But for me, the frustration is we've now been here for 30 minutes. We don't know how much longer it's going to take. It might take another five minutes, another four hours. I mean, and if you look at a formula like that one there, the operating. I mean, what's in. Can you click on the previous column Cell I this. So again, I this. I don't. So that clearly was there. We gave it that. That's a sum that it knew that in 2024 we were doing a sum to arrive at operating cash flow. And yet when you go to J, when you go to the exact same cell in J, it's done that. And you know, the net income is like no man's land there up in row 37. Why is it. It's like a little island all by itself.

Hey, it.

I would highly caution that nobody should be submitting a model yet like this to their boss. Obviously now it might fix it and make it perfect. It's still thinking. We just don't have a sense for, you know, where it's at and how much longer. But yes, we clearly have seen some errors. Do you want to point out one more thing that we noticed? You know, if you go down to the. The debt schedule. Zell, one quick thing that I noticed, you know, on the revolver piece of the debt, right. I mean, right click on sell three. So this is a row with no label. So I'm not really sure what it's doing, but it's. So what is it doing? It's taking total long term assets minus and then it's adding.

Why is the bank debt revolver equal to long term?

Well, no, I mean, weird. It's. It's taking a negative of negative, it's adding up and minusing. But listen, it's obviously in a bit of a spin here because its numbers don't make sense. But it's unclear. It's unclear how much longer we're going to need to wait to get something that might be useful. Listen, I don't want to sound too negative. It's extremely impressive that it's doing this and building something, but I think people, you know, should know you can't exactly take a historical set of financial statements and say, build me a rough, a quick and dirty five year forecast with some reasonable growth rate assumptions, reasonable schedules, and in five or 10 minutes we're not getting, you know, a beautiful model that we can then play with. Like we're going to have to troubleshoot this quite a bit before we can use it. And we don't know how much longer.

Prime example, there's a guy, Michael, on LinkedIn. I think it's Carney or something like, I can't remember the last name, but you know, he mentioned when I use Claude, I have 30 pages of prompts to build the model and I'm still doing 40 hours of rework. Right. He's trying to prove it can be done. You can get there. But it is not a go get your coffee. It's not a just trust it. You gotta know what you're doing and you gotta be willing to do a lot of reprompting and put time into it. I mean the whole tabs. I think the summary is Tabs has done a very good job too today.

Very good.

Yeah, been, been really impressed on the whole. There's a few things that you know, but despite it being impressive in general, you gotta know what you're doing and you gotta realize the more complex it is, the more time it's gonna take you and you have to decide, is that worth it?

I think Ian made a really good point which, which he may have made on a previous episode, which was in those battle work. Even though the prompt is quite a high level, there's a huge amount of text information saying, right, this is what I want you to do. This is an example. Here's an example. Answer with this information. So those sorts of workbooks give so much guidance versus this where the prompt is like kind of saying like, you need to understand financial modeling. Here's a rough starting point. Go. And agent's the only one so far that we've seen that mechanically has done a pretty good job actually.

Agree. They've all really struggled when there's limited structure and you're just asking them to go. Like the PVM is quite structured, did good. You know, the deferred revenue schedule is A fairly simple task compared to building a full model.

Right.

The Excel use cases are very structured. The analysis you have to do is less structured and you know, in general they haven't done quite as well. I'd say CAB did the best job on that by far. Far still had its issues, but was very impressive. I mean, I'd give it a 9 out of 10 on that one for sure, not higher. It was really good.

I think Tab AI has done the best of the third party tools, ignoring Agent from X from Microsoft. I think we're doing Shortcut Next, which is the one that, you know, has got a very high price entry point for one of the packages. So I feel like the third party bar has been raised by Tab AI, but Agent has already kind of taken.

I think it depends on where you look like. I think Tab AI did a better job on that ledger that you gave it, the analysis there than Excel Agent.

True.

I think it was on par on the structured cases. I think on building the model. Excel's done the best so far. So I think it depends on what you're doing. I think it's on par, just different.

Isn't that really interesting already though, that the Agent has done better on the task that requires more specialist knowledge? In one respect it is really interesting.

I think from a instruction standpoint and the fact that you can ask or add Agent. I think the most clean interface so far has been Tab AI. Train Slide is probably next on instructions and thing. Agent has work to do there. But it's interesting.

Yeah, listen, I'm excited about the future. They're all going to get better. They're still learning, you know, lots of reason to be excited and lots of reason to be very cautious and. And hopefully this is helpful at, you know, letting people see where they're at, what they're doing and with the experience that we're having run these tools.

All right, so I think we'll call that a wrap. Another great episode.

Thank you.

In.

And Giles, you say you wanted me to rap? Sorry, I wasn't sure.

Oh, yeah, go ahead if you want to wrap real quick to close this out.

Come on, let's go.

Give us a wrap.

If you think I can actually just rap freestyle, you've bought into those videos too much.

I'm very disappointed then. Giles, you're kind of like AI all hype and mo.

Oh, wait, I'm all hype.

So on that note, what we've determined is it's all just hype. No, in reality, it's impressive. Just like your videos. Are, Giles, but they take work. Really? The message here is it takes work. Right. Your videos don't just happen on demand. Nothing really does. AI is getting better and better, but you need to know what you're doing. Nothing's changed in that tab. AI has been great. One of the best tools we've tested. Been very impressed. And we'll keep going. So we'll see you next week. Until then, thank you.

See you, gents. Good job. Thanks, everyone.