333 What’s The Best AI Model for Accounting In 2024?

All right. Last week on the podcast, we put the new OpenAI model to the test. Reviewing a balance sheet as an accountant would can it identify the issues that I myself, a seasoned, enlightened accountant, human also, by the way, for the most part, would identify the same things that I did. And wouldn't you know it, it absolutely did. It passed with flying colors, the new OpenAI model, which got me thinking, hey, there's a whole bunch of models out there. We've got open source models that are actually better than closed source models at some things. Models that you can run locally on your computer. Are there more models that are capable of reviewing our financial statements? And are there like specific contexts where it's like, well, it's good enough for this little thing, so let's use them here as of today, which is a big old knockdown, drag out all the biggest AI models out there, biggest, some small ones too, that you can run locally, where we're gonna see how they all stack up to see if we can start offloading some of this, like technical stuff, like review work, a second set of eyes, like even a first review for a prepare. Can we offload some of this stuff to a large language model? Hot take, right? We're gonna test it today. Come on in. Let's see how it. So previously on the podcast. I don't even know whether to call it Jason Daly right now because we're in the middle of a rebrand to Jason on firms and there's just so much content in motion at any given point in time to where that like drawing a hard line between before and. It's just. It's impossible. You don't care. Okay, so previously on reviewing this balance sheet, what the balance sheet is. It's a balance sheet we ran off from the. The QuickBooks sample file. Craig's Design and Landscaping Services. If you are a QBO user and you have like an accountant profile. So qboa, when you first log in, you see a list of clients. That's what you're seeing. Then it's qboa. From the top right, you can select the sample company file. And this is super, super useful for kicking the tires on AI stuff. If you're considering is it capable of X, Y or Z? This little sample company is a very small SMB, not a huge volume of data, which in my opinion for testing this sort of thing is the place to start. This is not going to be your E comm company with like 10,000 monthly transactions. It being able to do this does not mean it can do that. But it is a signal that it's like, if it can do this, then we need to look more and see, like, where are the boundaries of what it can and can't do. But if you're doing similar things in your firms to find ways to use language models, this sample file that QBO offers, it's a great place to start. And when I first went through this balance sheet last week, basically I identified seven issues that were, I guess, potential issues. Because when you're reviewing a balance sheet, you don't have complete information. You're not looking at the ledger detail. You're doing a first pass to determine what things need to be looked into further. And that may be resolved by the ledger detail. And it may lead to a question that needs to be asked of the client. And that that kind of materiality, that judgment of knowing what's worth looking into and what isn't, is kind of like a secret sauce of a thing that right now you kind of just learn with experience. It could certainly be accelerated by somebody helping you and mentoring you and all that stuff. But what's useful here is not just to see, does the language model identify technical errors, it is to see how does it make that judgment. Because a huge part of being an accountant is when you make that judgment. When you, when you're like, it doesn't matter, when it does matter. So the seven things that I identified on this balance sheet as potential issues. As a reminder, one, there was a new savings account that didn't have a balance the previous month, got an $800 balance this month. And I'm like, I want to double check that. It wasn't like a bookkeeper that fat fingered the wrong account and just put it to the wrong place. I'm probably going to double check that with the bank statement. Number two, undeposited funds went up from 687 to 2062. I don't usually like undeposited funds and I don't want it tripling in size. So let's double check that. Number three, they got a new truck, 13,495 on the books. Let's double check the purchase docs to ensure it got booked correctly. Number four, there's a board of equalization payable that had a negative $21 balance last month. Why was there a negative balance last month? Somebody's getting fired. And a positive $209 balance this month. Sometimes there's reasons for this. You had a credit, you didn't want to reclassify it to a current Asset account, whatever. Still worth double checking though, right? Number five thing I noticed there's a new $4,000 quote, loan payable and current liabilities. Bad name. Also OpenAI the new Model 01 something they called out that I didn't. It's called loans payable. Let's double check. That's actually a current liability. It sounds like it may not be. Number six thing I noticed there's a new $25,000 note payable. Note payable is a bad name. At least give me the bank. But I'm going to double check the loan docs to ensure that it's been booked correctly. And. And then number seven, good old opening balance equity. It swung from positive 5,000 even last month to negative 9,905 this month. And what God's green professional accounting firm earth should we be seeing? $15,000 swings in opening balance equity in a month over month set of books? That's unacceptable. Also QuickBooks demo file. Bro, you're better than that. This should be the. This should be the pinnacle of properly done books. Actually, maybe not. Maybe it should be a hot mess like representative of the books when an SMB owner actually does where you see things exactly like opening balance equity this changing month over month. So maybe this actually is a good example. I identified those seven things the new open air model identified 10 things, none of which were wrong, seven of which were the things I identified. One was equity has a big negative balance. And I'm like I don't care about that. I actually care about the underlying issue. And then the two other things that called out were fine, like totally valid things that I didn't call out. I wouldn't call them high risk things. It was like, you know, you have accounts on there with no amounts for those accounts on the report, like either zeros or nothing. And that's actually. That one's actually probably a good note. Like I don't. From a formatting standpoint, I don't like putting stuff on the balance sheet that doesn't need to be there. But all in all, the new model which everybody has in Chat GPT, when you're talking with Chat GPT at the top, there's a model selector. You can choose which model you're talking with. We used 01 Preview, which is the biggest new model available. The cool thing with this is it does more like deep thinking reasoning which reduces hallucinations, makes it more likely you have a correct output and it's better at thinking that requires several steps of thought which maybe reviewing a balance sheet does. But what about all the other models out there? We've got big old models from Google, we've got models from Claude, We've also got smaller models which are increasingly getting really good. And that's actually a big deal because small models we can run on local hardware like your phone or your computer. Now, quick diatribe on running models locally because oftentimes accountants jump to I don't want to have to trust online security of anybody else. I just want to run it locally. I'll wait until the models are good enough to run locally on my machine. What I think the perpetual trap with this will be is the big models that are humongous and only capable of running efficiently on cloud hardware. Like to be clear, the biggest language models out there, like the biggest llama open source model to run that locally on a Windows machine. If you could even get that to work, which I don't know that you could, with a massive graphics card, like a big like gaming graphics card, you're maybe gonna generate a token or like a word or a part of a word per second, like it's gonna be slow as heck. And that's why there's value in smaller models being more capable. And the smaller models are getting better and better because they'll run faster, they consume less energy energy, they save a bit of the rainforest, all that stuff. But the problem with like sticking with the narrative that I'm just going to hold it out, hold out and wait until I can run language models locally because I just don't trust them. The problem with that is while the models you can run locally will continue to get better, so will the cloud models and it'll continue pushing the boundaries of what they can do for you. So if last week we actually hit a point where large language models can perform a useful review of financial statements, we might have just reached that. Like that's a genuine possibility. If we did, then do I want to sit on my hands until it's good enough to run locally 12 to 18 months from now, or do I want to use this for review? I can tell you my like, from my standpoint, I want to use it for review even though it's in the cloud. So yes, the local stuff will get better, but the cloud stuff's always going to be way ahead and it's going to be really hard to resist that human urge of, of, well, geez, this other one's way better and I could use that. Even though it's cloud based, it's gonna be really hard to not adopt the kind of workflow advantages that can offer us this episode of Sponsor in part by liveflow. True story was just shooting a Sunday YouTube video and whenever I write these, I'm like, what's a good, like ad partner that would fit in real nice and organically to this video? The video I think is going to be a banger. It's about how to better use your scheduling app to like automate and standardize everything that happens around meetings. And I thought what is a great tool to like standardize the prep work for meetings? Oh my gosh, Live Flow. So many situations. Think about like a client discovery call. Being able to use Live Flow to pull a huge spreadsheet, Excel or Google Sheets workbook worth of data out of QBO or zero, a whole bunch of tabs of information with the click of a button. Then literally select a different company that you're connected with, click of the button, pull all that data for that company too. Say you're headed into a client discovery meeting. The client books that on your calendar. You also book 30 minutes of prep time. You pull all that data from their accounting file into a spreadsheet. You've got an easy way to review it all, go through it with them in the meeting. Another cool example, LifeLow's got like over a hundred templates of like different dashboards and advisory stuff. One of them is like a four week cash flow dashboard. Like this big elaborate thing. You click the button, it pulls all the data from the accounting file. Let's say as one of my accounting packages, I do like weekly or monthly advisory calls with them. I use my booking tool for the client to schedule that live advisory meeting. I then book a prep meeting on my calendar, say 15 minutes or something like that. To get ready for that meeting, I hop into my spreadsheet app of choice. With the click of a button, I pulled that cash flow data into a spreadsheet. So I got like everything. And just like that, 90% of my prep work buddy, it's, it's there. And I can customize that template to be whatever I want, use it across a bunch of different clients. Imagine making those types of meetings so turnkey, where you just use Live Flow to pull all the data that you need for that meeting in with the click of a button. Man, so cool. Really fun Use case of Live Flow often forget liflow also does consolidations. So where this is a real headache is when you got to pull stuff from a bunch of different company files. You can do that with a click of a button as well. Even if they have different chart of accounts, set up the mapping. Once you could be heading into a big old board meeting with like a bunch of different companies involved. You can pull all the stuff that you need with a click of a button. Super cool. That's Lifeload. Learn more Check out the link down in the show notes this episode is sponsored in part by Team up, who will help you find talented folks in the Philippines to work for your accounting firm. They have recruiters on the ground in the Philippines. We'll find you awesome people. I'm looking at a bunch of them right here. Rona, that's her name. She's got a bachelor's in information systems. She went to work for a US firm as a virtual assistant. They're paying her 1070amonth. She had four years experience. She's handling people's inboxes, calendars and scheduling their social media. A firm hired her through Team Up. How about Aaron? He's got a BS in accounting as a cpa. Six years income tax US income tax prep experience. So he's doing everything business, personal hustle and ultra tax. US firm hired him. They're paying him 1964 per month. Farah she got an MBA and a CPA was last running an entire team of bookkeepers. Six years experience. US firm hired her, their painter. Just over three grand a month to run a team. A team gang. Whether these folks are in the us, whether they are abroad, they are capable of running any part of your accounting firm. Particularly as a lot of firms are moving towards like kind of a relationship manager model where you have an onshore person that is managing the client client relationship. That's really like the one blocker for some folks is they won't put offshore folks in direct contact with contact with clients though even that is kind of evolving. But our worldview around offshore work is oftentimes misshaped by the fact that menial tasks in other contexts like support will be delegated to India or something like that. The reality is these are super talented people that are capable of learning absolutely anything your onshore team is learning. And you've got Big four firms, you got regional firms, you got all sorts of firms employing tens of thousands of people in the Philippines that are absolute experts at their craft. I shared before my last four hires were in the Philippines and my firm completely changed the way we approach getting the work done. And that is again how I would build a new firm tomorrow. To learn more about Team up, check out the link down in the show notes. All right, so first we're gonna look at Google models. We're looking at a whole bunch of different stuff. But real quick, for perspective, what I wanted to do is actually show you what the quality difference of the models that we've gotten just in the last 12 months because we just showed off last week. We'll put a link to this in the show notes. It could be worth checking out that episode if you haven't yet before you listen to this one. We just showed off last week a model that well and truly like nailed a financial statement review, at least at a high level. That initial pass, making a list of things to go look into, which I would argue is a big part of the review process. Maybe 75% of it. There's still half. Do you fix it? There's still. How do you ask the question of a client and whether you should. But just identifying the potential issues, that's a really big part of it. But we're going to hop in a time machine here because I think the rate of improvement is really important, especially as we have two new OpenAI models on the horizon. We don't know what they'll call them yet, but basically think of them as 4.5 and 5, which are said to be way better than we have right now. And in general, anytime you get a new model from OpenAI, what was fringe and what the last model was barely capable of is table stakes for the next model. So a lot of people think right now we're going to have GPT 4.5 in the fall and then GPT 5 in Q1 of next year. Two generations of models from now what we're showing today, which is really impressive, is going to be beyond 10 table stakes, which means by the time we get to next April tax deadline. I don't know. I don't know if that means we're not going to have humans reviewing financial statements. I don't know what that means. But the way that you do technical reviews on accounting will have fundamentally changed. If we're already right now on the fringe of being able to chuck that stuff in and have it do the review like in less than 60 seconds, two moderate, two like model generations from now, you better believe it's going to do that stuff. I don't know at what level, but at a really, really high level. High enough to where we're using it. Right. And so that's why we're having the conversation here now because that impacts how we like design the org chart of our impacts a lot of things. But that's frankly that's overwhelming. So we're going to baby step into that and and kind of pull back and think through just what this means for our firms. And today. What I think is going to be fun to tease out is what models are capable of this stuff right now and what are not, and could I even do it locally? And is that something that may be on the horizon? So looking just at OpenAI's models, I would say that O1, the new model, honestly nailed it. Pretty much perfect. What if we go back to the 4O model and as we talked about last week, O1 is basically the 4O model but with some reasoning built in, like it has a conversation with itself to improve the quality of the output and reduce the likelihood of hallucinations and solve longer, more complex problems more comprehensively. Let's just go back one model though, to 4.0. What was the best model in the world two weeks ago and see how this does on the exact same balance sheet review test. So a reminder, the prompt that we're putting in here and you can test this yourself, here's the prompt. It says here's a draft balance sheet. You're the accountant doing a technical review on the financials to ensure their correctness. Identify any items that may be incorrect, then provide a list of items that should be looked into further. I'm not interested in the insights about the company or its financial health. This is purely a technical review to ensure the accounting has been completed correctly. Only list items if there's probably an error and then I copy paste the balance sheet in because you can't upload files to the new model yet. So I literally just drag select the balance sheet and it's this big ugly mass of text. But language models can make sense of that. So in the interest of fairness, while I could upload a PDF to the 4.0 model, I'm going to give it the text because that's the same thing that we did with the new model. So here's the response it gave. It identified 10 potential issues. The first potential issue identified by 4O, the previous best model number one total liabilities versus total number one total assets versus total liabilities and equity. It says the total assets 17558 should match the total liabilities and equity 17,558 which is correct. However, ensure that each component within these totals is accurately calculated and categorized. That is an absolutely useless, hot, hot garbage insight for me. It's saying that assets equal liabilities plus equity, which is great and how it should be. But you should make sure that each component has been calculated correct. That's your job, you numpty. That's a really bad insight. That's a. That's not good. Number two, Savings account missing prior period balance. The savings account only has a balance for the current period, 800, but does not show a balance for the prior period. This could be an oversight or missing data if there was a balance sheet that should have been carried over. If there's a balance, it should have been carried over. I'm realizing now one of the smart things that the other model did. In fact, let me flip back to that conversation and I'll show you what the newer model said about the same thing because it identified the same issue. But here's how it explained the issue. It said introduction of savings account without prior balance. Remember, we're reviewing the current month's financials issue. A Savings account with $800 appears in August but not in July. Explanation. The new account should have an opening entry if it was open during the period. Action. Confirm the opening of the savings account and ensure the initial deposit and any transfers are recorded properly. Back to what 4o said. It says it doesn't show a balance for the previous period. This could be an oversight or missing data if there was a balance that should have carried over. No, no, no. That's not the issue. What the issue is is we're. We're reviewing the current month's financials. We already closed last month. The issue is, was there actually a new account or not? The issue is not, are we missing data from last month? The new model nailed that, right? The old model did not. It said you're missing data from last month. That's not what I would go and review. I would go and review the fact that there's actually a savings account now with an $800 balance. I would say that's oh for two chat GPT4. Okay, remember this. This is. This is telling us how big of a step we've gotten in the last two weeks and how LLMs can help. Accounts number three. This is still from the last model 4.0 fixed assets truck. The truck's original cost is listed as $13,495 in the current period, but shows zero in the prior period. If the truck was purchased in the current period, the balance should reflect the acquisition correctly with no previous entry. Confirm if depreciation or purchase entry is missing. Like confirm it wasn't missing last period. I'm going to give it like half credit for that because it did call out that we should Ensure that it was booked correctly. But when I see a new truck on the balance sheet, my mind doesn't go to well, we better look to see if it should have been there last month. My mind goes to why is it here this month? Right. I'm going to give it half credit for that. Number four, Negative value and board of equalizations payable. Prior period. The board of equalization payable shows a negative balance of 2116. The prior period liabilities should not typically show negative balances unless there's a credit adjustment which should be investigated for accuracy. That's a pretty good answer. Even weirder is that it went positive this month. I'll give it full credit for that one. Loan payable and other current liabilities. That's number five. The loan payable amount 4000 is listed under current liabilities. Verify if it's current or should be split between current and long term. It's also a net new loan. It doesn't say anything about reviewing the docs. I guess that's an okay answer. It also called out the new notes payable of 25,000. Confirm if this is current or long term. That's a good answer. Equity balances. Oh, there's like two sub items here. Opening balance equity. Significant change in opening balance equity from positive 5 to negative 9 suggests posting errors or adjustments were made. The other model's answer was much better because it elaborated and said the opening balancing equity is usually just a initial one time thing. And it went as far as to say that it should be moved to retained earnings or a capital account. That was a much better answer from the new new one. It also called out. This is the older model. The net income for the Prior period is negative 127 but has an unusual balance that should be reviewed to ensure if it matches profit and loss statements. What is unusual about $127.93 net income balance? Oh, this is a good one. Number eight, Consistency and accounting basis. The balance sheet is labeled cash basis. But ensure that all items, especially liabilities like credit cards and loans are accounted for consistently under cash basis rules. Typically these would not appear under pure cash basis. That is not correct. Okay, so this isn't very good. It has two more undeposited funds. That's a good call out 10 missing retained earnings. Eh, the other model said that too. This is not usable is my takeaway here. So it maybe got like half or third credit. Whereas the new model I think nailed it like 10 out of 10. So even just going back one model, frankly it is not even close, which makes it laughable to go back further. So the newest model that we tested last week was the model 0101 preview specifically. Gosh, these namings are awful. The previous best model we had from OpenAI was 4.0. The model before that that was best was just four GPT4. I just plopped that same prompt into GPT4. Here's the response. So we're now two models back. Number one date Friday, September 13, 2024 falls under cash basis. The date line should only consist of a single specific date. I don't even know what that means. Number two Fixed Assets Truck the balance sheet shows a sudden appearance of a truck as a fixed asset. There should be a depreciation expense. Maybe. Number three, the current assets for the previous month July are higher than the current assets for the present month, August. This is not an error per se, but could be if there are no identifiable transactions leading to this reduction. You have all the amounts that make up the totals. This isn't good. Number eight the net income under return and retained earnings has shown a sharp decrease from July to August. That's a symptom. Like that's not a. That's not a specific technical error. I just want the technical errors not super impressive and a few things that are just outright wrong. Let's go back one more generation to GPT 3.5. This is specifically GPT 3.5 Turbo, which is an improved version of what first came out with the original ChatGPT in November of 2022. So almost two years ago. Here are the insights it came back with. Number one, the balance of the checking account seems low compared to the prior month. Verify if all transactions have been accurately recorded. Does it? And went down from 4321 to 1201? Maybe. Number two, the entry for savings account is missing as there should be a balance recorded for it. It was 800 this month and zero last month. I don't know what that means. The credit card's balance has decreased noticeably. Ensuring that all transact. Ensure that all what? No, it says ensuring that all transactions have been properly recorded. Credit card balance went from $158 to $123. I don't know that I'd sound the alarm over that. These are not great. They're also very concise and aren't particularly clear how you would remedy them. Today's episode is sponsored in part by Bookkeep, the tool that's gonna automate a whole bunch of your E Commerce accounting workflows. You Know apps like to get all up in your face and be like, come use me bookkeep. What people love about bookkeep, this is an actual user quote. Bookkeep works so well I forget I am using it. Don't you want the thing that just works? Here's the thing with Bookkeep. Bunch of their team, actually most of their team are accountants and bookkeepers. They're not software nerds that were looking to cash in on some sort of plugging a thing into another thing. Arbitrage. What's the difference with bookkeep? Well, they use sale data while other apps base their entries on payout data. This means that they're like inherently cash basis. Right. Bookkeep uses sale data. Means you have way more granular control over what you can do with it. Seeing very detailed information from each sale and giving you greater power for handling. Handling very complex scenarios where say you're selling across multiple e comm channels, taking various payment methods. You can map all that stuff so you're not just stuck with the payout data. Right. To handle tricky stuff like booking shopify revenue based on fulfillment date as opposed to the default which is the order date. Somebody listening to this just got really excited, man. No other app does that. Okay? Your payouts will always reconcile to your bank feeds to the penny. And you're never going to miss a payout because bookkeep will literally tell you every single day how much money each selling platform owes you. That is the power of using the sale data, not just the payout data. Right? If you're hustling e comm stuff, please just don't stop DIY and hacking together. You just, just don't. Okay? Get in touch with the folks at bookkeep. They're accountants and bookkeepers just like us. You will feel heard and understood. Check out the link down in the show notes. This episode is brought to you in part by Cloud Accountant Staffing. Hey, let's hire some talented people in the Philippines. Because that's what I did in my firm, you know why? Well, let's. Let's compare this to the traditional hiring method. Going with a traditional recruiter. Startup fees 15 grand. I don't know. Startup fee with cloud account staffing. Thousand bucks. That's it. Annual employee salary entre for a senior accountant. They say 75K. That's reasonable. That checks out. Annual salary for the equivalent Philippines hire. 30K. That was 75 versus 30 with traditional recruiters. You got to make a long term commitment. Cloud accountant staffing commit for two months. That's it hire in the Philippines, usually in about six weeks onshore, you're probably going to deal with two to three months or also maybe eternity and your offshore team benefits, payroll taxes. You don't have to manage that stuff yourself because it is no fun as opposed to working with super talented people in the Philippines, which is very fun. I definitely did it myself as I've shared, I built my firm tomorrow. Still what I would do hearing from a growing number of folks who tune into this stuff who are like had never really considered it before. Honestly thought you were a shill the first time I said it. Now I've made my first hire and I'm super happy with it. To which I say cool because it's totally how I would roll a firm. Still learn more about cloud account staffing. Check out the link down the show notes so this those responses. That's the quality of the responses we had when we first got ChatGPT and there's a lot of people when we first got ChatGPT and it became the fastest growing app of all time, the win out rushed to judgment and said this thing stinks. It's never going to be any good at accounting. Just look how far with just OpenAI's models we've gotten from investigate the credit card balance going from 158 to 123 to now it nailing all the same things that I did and actually giving really useful tips for why it's probably wrong and how to fix it. So OpenAI models have come a long ways and not a lot of time seemingly will just continue to get better. Which is part of why we're having this conversation. Seems pretty darn good today, but six months from now, like two models from now. Wow. Like what are all the ways we can use this? Because that's not very far from now. But I know we probably got some some Google fanboys, fan girls, fan people in the audience, some open source nerds. Let's say you want to run this stuff locally. We are going to run the exact same prompt. Uh, let's see, what should we start with? Let's start with Claude because Claude has been like the other model that's really, really good and kind of pushing the limits of what's possible. And there's been big chunks of time where I've just used cloud and not chatgpt because it is really good. So let's go to the best of the best cloud option right now. The model specifically is 3.5 sonnet. The exact same balance sheet prompt asking it to review it. Just give me technical insights, not like insights about the financial health of the company. Let's see how Claude does. This should be good. I mean, a month ago everyone said this was the best language model you could use. But as we're seeing with this testing, a lot has changed in the last couple weeks. All right, number one. Oh, it only gave us five things. Number one, the savings account balance is missing for July 31, 2024. So because there wasn't a prior month balance, it says it's missing. Number two, the truck asset appears in August but not July without explanation. It's a balance sheet. What would the explanation be? Number three, the opening balance equity shows a significant unexplained change. I don't know what explanation you're looking for. It's a balance sheet, bro. Number four, the board of equalization payable has a negative balance in July, which is unusual for a payable account. Then number five, total equity for August doesn't match the sum of its components. Pardon? Opening balance equity plus retained earnings plus net income. You're telling me a QuickBooks generated financial statement doesn't math? Right? That has never happened. Sometimes I feel for the non accountants that tune in into this podcast and just don't get the humor, you know, that's just wrong. It does. It totals just fine. Good job QuickBooks. You did it this time. So that was a month ago. The best language model that we had that was as good as it could review a balance sheet. It's missing quite a few things that I had brought up that were worth looking into. Not even close. Honestly not even close. Which doesn't give me actually much hope for the Google models and the other open source models. But let's do it anyway. If anything, this is becoming an amazing advertisement for ChatGPT and how good this new model is that we just got. Boy, I don't know about you, I feel like I've like rekindled the spark that I had early days chatgpt of like we just discovered something that's pretty wild and could do a lot for us. Let's go to Gemini 1.5 Pro. This is the biggest, most performant Google model there is. Best like general purpose model. Let's see how it does. I would expect it to not do as well as Click Claude that we just heard from. Number one, cash basis versus balance sheet. The header states cash basis, but a balance sheet should be prepared under the accrual basis of accounting. This is a fundamental issue and needs immediate correction. Is it worth even Continuing truck depreciation. It says there should be depreciation if you have a truck. That's actually not a bad note. Loan payable classification. Part of the $25,000 note payable might be current negative equity. While not technically an error, the large negative equity balance raises serious concerns about the company's financial health. So important here is a language model's ability to follow my instructions. I even used the words financial health in the prompt and said, I don't want you to comment on financial health. So this is not like complying to what I'm. What I'm asking for. It brought up the negative Board of Equalization Payable. That's good. It told me that the inclusion of a date and time stamp on the report is unusual and unnecessary. That's how QuickBooks prints these things out. That's actually kind of funny. And then title consistency. Hang on. The title refers to Craig's Design and Landscaping Services, but the balance sheet itself lacks a formal company name. This should be made consistent. I have no idea what that, what that could possibly mean. Okay, that's the best Google model. I'm, I'm honestly, I've never been super impressed with Google models. I hear people say they use the Google models for writing, and that's an entirely different challenge, to be able to write something in a natural tone and all that, but for technical things. Not impressed. One other Google model to test, just really quick. This is one of the Gamma models, which are smaller models meant to run on local hardware, like on a smartphone. If you're a nerd. This is the Gamma2.27 billion parameter model. This is the sort of model that you could run locally on your machine. And then after this, we're going to test two open source models. So this is a smaller model that you could run locally, which should ought to put in perspective, like how useful is a cloud model for this versus a local model for this right now. First issue, it called out cash basis and the title suggests cash basis accounting method. But the inclusion of accounts payable and notes payable, which are accrual accounting concepts, is inconsistent. There is no accounts payable in this. There is a notes payable, but that's not a problem. Total bank accounts and total other current assets are repeated for both August 31st and July 31st. But the individual line items only show data for August 31st. That's a QuickBooks report formatting thing. It didn't put a zero for some of these. It just leaves the line empty. I would, you know what, I would agree with You. I. I don't like that either. Arizona Department of Revenue payable has a balance of 0 for August while other liabilities have non 0 balances. I don't. The fact that it went down to zero, that's. That's not a flag for me. Calls out the negative board of Equalization payable. Loan payable is under other current liabilities but lacks a maturity date. That's. That's a weird thing to have on a balance sheet. The total liabilities and equity do not equal the total assets for both periods. That'd be a good note if it were the case. But they do equal so that is a bad note. Okay. Itty bitty model didn't really do well there. Two more models to test. The open source monsters. The biggest open source models you can get. First we will test. It's called Mistral. We're testing Mistral Large 2 and until we got the Llama models from Facebook, these were the biggest and best open source models you could get your hands on. Let's see how this one does. First one calls out the board of Equalization payable being negative good. Second negative equity. The opening balance equity and total equity are negative. While not impossible, negative equity is unusual and should be investigated further. That's kind of your job. And number three, cash basis but accruals present. The balance sheet is stated as being on cash basis but includes things like undeposited funds and loans payable which are typically accrual based accounts. That is not correct sir. Date discrepancy. The balance sheet is dated as of October 31st but review date is September 13th. That's the date that the report was printed. Ensure that all Transactions up to August 31st are included. Not super impressive. Last one. Let's try the latest Llama model which is the best open Source model now. 405 billion parameter model. You could not run this locally like you just couldn't. Even on the best Apple hardware which is like the best hardware for running language models locally. I don't. I don't think that you could actually run this but for open source like see how close it gets. Number one, the savings account balance has no comparative figure for the previous period. That's true but it doesn't act like. Give me a suggestion for how I could confirm that. No accumulated depreciation recorded for the truck. Not a bad note. Calls out the negative board of equalization payable says the loan payable account has no comparative figure for the previous month. That's not a problem. The problem is it's a new thing, and we need to confirm that the new thing is correct. Correct. The prop. This is. This is what actually GPT4O was doing. It's saying something's wrong because there's nothing in the previous month. And that. That's not wrong. That's just something that we need to double check. The rest of these aren't bad, but it is a long, long way off the new open AI model. So how's this? Let's make a deal tomorrow. Let's stop this faffing around and let's go deep on just how much the new open AI model can do. Can it actually follow through and like, say, review the general ledger detail and propose a resolution, how to fix it? Could it actually do that? Start with the top level things to look into, but then also do the next step. Dig into it and say how to fix that issue or how to pose a question to a client. Even like that, judgment is very important. Tune in tomorrow. We're going to dig into that. If it crushes it, boy, I don't. I feel like I got to shout that from the mountaintops because that feels like a big deal. But tune in tomorrow and we're going to see how it does. I'll see you there.