148 The State of Accounting AI

Team, I am off celebrating Thanksgiving this week. But this last Sunday on the main channel we did a kind of a State of Accounting AI Q&A to celebrate hitting 10,000 subs on the main YouTube channel. Exciting times. We crowdsourced a whole bunch of questions from big brain questions to AI questions that people are too afraid to ask. Ran through those and we're gonna put em here on the podcast feed. Cause I think it was a good discussion of just what's possible today and what isn't. So without further ado, here's the big AI Q&A round. We hit 10k subs. So let's do an AI Q&A. Ah, so I put out on social media, what's the stuff you want AI to do for you right now? And you're not sure if it can led to a great discussion. So it's just gonna be a bunch of thoughts on the state of A accounting and ideas for stuff that you can actually do today. Audit automation. Can AI fully automate audits? Analyzing data and flagging discrepancies? Can it do the whole thing in like one fell swoop? Like no and probably never. But that kind of misses the point. We've actually got a number of really interesting AI like audit software companies building products. Now the reality of how all this stuff changes is it's not like boom, we just fixed this thing. It is the tasks that require human work change over time. So. So AI tech's absolutely going to eat into the stuff that we have to do. One really good example is audit interviews. All the questions that you have to ask. We can now record all of that stuff and AI can pretty reliably pull out of those interviews information to like tick a box to say that we have this and reference the exact quotation. Like that's an example of how AI will definitely change audit. But there's still gonna be a bunch of stuff for us humans to do. Predictive analysis. Can AI predict financial trends based on historical data? It might be able to. Like it will try. It will make something up. But just blindly asking it to do that probably isn't the right way to use AI. I approach it from the standpoint that I need to instruct it exactly on what my expectations are for how it will do that process. Think of it like having a junior person on your team or an SOP in your firm make really explicit guidelines for the right way to do that thing and then have it do that thing that way. We put this mystique on like this black box, like magic wand AI is just going to go do this stuff. The reality is as long as it's a black box, people are going to be annoyed with the output because they have different expectations for what that output should be. So the better approach to this and many AI problems is to really explicitly outline what you want and then it can do it pretty darn well in a totally scalable, low cost way. So if you're thinking about predictive analytics, that's probably how I would approach it. Fraud detection how advanced can AI be in detecting sophisticated financial frauds? Kind of the same as the previous answer. Honestly, how much can you instruct it on what exactly to look for? Sure, maybe it's got like some novel capability. If you just chuck at some stuff and say find things, like maybe it will do that and maybe it's still even worth doing that just in case it does find something. But the better version to me is a more procedural approach where you have it do things step by step, similar to like what you do with human beings. I think oftentimes we may not discover fraud because, you know, of sampling risk or just like not following all the procedures. From that standpoint, this could potentially be much better than humans because you can do that stuff at scale very quickly. But right now I would only trust it to be as good as the explicit instructions that you can give it. Tax optimization Can AI offer real time tax advice for various scenarios? This one's like brutal just due to the sheer number of variables there is. And I'm thinking specifically US income tax. But you know what, like if we can build nasty old tax software that can handle all this complexity and calculate it all, I think we can totally instruct an AI in how to do the same thing in time. In reality, how that probably comes together is not this one magic model that knows anything and everything about all things tax. What it probably actually is is a collection of a bunch of modules, like a more like modular approach. So you've got models for this, this, this and this. Right now the best AI applications you basically talk with like a controller model and then the controller model delegates tasks out to expert models for specific things. So if you think about something just astronomically complex like US income tax prep, the solution there probably ends up being a bunch of distinct models that all know how to do specific things very well and then this sort of orchestrator model that works across those. Got a ways to go till we get there. But I do think AI has significantly reduced the cost to building a tax like prep platform from scratch. Client advisory can AI transition to offering strategic financial advice. I'm gonna sound like a broken record there. How do you proceduralize advisory? There's some people who say like, oh, advisory is just, I've done this for 30 years. And that's just the, you know, that like whole gut feeling, shooting from the hip thing. And there's people that definitely do advisory that way. And to get an AI model to do it the exact same way that you would, that's not realistic. It's also not to say that it couldn't do it better. It would just be different. And much of what people want from AI is for it to do it how you would do it. And if it doesn't, you think it's bad. That's just, that's human nature. It's the exact same thing that we do with staff. Right. So I would say to the degree that you have systematized financial advisory, AI can absolutely help you with that because it's really good at following specific instructions. How far is AI from being trained on a given tax code and set of accounting standards and being able to spit out better than average interpretation application advice from a CPA closer or further than self driving cars? I'll be honest, I think in many situations we're probably already here. Now what's admittedly really hard, and this is very region specific and tax jurisdiction specific, but my background is US Tax. And the hard thing in US Tax is actually getting to what is the authoritative literature that I'm going to base my decision on. Because you've got the tax code, you got regulations, you got like court cases, all of these different things that may have commentary on the same subject. If this was like an IKEA build instruction manual and there was one page about how deducting your office and home works and that was it, like great. It would be phenomenal at that. The problem with tax law for us is like it's a collection of all these different things and many of those things give different answers for the same thing or creates a conflict with kind of some other thing. So right now I would argue it's actually very good at this if you give it specific context. So we did a whole video on how to use AI for research. I'll link it up here. And it basically involves pointing AI to the authoritative subject matter that you want it to use. And GPT4, the latest model, 3.5 was not good at this, but 4 is really good at sticking to specific context. What I don't want is it using its general knowledge to answer a question. What I do want is to point it to this and say, according to this, what is the answer? Or according to this, Give me the 10 most relevant word for word excerpts from this document to help me make a decision, which is. Which is really like teaching it how to research like a human. And that stuff, I would argue pulling the most relevant stuff, I'd argue today is every bit as good as a human, if not better. At the very least, it's a killer second set of eyes. But the harder thing, and this may be different in different places where tax looks different, maybe you've got just a single set of rules to work with that probably works great today in the US the hard thing is, like, what the source of truth is. And getting all of those things in one place, honestly, huge opportunity for traditional research providers who have all that stuff organized already. Unfortunately, they're usually the slowest ones to move. Right. I want something very simple, just that. AI what can enter into a portal, huh? And download all the bills for us, or download all bank statements without wasting our time. Oh, preach, buddy. Not having to log into a kajillion different systems and download that stuff for you. It is. It's really. It's not very simple, unfortunately, because it requires supporting every little, like, mom pop, like anybody that's got a website, like, logging into this thing. There's just the notion of supporting tens of thousands of different user interfaces to go download every bill imaginable. Kind of sucks, I will say. And it's like, it's impossible to build from a code standpoint, that would just be astronomically complex. Right? And then every single one of those sites can change, and you've just got this really fiddly thing that breaks all the time. This is actually a killer application for AI agents. If you listen to my podcast, we've talked quite a bit about autonomous AI agents, which you basically task them with a thing to do, and they browse the Internet to go do that thing for you. And those AI agents are probably gonna be fundamentally more robust than somebody hard coding how to log into a portal and download a bill. And maybe it's a combination of the two, because as soon as something changes on the website, that breaks. But an AI agent can reason like, okay, the login button is over here, or this thing got renamed to that. It stands to reason those agents could make this stuff much more robust. But I would argue that right now, while we fixate on this in, like, the course of doing monthly closes for clients, what we're really missing is a explicit pre Accounting process that is a standalone process where it is somebody's job to go out and capture all of the things that that can't be captured in an automated way. Because you know what, this is an admin process and it takes somebody like 30 seconds to do each of these things. Why it's annoying is that we haven't made it somebody's job to do yet for us. And then you get to the 13th in the month and you're in a scramble and you're like, oh, I gotta go grab this and this and this. Think about refining your pre accounting process because this is not an expensive thing to pay somebody to go out and do. It's not fun. No. And I do want to automate it. But if you're a big brain pro, if you're working with clients and stuff like that, you should not be wasting time going out and logging into all these systems and grabbing stuff stuff yourself. Actually, next week sometime right around here, we're actually doing a dedicated pre accounting episode on my daily podcast. Check that out. Okay. Can AI look at two documents, a bank feed from the accounting software in PDF form, and a bank statement in PDF form and show in a table which transaction is missing? Man, I love that. Like, this is the beauty to me of AI is these really fiddly things that we're just stuck on where what if I could just chuck this into a magical machine and it helps me. This is the personal productivity side of AI, where it happened because something else broke. But man, if I had an assistant that could help me with it, how cool would that be? Now, the practical thing that makes this hard today is the model that's really good at doing this, which is ChatGPT's data analysis model. If you're on the paid ChatGPT, in the dropdowns, you'll see data analysis. Unless for your version, they've now gotten rid of those dropdowns cause they're talking about rolling them all up into one. But unfortunately, with ChatGPT, the only real privacy settings we have is they toggle under settings that says don't use my prompts to train the model. Now for you. Is that enough to be able to put potentially sensitive information in there if you know it's not retained? That's like a firm policy decision that you all need to make in each of your firms. I did a podcast episode on how to develop that policy for your firm. Highly recommend doing that. Over 80% of firms still have not. And so I'll preface it with that as right now the hard thing is, can you do that securely? Second part of that answer is absolutely yes. Can you do that? Now the best thing to do is probably try to get both of those sets of data in CSV form and chatgpt can actually do that for you also. So you can upload a document like a bank statement and say, give me a CSV of all the transactions here with the format, date, description and amount. Now normally what I'll do in practice is I will copy and paste the transactions in because then I can remove any sensitive or identifying bits of information, but it will absolutely reliably give you a CSV that's 100% correct. And then when you've got this second document, that is what is in the accounting system, you can also use ChatGPT to get that to a CSV. Once I have those two CSV files, I'm going to upload both of those to ChatGPT and do just what you said, go through these line by line. I would probably say identify any unique records in each of these according to their date and amounts and it'll absolutely do this. The big blocker right now is being mindful of data sensitivity. Is there an easy way to redact those documents? Otherwise, like, the exciting thing is AI can absolutely do that sort of thing for you today. Like now, can AI create a searchable source of truth for our company's SharePoint FO show? A while back on this channel we did like a custom how to build your own custom ChatGPT video. It was with a third party tool where you just, you upload your context, your documents and it makes it searchable and chattable. Microsoft's coming with their own version of this stuff. They call it 365 copilot. It's now available for select enterprise users, but hasn't made it down to us normies quite yet. I don't know if that includes SharePoint or not, but the state of things right now is in pretty much every situation, yes, you could like dump out all of your data from a system and put it in another system that enables this, or you can wait until that system gives you a chatbot for it. But it could be a long wait. I mean, Microsoft was teasing this functionality back in February and we still don't have it in most people's hands. And honestly, it's a good, a great reason to be talking with our vendors to say, hey, how are you leaning into AI? Like when can we expect some meaningful developments here? Because the rate at which things are changing has, has Made that honestly a huge driver of our future success. If the tools that you are on right now are not leaning into AI man, that is going to hurt when you've got other tools in the market that are taking the lead on that stuff and saving people a bunch of time. So that's absolutely a conversation we need to be having with our tech partners right now and rewarding those who are shipping awesome AI stuff. It's something that I try really hard to do on my channels. When somebody ships an awesome AI thing, we need to be talking about that, we need to be rewarding that because it is keeping us accountants relevant. Can AI develop a 7 email series funnel for lead generation? Very specific. So here's how I think about this kind of problem and how to get it to complete a task like this. Two sides to achieving that task. I can tell you how not to do it, is go to ChatGPT and say develop a 70 mil. Like, the quality of that's just not gonna be great through the lens of the user. Because ultimately what we actually want from AI is for it to do it the way that I would do it. Which, who knows if that's better than the way they would do that or not. But the reality is, like, we're looking for something that aligns with a very specific expectation. So if I'm coming to it with a task like that, I would give it two things. First, I would give it a framework for what is the correct way to do this? What is the best way to build a seven email series funnel for lead generation? Maybe there's like a thought leader that I really like that has put together some content along those lines. I'm gonna give it an outline of, like, here are best practices for how to build this out. Second thing is the actual meat of what goes into those emails. Do you have blog posts? Do you have titles? Do you have kind of a general flow of what you think will be in each of those funnels? Any other body of context that you have that can inform what goes into them, provide that as well. So you've provided the technical framework for the right way to do it, and then you've provided the subject matter for what will go into it. And this is honestly the same way that, like our brains work the way that we would approach solving this problem. But chuck that in there and you will actually get something pretty darn good that you can use as a draft to kind of iterate on and use as, you know, a starting point for finalizing it yourself. And if your brain is pushing back and saying, well, why don't I just do it myself from scratch? Because that's more work to do it from scratch. And because doing it from scratch doesn't teach you anything about working with large language models. The reality is these things are like personal assistants that are going to get more and more powerful, connect better with the other tools that we use. Whether you're doing it via chat, whether you're doing it via speech, these things aren't going away. And it's genuinely a learned skill that will make you more productive. I honestly think of it as akin to like computers first coming around and being like, well, is this really saving me time right now? The answer is 100, like 100% of the time. No, it won't. Like, there's going to be times where you try to use, you know, chatgpt or Bing Chat for something and it will fall on its face and it won't save you any time. But you're investing in a skill that is going to become more and more valuable over time. And each time you exercise that muscle, the next time you come back, it's more likely that you'll know how to use the tool in a way that will get you to where you want to go. I'm curious to know if AI can simplify complex tax regulations and automate tax calculations. It would revolutionize the way we work and save us valuable time. What's your take on this? It depends on like, the scope of complexity. I mean, the answer in general is yes. And so if you took a very small, simple tax calculation, like a little regional excise tax or something like that, where maybe there's three pages of rules for it and a one page form, that's something that it could smoke today for sure. What gets really hard is when you have these massive bodies of literature that define what is correct. And oftentimes those things conflict. And getting access to all of those things even is a challenge, unless you've got like, you know, this proprietary research platform that has all this stuff in it already. So on the subject of can it simplify complex tax regulations? Yes. Like, it actually is really good at this, putting this stuff in layman's terms, like making logical connections between different things across a big body of text. I mean, how many times have you been doing tax research when you're reading through and you think you got to your answer and then seven pages later there's like an exception that changes what the correct answer is for you? It is like the worst part of tax research. But Having a research assistant that can go through a larger body of stuff for you as a second set of eyes, that is super, super helpful. And we have the tech to do it today. But as is the and a bunch of applications right now, it hasn't really been productized yet where it needs to be productized. So right now the best place to build this is into the tax research platforms that we use. But we don't yet have a super meaningful implementation of this yet. I can like upload five documents to ChatGPT and show you what it can do and what it doesn't do well. And that's literally the same tech that is like working on the backend. When this stuff gets productized into these systems, we're just waiting for everybody to get caught up on this. And honestly, us talking about it and sharing ideas and shining a light on what the most valuable applications would be is helping to move all of this stuff forward. Like, we have a lot of developers and product companies that watch this stuff. We've absolutely helped push some cool AI stuff into the market. So let's keep having these conversations. Bookkeeping. When will AI make it better? Not this. Suggest a category based on where I put it last time. Rubbish. Read the receipt, extract the data, match it to the bank transaction and categorize it. I think we're a few years off, but it's coming. It totally is coming. In fact, the technology that'll be the biggest driver of this is something called embeddings and vector search. Did a whole podcast on that up here how it's gonna take us a big step closer to fully automated bookkeeping. Biggest blocker we have, at least in the US from it becoming fully automated is the fact that you can have, depending on different context, a Starbucks transaction categorized as like four or five different things for tax purposes. So oftentimes classification is in the eye of the beholder. Maybe management wants it one way, your tax preparer wants it another way. That is like the last 1% that we're gonna struggle with. But certainly big things like receipt. A receipt from Costco that has 20 different things on it. Where right now maybe that just gets chucked into office expense. This specifically is going to get way better and way more like context driven where you can give it the receipt and it can classify that stuff automatically. That that podcast, if you're interested in geeking on that stuff. It's kind of a walk through how that technology works and how it's currently being built into the tools that we use. Pull PDFs of a capital account statement from various portals, extract and structure the data from those PDFs and load that data into an upload template for accounting software. Okay, let's talk about the last part of that. Extract and structured data. I think it's extract structured data from those PDFs and then load that data into like a fixed format. That's something ChatGPT today does really well, actually. So if you have those reports in PDF form, the big blocker right now is are you able to redact it? But if you give ChatGPT unstructured data like that, but then define here's exactly the structure that I want it to be in, it actually does that really, really reliably for the first bit. Pulling PDFs from various portals. I'd point back to the pre accounting discussion of like, that needs to be a part of your process where you have admin support team members at the start of each month or whenever those docs drop, they've got an explicit list of pre accounting tasks that are super detailed and go as far as where do I pull this thing and where do I save it for the file system? So that when your senior team, like your pros, roll up and they're doing a month on close, you're doing a tax turn or whatever it is, they got all those goodies there in one place. This is one area where, yeah, I would love to automate it, but we get so focused on the fact that we can automate maybe half of it and instead of like actually making a process for it, we're just like, I guess we'll just automate this half when the fact that you can't automate it isn't a reason to not build a process. A manual process is better than a broken automated process, especially if it's a manual process that an admin can do. And eventually that stuff probably gets automated, but that's not a reason to not have a manual process for it today. Okay, two more what do you think of a technical accounting bot that helps firms slash accountants to keep up with updated guidance? For sure. Like, we need it. The hard thing here is when you say technical accounting bot, like, what body of information does that include? If you're a tax pro, that's a bunch of different things. If you're accounting pro, that's a bunch of different things. What we'll ultimately have, I think, is really granular control over the context that that bot will pull from. So do I just want it to pull from GAAP or from IRS regulations or IRS Publications or the tax code. Ultimately, once we can have really granular control over what context it will refer to, that's gonna make these bots much better. And the big thing that I think a lot of developers have missed the point on thus far is at the end of the day, I don't want an answer from the AI. What I want is all the information that will help me make my own answer as efficiently as possible. And so that's probably a list of word for word quotations with citations to where it got it so that I can make my own answer. Because AI will never reach a level of authorit where we can just blindly trust it, even 10 years from now, like when it's way, way better and probably always right. If your client gets audited by the IRS or something like that, and they come and look into it, you can't point at the AI and say, well, the AI said it was right. At the end of the day, it's your professional responsibility to ensure that the answer you got to aligns with like the technical literature. And so what I don't want is this black box. Like, give me an impressive answer. Cause right now that doesn't actually save me time. I need to know how it got to the end. Last one. Why use ChatGPT versus something like Intuit Assist? So Intuit's got an AI chatbot coming to QuickBooks, among other products. Why would we still use something like ChatGPT when we have tools like that and we're kind of like barreling towards this really annoying state of things where you have a chatbot and all in every single tool that you use. Right? And that's just gonna be really irritating. Ultimately there's probably some consolidation that happens there where ChatGPT, either through plugins or through this new thing, they just launched custom GPTs where ChatGPT can see into the other systems that you use. For example, your accounting ledger, Pull that information in, connect it with five other apps, and now inside of one chat experience, it can see into all that stuff and even write back to those things. So ultimately, why would I not use Intuit assistant instead use ChatGPT? It's gonna come down to what data set am I looking to manipulate? Is it the general ledger? Is it my email? And the more all encompassing, the more context it has access to, the better. Interestingly, right now in public accounting, I think the practice management systems are in a really unique place where they have access to all of our organizational email, our files, our projects, our meeting transcripts, team chat. It already has all that context, and many of them actually authenticate with accounting ledgers also. So if they support uncategorized transactions or something like that, there's already an accounting ledger connection there. So who's in the driver's seat to build a killer chatbot that can see across all of that stuff? Probably our practice management system. Now devil's in the details of how well it is implemented. It's kind of that issue of like what is better the best of breed tool for specifically chatting with a ledger or something like that versus a more holistic tool that maybe doesn't do those specific things quite as well. So devil will be in the details of like how well each of those things are implemented. But all of these things buddies stand to make me much more productive. So I am pretty excited about that gang 10,000 subs on a rinky dink little channel about running accounting firms. Thank you all so much for hanging out here. If you enjoyed this more like conversational format. This is what we do over on the daily podcast, like almost every day. Just talk and chop about running accounting firms. Running accounting firms for me was always intimidating because I didn't know other people that did it. I didn't have a mental model for the best ways to do it and didn't know where to have those conversations. My hope is through this channel and through the podcast, we're having very nuanced conversations every day to kind of demystify all of this. Pull more people into running their own firms and get really high quality information in front of you about the best ways to run firms sustainably. Cause we've all seen the versions that are not sustainable, right? Thanks for coming along on the ride and I'll see you when I see you. Bye.