136 Using Internal AI Chatbots in Accounting Firms

Come on in. It is Q and A Friday where I answer your questions about running accounting firms in a more sustainable way. A way that'll maybe save your marriage and at least ensure your kids can identify you in a lineup. Today we're talking about whether you can have AI chatbots like just inside of your system, just in your goodies so that nobody else can see. Another question. Are there like any totally secure ways you can use language, large language models right now where you just cut the team loose? Somebody bought a firm but doesn't know anything about accounting? Got some advice there, Yikers? Should your practice management system integrate with your tax software? A whole bunch of stuff. Come on in. Let's get our Q and A on. Okay, got a bunch of questions about LLMs and AI chatbots and all that stuff because we've kind of been talking about that this week and for whatever reason it's been seeming like they're more part of the general public discussion lately. So some helpful stuff for how to use this stuff in your firm and cutting, also cutting through some misinformation. Got this question via dm. Quick question. Do chatbots totally and fully live within your system? So you aren't training the larger AI model on your proprietary stuff with client info? And is there such thing as an SoC2 compliant chat bot? So as with all, you know, all AI things, it depends ultimately on the interface and the chat bot that you're using to work with it. So many of these systems use the same large language models behind the scenes and that could be GPT. Most commonly that's GPT 3.5 and GPT 4. And if you just roll up to ChatGPT for example, and, and you're chatting with it with the default settings, your information is like your prompts are being trained into the model. But ultimately the way that we will work with these tools most long term is in the context of where all of our other information lives. So for example, as supposedly Microsoft 365 copilot went live today for people on enterprise plans. I'm recording this on a Wednesday the 1st. And that is an example of an AI assistant that is that lives within like your organization's sort of data. And so that is a, a chat bot that can see into all of the other organizational Microsoft 365 data that you have and it just stays within there. It's an extension of their Bing Enterprise Chat, which is an AI chat bot that if you use Microsoft 365 you already got access to it is their chat bot with quote unquote, a commercial data protection. And the big thing that they do that you always need to be looking out for is your prompts are not used to train the model. And so, you know, the notion of does this stuff all just live within my own data? Like back in the on premise days where you had a server, like that was a little more clearly defined, right? Like that stuff just lived on your server. Now as we're using cloud services, that looks more like, you know, for Microsoft 365, does that data just live within my 365 organization and for their AI chatbots, the answer is yes. For something like ChatGPT, that's just like a third party service that you're going out to, right? But the downside of that is it can't see into the rest of your firm's data. So that's almost like this sort of intermediary step where ChatGPT is cool right now, but ultimately the long term applications are the ones that have all the context of our data. And so that's probably, you know, chat within your practice management system or within your Microsoft 365. It could potentially be within ChatGPT courtesy of plugins. So there's over a thousand ChatGPT plugins now. Maybe there's a future timeline where we have plugins for our practice management system and QuickBooks and all these other things that actually make ChatGPT the interface that you use. But I think the more likely timeline right now is that long term the way that we work with chat is embedded in our in like the systems where that information is. Because if that chatbot can see into all that information, it gets way more helpful. And so really now we're just now seeing the first kind of mainstream application of this with Microsoft 365 copilot. Google did this with their chat assistant Bard about a month ago. Now they went live with its ability to see into your email and your calendar and all the other organizational data. And it's been, okay, it's been a little bumpy. There's been examples of, you know, it hallucinating entire emails. So like we're at a stage. And to be clear, the Palm model that Bard is built on right now is much worse than GPT4. GPT4 was a big step forward and hallucination. So it's still early days for this stuff. But ultimately how we will work with these things will be within our own systems. Like that's, those are the bots that we're going to be working with more than anything else. Somebody else had chucked the question. And I'm against people putting sensitive info into Chat GPT for this is a no, no. What's my take? If you haven't heard it, on Monday we did an episode on this building an AI usage policy for your firm. And it kind of outlines the three different types of information depending on their level of sense sensitivity and then how you define what's inbounds with a specific app like ChatGPT for each of those three types of information, something you got to think through to kind of liberate your team. We also had the unveiling of our guest, Nervous Ned there. Maybe we ought to chuck this question at Nervous Ned just to get his take. We chatted with him quite a bit on Monday. So I'm going to ask him, is there ever a situation where we can chuck sensitive client info into a large language model? If you missed Monday's episode, this is a character I've created with chat GPT and with GPT4. I basically gave him some parameters of here's who you are and, and kind of your worldview and it plays that character within those boundaries. And then I'm gonna use a tool called 11 Labs to generate a voice for his response.

Whoa, whoa, whoa, hold your horses there. Chuck sensitive info into a large language model. That's like throwing your saddle into the fire because you think it'll make a good seat warmer. Not on my watch.

Interestingly, you'll find that Nervous Ned, like many people, like to use analogies that don't particularly make sense to like, round out their worldview.

Sensitive client information is like family heirlooms. You don't just pass them around at a garage sale. Sure, you can strip down the information to the nuts and bolts, redact it until it's as unrecognizable as my Aunt Bertha without her signature beehive hairdo. But even then you're playing a high stakes game of Old Maid. The way I see it, if you wouldn't shout it from the rooftops, don't feed it to the chatterbox AI. That's a line you just don't cross. Not without a ten foot pole anyway. Keep it safe, keep it secure and keep it out of the language models. I say keep your ledgers close and your client info closer. Stay cautious.

Nervous Ned honestly is a fairly realistic proxy of like, I don't know, just the general fears of the change and moving too fast and all that. One interesting tidbit there. He mentioned redacting it until it's as unrecognizable as my Aunt Bertha without her signature beehive hairdo. On Monday, one of the things we talked about, one of the types of information is redacted sensitive information, which isn't usually considered sensitive information. And sometimes something can be redacted into oblivion to where it's not helpful anymore. So for example, maybe the people's names and the businesses names are actually a key part of the task that you're asking that thing to do. That is a totally genuine situation where redacting is impossible. So good point, Ned. I got one question here about Is it realistic to run AI models locally? And right now with the current state of models, they are super, super resource intensive and they go beyond most of them anyway. Go beyond the scale of like what's possible possible to be run very efficiently on your local machine. Now all that's being said, it's getting better. One big problem, unfortunately, is on Windows PCs, unless you're running a big sweaty Nvidia graphics card because your pwning noobs at night or mining crypto. Windows machines in particular are pretty brutal when it comes to their ability to run large language models because they're doing it off the cpu, which is like super not optimized for this task. Getting this episode is sponsored in part by Live Flow. Live Flow is the easiest way to sync that QuickBooks data back and forth to your spreadsheets. You may have seen this actually had a big announcement lately. So this fall G2 gave them the top spot in their fall 2023 report as the leader in the financial analysis category. That's right, they won number one. Nice work. If you've been around my channels for a while, you've seen life float kindly. They have spoiled sponsored quite a bit of stuff. And I'm not. I mean, I'm not saying I'm taking credit for it, but that was probably why I'm not familiar with Live Flow. Super easy way to sync that stuff. Sync your QuickBooks data back and forth with Google Sheets. They got a whole pile of templates too to make the process of building that stuff for the first time as easy as possible for you. Stuff for managing cache, AP KPIs, like everything you can imagine. Sync that data into your existing sheets to make them smarter. Get it to auto sync or build your like custom new sheets that talk with QuickBooks totally from scratch. Pretty cool tool. Check that one out at Lflow IO gang. This episode is sponsored in part by Copilot. Are you sick of Copilot ads? On this podcast yet. Because you shouldn't be. Because bless their hearts, they're making all of this magic come together and all of this possible. You know, Okay, I know we got some hyper nerds in the audience out there. If you saw our practice management report that we released recently, Copilot was on the list. Because they've got this hyper flexible approach to building a client portal and they're not getting in all the other stuff, the workflow and all that. All they want to make is a super flexible portal platform for you. And if you are a mega nerd, boy, wouldn't it be nice if all these tools in our space had killer APIs so that we could automate all this stuff automatically? Wouldn't that be nice? Not a thing though, right? Wrong, it's a thing. If you use Copilot, they've actually got a killer API that lets you work with pretty much everything in the entire platform. So the notion of building some like fully automated stuff where before that wouldn't have been possible. It is. It's possible. With Copilot, it's possible. But let's say you're just a little bit nerd, not mega nerd. Your little nerd. Through stuff like Zapier and make, you can actually build a bunch of your own automations. In fact, I got the Zapier page up here and there's. What the. Geez. There's an absolutely shocking number of triggers and actions. Hang on, pause, quick count, break. 35 triggers and actions in Zapier for Copilot. If you like building your own automations. Pretty cool, right? Hey, if that is exciting to you, check out the link to Copilot in the show notes. And thank you to Copilot for your continued support. Unfortunately, Apple Silicon right now is killer at this stuff. Like really, really good. Problem is most accountants aren't running Apples right now. And if you are, great job. The problem is Apple's like way behind everybody else on large language model stuff. But ultimately long term, is the solution going to be running large language models locally on your machine? I don't know that it actually will be for the same reason that all of our other software has gone cloud based. And so big upside obviously is you have way more hardware at your disposal if you are doing it through the cloud. But there were a lot of people that said in the past that like the cloud is fundamentally just not secure, so we're not going to move our software there. Well, we've kind of solved for that and so the solution is just giving us Secure ways to use language models in the cloud. It's not to take it all in house and run it locally. You can do it and it's slow. But for all the same reasons that cloud software in most situations is going to be preferred to local software, like the ability to get on like ongoing updates. The fact that software vendors are much more incentivized to build this stuff because they can charge you monthly rather than just charge you once and then have to give you support until the end of time long term. The reality is I think we'll just be working with large language models in the cloud with very limited exception, which if anything further opens up the delta of how useful desktop software can be compared to cloud software. Because if you got, if you got a legacy desktop application like QuickBooks Desktop or these tax softwares, how are you then going to bolt a cloud based AI chatbot into it to be very helpful? It's possible, but it certainly doesn't do desktop software any favors. What I'm super jazzed about, and I brought this up a few times, is voice assistants incorporating large, large language models. I for the first time used ChatGPTs built in voice functionality kind of talking back and forth. I was working out the other morning for an hour. I know, I know. And for pretty much that whole hour I was chatting back and forth with ChatGPT, like seeing like can I work through this problem I'm having right now or can I build out an outline for this thing? Just like trying to approach it in an open minded way to say can I actually accomplish something with just this voice assistant in this little window of time? And for the thing I was troubleshooting, it was okay. I think it's, it's very, probably going to be very situational right now. Whether that's useful to you or not. Right now what it is phenomenal at is writing and putting together creative things and outlining and all of that. It will do that very well for you. And if there's like a situation where you can kind of passively dictate and work through with that assistant, what you're trying to do, pretty darn good. For more technical stuff, not quite so much. And especially not if it's something that requires it seeing into some other source of data, right? Like that's where Chat GPT kind of falls over right now is it's like this own kind of siloed tool that's not connected to the rest of your data. Now something that really popped on LinkedIn this last week was this Thing kind of framed as like this, I don't know, shocking expose that some people, they said built a model, they didn't really build a model to do this. They basically set up an automation that would generate disinformation that was really convincing, like at scale, like a huge volume, like 200 articles a day. And we've talked about on this podcast like how you can generate thousands of articles a day with AI and it's like, oh, don't worry, they didn't actually release it gang like, man. Hopefully we've been stressing how obviously this is going to be super problematic. I mean, Sam Altman, OpenAI CEO said the biggest problem we're probably going to face is long before it reaches, you know, super intelligence or general intelligence, the ability of the AI to kind of go out and solve novel problems that humans haven't been able to figure out long before we reach that it's going to be super convincing, which will be really problematic because you don't know what's real and what's not. You don't know if you're talking with an actual person or not. So like, yes, I mean people are already doing this. Like disinformation is getting pumped into the ether en masse. And we've talked about how Google is deranking really low quality articles and its search results, whether it's going to catch everything and whether ultimately AI writing is going to end up being like better than human writing. Arguably right now it is kind of remains to be seen. But what do we. And there's all sorts of tie ins here with like from a political standpoint, you know, are we post truth and all these things and it kind of becomes this big complex discussion. But what is maybe like some of the net effect to take away from this? What does it put a premium on? Puts a premium on being able to tie something back to an individual who has built trust. Like that's why I show up on camera. That's why I'm not a fan of like anon folks going on social media and then trying to get you to actually listen to them. Like I'm not super afraid of this because I already have trust with the people that follow me on social media and listen to the podcast and all that stuff. And in my mind that just gets more valuable when you have that trust because something coming from me has a totally different value than something that just shows up in a feed. Right. And I know we've also been talking about, you know, the ability to like deep fake yourself or deep fake Other people saying things. But this is actually an opportunity for platforms like YouTube and LinkedIn and stuff like that to be the kind of the verifier of this being truly from that person. There's been issues with like TikTok ads where they'll have like, Mr. Beast and like other prominent people offering to give away this and that, and those ads are put out there without those people's consent. But if you can tune into a YouTube channel or my podcast feed, and the fact that you're coming to that channel is verification that this is actually coming from the person that you think it's coming from, that feels like kind of a moat to disinformation. Now, do accounts get hacked and stuff like that? Like, yes, absolutely. But that to me, like, if we're thinking through what's on the other side of this when it's really hard to make sense of all of it, that's a great rationale to like, start building trust on social media, to have your own channels that people can trust. Because, I mean, the like, tried and true method to undermining something that is real and something that is fact is putting so much information out that people just throw up their hands and they say, well, there's so much conflicting information, like, how do we even know what to believe? And it's like, no, you're missing it. That is the strategy. They put out so much information that you throw your hands up when it's like, no, we actually know this. And so more and more, I think people will get information from folks that they trust. And that's, I mean, simultaneously a good thing and a bad thing, right? Because there's people who are going to have bad motives, going to be people who ultimately won't tell the truth. But in terms of what is our place long term as human beings and advisors, it is in my mind, yet another case for you to be out there communicating what's important to you, to the folks you do business with. Even like kind of the broader Internet, like, as a personality out there that can be trusted, that actually knows what they're saying, as opposed to folks who have zero stakes and can just post something on social media and say, yes, you can write off your G wagon, right? Like there is no stakes there. You have stakes because you actually do this, you're a professional. And the ability for somebody to come to a trusted channel, like, to me, that's ultimately the solution for that. It's that time of year again, gang. Halloween? Nope. Thanksgiving. I guess it's soon. But worse than all that it is time to fulfill your annual continuing education requirements. That old chestnut this episode is sponsored in part by Pronto Tax. We know how that that CPE deadline usually feels, right? More boring than watching paint dry. CPE really do be like that, especially virtual cpe. Man, you usually can't even see the person that's talking. It's like this awful like set of slides with these fade animations between each of the slides. And then you hear this like super corporate voice, which frankly might even be AI. Actually, if it isn't, it absolutely could be. Well, are you ready for better cpe? Me too. If you're ready for a continuing education experience that's different than anything else out there, and not incredibly boring, yet still highly educational, you could be ready to give Pronto Tax School a try. The people at Pronto are a little crazy. What? Their training's not for everyone. Things can get spicy. But the people at Pronto really know their stuff when it comes to taxes, and they make a valiant effort to make completing your annual continuing edge at least halfway entertaining. They're approved by irs, CTEC and NASBA so that you know you're getting valid credits that match your credential. And over 4,000 tax pros choose Pronto's CE slash solutions every single year. Visit Pronto tax school.com or check out the link in the show notes to learn more. This episode is sponsored in part by the fine folks at Cloud Accountant Staffing. Do you hire accountants? Bless your little heart. Not the best part of the job in my opinion. Not something I ever enjoyed. Well listen, you can build your accounting dream team with talented offshore accountants in the Philippines that work 100% full time for your firm. Their accountants aren't freelancing or contracting for multiple firms. They're all yours. They work exclusively for you and are incentivized to stay with you and your team long term. They're not going to get swiped. Cloud account staffing is 100% dedicated to the accounting industry and founded by a former accounting firm owner that understands your business, knows your pain points. They had to hire some accountants and they said, you know what? We're going to build our own pipeline in the Philippines, going to pull in some super talented people and then open that up to other firms. Basically that's the story we've been talking about a lot about staffing, building more resilient staff staffing pipelines for your firms. I had staff in the Philippines like totally red pilled me to like oh geez. Like we need to globalize the way that we get our work done. Check these folks out link in the show description Cloud accountantstaffing.com okay enough AI stuff. SMB buyer on Twitter I love this Tweeted me I've got a topic suggestion for your podcast. I bought an accounting firm and know nothing about accounting. How would you advise not screwing this up? Selfish suggestion because that's me right now. Thanks for all your content has been invaluable. This is very self serving but like listen to this podcast like this is very inside baseball about the day to day of running accounting firms. Bless your heart you weirdo. Going out and buying an accounting firm. I'm not anti normies non accountants coming in and buying firms. In fact I think it's probably a good thing like an injection of of fresh ideas and the reality is like you come into it and you, you get beat up a little bit. Super relevant to this was yesterday's podcast with Patrick Dichter who's a guy, a super smart guy. I hope you enjoyed that. A non accountant who came in and and what was particularly interesting to me, what we talked about yesterday was how do you run an accounting firm if you're not the biggest brain technical expert? Because that's the place most of us come from. Like we are the boss. We sign everything at the end of the day. Or at least that's how we started. How do you delegate that? Quality assurance basically is what it is. And so that was great to hear from him on that subject. How do you not screw it up? That is a great question. You're already doing the right things. Be in community with other people that run firms like go to conferences, plug into thought leadership. Not because you're going to agree with all of it, but because the more of that stuff you're able to consume, the more it rounds out your worldview. So we all have like this single tunnel of kind of our own lived experience and what has gone well and not gone well in running accounting firms. And the most useful thing that I ever did was got out of that bubble and talked to as many other people that did what I do as possible because it multiplied that worldview I was able to have and overnight it turned me into a honestly a real a hole. Know it all inside of my firm because I was like so staunchly confident on certain things like, like no, this is the wrong path. We will get it wrong. Because I've talked with 10 people that actually did this and it went totally sideways. They're going to talk with people where it's like yeah No, I can see how you got that wrong because I would have done this or that differently. But that is all golden information that is all like seasoning the decision making process that you have and that kind of internal mental model you have for how you do stuff within a firm. I mean, I shared, what was it, two or three months back, I think when I was in Australia, we shot a pod where I shared like Man, 5 years ago I didn't know squad about any of this stuff. And I was still the guy who was honestly in the same situation you are in Tax Zach, not really knowing anything about how to run that business. And now people hold me out as an expert and it's honestly just because I think I have more of these conversations than anybody else and then I get to reflect all that stuff and what I learn to other people. So like I'm very transparent about ultimately what I do is I steal the best ideas I can find and re share that stuff and try to be transparent about what I think works and doesn't work. And hopefully it makes us all a little bit better. So don't like put your, don't have your head down so much that you're going to try to figure it out yourself because there's a lot to be gained by rubbing shoulders with other people who do this stuff as well, both online and locally. You're not going to get the same level of thought leadership locally and then make a couple friends who do this stuff that made a huge difference for me. Like make a couple of friends who are no bs, people who will give you the real story on, on how things go and who can also be like a sounding board and even just like a release when stuff's feeling like a lot. Because when you run accounting firms like there are dark days. It is, it's really hard for a lot of reasons. Maybe that key person just left and you're totally in a lurch. Maybe you let somebody down or you know, you've got an upset client and you know how you manage these things. Definitely some of that gets better and it's easier for some folks than others. But there's going to be really hard times like, and change management in an accounting firm like is absolutely brutal. That's the one thing that I think everybody always underestimates is the notion that you can step into a firm, modernize it and then be in this fundamentally different place. Like this isn't, this isn't a storage facility where I just have to swap out the human like gate operator for a computerized system. And now I've got this automated business. Like, the reality is doing accounting and tax work is still a human driven process. And the output's only ever going to be as good as the humans who are engaged in that process and what you can get them bought into. But good on you for doing it. Stick like, stick to it. It gets better. Keep trying to find a more valuable customer who will pay you better for what you do. That cures a lot of ails. The firm that's getting 25% of what this other firm is getting for the same work, they're feeling a lot of things. They're like, we don't have enough staff. I can't stop working because I have to work late just to pay the bills. Like a more specific customer that'll pay you better for what you do. Like, that makes a lot of things look good. Your capacity, your workflow. That's kind of the gold star when it comes to like, what, what needs to be my North Star for what our firm focuses on. And it seems really obvious, but it's also easy to fixate on like workflow and all these other things. Right? Okay, this is a biggie, gang. I got this question and it is a great question and I need to shout it from the rooftops so everybody hears it. This is super, super relevant to us audience folks who touch tax at all. But I have to think it also applies elsewhere as you're getting into regulatory stuff. So the question was essentially after we released the practice management system report, do any of these systems integrate with my tax software so that it will sync address changes and all of that stuff across my tax software to my practice management system? So all that is tied together and one source of truth. And if you know, you know, right? You know. But two things I'm going to say about that. One is this feels really important because the maker of your tax software wants you to buy their practice management software, which does quote, integrate. That is why it feels so important. Back in the day, when I was shopping for a new pm, I. I couldn't imagine ever going to a system that didn't have that integration because it would just be such a waste of time and like being able to zoom out. I felt that way because that's what the people within that organization would tell me who were trying to sell me on the practice management system. Now let me tell you about that integration because it ain't what you think it is. Not all integrations are made the same as all accountants know. In my experience, these integrations from the tax software are generally one way integrations. They're not gonna go as far as like pushing updated data like down to K1 recipients. In my experience there were also dependence upon like these pop up dialogues that everybody had to know like what the right thing to click in a certain situation was to sync that data, otherwise it wouldn't sync. And then you have issues with, well, can it sync when the return is in the process of being finalized? You got all these complications that make that integration absolutely not what you're thinking. It is like you still have a big data management issue no matter what. So the reality is behind the scenes that stuff is not sinking in a seamless way at all. Second is, and this was a huge lesson for me, there are so many other reasons to choose a software platform besides integrations. Integrations, as far as I'm concerned, are marketing. Now. There are certain mission critical integrations that I absolutely want. Don't want my build.com to talk to my QuickBooks. Oh yes I do, because that would be a whole lot of double entry. But the reason that people go out and they advertise this integration that they built is so that when you use tool A, you say, oh, of all these other options for this other thing that I'm going to buy, only one of them integrates with the tool I already use. So I will choose that one. And 90% of the time that's a really bad reason to choose a tool because you are picking that tool to do a job and the job of that tool is not simply to sync data with that other tool, otherwise you wouldn't be buying that other tool. Right. So fundamentally, how well does that tool do that job in a vacuum? Does that integration matter? Should it be a factor? Probably. But I think our heads, and this was me, I mean, I made this massive mistake and that's why it's so triggering to me. I way overvalued the importance of that relative to simply the functionality of that tool. So when we're talking about practice management systems and what, you know, more modern cloud practice management systems offer that you don't get from maybe a suite that integrates with your tax software, we're talking about stuff like automated client requests so you're not having to build these lists and follow up with clients and all that. A super slick client portal, something that feels really, really easy for your clients to use so they're not going to be annoyed about it. Being able to present a unified online experience to your clients. Imagine that. I mean, it's Quaint that we call these things portals when what they are is a website. And all of the services that we use have a website that lets us go do things like update our payment method, stuff like that. But when you start thinking about something like, where do I log in and change my street address so that'll go back to your system, Something like that right now in our profession is like the Holy Grail. Like, nobody has put that man on Mars yet. Like, the ability for a client to log in, update an address, and then it just magically go everywhere that it needs to go. And for people outside of our profession, they're like, why is that? What? That's ridiculous. Why is. Accountants are so dumb, buddy, let me tell you, it's harder than you think it is. But I'm subject to the tools that I can use to do my job. And let me tell you, I want that to be a reality 10 times more then this engineer out here who's just like, sitting on the outside saying, oh, accounting stinks and is ripe for disruption. Please come disrupt it. So what was the second point there? The second point there was choose a tool if it's the best tool to do that job. Not based on integrations. Integrations can be a factor, but don't choose a tool because of integrations. And if y' all are raising your hands in the car, put them back on the wheel and then comment with a raised hands in the comments. Because, man, have I seen so many dead bodies on that road. Mine, myself included, unfortunately. I'm sorry, I got a little worked up at that one. But if you've gone down the wrong path on that, you know exactly what I'm saying. Hey, thanks for coming and hanging this week. You got any questions, keep checking them in the comments. This is consulting. People always ask, do I do any consulting? I say, no, but put the questions out publicly. I will answer them in a way where everybody else can benefit from them, which is just fundamentally better than saying, oh, you gotta. You're gonna pay me this money to go give you this secret advice? More we can do that stuff transparently, the more it helps everybody. So keep those questions coming. They're not just for you. They're for other people to benefit from hearing them too. I'll see you next week. By.