410 Accounting AI Security Q&A!

Gang, today, kind of a different one. We are going to go through a whole bunch of questions on AI Security. What are the questions that we're too embarrassed to ask? Is it okay to put confidential client information into an AI tool? I know we're all excited to lean into this new stuff, but we wanna do it responsibly, right? So let's like get into the weeds a little more than we usually would on how do we determine this for ourselves? What's okay to use, what's. What's responsible, what's not responsible. Because there is like so much misinformation out there. Outdated information about what is secure and what is not when this stuff is known. There is nothing mysterious about this. Besides the general distrust people have for software companies where a software company will say something and then they'll just be. Person will just be like, well, I don't believe you. That mysteriousness, I can't help you with that. But this stuff is knowable. We can make decisions on it. And it means you can, you could put this stuff to work responsibly. So let's like do the work of cutting through the fluff, actually making a decision so that we got a clear path forward. Come on and let's talk about it. So put the call out on social media. I basically just said, what are your most embarrassing AI security questions? A lot of conversation lately where it's like, to the end of, I would love some thought leader to actually come out and say, here's what's okay and here's what's not okay. And I can tell you it's a little bit tricky because nobody wants to be on the hook, first of all. So like companies won't come out and say, well, like, here is the right and wrong thing. Actually mad respect to right tool, not right tool, RightWorks, who put together that AI usage policy with me, like, they stood behind it. Big company. We could put the link to that down in the show notes. They said, if you're going to use AI, here's a policy you can swipe to use in your firm. Because so few people are like willing to come out and say that, right? And part of the issue here is saying that AI is okay is a very unpopular take. Like, I have talked with so many accounting firms in the last couple of weeks that are using AI for everything from reviewing tax returns to reviewing financial statements to drafting executive summary, doing all sorts of really cool stuff, but they are afraid to talk about it publicly because they think they're gonna get canceled because this AI Boogeyman is still out there kind of saying, like, it's unthinkable to ever put sensitive client information into, quote, into an AI model. And that's like the most important thing we can dispel here probably. And if you listen to the podcast, you've probably heard me rail on this at length. But when it comes to AI models, the only thing that makes large language models different from any other tool that we use is whether or not your prompts or the things that you give it are trained into the model. Cause we don't want that, right? Like if I upload a profit and loss and I ask it to take a look at it, I don't want that profit and loss to then be trained into the model. Now it's a whole different conversation of whether that would even be retrievable, because language models don't really work that way to be able to retrieve a specific thing. And the examples we've seen in the wild of that, like ChatGPT being able to regurgitate, you know, say a big part of a New York Times article verbatim. Where that's happened has been because that article, the New York Times will publish it and the free portion of it will get distributed 2,000 other places online, reshared. And so it goes into the training data several thousand times. And so when, when the language model is like, okay, what's the most likely next word here? It's gonna make something that looks like that article because it was trained into the data so many times. If something of yours went into the model, something slipped through, it's realistic that that is actually retrievable. When, if we think about how these models work, they generate words, text, a token at a time. So that's one word or part of a word at a time. You really think one thing slipping through is gonna make it so that the most likely next part of a word will be some coherent, sensitive thing? Almost certainly not. But let's just assume that could happen. We're gonna take the conservative path here and say, no, none of our stuff should be trained into these models. And as long as that AI tool is not using your prompts to train the model, then it looks like any other cloud app that you use. Like, do you still need to do security due diligence on it? Like, yes. Should you probably have multi factor authentication? Absolutely. If I stop using the service, do I want all of that data to be deleted? Yeah. And so the only thing you gotta look out for with AI tools is, are my prompts being trained into the model for all of the leading business tiers of these services. Now, Anthropic, ChatGPT, Microsoft, Google, Writeworks has another really good comparison of all the leading models and the security rules around these models. Your prompts are not being trained into the models. And so there is a misunderstanding right now. That AI model means it's this amorphous thing that's constantly learning and gobbling things up. It is gobbling things up, but it's not gobbling up the things that you give it in most cases. Now, free versions of some of these models, they will do that. So watch out for that for sure. And that's why I send people to ChatGPT teams, specifically the team plan this. It's like very well understood exactly how the security works. Nothing is getting retained. It's not that expensive. ChatGPT is as far as AI assistants go. Like it's still just so much more performance than any of the other ones. And I want Microsoft to nail this cause that's easiest in most of our work setups if we're using Microsoft 365. But it's Microsoft stuff is so, so far off, like the other really good stuff. And they tell you well behind the scenes we're using the same models. Well then why does it stink so bad? How is it that when you use GPT4O, it's so much worse than when I use it in ChatGPT. I digress. ChatGPT right now it is like the alpha, the best experience. And so if we know that our prompts are not being trained into the model, it literally looks like every other cloud app that we use in our firm. And we use a bunch of cloud apps. We use practice management systems to store all sorts of sensitive things. Tax returns, Social Security, like all of these things, an AI app, it needs to go through the exact same due diligence process as your practice management system. Anything else, and if you put them all like this is a. Especially if you have like an IT group that's pushing back and saying like, no, this is irresponsible. You shouldn't be doing this. Put all of your apps in a grid, compare that security functionality and you will find that today's leading AI apps are more secure than than most of the accounting apps we use every day. Today's episode is sponsored in part by Kanopy, the practice management system Canape. They're one of the earliest like really solid cloud PMs that the accounting space got and they continue to somehow be like very progressive and kind of at the leading edge of cool new stuff. Unlike a lot of very early cloud tools where they just kind of end up looking long in the teeth, man. Canopy has one of the, if not the most complete feature sets in the biz. And everything that they've got in their practice management system, just super polished, super professional, something you can proudly put in front of your clients and your clients not have to be like, give you some grace because they just expect that the tools that you use will suck. Oh my gosh, how long has that been a problem in the accounting industry where they're like, yeah, it's cute. Oh wow, look at that portal. I'm just impressed my accountant has a portal. It's like, no, we should have really good tech that is professional and works as well as any other apps that person uses. That's been my experience with Canopy. Just polished from end to end. One of the most comprehensive systems out there. To learn more about Canopy, check out the link down in the show notes. Think about your QuickBooks. You know that you run payroll and it has all sorts of sensitive information. Is that all locked down by Multi Factor Authentication like most of the practice management systems we use? And so let's, let's talk about one thing to look out for in your apps. In fact to get to the questions here. Cause that is the point of this podcast. Derek Foote had asked what's like a general guide of where and what to look for for security disclosures of any kind, like as we're looking at an app, so that if we're looking into a specific platform, we have sort of a roadmap of questions we need to answer. Now I will say I think the best thing that we can do honestly is to have a trusted IT partner that can help you with this stuff. But not everybody does. And it is worth educating ourselves a bit on like what are the things to look out for. I can tell you the three biggest things that I look at. One, can I delete my account and all the data be deleted from their system? I want to be able to leave some SaaS solutions, especially early stage SaaS solutions. They don't have a way for you to kill your account, which means anything you put in there is just kind of like sitting there in perpetuity. I don't want that. I want a way to kill it in the end. Two, I want Multi Factor authentication. Most security breaches happen as a result of somebody getting access to a system, a login that they shouldn't have access to. So like Phishing attacks. Usually they're going for your login credentials so that they can go log in and get stuff out of the system directly. Multi factor authentication eliminates 95% of this type of breach. And I know it's a hassle to fuss with that stuff when you're logging into things, but just removes such a huge percentage of breaches that it's still worth doing. Third thing I look at is, do they have an AICPA SOC 2 certification? Now important here you have SOC 2 certifications from the AICPA. There is both a type 1 and a type 2 certification. As you might imagine, there are different levels of veracity. Now, oftentimes software companies will talk about their SoC2 certification and, and conflate that with a SoC2 Type 2 certification. If all they say is SoC2, it means they only have Type 1, because Type 2 and Type 3, as you would imagine, are harder to get. Third thing I'm looking for is, does the company have an AICPA SOC 2 certification, either a type 1 or a type 2 certification? One point of confusion here, it's called SOC 2, but there's both SOC 2 type 1 and SOC 2 type 2. And oftentimes a software company will say, no, we're SOC 2 certified, conflating that with type 2 certified, when that's different than type 1. And usually this stuff is shared publicly. Like you can get a copy of the certificate. The software company will talk about their certification and the importance of it. Sometimes there's actually some misleading language there when what it actually is is just a blog post talking about the value of it, but not actually saying here's our certificate, like we actually have it. And also if they don't acknowledge Whether it's type 1 or type 2, it means it's type 1. Sometimes, like the rep that you're asking at a software company, they don't actually know that. Like when you say, is it type 2? The only thing they know is that it's SOC2. And they'll be like, yeah, but SOC2 type 1 and SOC2 type 2 are two different things. So at the end of last year, I put out recommendations for nine practice management systems. I said, these are the only practice management systems that your firms should, should run on. To my knowledge, only two of those nine have achieved type 2 SOC2 certification, whereas most of the others have type 1. Some of them don't have it at all. And is that an argument for them not being on my list? Potentially. So to recap, the three things I would look at like asterisking this with. It's good to have some IT professional help here too to speak into this. Number one, can I delete all my information off of their servers without having to call somebody or whatever? Can I, like, is that a thing I can do? Number two, can I enable multi factor authentication, since that's where most breaches come from. And number two, do they have a SoC2 certification, either type 1 or type 2? And so if you look at the leading AI models right now, the leading AI chat assistance, the ChatGPT team plan, comes with SoC2 type 2 compliance, as does Claude, as does Gemini. It's hard to tell on copilot, but basically, if you're putting any when it comes to Microsoft, if you're putting sensitive information into your Microsoft 365 anywhere, the copilot you're working with, like, you can basically assume the same level of security with that copilot as, you know, putting something in OneDrive or something like that. So people are nervous to outright say, like, yeah, you should go use these AI tools because it does depend. Like, you could set the policy in your firm to say, all of our tools need to be SOC 2 type 2 certified. And if that's the case, not every AI tool will be enough there. Just like most PMs won't be enough. So there is a level of like, you kind of gotta decide for yourself to a degree. And so the thing that I come back to is the leading AI assistants right now are more secure than 80% of the tools in our space. This episode is sponsored in part by Cloud Accountant staffing. Nothing grinds my gears quite like a vendor in the accounting space that knows absolutely nothing about how accountants or accounting firms work. And there are all too many of them. All of them. There's a ton of smart people in software and all that, but you definitely know when there's a sales rep or something that's just tone deaf and doesn't understand what you do. Cloud accountant staffing. This was their experience trying to offshore, trying to hire talented folks in the Philippines. The founder was an accounting firm owner and had tremendous luck hiring talented people in the Philippines like I did and eventually just started doing it for some other firms. And it grew and they started advertising on my channels and now it is a proper big boy business. And the reason they're crushing it is cause they get what it's like to run a firm and how to do this stuff and how to even work those people into your team successfully. If that's something you're exploring. To learn more about cloud accountant staffing, check out the link down in the show notes, Richard Crane asked the question which versions are safe to put client data into with no scrubbing? It ultimately comes down to where you draw the line on what is safe. If you have a practice management system that's say SOC 2 type 1 certified, you can get that in most of the leading chat assistants you use right now. Obviously safe is like, well, what does that mean? Does that mean when I put it in there once, it's there forever until we kill the account for our practice management system? That's usually how it works. Most people are not actually doing the data retention thing where they dump stuff that's older than three years or however long you need to go out. Right now, if you are on Microsoft 365, if you use Google Workspace in both, you can get AI assistants that are plenty secure. They are just as secure as all the other places in those environments where you store documents. ChatGPT specifically the team plan. In my opinion, it comes with all the security that I need to do that stuff. Another really cool feature ChatGPT has is these ephemeral chats. There's a toggle where as you have a conversation and as soon as you leave that conversation, it's gone, it's deleted forever. Poof. And I actually wish most of our services had something like this. Cause a lot of the docs that we have, I guess accounting is kind of unique where a lot of the stuff that we generate we want to hang onto as a workpaper. But a lot of stuff can and should be ephemeral. Like if I don't have to hang on to anything longer than I have to, I'd rather hot potato it and just not have it. Like it's kind of just additional liability as long as you have it in your possession beyond like where it's actually helpful or required to do so. Right. On that one, I would probably point you to writeworks Comparison, which we've got linked in the show. Notes compares the security of ChatGPT. Claude Gemini, copilot and. And Spark, right. Works his own tool. Now, like zooming out a bit I would say. I think the only one worth using right now in my opinion is chatgpt because it is just a lot better as a general tool. There are specific jobs some other tools can do better right now, but the best all in one by a mile still for me is ChatGPT. So if it's me, I'm not even looking any further than that because security, everything has to pass the security bar. But then beyond that it has to meet like the usefulness test. Is it actually gonna save me time? Is it worth learning? ChatGPT like 100% in my opinion is good enough to be investing in today. Another good question here from John. Let's say John I. Cause I'm gonna butcher this name. Sorry. Number one, are you able to put confidential information that won't get stored in the model or do you have to anonymize it? Covered this for the leading AI business, AI apps today, information is not trained into the model. So what you're looking for is prompt training or model training. And on pretty much all the pricing pages for these tools you will see the tiers at which prompt training is not a thing. So are you able to put confidential information in that won't get stored in the model? Yes, you can do that. One other aside, OpenAI, a lot of people say like quote open source models, there are open source models. OpenAI does not put out open source models. They did in the old days. Now it's all closed source. But open source model does not mean data is being trained into the model. That's a totally different thing. I think people think of that as like, oh, the door's open. Or they hear OpenAI and they're like, oh, oh, it's gobbling everything up that we give it. It doesn't work that way. Prompts are either trained into the model or they're not an open source model. Like Llama. There's a bunch of Llama models that were put together by Meta, the company behind Facebook. They're all trained on a huge volume of data at the cost of billions of dollars. More things aren't seeping into it as people use it unless those models are being retrained. Also, the people who are telling you that they're training their own models, they just aren't. When Llama's dropping a model that they burned a half billion dollars on training every four months, I can assure you your accounting vendor isn't training their own model. Everybody's using the same models. There are quote public models, meaning you can bounce stuff off the model and get a response, but nothing is retained. That's kind of the really exciting thing with this current age of AI is back in the old days, it was the companies that had all the data and the money to train their own models that had the advantage. These days everybody's using the same models like Salesforce, humongous companies, down to itty bitty Tiny companies, everybody's using the same models because they can't match the investment and all that that goes into making a model this good. Okay, second question from John here. Does it matter if the data comes through as a Word Doc, Excel file or SharePoint link, for example? Look into the specific service. I can tell you for ChatGPT, they have a whole support article on this FAQs. How are chat messages retained versus files retained? And the short answer is it. It is retained as long as you let it exist inside of your chatgpt so you can archive it and still go back and search and find it. Or you can delete it and it deletes it. It deletes the messages and when you hit delete, it also deletes all the files so there is no different retention of files versus messages. And then number three, are the custom GPTs held to the same security measures as just normal messages? Yes, OpenAI on their help center has some good supporting articles around this. Basically, as long as the custom GPT exists, it has to have that data to still work. But if you delete it, it all goes away. Similarly, ChatGPT now has what it calls projects. And inside of projects it's actually super cool. You can upload files like the HR manual, you know, for the accounting firm, and any conversation you have inside of that project then knows all the information that you uploaded to it. So that could be, say, you know, I always reference Oregon publication 17 and a half because it has all sorts of Oregon specific stuff. You could have a project where you upload all your own copies of authoritative documentation or whatever collection of, you know, documentation from the. People always get bent out of shape when I say authoritative, as if I don't know the difference. I do, but you know what I mean. Whatever collection of literature from that tax jurisdiction you want to put in there, and then any conversation you have inside of that property project looks to those documents for answers. But if that were something secure that we put in that project, which you certainly could, it would be there until you deleted that project. Now by default, I believe ChatGPT, the very oldest messages, I think beyond some certain time span, they self destruct. But the short of it is as long as you can access it in the app, it's still there. It has to be there. But if you delete it, it's gone. It's as simple as that. And there's a number of versions of this question which I think people have like kind of rattling around in their head and they maybe just haven't gotten specific enough or thought about it enough to really pin down what it is that really needs to be decided. So the question how can we ensure client data remains completely secure when integrating AI tools into accounting workflows? The answer is the same way you do it with any other app, with the asterisk and ensure there isn't data training, ensure prompts aren't trained into the model. If you're using the tools we're talking about here today, that's never the case. They're for sure, like off the wall, like weird random websites where you can do stuff. And I'm sure they do all sorts of wild training. Don't use those with your client data. That ought to go without saying, right? But to ensure that that data is managed responsibly, Byron Patrick chimed in here. He said, follow the same process. AI might be a new car, but the rules of the road remain the same. And if you aren't sure what those rules ought to be, that's a bigger problem around having some technical expertise that can speak into what are the right and wrong tools to use. Because the right answer is not what everybody else does. It really isn't. And that's part of the problem right now is there's all this group think that says AI's irresponsible, you can't use this. That is paper thin. And everybody's just echoing what other people are saying. And some old information where beginning of ChatGPT. Yeah, every single prompt was trained into the model back in the day, but that hasn't been the case now for quite a while. And you have, I mean, HIPAA protected information being used. You have government agencies using ChatGPT yet we've got like accountants sitting on the sideline being like, well that's unthinkable that you would ever put sensitive client information in there. And people said the same thing about the cloud. Unthinkable to think you would ever put sensitive client docs into, you know, a server that wasn't in your office. I get all that. It takes some time for that like ship to swing around and kind of get our heads around the direction that we're headed in. But I'm so eager and willing to have these conversations because the more we have these conversations and get down to first principles, what is secure and what is not, the more that aids AI adoption. And there's just an abundance of people out there that don't have any expertise in this or like actually know how this works that kinda just keep echoing the same things over and over. Again, when we are literally, like, living through the biggest technological revolution ever and we're being blocked by, like, the AI spookies, something that's not real. So, like, man, let's do the work. If you got stuff you still need to work through or answers you need to get to, like, do the work of getting to those answers so that we can unblock our team and start leaning into this stuff. Cause, man, like, it is gonna be a wild year. Like, how good the new models are. A lot of really just really cool stuff happening. If there's any other questions I can answer, please pop them down in the comments. The more embarrassing the question, the better, because somebody else probably has it. I'm happy to keep having this conversation if it means one firm flips on ChatGPT and they're like, here are the boundaries. Let's do it. I think it's worth having these conversations for that reason, gang. Thanks for being here. Thanks for hanging. This is a very polarizing thing. I don't think it needs to be. It can come down to real, real answers. Thanks for being here. I'll see you in the next one.