Ep 541: AI & Trust: When 98% accuracy won't cut it and how Sage can fix it

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life.

What if good isn't good enough? Right? I think it's something that business leaders are constantly thinking about when it comes to AI. The they're like, hey, if we get this right, most of the time, let's go ahead and roll this out to the entire organization. And sometimes that might be okay, right? If you're doing strategy, creative work, content, production, but what about your books? What about your finances? Sometimes being 90 or 95% correct could be bad. It could be a recipe for disaster when it comes to your business's AI plan in 2025 and beyond. That's why today I'm excited to talk a little bit about trust in AI and how. Well, if you're watching our video, our livestream, you see, I am at the Sage Future Conference here in Atlanta and how Sage is really helping increase everyone's ability to trust their AI. All right, I'm excited for this conversation. I hope you are, too. What's going on, y'?

All?

My name is Jordan Wilson. I'm the host of Everyday AI. This is your daily livestream podcast and free daily newsletter helping everyday business leaders like you and me not just learn what's happening in the world of AI, but how we can actually leverage it to grow our companies and our careers. So make sure, if you haven't already, go to our website at your everydayai. Com. We're going to be recapping today's conversation and a whole lot more, but like I said, you can probably see a little different setup here. I am in Atlanta at the Sage Future Conference, and I'm excited to welcome our guest for today, Aaron Harris, the CTO of Sage. Aaron, thank you so much for joining the Everyday AI Show.

Thanks for having me. Really excited to be here.

Yeah. On the road in your hometown, actually. But for, you know, for those of our audience that maybe don't know Sage, tell us what Sage is.

Sure. Yeah. So Sage is a global software company that focuses on accounting, hr, payroll, manufacturing, sort of all of the things that a finance and accounting team needs to run the back office of the business. So we're actually a British company. We're headquartered in Newcastle, England, but we've got offices all over the world. And our headquarters here in North America is obviously here in Atlanta. It's a company that's been around for a while. We've. We've been building accounting software for more than 40 years now. The company that I co founded that sort of got acquired in, we started more than 25 years ago. So we've been doing this for a while. We're not as well known in the US because the US is a huge market and there's, there's, there's sort of more players in this market. But if you go to the UK or Spain or France or Germany, like some of those countries, we're a bit of a household name within the accounting industry.

Yeah. And let's just kind of skip to the end. When it comes to the intersection of AI and accounting, why is, you know, sometimes being 90 or 95% accurate? Why does that not work?

Yeah. So, I mean, there's, there's so many ways to address that question. I think the first way to start is that, you know, look at the CFO in a company and the finance team that, that CFO trades on trust. Right. Their, their job is to create confidence not only within the business but with stakeholders, whether it's investors, creditors. Right. That you can rely on the accuracy of the financial reports that, that they're issuing. Internal stakeholders can rely on the forecasts and the budgets. Right. That are being provided. And the minute that CFO put something out that, you know, has a mistake in it, they're going to lose that credibility, they lose that trust. And so the bar is incredibly high. One of the things that's kind of interesting in the mindset of a CFO and a finance team, if, if I'm a penny off in, you know, the basic equations of accounting, they will hunt that penny down, right, for days until they find it. They may not ever give up until they find that penny. And so 99%. Yeah. That's not going to cut it. Right. It's got to be perfect.

Yeah. And not only that. Right. And I'm sure many of our audience can relate to this large language models by themselves. Right. So if you're using something like ChatGPT. Not always the best at math, right?

Yeah, no, not at all. Not at all. Yeah. In fact, you know, large language models are trained. I mean, this is what makes them so amazing. They're trained to be creative. We don't want creative accounting and we certainly don't want them doing math. Right. So, you know, in the way that we build AI, this is something that I often tell audiences is one of the first rules of building trusted AI is not to use AI when traditional development will work better. So if we want AI to do math, we're going to give AI, a calculator to do that math with.

And, you know, I want to get more into this, this piece of trust. But, you know, I saw your keynote, and when you were showing some of the things on screen, I'm like, wait, I need that. Especially when it came to Sage Copilot. So can you explain a little bit for maybe our audience that doesn't Sage and your AI offerings. What the heck? Like, how can you do that with such high accuracy and, you know, help people kind of close the books, you know, much faster? You know, I think you said originally it was like two to three weeks, and now down to like two to three days.

Yeah, yeah. We're still not. We want to get rid of it. Right, right. It's a relic. Right. It's archaic. We want to get rid of this thing. So we, we think about this in, in waves, right? In the way that we build and deploy AI and, and the first wave we call tas, this is AI that you don't necessarily see at work. So the first thing that we built in the world of accounts payable automation was AI to read and categorize an invoice. And my first conversations with a data science team was, okay, there's a bunch of models from big players that are built just to do this. Why are we not using one of those? And what they had to convince me of was that those models just weren't good enough. Right. They were about 80%, 75, 80% accurate. But in addition to that, you know, that wasn't an even. That wasn't evenly. Right. Deployed accuracy. They really struggled to find the total on an invoice. Right. They might be 30 or 40% accurate on that. Turns out it's kind of hard to find. And so ultimately we had to go build our own models. Models, plural. Right. We've got five models. Just, you know, some of them are looking for the total. Some of them are checking the work of the other models to make sure that, yeah, you really did find. Find the total. So, you know, it ended up being dozens of models. Now, you know, the, the, the limitation of this approach is that you can't really interact with that AI. And that AI has to be very, very carefully orchestrated, scripted. You know, it does what it's told to do exactly the way it's told to do it. We can't really get to the next level of automation until you can interact with AI. And so that's the big breakthrough with large language models that sit behind Sage co. Now you can be directive of the way AI works and sort of how it gets its job done. And so that's, you know, it's a, is a huge breakthrough and it's really a change of psychology, if you will, in the way you design the product. When we are in that task based phase, you know, a lot of customers don't even realize how much AI is actually operating behind the scenes. Right. They don't realize that until they're confronted with a conversational interface. And so that like totally changes the way you design. You've got to design for confidence now.

Yeah. And how did we get to the point where you have a product like Sage Copilot that can accurately, you know, take advantage of, you know, the powers of generative AI, yet work in a more almost deterministic. Right. Like, like way. But you know, I saw that, you know, there's been billions of predictions, millions of documents that you've to, to get here. So without, you know, going into. Because I'm sure we could talk about this part for hours. But how did we get to the point where, yes, you can feel confident as one of the global leaders in the space to say, yeah, you can go use AI for some of your most important financial tasks.

Yeah. So I think there's, there's two parts to that question or two answers to that question. But, but the, you know, the first I guess I will get back to is how do you design the product? Right. You sort of have to like, you have to be credible and believable with your customers. And so if, if I stood in front of, you know, our customers today and said, you know, don't worry, it's 100% accurate, like, you know, we've got it, they, they wouldn't believe at all. Right? So you've got to adapt the experience, you've got to design around this, this understanding that hey, we're going to get that large language model to be incredibly accurate. But what we're going to do is we're going to over index on understanding, okay, have we met that level? And if not, how do we engage a human properly to review the work? And so that is such a huge, huge part of it. But the other part of it is that again, the off the shelf models, as good as they are, as magical as they are, as powerful as they are, they're not quite good enough for what we needed. We need a large language model that knows in depth, we, you know, with expertise how our products work. It needs to be an expert at our APIs. Right. So in, in the process of completing a task it's probably going to write some code on the fly, right, to use an API. And so like it's not going to work if it hallucinates in the process of, of you know, building that API request. And so all we found was these off the shelf models are pretty brilliant. But you know, two problems. First, as brilliant as they are, they still make mistakes. But second, you know, they're incredibly expensive, right. So if we wanted to go about, you know, not just operating these models, but sort of get into the world of building one of these gigantic large leg language models, I'd never get the budget to do that, right? What? A hundred million dollars?

A lot of money to train those. Yeah. Trillion parameters costs a lot.

Yeah, yeah. And so, you know, fast forward two years and the, the, you know, the efficiency and the capability of fine tuning these models has increased rapidly. Right. So costs have come down, efficiencies is increased, but also the tools available to companies like us have gotten better and better and better. And so we took GPT, as you mentioned, trillions of parameters, GPT4, we think 2 trillion probably. We started with a model that's 7 billion parameters and we fine tuned it from there. And when you're fine tuning like, you can sort of like slough off the stuff that you don't want it to do, right? We, we, we train our model to like not accept toxic prompts, right. We, we train it to be pleasant in the way that it interacts. And then, you know, if the conversation's not about accounting, then we don't want to have that conversation. So we can get it down to a 7 billion parameter model and we can fine tune it to be really, really, really good at these accounting tasks we want it to do.

You know, it's interesting because I'm sure there's a lot of people in our audience, specifically you know, CFOs, people who work in finance, that maybe their first or one of their first interactions with, you know, AI, they were probably saw a result and they're like, I'm never going to touch it again.

Right.

Because some of the earlier. Right. Even if you say something like GPT4 or some of these, you know, earlier trillion parameter models, they couldn't do basic math, right. So I think even a lot of people that I talk to, they kind of wrote off, you know, hey, we're not going to use AI in this department anymore. But it sounds like the one, the models that you have powering your sage copilot, I mean it sounds like it has like a, like a PhD in CPA.

Right.

Like talk a little bit more about how you were able to address that trust issue by essentially going through and training this 7 billion parameter model to become an expert.

Yeah, and I want to talk about that first experience too. But, but so what we've done is we've taken that base model and then we've, we've trained in all the product documentation and sort of loads of material around that product documentation about practices and how products work, you know, all the developer code. We've trained it on accounting textbooks and accounting exams. We've trained it on content that helps it to sort of understand and speak in the vernacular of, you know, accountants and financial analysts. And one of the things that's super exciting that, that we announced today is we're partnering with the AICPA, which is the industry association that accredits CPAs. They're now going to make their, you know, professional content available to us to train into the model. Now it's a proof of concept. You know, we're going to be conservative as we are when it comes to AI. So I can't sit here and say like, this is exactly what we're predicting. But I think it's an incredible signal that, you know, the accounting industry, which, you know, early on, like, you know, the headlines were saying accountants aren't going to exist anymore. Right. I think it's incredibly interesting that the accounting industry is, is not just sort of embracing AI, they're contributing to the development of, of AI models. But I, I want to kind of come back to that first experience because it's so critical. You're absolutely right. If a CFO uses our, our co pilot and their first, their first experiences, it makes a big mistake. They'll never like, we won't get another chance. I grew up in, in Silicon Valley, probably like a lot of people on your, in your audience and you know, the mantra in Silicon Valley has always been move fast and break things. When you're building this kind of AI in this industry, we've got to have a completely different culture. So I talk about, it's kind of pithy but like accept humility, embrace responsibility. We have to have a different mindset. We can't rush AI to our customers. If they have that bad experience, they won't come back.

So you've talked a little bit on how you were able to increase accuracy, right, by creating and fine tuning your own model, trained on everything that anyone in finance, cpa, et cetera, really cares about. But what about on the back end? What about, you Know, observability, like traceability, you know, how does, you know, Sage Copilot and some of the things that you announced today, specifically kind of this trust label, how does that address it?

Yeah, so one of the things that we had to do very early is, you know, we had to build our own infrastructure for machine learning. So I kind of like to compare this to the early days of software as a service. So if you go back, you know, to the very earliest days, you know, all of us, you know, Salesforce included and there are other pioneers that are kind of still around, we had to set out an objective for our developers that we would release a new version of the code on a weekly basis. We would upgrade every customer to that, to that next version of the code automatically. So everybody would always be on the same version of the code. And we had to do this without disruption. This sounds normal now, like 25 years ago. That was not normal. Like, that was an incredibly provocative thing to say. So if you fast forward, when we started building AI, I had to give the engineers a different mission that was even more provocative, I believe. It's like, I don't want a weekly release. I want you to automate the training of this AI and detect when it's improved enough and then automatically update the version. But it gets worse, Mr. Developer, like, you need to be able to do this on a customer by customer basis, right? So we're going to have some big models that are trained, you know, from, from the collective. But a lot of what we do, we need to train from the individual customer on a customer by customer by customer basis. So you've got to this infrastructure that can automate all that, but do it in a safe way. So we built all the automation. This is why we've got tens of thousands of models in production today. But what we also did was we built in all of these safety mechanisms, all of these controls. So we've got systems that detect model drift and launch a process to get a data scientist involved. We've got a couple of safety mechanisms that detect hallucinations. This is where we've actually gotten some of our patents on this. So we call this whole thing the Sage AI Factory. And you know, if you see me talk to our customers or analysts, right, partners about AI, I'm invariably going to talk about the Sage AI factory because I think it's so important to understand behind the scenes, like how does, how does the factory work? Right. How does this stuff get built and how do I know that you're taking steps to make sure that it's safe.

So I'm curious how many total organizations you have using kind of the AI Copilot features inside Sage right now.

So we have tens of thousands using Copilot in various places around the world. We started small. We started with sort of small businesses that have simple accounting needs. We started with small accounting firms that tend to serve those businesses. And we started on sort of the early capabilities, and over time we've expanded it to more products, you know, more countries. But we've also sort of gone into now more sophisticated businesses. So we launched Early Access for Sage Intact at about, about six, six or eight months ago. And I guess the thing that I would want to reinforce here is we're being very deliberate about how we open that to more. To more customers. Right. We're just, we're so. We're so careful to not have that first bad experience. And so we're gonna be very deliberate in the way we roll this out to more customers over time.

Yeah, it seems like a super strategic rollout and obviously makes sense when trust is paramount. Right. And you can't, like you said, you can't have someone have that first bad experience where something goes wrong because the stakes are so high, you know, so I'm curious throughout the iterations of Sage Copilot, and then, you know, obviously another wave coming with what was announced here at Sage, maybe some of the initial, maybe obstacles that you were able to overcome, and then maybe what do you think is. Is next? Right, in terms of not like, hey, what's, what's the exact product roadmap, which I know you guys did lay that out a little bit. But in terms of trust, in terms of reliability, what have you guys already been able to overcome? And then what's next to overcome?

Yeah, yeah. So. So if you'll forgive me, I'm going to tell a story.

Please do.

And, and I promise it's setting, setting up for the answer. So, so I've been talking about, you know, Sage Intact, and I was one of the co founders there. So 25 years ago, we just started the product. We just launched the product. Our biggest objective or the biggest obstacle to, to getting customers to buy the product is that they were not willing to put their data in the cloud. And you know, we sort of scratch our heads at that now, but sort of, you have to imagine how new this was at the time. And we would, we would use all the arguments you'd expect. We can spend more on security than you can. Our livelihood depends on keeping your data safe. You Trust a bank with your money, why not trust us with your data? It just wasn't working. And so what we ultimately ended up doing, and I think it's pretty clever, we put a button in the product that said, see my data? And when a user clicked that button, we would pop up a window that had a webcam in the data center, pointing at the server that had their data on it. Now, we only had one server at that time with customer data. So, you know, this was pretty easy to do then. This was actually pretty awesome. It actually kind of worked. Thankfully, we reached a point where it was less needed when we turned it off. Now, the reason why we turned it off, and I promise, again, true story, we had a technician in the data center who was doing some cabling and doing some maintenance, sort of bending over, moving around a bit, probably should have been wearing a better belt. And, you know, sales rep chose at that point to click in front of a prospect, right? You know, show my data, right? And they got the full transparency, the full transparency.

They saw the data and then some, right?

And so, like, this is kind of like an old story about building trust. And, you know, how, you know, how transparency plays into that today. It's still true. If you talk to companies today and they're evaluating your software and they know that AI is a big part of the offering, they want to know that you're going to keep your data safe, right? They're going to want to know that they can trust the technology. And the problem with AI is the industry and the technology is moving so fast that sort of the regulatory environment around it hasn't kept up. So we can't rely on a lot of sort of external signals that you can trust the AI. We do use some, and I'll get into that. So what we determined was, well, we need to put a button in the product. We're going to put it in each AI feature where a user can click it. And what's going to happen is instead of that webcam, we pop up what we call the Sage Trust label. And in that trust label, this is kind of like a nutrition label, we're going to be super transparent about. Okay, what are the models that we use to build this? Are we training our own models? If so, how do we use your data and the training of those models? You know, what are the steps we're taking to keep your data safe? What are the safeguards in place to defend against issues of bias or other ethical concerns? And we're putting it in a nice, easy to read format. And if they want to learn more, we give them a button where they can click and go out to read all of our AI commitments. Kind of solving the same problem, really. So, yeah, we announced that today. We're encouraging other vendors in the industry to kind of follow suit. We can't wait for the regulatory bodies to, to catch up. So we've got to plow ahead with something we think simplifies this and sort of signals to the customer. Here's why you can trust us.

What do you think is going to be? Or maybe this is it, right? Because I think every single, you know, company that, that's trying to, you know, put out responsible, trustworthy AI products, there's always that, that, that hurdle, right, is with a trust label, is this something that you think is going to be not like the last hurdle, because there's always innovation, there's always, you know, new capabilities. But is this going to be one of those, you know, those hurdles that after you get over it, you're like, wow, this made quite a difference for people in finance to be able to trust AI in their data.

I think it's going to help a lot. I mean, I, I think that, you know, the first thing that it's going to do is it's going to get the CTO off the phone. Like, you know, I. So, you know, when you get a customer who's evaluating your software, and we've, we have millions of customers, by the way, right? We've got 4,000 people in sales. If we get a sophisticated customer that's got a question about AI, they're going to ask me to get on the phone and talk to that customer and explain it. If we've got this trust label, it just makes it easy for people to understand and evaluate. So I think that's critical. I don't think it's going to go away. I mean, I think it's going to evolve and change. Is this the moment where we solve the issue for good? Is there going to be a point where I can turn off that button the way I turned off the webcam? I don't know, but I guess everything is happening so fast with AI, it's progressing so fast. And I think we all need to be a bit honest, right, that there's going to continue to be lots of reasons to not trust AI, right? It's just broader than just the accounting field. So I think we've got to have this mindset that this is one step in the journey and we're going to have to continue evaluating and Looking at, you know, okay, how are people feeling about trust in AI and what are the new things that are causing them to not trust it? And we're going to have to just keep adapting as we go.

So, Aaron, we've. We've talked about a lot in today's conversation. So, you know, everything from trust and transparency to even how Sage is not just, you know, building their own models, but how they're showing their work to customers on how it's being used and how it's being implemented. But, you know, as we wrap up, what do you think is the one most important takeaway for those people? Whether they're, you know, a cpa, whether they're a CFO at a huge organization? What's the one biggest takeaway that you want people to know from, you know, Sage Future here when it comes to trust in AI?

So I think, you know, the one thing is the future is the future that we build, and sort of taking on the responsibility to build that future is pretty serious. And so that's why I'm having this big conversation with you. What we've learned through all of our conversations is the biggest signal of trust is the company behind the AI. Right. I. I, like, I'm not a sophisticated person. Maybe I don't know how to evaluate this, but, hey, that's a brand that I trust. And it's got to change your mindset, right? You've got to be transparent. You've got to be credible. You've got to be willing to admit that, hey, AI is not foolproof. It's. It's. It's gonna make pro, it's gonna have problems. And then you've got to make these commitments that you publish, and you've got to stand behind them.

I think that was such an insightful look into not just what's happening here at Sage and their copilot and everything happening at Sage Future, but also just the industry as a whole. I think it's important for people to hear, yes, authenticity and transparency are just as important as productivity gains and everything else that you get from AI. So, Aaron, thank you so much for taking time out of the very busy Sage Future conference to join us. We really appreciate it.

Oh, thanks for giving me a chance to talk about my favorite topic.

I love it. I love it. So there was a lot happening in this conversation. Maybe you missed a golden nugget that Aaron just dropped on us. Don't worry. We're going to be recapping it all in our newsletter as well as everything else that was announced. Here at Sage Future. And it's not just if you're a CFO or a cpa, even if you are a small, medium sized business owner. A lot to know. That was just announced. It's all going to be in our newsletter. So thank you for tuning in. Please join us back tomorrow in Every Day for more Everyday AI. Thanks y'.

All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everyday AI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.