438 The Race to AGI And It's Impact on Accounting Firms

Hey, we got a fun one today. Recently I had the pleasure of speaking in Melbourne, Australia at the AI for Accounting Summit. And I'm just going to show you, show you the whole thing right now. The keynote. Lucky day for you. This was a couple months ago. For anything that is like referencing dates, basically we ran through like what a crazy year 2025 has been for AI already and what accountants should be looking out for. If you've never seen me speak in person, this might be a fun one for you. Very stammery. I pace around a lot to make people laugh. It occasionally works. It was a great conference. And actually down in the show notes, I will link you a way to get some of the other talks because there are a bunch of smart people presenting here. But without further ado, here's my talk from that conference that was put together by ator. We've had a unbelievable past 60 days or so in AI announcements and it makes me so happy that so many of you are already AI users. This is probably the first room I've spoken to where over 50% of people were like, yeah, this, like, I use this all the time. Like a disappointing percentage of the time. It's the opposite. It's like, what's a chat GPT like? It's just very entry level. Folks haven't played with it yet. Um, but we're gonna run through just the wild stuff that's happened in the last 60 days because I think it dictates what 2025 probably looks like and over the arc of today, this is like, we're starting big, we're starting wide. But what I love about the programming structure is throughout the day we're getting more hands on, more practical, more like, what do I take back to my team? So without further ado, this is the big problem to me about AI and AI implementation in your firm is how do I invest in something that right now feels speculative? How many people in the room are running, you know, twice as profitable accounting firms thanks to AI right now, or even 50%, like nobody is. So in many ways it still feels speculative, right? Any other tool. If I'm going to pick up Carbon, you know, I know other people that use carbon and that workflow tool, whatever it is, like it's supposed to improve this or that, you know, 20%, something like that. This is just much more abstract. Another problem, it keeps changing. My goodness sakes, like the rate at which this stuff changes and you try something one day and it doesn't work, then the next week you have a Different model with a different prompting strategy, like the rate at which it changes, you know, for. For more of the AI nerds, which I definitely am, and I suspect there's a lot of those in this room. Like, it can be exciting for us, but you take that back to your team and they're like, man, chill out. Like, this is too much for me. And unlike most of the apps that we use, it doesn't come with a manual. Like, you're setting up carbon, let's say you're rolling that out in your firm. You got an implementation expert. They tell you what all the buttons do, and every time you hit the button, the button does the same thing. This AI does not work that way. And so it requires almost a different part of your brain to get to grips with it. So it's just an inherently difficult thing to roll out. A bit about myself. Most people know me from. From YouTube, from the. The vaguely helpful things that I share on LinkedIn. But my background is in running accounting firms, so I had a team of about 40 in the states a couple years ago. I am American, right? Didn't call that out. You probably couldn't tell, but I sold that firm a couple years ago to just focus on thought leadership for small accounting firms because I didn't see anybody doing it in a way that, I don't know. That got me excited. That felt like it was of the times, and they understood tech and were, like, maybe not super thrilled with what most accounting firms looked like at that point in time. So these days, I do talks like this. I create stuff online. I run a. A online community for about 500 firms called Realize where it's basically a space to help folks make connections with other firm owners. Because for me, that was the single best thing I ever did, was make some friends that run accounting firms. And then last, Jason on Firms is my podcast. Any podcast listeners here? I asked not to see who listens, but who doesn't listen, so I. I saw quite a few of you there. Jason on Firms, we talk pretty much every day about running small accounting firms, and about 1500 people tune into that one every day. All right, what we're going to talk about today. First off, let's go to the moon. The global race for artificial super intelligence. Why 2025 is probably going to look different to 2024, and then how you navigate all that when everything keeps changing. Okay, let's go to the moon. This is going to be a news roundup of a whole lot of stuff that happened in the last 60 days. First 60 days ago. This is from a live stream from OpenAI. I'm going to assume a lot of you folks know some of these characters and the names based on that survey response, but they were doing a live Stream. This was December 20th and it was, as most of these things are, it's just kind of a real awkward little, little live stream with a few nerdy guys talking about AI things. And they announced that their unreleased model O3 had basically beaten this benchmark that was supposed to be representative of what is, you know, AGI or artificial general intelligence. So artificial general general intelligence as most people define it, that is a level of intelligence where it can perform things more or less at a human level. So you give it kind of an open ended task and it will go and complete that task more or less as you would expect a human to do. And that's obviously that's a really big threshold, like the notion of humanity making a thing that is as smart as humans. Like that's a really big kind of tipping point. And so I've got this benchmark and we're gonna be talking a lot about benchmarks because AI people love benchmarks. This benchmark called ARC AGI, where essentially it's, it's a series of questions and when you pass a certain threshold there's this notion that like if it can score better than a certain number, then it's, it's AGI. And so they, they, they're talking about this benchmark their new model scored. I think it was like an 88. Yeah, 88 on the new benchmark. And they said humans are, are, are usually going to score an 80. And they're like, okay, that's it, see you later. And, and the, the stream just ended and it was like, that's it. Like we just made like potentially the most powerful thing that humanity's ever made. And it's like just kind of an awkward back and forth and then the live stream just ends. They've kind of hemmed and hawed to say like, ah, this probably isn't really AGI. We need to go back and make a better benchmark. And so I don't think they're actually coming out and saying that, but that was definitely the implication. The notion that we've made an AI system that is as smart as humans, like that is a huge, huge breakthrough. That changes, not just accounting, but that changes everything very quickly after that. This is 44 days ago Sam Altman posts on his blog. He says basically, AGI as we understand it today, we think we know how to build that. And we're turning our focus now to superintelligence. Superintelligence being. I've got a definition here because it was a little on the nose. Superintelligence is a hypothetical agent that possesses intelligence surpassing that of the brightest and most gifted human human minds. So superintelligence is like, it's better than you at everything. So that is, that is the focus now. And they think this is where we're headed now. Obviously a bit of a conflict there. This is what they do. Can you believe everything that they're saying? Are they just trying to hype this up? But you can see how this is kind of a significant tipping point that goes just beyond tech and Silicon Valley stuff. But it raises to the level of, yikes, who's going to get there first. Because there's this kind of, this notion that some people subscribe to that says once you get there, there's maybe this potential like takeoff scenario where you have this super powerful thing and now you have this advantage that no one else can catch up to because this super intelligent thing can unlock all of these things for you. And so very obviously the governments are getting involved. And that's been a really big trend the last couple months is more like a state level involvement in AI development. So 29 days ago we had a new US president inaugurated. And then the same day we got deep seq r1 from a Chinese company, which I'm sure many of you heard about. More benchmarks. Here's six benchmarks, just a bit as Deep seeks the one on the left. And it's pretty good. It's better than at the time. OpenAI's best model in most, in most ways, which is a little mind boggling, was a huge surprise. There were some other things in there, like the efficiency of how it was trained. And it created some issues for Nvidia that you may have heard about. But the wild thing is it's open source. They gave it away. They said, you know, OpenAI, who's made this language model that is arguably the crowning achievement of technology. And like what humanity has developed, like you could make an argument, it's, it's a lot of different things, but OpenAI now valued at over $300 billion their creme de la creme. Very best model. Somebody comes out and says, here it is, you can just have it, you can run it yourself if you want. That is wild. So it scared a lot of people, as you would imagine. So the very next day, the US says, well, we're not gonna be beaten because this is, it's just a very American thing if you're not the alpha dog. So they come out and they say, we're gonna commit $500 billion to AI which that's just a, that's a number that's big enough to where it's hard to even like get perspective of what that is. For reference, the annual US defense budget is $800 billion and they're talking about pumping 500 billion into AI. The space race. The space race is like, that's the thing they teach you when you're growing up about how the Americans got to the moon first. And we're all very, very proud of it. We won the space race in today's dollars. We spent about $150 billion over 12 years in today's money on the space race. So this is an order of magnitude more than that. Now the real story here, it's not the US putting up the money, it's private money that's being put up. The US is saying, oh, we're gonna, you know, help them get through regulations and make sure that the data centers have enough power and all these things. So a lot of this is kind of optics, but that's still a really big number. So like the, the excitement around this has gotten to the level where states are starting to get involved, countries are getting involved, saying we see the power of this and we want to get there first. 26 days ago, only a couple days after that, OpenAI launches operator. This is their, their first agent basically. So there's a built in browser. You tell it to do something for you, go make me a dinner reservation at whatever place it will then go navigate the web with a built in browser, navigates the web and does your bidding. And this is kind of, kind of the holy grail of AI applications, right? You have this, this agent that is going out and doing work the same way that a human user would. Now right now it's really expensive, it's really slow and it really sucks and it's not very good. But you can see how as soon as this thing is like kind of sort of good enough, I mean, that massively changes how you work. That means work for you probably looks more like kind of orchestrating these different things that are handling tasks for you behind the scenes. If you play with ChatGPT much, you probably saw a couple months ago they released a recurring tasks, the notion that you can tell it to do a thing. So let's say I wanted to pull the bitcoin price from the web at 8am every Monday morning. I could tell it to go do that and it will run off and run that every single week. But you can see how there's a future where when operator type agents get more powerful that recurring task, it could be great. Go out and do this workflow thing for me every, you know, Wednesday at noon or something like that. Now a lot of technical hurdles to still get over there from it being faster, just kind of more coherent in general issues like managing authentication. I mean, we log into a whole bunch of apps. How do you get the agent to manage that? So lots of things to handle there still. But a very exciting indication of, of where we're likely headed. Humanity's Last Exam four days later this is a paper that came out that said the benchmarks we have right now aren't hard enough. All the language models, they are doing too well on the benchmarks that we have. So we're making a super, super hard one. So hard that the best model, DeepSeek R1, can only score a 9.4% on it. And this Humanity's Last Exam, which, that's a very ominous title, basically what they did is they went around to a whole bunch of experts in various domains and said, help us build a question set that's representative of like the boundaries of what people can do within that domain. So like the hardest questions you can come up with, the thinking being. This is an article from the New York Times. When AI passes this test, look out the thinking being, if AI can pass this, then it's like, well, it's every bit as good as the leading experts in each of these domains. Humanity's Last Exam. What a name. Today's episode is sponsored in part by the folks at Cloud Accountant Staffing who help you find talented accountants in the Philippines. We get a tremendous amount of inbound from anyone and everyone that's got like offshore staff that wants to come and work for us. Right? So how do you cut through the noise and know who's legit? Well, for one, I'm gonna let two offshoring groups sponsor my content. And I now know tens tons of firms that have used each of them to great success. Anytime I hear about them using them, I always ask, how did it go? How was the experience? Have heard nothing but positives. That's why we've, we've stuck with them. So how's that for a filter? Because I have talked with a lot of them that ought to put them ahead of most. But the other thing to look out for in any of your vendors is are these folks turning up at accounting conferences and having real conversations with real accountants to understand what we need? I take that accountant staffing like they are making the rounds this year. If you are out on the circuit at all like I am, you're going to see them. And that is a great signal that they care that they're not some like random offshoring group that are just throwing work at people with all sorts of various skill sets. Like they are very focused on the accounting firm and exactly what your needs are. They're actually run by folks that used to run an accounting firm. Fancy that. Let's call it accountant staffing. To learn more about them, check out the link down in the show notes. This episode is sponsored in part by Makers Hub gang. It has never been more important than it is today to ensure that you have hitched your accounting firm wagon to the tech partners that are going to take your hand and lead you into the AI era. I don't care if you think AI is going to going to be the end of everything or the start of something better. AI is already here, it's already happening. And Makers Hub is at like the bleeding edge of what AI can enable for the accounts payable process. I know, wow, sexy. Listen, just be happy that they're doing it so you don't have to. These and AI vision models and LLMs to push, you know, automated extraction from bills and kind of the subsequent logic further than we've ever been able to do before with just pure OCR technology. And honestly the bigger thing is when you have somebody that knows their stuff like that. And Makers Hub has come around really just in the last couple of years. So really in the age of generative AI, I have a high degree of confidence that whatever is around the corner when it comes to AI, they are in the driver's seat to lean into that stuff in a way that oftentimes the incumbents are not Makers up. You're also going to see them out at a whole bunch of accounting conferences this year. So they are having conversations with accountants. They understand the problem of the firm, how you run a scalable bill pay service. So if that's something that your firm does or if you got clients with gnarly bill pay issues, Makers Hub does do it. This is the platform of choice. Learn more about it down in the show notes. Remember O3, that was the model we talked about first, where they said, oh, this might have kind of sort of been artificial general intelligence. 18 days ago they released O3 mini. So not the whole thing, but a mini version. If you played with it in ChatGPT, it's really good at some things. It is better than the previous best model 01 and roughly three times as efficient. So when you're using ChatGPT, you will run into usage limits of how many times they'll let you use a model. This is better than the previous best model and three times as efficient. The full OH1 model, the last one, it came out 57 days before this. 57 days and we got something that's better and three times as efficient. Like what have you done in the last 57 days? That is a very quick turnaround. There's so many things that we also are not going to cover here. Like the last 60 days have just been wild. 16 days ago is a really big one for us. OpenAI launches deep research. This is part of their $200 a month pro plan. Basically you cut it loose, it goes and researches a problem for 10ish minutes at a time and we'll come back to you with a report of up to 10 pages in length. And to do this, it will pull sources from the web, it can use documentation that you give it, but it is doing very technical, high level things and you have like PhD level people saying, you know, maybe for the first time this is pretty darn good. Like this is actually producing something that's representative of what we would ask our peers to produce. Now, remember this benchmark Humanities last exam, Deep seq scored a 9.4%. OpenAI's deep research comes in at a 26.6% and that oh3 mini model scores a 13% as well. So in the matter of a couple of weeks, we've gone from 9% to 26%. Now another big one. Any Perplexity users here? Yeah, so Perplexity's been kind of probably like the best AI search engine, like a Google alternative sort of tool. This is four days ago. Making AI slide decks is the worst. Like people will. Some conferences will tell you you gotta submit a slide deck a month before the conference starts. And you're like, I can, but I'm not gonna use it. This was just four days ago. So Perplexity releases their own version of, of Deep Research and it's really good. So coming back to this, this benchmark, it slots in right here with the 21.6%. But the catch is instead of taking 10 minutes to run, it only takes three minutes to run a query and it's free. So five queries a day are free. So We've gone from, you know, the O1 models and Deepseek R1 only being able to do 9% of this, this, you know, benchmark to now you have something that is free for anybody to use that scores a 21. I can tell you like not to overstate this. In the US we have, we have an income tax system that's just a total mess. Requires a bunch of research just to get a tax return out the door. I imagine you have, you have some version of this as well. Perplexity's Deep Research is the, the best most powerful research tool we have ever had. And it's free. Like this is changing so much of how people work already. Like it is wild how good it is. Like it is, it is ready to use today specifically on Perplexity. One thing to look out for all things Perplexity. Get trained, get all data gets retained to be trained into the model if they want. So watch out for any client specific stuff. Usually when it comes to research, that's pretty easy to not have anything in the prompt that is identifying but something to look out for. All right, things that we didn't get to after Deepseak R1, there were several other Chinese models of similar performance levels that came out in like the following week or two, but they didn't get the headlines because Deepsea got to it first. O3 mini. It scores better than all but five US competitive coders because competitive coding is a thing. Evidently it smoked a bunch of high schoolers at, at informatics. What do you call high school? That's like grade nine, nine through 12. Okay, you call it high school. So do we. All right, basically they have these, these Olympiads. It's five events around the world and all these different domains and the sharpest high schoolers from around the world come together and they kind of take this big sweaty exam and OpenAI shows up and they're like, hey, do you mind if we have our computer do this too? And the previous Model 01, it finished kind of middle of the pack. The new model OH three, it smokes almost everybody. Which just imagine being one of those high schoolers like you've worked your whole life. You are off the charts, unbelievably good at this stuff. You go to this amazing, amazing event and somebody shows up with like a calculator and they're like, yeah, sorry, we've got this thing. It's pretty good. I don't know if you've been following the news headlines in the next couple hours. Elon Musk is Launching their next model, which is supposedly trained on the largest GPU cluster in the world. There's just a lot going on Google. We didn't even talk about Google. Their latest generation of Gemini models. Really, really good on some kind of head to head large language model leaderboards. They actually have the highest rated models right now. Google does robot stuff. Anybody been following robot stuff? Look at this little guy hopping on one leg. We're seeing some wild, that's just, that's wrong. We're seeing some wild robotic stuff as well. But we're just, we're going to cruise right past that. And then France announces they're going to kick $100 billion into lay. They're excited about it too. And everybody, it seems like, is getting in on the act. And we've got this sort of global like, you know, next space race sort of thing. Everybody's in a race to get to this, I don't know, super intelligence, general intelligence. Nobody really knows exactly what they're racing for, but they're all really excited. It's turned into this big competition, the space race. It may be a great analogy because you know what we got from the space race when we didn't go back for 50 years? We got some moon rocks. There's a moon rock, there's a smaller moon rock that's a flatter moon rock for like skipping on moon lakes. But nothing about this, like it's still a done deal. It's amazing to me you look at something like this that kind of consumed the whole world, this space race. Did you know a couple years ago they found a box that still had some, some moon rock samples in it that they hadn't opened. Like it was bouncing around in the trunk of the space shuttle or something. And they had this big event and they opened up these, these samples that they hadn't, hadn't even opened before. I actually think it's like a great cautionary tale where it's like just because everybody's super excited and it becomes this big geopolitical thing, it becomes a race against everybody else. But that doesn't necessarily mean that it's any more legitimate now. It sure seems like this will change just about everything about the way we work, but nothing is for sure. And the further we try to look out ahead and, and say, well, I know how this or that is going to happen, the more we kind of miss the point. I think like one thing I think tech companies do a lot and this humanities last exam is a great, a great example of this, is they try to solve 200 IQ problems. When you and I's days are filled with like 60 IQ problems. My problems are my clients still won't respond to my email. We just, the government just passed a new regulation. It's going to be a big headache for my clients. Trent. It's taken Trent way too long to get through his projects. Like, I have very basic problems day to day. And as exciting as. As things like humanities last exam are, that's also not what I do. So the actual adoption of where it's useful and what it will do for me versus what it will do for a benchmark like this, those can be two very different things. Like, it is a very sort of jagged adoption that we're looking at. And it is worth remembering that as we have AI experiences that can be disappointing, it's really the only tool out there that you would buy and then just won't do the thing that you want it to do. Like, normally you go out and you buy a tool because it does a specific job. You get it going, you're like, great, that works. You've probably seen this with other folks trying out AI tools. They will try one thing, it falls on its face and they're like, this thing's dumb. I'm not using this. It's not about what it doesn't do as much as it is about what it will do. And that's just a very different way to approach software adoption. Okay, why? I think 25. 2025 probably looks different than 2024. Few thoughts here. One, the tech is just wildly better. It's just so, so much better than it was even six months ago. This is a little test we ran on LinkedIn where I had a language model write a set of review notes for a balance sheet. And I polled people and I said, how could you use this? Because the review notes that I wrote honestly were pretty identical to what I would have written 80% of people, and this is accountants on the Internet. So a whole lot of people that are quick to get pitchforks out. 80% of people said this would be a tremendous resource for my juniors as kind of an initial review, because a lot of these review points come across their desk every single day. It's the sort of stuff that we've identified where if you can use an AI to do the same thing before it goes to the reviewer's desk, that's really helpful. Number two, it's getting more believable. Whether this should make it believable or not is another question. But people see it more People are talking about it, more people are feeling less like, well this is just another crypto sort of thing. Like people love to say that, so it's getting more believable. At number three, it's less new. This is a comparison by a company called writeworks of sort of the security features of the leading models. There's a whole lot of like new spookiness around it. Kind of like we had with the cloud and actually similar to the cloud, there was a lot of messed up stuff. Early days cloud, there was a lot of ways cloud was done wrong. Very similar to AI. The very first version of Chat GPT, everything you would put into it was trained into the model. So there's a lot of somewhat outdated information that still is, is circulated but, but I think we're, we're sort of working through that to figure out how to do this responsibly. Number four, other people are doing it. As much as I would love for the decision making to be like this, first principles based, like, no, it's right for us because of X, Y, Z. A lot of us are just like, well Jim uses it and I saw, I saw Tina was using it, so it's probably cool for us last, it's becoming the new normal. It is like part of culture in a really weird way. This is a quote from Paul Graham, a software guy. Learn something interesting from the CEO of a fairly big tech company. Usually 28 year olds are more productive programmers than 22 year olds because they have more experience. But apparently 22 year olds are now as good as 28 year olds because they're more at ease using AI. Now if ChatGPT had come out when you or I were in high school or college, I mean, we would have been wizards with that thing, right? Right now you've got a whole generation of kids coming out of school who have spent hundreds of hours with AI tools we probably never even heard of. And those people are entering the workforce and then what do we do? We sit down, we have a meeting with all the big brains in the firm and we say, is this good enough to roll out to the team? And you're like, well, it's not really helpful to me. Meanwhile, you've got this like entire new part of the workforce with an entirely different skill set. It's like the first time you saw an iPad kit and they can just like hustle a touchscreen and you're like, holy smokes, if that's ever a valuable life skill, I'm going to be in big trouble. Like, it's an entirely new skill set the kids are coming out of school with that many of us haven't developed yet. Okay, how do you actually navigate this? This is kind of more the point of the rest of the day. Just a few thoughts I wanted to share in wrapping up, because it is. It's just really hard to know what to do. I mean, bless software folks. Like, what an unbelievably hard time to build software where it's like, what is this all going to look like in 3? I got no idea. So it is very hard. And everybody's got a crystal ball. Maybe it's just social media, but it seems like everybody's got a crystal ball. It is worth recognizing that nobody has a perfect crystal ball. And it doesn't matter how many opinions you have. All that really matters is ultimately, what are you doing with it? Like, how are you applying it? What will it do for you today? That's ultimately what matters. I'm very excited for Mary to talk about Carbon's annual AI report. One of the big takeaways for me from that report, if I could spoil her whole talk, is the percentage of people who are very excited versus the percentage of people who are actively training their team. It's really disappointing. A lot of people are excited, but we're not necessarily investing in it yet. And investing in looks like using it. It looks like just getting to grips with what works and what doesn't work. And no matter where it goes, as long as we are engaging in it, like we're in the driver's seat to take advantage of it, it's worth remembering that we are all products of what we consume. Be it on social media, be it the books we read, the movies we watch, we are. We just become products of the things that we consume. And it makes it really, really important to. To think about how you curate your info. Diet AI is only problematic if you're surprised by it. Like, the thing we got to avoid and the accountants that I would love to reach out to and try to get to a better place are the ones who are kind of hiding under a rock and don't want to engage with this stuff. It's a lot. You're not going to, like, sit down and read all the news stories every single day. Just make sure wherever it is that you consume, you've got somebody in there giving you helpful, practical stuff, not the hyperbole. Like, there's just so much of that that's just like, slop and hype and it's not real. Like, find who Those people are who can give you really practical tips and just have them kind of in that loop of how you learn. Because you're not going to sit down and go, you know, go on a retreat for a week and deep dive this stuff. You probably not. You got, you got enough things to do. So think about how you can work that into just how you learn on an ongoing basis. Last man. The best way to learn is to do it with other people. We're in a room full of, I mean, people that took time away from their firms, from their families, paid money to come and talk about accounting. AI, yikes. I mean, one of the only rooms in the world where that's like a cool thing. Like you don't talk about that parties. You don't tell the person sitting next to you on the airplane about that. That's something you keep to yourself. But this is a very rare, safe space. And if you can make a few friends who are trying to figure this stuff out as well, like, it makes it so much easier to learn. Like, you will learn so much faster. One last thought. So my firm, it was an 80 year old firm that I'd bought and the original managing partner, the last 15 years he was at the firm, everybody else in the firm had a computer on their desk and he didn't, he was like, I can wait this out. And part of it was because he thought, he said he was going to retire and then he never did because that's just what accountants do too. They stay around forever. But he was like, no, I can wait this one out. And this is very much that. To me, this is a new way of working. We don't put, you know, 40 words per minute typing speed on a job requirement anymore. We don't put proficiency with Microsoft Office applications on, on a job description anymore. To me, this is like becoming such a core part of work where it's something that not only you need to learn, but we need to keep our teams up to speed on to ensure that we're relevant for wherever this takes us. That's all I got. Thank you. How'd I do? Like I mentioned, up top, you can get access to the other talks from this conference that were very, very good. I'll put the link down in the show notes and I'll see you in the next one.