How AI and Technology can Revolutionize Your Law Firm with Jay McAllister

Welcome to Accountants Law Pod, where accounting professionals and law firms converge. Hosted by Lynda Artesani, Sarah Prevost, and Stephen Libhart.

Hello everyone, this is the Accountants Law Pod. This is a fairly atypical setup here. We have our hosts and normally they would be the ones to kick things off, but of course in typical Jay McAllister fashion, here I am as a guest. Beginning our festivities of the day, I'd like to kick it off by first introducing our hosts to give some comments before we have this discussion, which we will talk about the AI age that we live in and how it is affecting legal practices today. My hosts, kick it off, please.

I adore you. You are so good.

I don't know.

I think that comes on our stuff and introduces himself.

Yeah, I'm, I'm outta here. You did a good job, Jay.

All right.

AI Jay.

How do we know he's not real Jay? And he can make—

Exactly.

AI Jay.

Yeah.

Oh.

I'm expecting Jay to come from the side somewhere like—

Well, for our viewers, they'll have to wait until the end of the episode to figure out whether—

There you go. Said like a true marketer.

So true. Well, you're so magnanimous. Like, you got to introduce yourself, right? That's just the way it goes. I love it.

You know, in certain cases, this can be an effective intro. So we'll see how this is. But, you know, hey, on the spot, I like the way this is done. You know, this was not planned. So, you know, we decided that probably what, 10 seconds before?

Yeah, have fun with it. As you can see, I couldn't hold— I couldn't do a poker face on that one. This was cracking me up. Well, welcome back.

Yeah, welcome back. You know, it's funny, Sarah, last time we were together, Tierra literally had to bring the hook in and tell us, okay, time's up. We need to end it. We would have kept going and going and going.

As you can see, Steve, we missed you on that last one, but you can see why we just—

Absolutely. I've been looking forward. Yeah, yeah.

Well, we were—

All right, okay, who paid you guys to say these nice things?

Oh, I know, I don't know.

No, no, no, no. Let's see. Actually, we got a lot great feedback on the one that you—

Somebody in the blue category. No, I'm kidding. You know, I got to say, prior to us even starting this, it was funny. We were having a conversation about a wonderful colleague of ours, and we were joking around that he's AI because we just don't know how he does it.

Yeah, I hold strong to that belief. I'll let you know if that changes or updates because I am someone who is— I have a background in science. So I do believe in the scientific method and being disproven if compelling evidence presents itself. But right now, I must say the evidence is all pointing towards the fact that they are—

Wow.

Oh, that's true. Well, we are talking about AI today. What did we want to dive in? There's so much that has changed since we were—

Don't be afraid of it. Use it. Oh my God, it's wonderful.

That's deep. But since Lynda and I spoke with Jay,— it has rapidly changed.

Oh my gosh, it's just, it's going crazy. And I, you know, I send you guys in our backdoor channel articles that I find, and they're like, it's now doing this and it's now doing that. And it's just like every day there's something new. Well, what about— okay, can I ask this right off the bat, Jay?

What about the—

and the name's escaping me, but the new Chinese company? What do you know about this?

Oh, don't get me started. So one thing I must say, it's, it's very difficult nowadays So I do quite a bit of speaking engagements. I had a talk that actually Sarah and Lynda were able to, that's how we connected originally after I talked at Clio Con.

Yeah.

And I was talking about AI and, you know, I talked about the framework that I created. I think we went over the framework the last time I was here on the last episode.

Mm-hmm.

The Leverage AI framework that helps legal professionals to identify what tasks would be suitable to out, to be outsourced to AI. Well, the interesting thing about it is that we are sort of in a new frontier where part of my preparation when I speak is that I will enter into this phase where I just binge-watch content, read articles voraciously for about a month leading up to my talk just to make sure I'm up to date on all the latest developments. And one thing I'll have to, I have to say about AI is this task has become so difficult. Like I'll literally, I've spoken, I just spoke for the Institute of Continuing Legal Education 2 weeks ago.

Mm-hmm.

And I spoke about a case in which maybe you've heard of it in Minnesota, which a Stanford ethicist was called in in order to weigh in on this particular matter, which dealt with using deepfakes in political elections. He was supporting this existing legislation and saying that it applies to AI. And the interesting thing about it is like within a week of me talking about this, of me researching this and then being able to present this, the information had already changed materially as far as, you know, how, you know, what his new involvement was. Originally, the court wanted nothing to do with him once he had submitted this information, this ethicist. It turns out he did. He cited cases that were fabricated, which is a career-ending, usually a career-ending move as an AI ethicist. Yeah. And you now, are caught using, I think it was ChatGPT, without confirming the output.

Why?

You're right.

It wants to please you.

That's why. So, you know, but he was actually brought back in to weigh in on the discussion. So, you know, AI is changing so rapidly. It's very difficult to keep up to date. And, you know, from a day-to-day standpoint, I mean, OpenAI just released a bunch of new features. We can talk about that specifically how it pertains to legal context, but, You know, I think, so you mentioned Steve Deepseek. So Deepseek is that, Guangzhou, China is where Deepseek comes out of. And a private company, essentially the whole premise behind why this is making headlines is that this is the first time that we've seen a private company create a large language model. Is it private? Right, there's some air quotes there. Create a large language model that is on par with, and in some cases exceeds the capabilities of our, you know, best-in-breed tools today, like Anthropic's Claude, ChatGPT, and Gemini.

Right.

So, and the interesting thing about it, the reason why it's creating all these ripples is that it's done so at a fraction of the budget that it costs to research and develop these other platforms. So for example, With DeepSeq, they're now on version R1, is the new version. That's their new advanced reasoning model that's causing all of these stories to be put out. But the cost to build that, to research and develop and bring the tool to market was $5.6 million.

That's all?

$5.6 million, right, that's all. Now, it's sort of a hotly contested topic because there have been many experts that have weighed in on the conversation and they estimate the cost to be closer to $2 billion. But, you know, you figure, you know, $5.6 million plus or minus $2 billion, just a small—

What do you believe when something like this comes out?

You know, it's like— Well, the thing is, it doesn't matter because either way, even at $2 billion, that's significantly less than the cost of bringing something like ChatGPT to the marketplace. Like, we have the best minds in the world getting together for years and years to develop these tools. And then here comes this private company, we can talk about that aspect as well, out of China, who then creates this tool that rivals the capabilities of these platforms.

I think they got some of— they copied some of what's already existing because, I mean, the fact that they did it that way, and I know it uses less energy, which is another big bonus of Deepseek, but, and I know it looks at the data in a different way, a way that I guess there was one way that ChatGPT's doing it, and then there's another way that Deepseek's doing it, which is able to do things faster, better, even though they tested it and it didn't do so well on the testing that I saw.

Oh.

So that, that's fascinating. And I, it's all fascinating, but it's like, which one do you use? And I, I just test them. Like, I like Perplexity. That's one of my favorite ones, but it depends on what I'm doing. So there's different ones for different things. I, I could imagine for law firms, it's the same thing, right? They're gonna use different— if you're gonna use it for research though, shouldn't you be using it in a product that does research for legal that's gonna cite I want to see the actual cases that— I don't want made-up cases. I need to see the actual if I'm going to use it in court.

Right.

Yeah.

And, you know, that gets into the conversation around the utility. And, you know, there are various use cases for AI tools that do not leverage RAG or retrieval augmented generation, essentially showing you their work, pointing you to the primary sources that it's using to derive its output from. From the DeepSeq standpoint, since we were talking about that, they've, you know, they've actually said that they've used a process called distillation in order to train their model. Now, distillation, what it essentially is, is feeding AI-generated content into the model in order to sort of create a synthetic dataset.

Ah.

So because, you know, ChatGPT had such a large dataset, because Google, who actually has one of the largest datasets available— Yeah. Because they have all the search information, that we've output. You know, because they have these access to these large datasets, they can create an AI model that has much more variance in the information that's being used to train it. Deepseek did not necessarily have that, and it's primarily targeted actually at the Western market.

Mm-hmm.

So what they did is use the technology called distillation, which OpenAI is actively pursuing litigation against Deepseek saying that they used, uh, in a way that's not intended, which is that distillation. They don't allow you to use their outputs in order to train their own model. So whether that was the case or not is still— a jury's still out on that. But either way, um, you know, it's impressive what they've been able to do in such a short time span and so cheaply. They—

I was gonna say, that's inexpensive, and also it's AI to AI, essentially.

It is not using as much, because I guess AI takes a lot of energy to produce and this is a lot less, whatever the way they did it. And I, and I read about it and I'm like, oh, I had to reread it and reread it to kind of grasp what it was trying to tell you in the articles. But it's fascinating.

So there's sort of, you know, it's like, who, who is, who wins in this scenario? So not only did they develop the platform for a cheaper cost, much cheaper, I mean, we're talking about pennies on the dollar, but also the, the usage of the platform is, is much cheaper. So this is for both the end consumers where the platform is free But then even for developers who are using it to tap into the API, the costs are significantly reduced from tools like ChatGPT or Gemini. You're talking like a tenth of the cost in order to— like, they usually— the way they measure this is in hundreds of thousands of tokens. That's how they measure the output. So when you query their model and ask it to deliver information to your tool, they bill you based on that.

Okay.

And Deepseek is at such a low threshold that That's where, that's really why this tool started making headlines is because people, you know, these companies and these hedge funds in the US who have made these huge bets on AI companies and put plenty of their portfolio into those companies.

Yeah.

They're seeing a major threat of Deepseek entering the market because they're doing it so cheaply. Now it's going to likely force those other companies to, you know, play the catch-up game. Play that game. Now it's race to the bottom.

Yeah. Well, today we just saw the politics of TikTok and China and all of that go on. And now here's Deepseek and how do we trust it? How do we know that they're just not gathering all of our information to use against us? I mean, why do you, like you said, this comes on so fast, so fast. And it's like, how do you embrace it? How do you learn it? How do you trust it?

Well, I think too, Steve, to add, I feel like what Lynda brought up was a very valid point of if you're an attorney—

Right, right.

—using tools like this where those tools are vetted and like Jay had brought up—

Exactly.

—they show you, they cite you the relative space. But to me, I start to wonder, it sounds so synthetic, AI to AI. It's like, how does my AI bot said this to your AI bot? And next you know, you've got this—

Well, so with that said, let's let's use LexisNexis as an example.

Um, is Deepseek saying, okay, I vetted this through LexisNexis, and then you've got to go to LexisNexis to look it up to prove that you're getting the right information to file your case? I mean, where does it— you know, where's the peace of mind with this?

Yeah, so, so Lexis, from, from my understanding, they don't use Deepseek, and many American companies are not switching their model even though it's substantially, substantially cheaper. Deepseek generated the track record that these other vendors have. But there's actually sort of 3 abstraction layers you have to look at when it comes to confidentiality with AI tools.

Okay.

There's sort of the deep tech lens. Now deep tech, these are the companies like OpenAI, like Deepseek, like Google Gemini. They're the one creating the large language models. So, so that's sort of the base layer. So what does their vendor agreements look like? What does their confidentiality clause look like? What does it say that's in there? Now there's this whole separate argument about, okay, it might say X, but do they do Y? And that's where you sort of have to take a leap of faith and look at the company's portfolio and then also look at the, you know, what is the motive for that company? Like what are they trying to build? And I'll explain what I mean by that in a little bit. The second abstraction layer is what cloud provider is hosting this information. So in many cases— Wow. For example, OpenAI, they don't run their own servers. They don't have their own hardware that run, you know, Lynda, where you mentioned, you know, the power that it takes to run these platforms. They don't have their own hardware. So they actually use Microsoft, and Microsoft has huge data centers all over the world with GPUs inside of them, which are essentially the computing units that perform the calculations and deliver your output. So then you have to look at, okay, so there's— Mm-hmm. OpenAI as one layer, and then they're hosting their information on Microsoft servers. So then what does Microsoft say they're going to do with the data?

Yeah.

And then you have the final level, which is what we call the wrapper. Now the wrapper is, you can, you can think of the large language model as the engine. That's like the, like from a car analogy, since I'm a gearhead, as you can see behind me, Corvettes everywhere. So the engine is like, okay, what engine is running your car? You can have any body style at, you know, put on top of an engine., but the engine is the core technology there. But the wrapper is like, okay, do you, or do you have an F-150 or is it a Ford Taurus or is it a Mustang? Like what body are you putting around that engine? And that's the tools like Spellbook, that's the tools like Lexis AI, the, you know, the, the end users' tools that they're interfacing with. So you really have to understand those 3 factors at play for each tool that you're using to determine how you're going to safeguard your client's information.

Interesting.

Now, when it comes to Deepseek, because of the fact that, like, you know, the air quotes we had referenced earlier, it is theorized, and this is all speculation, but it is theorized that the Chinese government likely had some involvement in at least the financial aspect of subsidizing this platform. It's very improbable that a private company Yeah. Be able to amass the resources, the talent, you know, to build something like this, like Deepseek. So because it can be speculated that the Chinese government had involvement with this platform, obviously we have a sort of a rocky relation with the Chinese government and privacy. That's why you're seeing, you know, the TikTok ban or companies like DJI being accused of sending information back to China about its US users. This is sort of a, a rocky scenario now. So if that were the case, that they subsidize the platform, then it is possible that information could be, you know, gathered from the tool and sent back to the government.

Yeah.

It is also possible, I was on a webinar the other day with some really bright minds, you know, who are on the forefront of this. They brought up an interesting use case that it is also possible that right now that the way the tool works, since, you know, folks have, You know, the whole concept behind Deepseek is that it's open source, which is, you know, another big component there, meaning that we can see, it's not a black box like ChatGPT or Gemini. We can actually see the code that powers Deepseek.

Oh wow, okay.

Yeah, we can see what's going on behind the scenes. This is huge when it comes to AI models because typically, you know, it's like you input your prompt and then out comes your response and you don't know what happened. You don't know where the data went.

Yeah, yeah.

With Deepseek, we can actually read each line of code. They publish it available for, every day. Okay, maybe not like normal people, like, but very—

I was gonna say, who reads code?

Yeah, I mean—

You go in there— Yeah, not, not, we're not gonna be—

You turn up the paper in your coffee and like read DeepSeas code.

So why do you suppose they did that?

So, you know, they want to create this environment. So, so open source, the way it works is typically that's where you see the most innovation happen. When you have something that's open source, you democratize that technology, you give power to the people to be able to create innovate and enhance the tool. So now you sort of crowdsource your development instead of it being focused on one company who has to make all of the developments and improvements to the platform, the community as a whole can do so. Now, and this is a big win for consumers because typically that those additional minds that are now able to tap into it, they can create some really unique things that one company on its own could not create. So, you know, the way this works is that When you have a tool like this that's out there in the marketplace and it's open source, now you start to wonder— oh, that's right. I want to bring the point up that I heard on this webinar that I was on the other day. The point was that we're wondering if since users can see the code that's running and we can see that right now in its current state, it doesn't appear that there's anything malicious in the code base or anything that would insinuate that information is being extracted and sent to the Chinese government. However, it's possible that the play could be that there is some latent code which will at a certain time, 2 years from down the line, who knows what it might be, then start sending information once we've already gained this tool's trust and we've already sort of—

Trust, yeah.

People aren't really following it as closely as far as the confidentiality component.

Kind of like it's been there, so I don't need to mind it anymore.

Yeah, when's the last time you looked at Microsoft's privacy policy? Do you use your Outlook email without Or do you like check to see their terms and conditions?

I should read the Google one. No, I haven't. You have to, you don't even get a choice. You just say, okay, okay, okay, right?

Yeah, I mean, you have to sign it.

Yeah.

So, you know, so that becomes an interesting angle. It's like, so my take right now is that I think people should, you know, despite all of these potential security concerns that might be there, I think people should gain familiarity with using the platform because it just adds to your diversity for, how these tools function. You know, the, the way Deepseek is set up now, it's sort of like ChatGPT in its early days where like you don't get all the, the complex canvas capabilities and the projects.

Yeah.

It's very simple, like just question and answer.

Yeah.

But the really cool thing that I like about it, one thing that it does much better than ChatGPT, and this alludes to the open source nature of it, is it actually shows you the reasoning model. So if I ask it to compare data between Vendor A and Vendor B because I'm looking to make a decision for my company, you'll see, and I, and I tell you to rank them based on, here's my criteria, rank these companies. You will see it'll say, okay, I'm analyzing their expertise in X area. Okay, I found this. Oh, but you're looking for a competitive analysis. Maybe I should look into this instead. And you'll see this line by line as it reasons its way through your problem. So you can actually, And from an attorney perspective, it's interesting because that's one of the main issues that lawyers have. They're skeptical about the reasoning behind how it derived its answer, but you can actually see the basis for how it created its logical arguments. And then underneath that is where your output will be.

That's impressive.

Wow. So, you know, the only caveat there I should say is that my take is that it is not a tool that you should use and enter client confidential information into. Ideally, you really shouldn't enter it into any of the platforms. There are some exceptions out there, especially if you're using the paid versions of ChatGPT. Again, this all comes down to trust and whether do you trust OpenAI, do you trust Microsoft? I tend to trust them that they'll do what they say in their privacy agreements because of the fact that their whole business model is centered around the fact that they're looking to create these solutions that people use over time, and the trust that they built, they, they, they don't want to erode. So I tend to believe them that when they say they're not going to use my, my, my data to backtrain the model, I tend to believe that they'll do so because if they did, that would cause, you know, some disastrous harm to their, their company and ultimately would lower their user base.

Yeah, that's, that's a lot of data or a lot of information on what— like, to be able to see the data, to see the information, how it reasons it out. Currently, you don't have— you can see reference, like, um, Lynda and I use one product on you, and it shows you reference codes of where it's pulled some of that data if you ask it some questions, or how would you look at this. Um, but I'm thinking of, like, you're— like you said at the very beginning, you're going into court, you're using these things as references, and you haven't looked further. Why aren't you using trusted resources that are built in and products that— not every product has it, I understand that, but the ones that do that are within the industry. I'm surprised at this profession because you're right. Why are you writing yourself off right then and there? Like, goodbye.

You know, it's like what's happening is that the output is so convincing that it lulls people.

It lulls people. Yeah.

You're in a hurry.

That seems intuitive enough to be true. Yeah. Like, I can tell— what if I told you that everything I just said about Deepseek was incorrect? But like, I said it confidently enough.

And you made us believe it, right? Yeah.

That it resonated, right? So—

Yeah, it wants to please you, like Lynda said. Well, it didn't want to please Steve though, Lynda.

I think with AI too, don't you think that it's going to grow a whole new industry of lawsuits and things that attorneys can do?

Oh, sure.

Part of this because like, artists, and then they copy. And there's so much that you can see that's going to develop into a different type of law practice to do with AI. I mean, you can see the writing on the wall with this stuff.

I mean, so I have a colleague of mine who's a voice actor, and, you know, there was just recently an event in her industry where a voice actor's sort of style and tone, and I guess you could say their likeness, was used captured by an AI tool. And then that tool created these, I think it was like for store jingles or something like that, or some store announcements or something. And they were used in order to train that model to then produce this stuff without her being involved. So now it's like, well, I want a piece of that because this is clearly, you know, it's using my likeness and they're saying, well, no, no, it's just, it was inspired by. Yeah. You know, it's definitely one of those—

Oh, so it's gonna change the language around trademark and copyright and inspired by. I mean, as it is now, what do you— you type something, it says— it gives you caveats to use in your writing style if you choose to use that, if you don't rewrite it.

I didn't think about that. That's been around, like, with Grammarly. All that stuff's been around for longer than—

Longer time.

You don't ever think of that as AI, but a lot of that—

Yeah.

That stuff that we've been working with for a while has been there.

Well, so, you know, you've heard the stories before about how artists sue each other over their songs sound alike. I wonder if AI could, like, if you could go in there and say, write a song "Write a song for me that's never, write a tune that's never been done before." And then make your millions, you know?

I wonder if that would fly in the Kendrick Lamar and Drake thing. 'Cause you know, he's writing his own stuff on Wednesday. I don't know about that.

You know what I'm gonna be doing all afternoon.

Calling up Kendrick Lamar, I think, right?

Yeah.

You got him on speed dial, right?

Yeah, yeah.

Let him know, hey, I'm a big fan, big fan.

Yeah.

Well, it's interesting because one of my colleagues and you all, Lynda and Sarah, you might know him, but Damian Riehl, he's an attorney and also legal technologist. He's an IP attorney specifically. And he actually, he's also a musician, which sort of factors in this story as well. So a man of many talents, clearly. Yeah. Hardest working man in showbiz. And what he did is he actually, and he's got a TED Talk, We should probably share it in the link, but he has a TED Talk where he had this idea where he was tired of artists fighting over, or over copyright infringement, saying that, "Hey, you took my idea, you took my song, my melodies." So what he did is he actually wrote an algorithm that would determine all the melodies that could exist under a certain length, and then he published those into the public domain. Yeah. So that he would own all those songs technically.

Ah, ha, ha.

So that he could sort of start a precedent and say, like, okay, all the slots have been taken. I'm not going to sue over this. So now we need to start really deal with the real question, which is that, like, because so many songs have been made over time, is it possible that I've never heard your song, but we just happened to land on the same melody choice because You know, it's just something that popped in my head, and there's only a finite amount of available melodies. So that's his whole concept, which is like, it's more about the, you know, the lyrics and the music and the tone, and less about the physical melodies, because there is a finite amount of those that exist.

Okay, so I have a question for you, 'cause I posted this. I was a little bit fascinated by it. So I posted this over the weekend, and it was about a song that we, it's very popular, but he took the music, the beat, one of the music tones in his song from an Australian, like where you walk across the street and it makes this noise. So I'm like, well, who is the creator of making the noise? And how does that work?

When you walk across the street?

In Australia, there's a noise that's made on the sidewalk when it's a crosswalk and lets you walk across.

No kidding.

And it does this this melody, which it hit this artist and he's like, I like this. So it got into the song. And Sierra could probably link— she probably saw because she sees my Instagram. It got almost 2,000 views from just my reposting, just for fun. I was just having fun.

I was like, wow.

I thought Sierra was you on Instagram.

No, it was me. And anyways, I was like, who made that tone? For the sidewalk crossing to cross the street that this guy now has in his song, which makes me— that's where I think my brain goes. And I think what Steve is going to— it's like, what point does this— because it keeps going down the rabbit hole of where authenticity comes from, or melody, like you're saying. How does this— because it's really, it's like that in legal. It's, it's how do you validate the information you've received to present in a court case How do you know where the reference— I mean, you'd have to go back to your regular research, right? You'd have to go through your regular means of research to ensure that those cases line up with what you're going to reference, I imagine.

I mean— Yeah, at least, you know, it's like these sites are there for a reason. That's how opposing counsel is figuring out that you're full of it, is that they're looking at your case sites and they're saying, I can't find these. So before you send those out to the, you know, to the tribunal, Why don't you actually take a moment to verify—

To verify.

—what we're about to present as true and accurate to your understanding. So—

Yeah, yeah.

Yeah. I thought you were originally talking about— what was it? Ice Ice Baby, because that was a direct Queen ripoff.

That was a direct Queen ripoff. Yeah, that was— this one is interesting. I'm sure, Thierry, I could get you the link too, but it was very fascinating to me. But also, I just— my brain was like, Well, who made that tune? Like, who made that crosswalk sound, right? Because people make things. I mean—

That's some poor engineer who wasn't even, like, looking to create—

Yeah, I know.

Now he's like—

He made a music melody that's in a famous song. There's a Jimmy Kimmel that talks about—

There's an entertainer that does— he takes a song and it just rolls into the next song and the next song. It's all songs you know. It's one beat and it just goes through, and every song has that same beat. And it's like, wow. But you hear it, they sound differently when you hear it independently, but when somebody puts it all together in a string, it's like, wow, that's beautiful.

Melody.

Yeah, same basic song, basically, yeah.

Yeah, but even then, it's like, aren't we all sort of informed and inspired by our collective society anyway? So if I hear a song and it sort of inspires me to think down a certain path, like, I think it's really difficult to try to say, like, it's an unrealistic it's an unrealistic goal to say that everything should be uniquely inspired and sort of siloed in its own— like, everything— art inspires art. And I think the greatest artists, they will say, like, even they've taken inspiration from X, Y, or Z. I mean, you talk—

Good point. Good point.

Film directors, they'll say, like, like, I'm a big M. Night Shyamalan fan, you know, and he'll talk about how, you know, various pieces that he's seen that sort of create these concepts in his head, and he uses those and pulls them together and synthesizes it. So now because there's multiple pieces of information, multiple people that's in one place, you know, that's, you're creating originality by combining these unlikely—

By combining them.

Way. Yeah. And that's really all AI is doing anyway. It's taking information that already exists.

Yeah.

And it's finding new ways to interconnect, or it's finding new relationships between ideas and concepts that likely would not be connected.

So in the legal profession, when you're, when you're looking at this as a firm and how to work with these tools, because you hear obviously the still the big scary thing about I'm going to lose my job or these jobs are going to go away, or so when you're in that space and you're looking from a consulting side and you're looking at their workflow and you're looking at improvement, what do you see is the most What do you feel like is the most tangible piece, or what is the piece that most attorneys are like, oh my gosh, I could actually do that, and that could be serviced by— like, I don't mean to downplay this, but people love to have those robot tools on their, their sites. To me, sometimes it makes sense depending on the type of tool you need, but it doesn't make sense for certain— I mean, I guess it just depends on the tool. I is kind of— and I'm curious what your through your lens, through your journey, what you have found most successful that has worked kind of like on the quick or maybe as more of a fascinating, like, wow, I could actually do that and that'd be easier.

I certainly believe that law firms should not implement AI just for the sake of implementing AI.

Perfect.

Because then it becomes a novelty.

Yeah.

And then it's something that's left on the shelf.

So true.

So you have to actually be looking to solve a constraint in the business model. It forces you to put on the mind of a business owner and focus on the business of law for a second rather than focusing on being a practitioner.

Yeah.

Where are those high-impact areas that the firm could leverage AI in order to create efficiency around and then targeting those. So like a big one, like, you know, many firms and Clio had a statistic at ClioCon that they released in their legal trends report. Which is that it was something like 70, over 70% of law firms are using AI in their—

Wow.

But no, but from my knowledge, so this statistic, when it was originally said, it did not really jive, it didn't sit well with me 'cause I'm like, 70%?

I know, yeah, that too.

I have many clients all over the US and although anecdotal, you know, we're talking about, you know, a client base of, you know, 50 or so firms. While anecdotal, you would think that would be a large enough sample size to at least see somewhere near that. I'm seeing somewhere near like 10 to 20%.

Interesting.

You know, the level of interest and the appetite my clients have. So I started to wonder, so what could be, you know, do I think they lied about their data? Not necessarily. It would seem to be probably a, you know, the core of Clio's focus is not necessarily the AI tools, it's being a competent case management system. So there's not like a huge incentive there. So like, what could be going on here? When I started thinking about it further and talking to my clients, what I realized is that one, the methodology— I talked to Joshua Lennon, he's their lawyer in residence— and the methodology that the questions that they asked was around, you know, specifically, are you using AI in your day to day? Now, there wasn't necessarily a follow-up about what types of tasks are you using AI for?

Yeah.

At least to my understanding.

Yeah.

So then the question I started asking my clients is, what type of tasks are you using AI for? And what I found in my clients is that most of these tasks are legal adjacent tasks or tasks that are not necessarily even related to the law firm at all. So technically, these lawyers are using AI, but they're not using it for legal production.

Mm-hmm.

Right? And a lot of that comes from the fact that you see these stories like the lawyers from New York or the ones from Minnesota, the AI ethicists that I told you about. You see these pitfalls of negligent misuse of AI.

Yeah.

Lawyers are— they're spot— they're trained to spot risks. So and, you know, spot issues. So that when they see these type of things, it tends to make them take, sort of take pause.

Well, and is one type of law to practice using it, but another type is not, you know, the type of law that they practice, maybe an, I mean, just for sake of conversation, maybe an immigration firm would hardly use AI, but a corporate transactional attorney probably would. You know, I wonder, You know, in your 50 clients, what's the mix and how are they using it? You know.

Great question. So—

Yeah, that's curious. I'm curious about that.

No, a lot of this AI use is precipitated by, like, there, there's usually a catalyst. In capitalist markets, they always drive towards efficiency. So nothing is done just for the sake of doing it. That's why you're seeing now is the first time in history that private companies are entering into the space race, like SpaceX, like Blue Origin. Why? It's because there's a business incentive there now. You know, private companies weren't going to space. We didn't send, you know, private companies didn't send Neil Armstrong to the moon in the '60s.

Mm-hmm.

Government did that because that's the type of projects government take on where there is no ROI. But from a business case, there has to be an ROI there. You have to be able to say, I was watching a Neil deGrasse Tyson podcast. And he was saying that right now Elon Musk is saying, our next frontier is we got to go to Mars. He doesn't think that's going to happen. He's like, from a business standpoint, you go to your investors and your shareholders and you say, okay, we want to go to Mars. And they say, okay, great. Why? He's like, well, because it's the next thing, the next frontier for humanity. Okay, great. Is it safe? No, likely people will die. Okay, well, what's the ROI? There is none, negative. And it's like, that's a 5-minute meeting. That's like— We're not doing it. So when it comes to the AI component, you have to have that driving economic force. We saw that first probably about 3 to 4 years ago from in the insurance defense sector, because insurance defense is unique in the fact that the insurance carriers have tremendous amounts of data. As we all know, insurance companies, they collect information so that they can properly price their premiums and things like that. But also on the legal side, they want to understand how long legal tasks take so that they can make sure that they're getting the best deal. And then they put that downward pressure on the firms.

Yeah.

Who they already dictate pricing for and they say, hey, look, the, the, the other 100 firms that we have doing this type of work, it took them only 10 hours to review these contracts and you're trying to bill us for 50, not going to cut it. So those are the first firms that we saw implementing AI because because, you know, the other firms in the industry were doing it. So they had to, in order to remain competitive and keep getting that work from their insurance carrier clients, they had to adopt the AI. Then we saw it after, after that, the next phase was sort of big law partners who split off and opened their own boutiques. They knew that in order to compete and bring that same level of big law experience down to the average consumer, they realized that they could leverage these efficiencies afforded by AI to still produce higher quality work, give that big law feel, but do so at more affordable rates. And they, they really had to in order to compete with the, the, that structure, big law. So that was sort of the second phase. And now we're seeing, as these firms have started to implement these items, and now you're seeing, you know, defense firms implementing it, it's sort of putting pressure on the marketplace because consumers are realizing that law firms can and should leverage AI. Mm-hmm. So now they're demanding, expecting, and in some cases demanding that their lawyers use AI because they realize these tasks can be automated.

Automated, yeah.

They need these tasks. I think another, Clio actually did a study where they broke down all the tasks that lawyers typically do and they found that 74% of all legal tasks could be outsourced to AI, could be fulfilled completely by AI, which is, What kind, I'm curious, like what types of tasks would those be?

Just outta curiosity, like what are the buckets?

Yeah, so they actually created a whole, they referenced a, I can't remember what the study is, but there's an organization that publishes a study every year.

You think it's the paperwork side?

What was that?

You think it's the paperwork side of it? The, you know, all the paperwork they have to fill out or?

Well, certainly the paperwork, but as far as the tasks, this organization, they create essentially task codes They sort of enumerate all the various tasks lawyers do. And then when they looked at those task codes and then applied AI to those particular items, they found that that's where they found the 74% of those tasks.

That's a high percentage. I mean, that's a real high percentage.

Yeah. Well, writing briefs alone. Well, I think— You know, once you've got the information. Yeah.

Yeah.

I think about like what you said at the beginning, because we've seen this, we have a client that is in this space. I think about this as I understand that the insurance companies are going to come back and say, or any type of industry will eventually come back and say, it shouldn't take you this long. It should have only taken you this because of the aggregated. Who's to say the aggregated data was accurate to begin with?

Do you know what I mean? Well, when you're State Farm, you have the tremendous ability to push these firms.

And I'm wondering if there's going to be some eventual pushback the opposite way. I would imagine so because One complexity of one, and we see this in our profession, what looks straightforward of, say, getting into your file and getting it kind of working and smooth, and you've experienced this, Jay, what looks daunting when you find out the human capital behind it might be the problem or other elements might be the problem. Telling me that it's gonna take X versus, you know, what it is, that's a really tough conversation. That's basically telling somebody how they're going to work. And I think—

It is.

When it's hard, right? So when I think of AI in the sense of like, okay, is AI really a good use case for an attorney to have it kind of vet out the type of clientele it wants before it gets to them on a call? Is that a good use case? Or like you're saying, like, look at certain tasks But you still need, I feel like strongly, I still need some oversight to what is happening. I can't control, I don't trust what the bot is going to do.

And you shouldn't. You shouldn't.

Okay. Yeah.

You know, in the current state of AI, it's so Stanford, they had this highly controversial paper that they, research paper that they put out in 2024, which they compared some of the top legal technology AI tools. Especially many of them who cited that they have near zero, or I forget the exact terminology, but basically no hallucination. And they found that actually hallucination was rampant even in some of the best tools, tools like Thomson Reuters or Lexis AI Plus. Thomson Reuters had, I think it was like a 43% hallucination rate.

Wow.

Even though being constrained to primary data sources and being trained on legal, primary legal documents, rulings, pleadings, opinions, decisions. So, so, you know, so, so the thing is you have to always with the current state of AI, I don't know if we'll ever get there. I would assume probably would, but at least right now you always must check the output. Trust but verify.

Mm-hmm.

So there's no AI platform. I don't care who they are.

Yeah.

What they were trained on, how, what, how good their claims seem as far as hallucination. They all hallucinate to some degree. And you know, you, you have to check, you have to check the output. Like these tools are made to augment the, the role of an attorney, not to replace them, which from an attorney perspective, that's great. Right? Because you still are necessary. You're a critical part of the process. I would say AI is the paint. AI is the paintbrush. The attorney is the painter. So the paintbrush does nothing without the attorney.

Yeah.

You still need that person to guide and direct. And to craft and to envision the overall strategy behind it. The paintbrush doesn't just get up and start, you know, it doesn't create the Monet on its own.

It's just sitting there on the table.

I think of it because I read the—

Not yet until the AI gets better.

Yeah. I read the book, the Elon Musk autobiography. I forget who it's by, but I read it and they talked about Tesla and how his— he dictated from being the boss of it that I want a lot of automation in the creation of the car because I want the car to be made really quickly, efficiently, and all that. Then he actually got down on the floor and saw some of the robotic arms doing things that a human could do in 2 minutes, and the robot was taking 5 minutes.

Oh, really? Wow.

And he looked at it and he said, delete, get rid of it. I want a human doing that because I know— delete, because it's gonna be faster. It— the hand could move faster and the person think, think where the robot on some of the tasks, it was slowing the job down and he had to get a certain amount of cars out the door at a certain amount of time. And he eliminated a lot of time in building the car by taking away the robot and putting the person in. But nobody in the line or any of the people running it ever thought that because they knew he just wanted it automated. So they were going to leave it in place. It was doing the job eventually, but it was slowing the production down. And he went right down. And he went right down. He's like, take that off, take that off, and put a person in place.

Well, part 2 of the autobiography is that he brings in the Optimus robots and then re-replaces the tech.

Yeah, yeah.

Well, there you have it.

Yeah. So it, it's interesting, I guess if you're a firm right now who is trying to even begin this process of looking at implementing AI to, to your point in a more really efficient way, not in a way just for the case of having it, 'cause you're just gonna, it's not gonna work. You're not implementing it very well then. I guess it's just, do you come up like, from the beginning, are you coming up with a use case scenario for yourself? Like, okay, if I need these tasks done, what is the best way to make this more efficient? Or are you looking at it— how do you take the— how do you bite the apple, essentially?

Yeah, and it's a big apple, right? So people are figuring, worrying, they're wondering how they— where do they start?

Where do they start?

I'll just quickly run through the Leverage AI framework because that's the best way that I found inform law firms how to determine what tasks are suitable. So you start off with the L, the L in leverage, limit identifiable. You have to have some sort of constraint that you're looking to solve for in the business model. So for example, I'll give an example of a client I have, personal injury firm. They do a lot of toxic mold exposure work where they're looking to determine first whether the, whether that client is going to be a good fit for their firm. Most of that work is personal injury. So it's, it's very similar to personal injury, sort of.

Yeah. Yeah.

You know, it's all contingency. So the firm must have a very strong vetting process. But what they figured is that it was taking them an inordinate amount of time just to get through that vetting process. And they were talking to so many people that never even translated into clients, but there was such specific information that needed to be gathered. Things like, is the client currently still paying rent in that property? If they're not, if did they, did they miss a month? Did they, then it invalidates them. Do they still live in the property? If not, invalidates. Do they have pictures of the mold? If not, invalid. So it's like there's all these nuanced things that—

Yeah.

Identify, but it was taking a human operator so long to do so. So we implemented, for that firm, we implemented an AI tool to go through that process.

Wow.

On the front end, ask targeted questions and then based on the responses, ask other questions that would then vet that person so the firm knows whether they're even worth talking to for the next steps. Okay, so that's the L, identify your limit.

Okay.

The V is variance allowed in your output. So as we know, large language models and generative AI in general are stochastic, not deterministic, meaning the output can vary slightly based on the way you ask the prompt. If there's any bias in the prompt, you could even ask the same question twice and have a slightly different answer.

Yeah.

So you need to understand for the output that you're ultimately looking to create, Is variance allowed?

Mm-hmm.

Not, then that would, that particular task would not be a good use case for AI. The V, I'm sorry, I already said the V, the R is a repeatable process. Do you have a process that you've already identified that you can then instruct the AI to go through? Do step 1, step 2, step 3, step 4.

Okay.

If you don't have that or if it's not possible to do that, then that particular task you identify would not be a good use case for AI. Now, my belief is that most tasks in law firms can be broken down into those steps. Many lawyers, they feel that breaking step— breaking processes into steps sort of commoditizes their work, but it's the opposite. It allows them to create structure that then allows them to plug in their, you know, the, the various bright minds in that firm to perform that sophisticated legal analysis. But like, no one wants to do the rote tasks that are associated with getting you up to that point.

Yeah.

Yeah. So create the steps. G is goal-oriented. Is there a point that we know the task is complete? Can we point to something and say, once this is done, the task is complete? If not, not a good use case for AI. The A, so we did LVRG, leverage.

Mm-hmm.

The A stands for associate-friendly. So at the current state of AI, it needs to be something that you would entrust an associate, a first-year associate to do. We're not talking about, analyzing an entire case or arguments and coming up with your final, you know, your final brief. We're talking about these are tasks that you would likely feel comfortable having a first-year associate that you've never worked with before complete for you so that you can then check the output. And then the I is integratable data. So, and integratable is not a word, but I couldn't figure out any other better. I really need to put this AI, make it nice.

Ask AI what the word should be.

You know what? And it didn't come out.

It didn't come out, huh? Okay.

But the integratable data just means like if there's any external data we need to use in order to inform the AI, do you have it all in one place that we can use, that we can easily tap the AI into? So for example, I have a client who's using AI to create discovery resolution letters. So basically, you know, if a firm is, if a firm is in litigation, this firm is a, family law firm, they will— let's say there's a dispute about one of the discoveries, an interrogatory that was requested from the other side, and the other side disputes that having relevance to the case. Well, that firm needs to show that the lawyers took a good faith effort at resolving the issue before getting the court involved.

Yeah.

That process involves you going through all of your communication, listing out what discovery items were requested, what the status of them were, and then for the disputes, what items are disputed. That means that the firm has to go through all their emails and figure out that information. So what we did is we actually used custom GPT, so using ChatGPT, that you could tie all the emails into, again, that integratable data source. We had those emails output in one file and then we were able to share that entire corpus of emails into ChatGPT so it could then pull out all the relevant information.

I love it.

That's sort of the, you know, that's the framework. Framework that you should use when you're thinking about AI. You start at the top, first of all, figuring out what constraint actually exists in the business model, what thing is inefficient that is, is causing the firm to expend an unreasonable amount of time. You start from, from there and you work your way through the rest of those acronyms. And as long as it satisfies all those conditions, that task is likely a very good candidate to be outsourced to AI.

I like that example. I really do. Yeah.

You could be speaking on this at ClioCon perhaps, maybe, I hope. Oh, you don't know yet.

Good, yeah. They, yeah, maybe I'm gonna, yeah, I'm gonna get on that.

Yeah, yeah. I would imagine. Well, I was gonna say the I, the integratable, you can be part of the cool kids club on the Gen Z who makes up all kinds of words I don't even know. I'm like, what is that? That's now the new word for cool?

Do you guys know mid? Do you guys know that one? Which one?

Mid.

Oh.

Mid is the new— it's like, uh, it's just like very bad. It's like, oh, that's mid. Like, that's mid. Yeah, you have a bagel and it's like not that great. Uh, this bagel's mid. Mid. Yeah.

Okay. Okay. I'm like, because my husband knows way more than I do. He pays a lot more attention to my son's wording of things. And I'm like, you ask these questions? And he's like, yeah. And And before you know it, I was like, I, I don't know. Yeah.

I have a feeling that, you know, Tiara's in the art space, so I have a feeling she knows that one.

She knows, she knows that one. Yeah, yeah, yeah. This, that framework, I mean, I don't know, Lynda, I, if you, I'm sitting there going, okay, how can I just make sure? I know, like it, it just for your own business, for any business model, you have really bite-sized down such a great way to take such a huge thing and understand where does it meticulously go through this series. Yeah, it's brilliant, honestly.

Well, it was a result of a lot of failed AI implementations, and I realized if we're going to keep any clients, we need to figure out a way to create a repeatable process.

That's me.

So we're full trans—

I got you on that one.

A quick learner, quick learner.

I'm gonna start walking around and using that word now.

Oh, you don't wanna overdo it, Steve.

No, Steve, you don't wanna do that.

That'll be a dead giveaway that you've just learned it. You have to sort of slowly weave it in.

I think he's dead giving away that he just learned it.

Oh my God, that's hilarious. I love it. Gosh, thank you so much for joining us today.

Oh, my head is spinning. This is awesome.

You're welcome. And one other thing that I'll give you is that I just figured out that I was using the wrong microphone this entire time, and I just switched to my really good microphone. So I'm glad the last, you know, minute of the recording could be such high-quality audio.

I have to say, the sad part was, and this is kind of a funny inside joke with the three, the four of us on this side, is, you know, you're new to podcasting, you've made mistakes where you forget to record. Well, that did happen to my husband over the weekend.

Oh, did it? Oh, wow.

I was like, well, been there, done that. This is how you handle this.

Just say sorry.

So let's do it again. Just do it exactly the same way you did it and that'll turn out great.

That's exactly what we tell people. We won't do that anymore though.

Yeah.

Do you guys know I had a podcast at one point?

No. Yeah.

Along with everyone else and their mother, you know, back in 2020. Yeah, I had a pi— I did 5 episodes, which is I beat the average. The average is you do 3 and usually quit. So I got my 5 in. And I absolutely had a guest that I had in person live who drove an hour away to get to my office. I forgot to record. And I never I put it off for like 6 months. They're like, hey, I haven't seen the episode. I'm like, yep, working on it.

Oh, they didn't even know?

Oh boy. I gotta be honest, like, yeah, that's— I forgot to record. I felt like such an amateur and a failure. So, you know.

Yeah, it's rough. I mean, we all go there, like you said, every failed attempt.

And while we're human, you know.

Yeah, that's the whole thing.

Well, You guys did remember this time, right?

Yes.

Oh, yeah.

This morning, I checked.

Oh, yeah. It's now a rule. We record before we even start.

We record the very beginning. Yeah. Just not take a chance because we get talking and then all of a sudden it's like, okay, let's get started. And yeah, you don't hit record.

Screw up. So, yeah.

Well, thank you for coming.

Thank you. Thank you. Thank you. Yeah.

Wow. A lot of fun. A lot of fun.

To support the Accountants Law Pod, don't forget to like, share, and leave a review. And if you haven't already, hit that subscribe button on YouTube or wherever you enjoy your favorite podcasts. If you have questions or topic requests, shoot us an email at info@accountantslawpod.com or send us a message through our website, accountantslawpod.com. To join our weekly Accountants Law Lab mastermind sessions, head over to accountantslawlab.com to sign up. And to learn more about Jay and his services, follow the links in the show notes below.

And our show notes should probably include some fun stuff too, like we just talked about, just for fun.

And look for Jay on ClioCon Boston in the fall.

Oh, absolutely. Well, you'll see us. We will be there. Our team will be there. So you'll see— Yeah, we're bringing the whole team.

So you get the full proper trust experience.

Yeah.

Yeah, last year—

I'm not signing autographs.

We'll definitely be back. Last year was my first ClioCon and that was a lot of fun.

It's a good— and it's going to be in Boston. I mean, that's going to be so much fun. So.

Well, it'll be fun too, Jay, because if you want to crash the Proper Trust team, you're welcome to.

I will absolutely take you up on that. Steve, what about you? Are you going to be there?

Yeah, I'm going this year.

Yes.

And Jay, what city are you in?

I'm in Chicago here.

You're in Chicago.

Okay. Okay.

Oh, that's an easy flight to Logan.

[FOREIGN LANGUAGE] So my director of client service, which he'll be there as well, lived in Boston for many years, so we would go there. There's a phenomenally overrated restaurant called Legal Seafoods that we would go every time we visited. It is good food, but it's just not, you know, it's overrated.

It's overrated.

Yeah.

Yeah.

It's just one of those things. It's like every time we go, we would say we're going to find a local place that's, you know, that's where the real, like, Legal Seafoods.

You got to go to the North End and go for Italian.

Oh, yes, we did do that, actually. North End. Yeah, we found this nice little— it's like a speakeasy.

Yeah, there's a speakeasy. Yeah, yeah, yeah.

You go in the basement, there's like people smoking cigars right in your face. I'm not a smoker, but it was like great to have the authentic experience, you know? Yeah. Come on, just bring it on. 1913. It's— yeah, exactly.

Well, till next time, friends. Thank you. Bye, everybody. Bye-bye.

Bye-bye.