#38 - Legal Ops at Expedia: A Conversation with Mike Russell, Head of Global Legal Operations at Expedia Group

Hello everyone and welcome to Pearls on Gloves Off. I'm your host, Mary o'. Carroll. Today our guest is Mike Russell, who has spent over 30 years building teams and designing systems to optimize legal departments and currently serves as Head of Global Legal Operations for Expedia Group, where he's been since 2020. Prior to that, he established and led legal operations for Ingersoll Rand, now known as Trane Technologies, and before that spent over 15 years as director, Strategic Legal Technology for Liberty Mutual Insurance Enterprise Legal Services, transforming a corporate legal department of over 1800 professionals across 70 plus locations with over $1 billion in managed legal spend. Earlier in his career he focused on law firm infrastructure for document and knowledge management, time and billing, and telecommunications while serving as IT Director for the Spencer Fane and Wallace Saunders firms. Mike has been recognized as a law department champion in 2011 through an ILTA distinguished peer award, is actively involved with ILTA, the international legal Technology association, has served as faculty for the ACC Legal Ops Annual Symposium and Section Steering Committee. He also serves as a Pacific Northwest regional leader and DEIB council member. 4 o' clock today we get into a very hot topic. Metrics, KPIs and how do you measure value in legal. Now Mike has done a ton of thinking on this area and he even helped to write a book on this topic. We discuss why it's challenging to get to that nirvana of a GC dashboard with automated metrics because so many departments still have nothing today. What are the steps to get there? How do you know what to measure, what is an actionable metric, and how do you start to gather and report on the right set of data? We covered the idea of native reporting that comes with, let's say an E billing tool or a CLM versus the need to export that data into a BI warehouse and analytics tool in order to join it with data from other systems, your finance system, salesforce workday, et cetera. And finally, how does Legal Ops think about measuring its own impact? How can you think about measuring improvements in quality or reductions in risk? These are the million dollar questions that we try to tackle in our conversation today. Hey Mike, welcome. Thanks so much for being on the show today.

Thanks so much for having me. Glad to be here.

Wow, you have been in Legal Ops, legal tech, this world for a very long time. How many years since you've been in this area?

It's kind of crazy. I think Back to my first meeting of the an association in legal was in 1994. So that puts me right at 30 years this year.

Wow. You might be one of our most veteran of veterans in this field. Then in the beginning, you started at a law firm and you were kind of in knowledge management, tech roles. Is that correct?

Yeah, exactly. So back in the early mid-90s, everybody was making the jump to putting terminals on attorney's desk for the first time and figuring out that it was more than just word processing and the need for email and collaboration and all of that. So it kind of fell into Novell word perfect driven IT director role, which anybody who worked in Lob Prey, Y2K will remember those technologies and just kind of grew from there. But that was my early exposure to time and billing and the billable hour and UTBMS codes and all that fun stuff.

Wow. So you've seen probably more change in this space and specifically in technology than most of us, huh?

Yeah. And putting some of those early firms on the Internet for the first time was quite interesting because you had lawyers who would first of all tell you they didn't go to law school to learn how type. And here I am saying, go to the World Wide Web and use this wonderful thing called Westlaw. And it was really the Wild west in those days.

Wow, fascinating. We've come a long way now that I kind of know you were at a law firm and then eventually in house, both in operations and tech roles. A question I get asked all the time is, is it the same? Is legal operations the same at a law firm and is it the same in house? Curious what you think.

Yeah, it's interesting. I would say certainly over the last 10 or 15 years, it's grown to be a very close parallel, and we see so much of the same activities happening on both sides. But in the early days, from the law firm perspective, you did what the client wanted. You didn't ask questions, you didn't push back. Clients didn't ask how the sausage was made, just that it was good when the salvage was made and it caused the right and got them the right outcome. So it was really interesting moving into, I would say, the mid-2000s and certainly coming out of the first financial crisis. That's when I started to see a lot of collaboration. So suddenly we had project managers on both sides of the house, systems and technologies that would actually sort of talk to each other rather than just tossing some data over the fence. So it's been really interesting to see us really coming into alignment. And then obviously, with the rise of legal ops as a career path, suddenly there's a version of that in almost every legal entity that we See today.

I guess in my experience though, the law firm side was, at least when I was there was much more about profitability and utilization and realization rate, what's the right discount to be giving. And then when I came to the inside, I was really surprised because the goal was very different. It was all about more for less and better and faster and cheaper and efficiency, which really wasn't a word we heard a lot at the firm. We was that your experience and do you think that's changed kind of working in your more collaborative roles with law firms now?

Yeah, I would say very similar path. It was all about the partner at the end of the day, the partner check getting cut at the end of the year. That's what it was about, driving the firm while keeping the clients happy. But certainly as these concepts around of efficiency, process improvement, eliminating waste, those are still concepts I think that are alien to a lot of law firms unfortunately. But many are learning and are on a better path now. I would say the large global firms have obviously been there for some time and have a lot of really talented people doing that work already. You were right. As a small mid sized law firm, you would have these large corporate clients coming in to audit your work. They would look at how you leverage technology. And I specifically remember when we kind of started getting Windows 95 laptops out to people, they were very excited that you mean I can go to court and wait for my event to happen and actually be working on something else for someone else? And it was really exciting. And then the ability eventually to sort of be connected all the times everywhere became really important. But when those technologies first became accessible and even available at a reasonable budget, where you had 50 lawyers, a hundred lawyers, being equipped as road warriors as it were, that really made everybody more efficient and effective. But it wasn't until the client came along and started asking those questions did the light really go on and say, oh, this is what they're talking about, that we're able to handle this much more work because we're on a fixed fee or whatever the case may be.

How that was being measured prior to this recording. You of course sent me a book that you helped write. So if you want to show it on your screen right now, it. This thing is like a college textbook about legal KPIs. So admittedly I have not read it cover to cover because it is quite dense but really a valuable resource. So wanted to make sure we plug that. But tell me about that publication and how you got involved and your thoughts about legal metrics and KPIs generally, yeah.

So this is legal operations KPIs. You can go to legal operations KPIs.com and find it on Amazon, etc. And I had the great pleasure of collaborating with Maury Cabri and Julie Ruscha and we worked together over quite some time to really refine what people really wanted. That's certainly a common question I've gotten over the years is how do you capture data? What do you show to your gc? What is your board look for in legal data? And so many legal metrics tend to be spend driven or really lagging where they can be actionable. And then the answer is invariably that's nice, what are we going to do about it? What action are you going to take as a result? So the thought was to come up with a literal encyclopedia, many hundreds of pages as you said, with practical real life examples defining what are these things? How do you track and monitor them? And then what's particularly important and certainly to Maurice credit, how do you display them in a visual way? Because one of I think a very common legal ops challenge and certainly a law firm challenge in my time there was just getting rid of reports. People were tired of Piper in those days, PDFs and PowerPoints in these days, and trying to shift to something that was more click and mobile friendly. And I just want to know where am I at this matter, this case, this budget, my practice group, my team, there's so many commonalities there between firms and corporate legal departments. How do you show that in a visual way? So I think people will find in the book a lot of examples from folks like Edward Tufte and other experts on data visualization, how to take data metrics, make it more visual, make it more actionable, then you can go back and use whatever your BI tool is. If you're a tableau shop, Power BI or even Excel, you can do a lot of these things with the tools you already have. You don't need to invest in new technology to do it, but you do need access to your data and understand how to manipulate that data.

So many thoughts to dive into. I mean the first question I guess is are there a certain set of key performance indicators or metrics that you think are critically important that every department or every GC should be looking at? Or is that going to be different for everyone?

Yeah, I think some flavor of all the comments. Spend metrics. What have I been spending? What is it? What am I likely to be spending? How am I performing based on plan or budget? I Find a lot of finance departments, they like to compare months and quarters year over year. In the legal world, that's not always a great comparison because as we know there's massive spikes in work and it doesn't take into account court actions and litigation activities and things like that. So I'm really more interested and I find that finance leaders and chief legal officers are more interested in how are we performing based on how we thought we would do, based on a typical year, a typical quarter, all other factors being the same. No big major corporate activities or anything. Are we on plan or off plan? If we're off by how much and what's our countermeasure? What are we going to do about this little bit of red, shall we say? You're normally green, which is why I love making things visual. And then all of a sudden, oh, we were sued 57 more times last quarter than a year ago and that wasn't in plan. So what are we going to do about that? In terms of managing the litigation budget for it? Is the complexity of those cases such that we can kind of treat them as business as usual or do we see something scary in there that needs some immediate and higher level attention and we need to maybe repurpose some funds to that. But if you're not monitoring everything coming in the front door, which that might be another topic, we talk about legal front door. Right. As requests for legal services are coming in, are you assigning a complexity factor to that? Are you looking at how they compare to everything else you've done in the past and try to fit them into a bucket so you know the right way to resource them at the right cost to get whatever the desired outcome is?

Yeah. Oh gosh, we could go in so many directions. Mike, you just mentioned complexity factor, which I think is so important and really helps tell the story of what's actually going on as you can, as you said, to make it actionable. That is a tough one. How do you think about complexity? In the past I've thought about self reported complexity manager dictated levels of complexity actually like coming up with frameworks for what is a 1, 2, 3, 4, 5 in levels of complexity. How have you approached that and what's been successful?

Yeah, so I think part of it, when it's an internal request, you've got somebody out in the business making a request of legal typically and most generally the more complex parts of the business. So based on what department or division it's coming from, based on the matter or subject type, is this a scary complex data privacy issue or is this just a regulatory compliance, data privacy issue? So you can take an area of law and break it down into what's almost commodity work versus specialty work. And right there, oftentimes you can just apply a sort of an 8020 rule and figure out that 80% of the time we're getting the mostly routine stuff that often turns out to be a compliance exercise or regulatory exercise. We just need to implement and follow this new DSA or whatever is happening currently. Because that's what's coming at you all the time with litigation. I do find that, because especially companies that are serial litigants and they see the same types of work over and over, it becomes much easier because they have this massive history of what's gone on in the past. What were those outcomes? And then based on who's the plaintiff's attorney, where was the jurisdiction, what were the allegations in the case? Then you can really put together a plan or a matrix and put some automation around identifying that. And then if you like, you've got access to your dollars and cents spend data there as well. So then you can say, this is what this type of case has typically cost us in this jurisdiction, when sued by this plaintiff's attorney, you get all those things together and you can really put a nice box around it. And I think there's a number of sophisticated legal departments that are out there doing that in different industries. But if you're, if you're just starting out, my recommendation would be to think about what's considered commodity work. Go around on a listening tour. Ask your practice group leaders and your attorneys on the ground. What do you see day after day that frankly you're happy to do it, but it may not be the most exciting kind of work. Let's call that commodity work. And then let's talk about the meaty stuff. When that complex stuff comes in, how do you know it's complex? When do you know it's going to get to be complex? And then what do we do about it? And then try to start putting some rules in place and then eventually some automation.

Yeah, interesting. When I was at Google, we tried to think about complexity at some point and did some testing on having our attorney sort of self report as they were opening up the matter or requesting people were requesting help. What is this on a level of one to five, with five being the most complex and one being the most basic commodity work? Every everything was a four or five. Everything was always going to be a four or five. And so we took a step back and I actually got some great advice from someone and I wish I could remember who told me this. They said instead of using a scale from 1 to 5, increase it to 1 to 10, and what you will find is people will be a bit more accurate. Because even if they say it's a four, but it's really a two, you're directionally in the right place. They're not going to call it a 10 when it's a 2. Right. So at least getting the scale to 10 gave us directionally some more better indication about what the true complexity of something was. As people are getting started with metrics, I find one of the hardest things is you don't have a baseline. Right. If you don't have systems, if you don't have tools, if you haven't been measuring anything to date, what does the current state look like? And how do you start to gather that data and define it in a way such that you can have really good metrics one day? And, you know, I think about clm, obviously a lot. People come and say, we need to make a business case. We have to say the roi, we have to talk about how the turnaround time is going to be faster. And we say, what's the turnaround time today? No idea. We have no idea the volume, we have no idea the complexity. We have no idea what's coming in and how long it takes. So how do we make a business case? You know, what advice would you give to those folks who are starting from scratch and who have nothing? Like, what are the steps to kind of build up, to be able to do some of the things you're talking about?

Yeah, so when there's spend involved, probably there is data because somebody somewhere is keeping track of budgets. So you can always go to your finance department. And if you have no legal billing technology of any kind, somebody is presumably approving invoices and sending them for payment. So at minimum, there's a financial plan for the organization and somebody's been handling those invoices. So that's a great way to kind of start and figure out and sort through what you're probably already doing, just to figure out what firms am I using, what kind of work are they doing, and things like that. There's certainly a plethora of excellent consultants out there that can help people come in and understand the makeup of their organization. To your other point, when you've got business data, cycle times, how long do things take? Where do things sit? For how long? Sometimes it pays to just take a little time and do A little bit of a time motion study. A lot of folks in legal ops come from process improvement backgrounds and we sat there next to somebody and just shadowed them for half a day and say, tell me about your day. Where are your areas of friction? Where are your pain points? And when this work comes in, how much is there? Where does it come from? How often are you getting it? And then kind of what do you do with it? Are you compartmentalizing? Oh, you've got folders in Outlook, I've got the spreadsheet. I track my own work. You know, learn what people have in place for systems. Even if it's not an official platform or system of record, they're managing their work somehow. And you can then use that as a proxy for likely your organization wide current state, whether it's contracting or sales or procurement or whatever department you have to be dealing with. Some companies are fortunate. They may already have maybe an IT support or an HR support technology of some kind, so they can maybe piggyback on that and get access to frequently asked questions and then help build a little bit of business intelligence into that. But whatever you've got, go uncover all of the data that's hidden under those organizational rocks and try to find all the stove pipes that we all know exist and in most every organization.

I think systems are such an important part of getting to clean metrics and just being able to in a systematic and programmatic way define the data and then collect it almost automatically in the background rather than this manual process. What role do you think technology plays in the metrics and KPIs and creating sort of GC dashboard type things? Can you do it without systems? Are they pretty integral?

Yeah, I mean obviously you want to avoid as much manual work as possible and free people up to do more value added things. So once you get through that initial cycle of learning, this is how work gets done in my particular organization. And here's what I'm going to hone in on. Then you can leverage whatever tech may already be in place. A sales department probably is using a CRM of some kind. A procurement department is probably using a procure to pay or a PO management system of some kind of so you can really get a feel for, okay, maybe the contracts aren't in there and we don't necessarily know what's in the contracts, but we know we have a relationship with this supplier or this partner and here's the document that goes with that. Then you can start to implement some piece of tech that's going to help you Actually manage it. And a lot of times it can be a low lift. You can take a very simple work process, automate it and start to show data and then you can build your business case and then show that there is in fact ROI here. Because I'm closing deals faster, I'm reducing my cycle time, I'm keeping up with contracts that maybe are auto renewing and shouldn't be. The procurement side of the house, whatever it is. And then as always, back to spend management. Am I overpaying my law firms? Have we agreed to a discount that I'm not getting? How am I monitoring those things and enforcing billing guidelines and policies like that?

Yeah. And just thinking back, I mean, I think spend and contracts are probably the two biggest areas of focus when it comes to metrics, reporting and efficiency. Generally, as you think about low hanging fruit and where to start with the biggest impact in overheating. Calling a legal department. Let's talk about spend for a moment. When I started, sure you could go to finance and get some spend data, but it was such a black box in that it was. We knew how much we were spending with this firm or we knew how much we spent over the year on litigation. But how much does it cost on average to get to the first motion? Don't know. How much does ediscovery cost on average for this type of case? No idea. How much does it cost to file a patent? Don't know. Everything was kind of not broken out into individual units or parts of a case that we could then start to, like you said, not just report the weather of our spend overall has gone up or down year over year. It's what is actually happening, what are the stories within the patterns and what can we actually do about it? What are the actionable sort of things that you can do after looking at the trends and that I struggle to think that you could do that at all or very well without having an E billing tool or a system that is going to help you break things down into units such that whether it's in a platform or you take it out into a BI system of its own and you start manipulating the data to start rolling it up, rolling it down, seeing what, where the trends are. You really do need a system to be able to capture that data in a structured way and then to report and play with it. Is that sort of the journey as you start with you have something from finance, you got to figure out what you need and then put in systems.

Yeah, completely agree. Because if you don't have it broken down at a level that's usable, then either it becomes again a manual exercise or somebody has to go through and manually do that. And maybe that's okay once to prove the point, that's how much more detail we would have. But of course, that's why the lead standard and YouTube EMS codes exist. Just like medical coding. Just like any other industry that codifies work based on phase, task, activity and time. Sometimes we can't get away from the fact that we're still paying for work by time, but not necessarily. But at a minimum, I need to know where I am in this larger project. And if I'm in phase A, phase B, phase C, and within that the specific activities, then what level of person is doing that work? What's the cost of that person? How diverse is that person? All these little details start to come to light and, and it really requires an ELM type of system to do that. And then the same thing, of course, in the contract space, if I don't have good metadata and I don't have the technology to pull that metadata and tell me what's in these contracts, then I'm never going to be able to start bucketing and putting scope around each little piece of it.

And I always like to work backwards. I'm curious if that's also the way you think if you're going to embark on a CLM journey or an ELM or a spend management platform. There's a lot of work that goes into configuring it and all your processes correctly so that you're collecting the right data. It's collected in an automated way, it's collected in a consistent way such that you can actually report on it. And what you just rattled off were a bunch of questions that you want to answer, which is why you want to go and buy the software, right? And to have that data. I love starting with those questions because then you can start to work backwards to say, okay, if I want to measure, let's say, rates of one firm against the other, or turnaround time for a certain contract, or the volume of contract per individual working on it, what are the questions I need to ask? What is the data that is going to help me answer those questions? How do I get that data? How do I structure and define it? How do we create a process to then gather it and then what can the system do and how do I configure it? I mean, it's a long project and it's a long journey, which is why I always say the system is super important, but there's actually a ton of thought and prep work that goes into setting it up correctly. I always say the software should be the easy part. You just put that in place. But it's the thinking through how is this going to work? What are the processes, what is the data? How do we define it?

Yeah, I find that you get some key stakeholders in a room and ideally sketch out what is it you're trying to get out of this information. A lot of people will say we're data rich, but information poor. I've got a lot of data data, but it's not informing me to do anything about it other than like you said, tell the story of what happened last month, last quarter, last year. So get people in the room virtually or physically take some time to design something, use a, some simple whiteboarding type tool or an actual whiteboard and draw out what is it that I could see the delta from this to this and I had all these other supporting bits of metadata, then I could actually make a decision about the next steps based on that. Starting kind of from where you want to be. And it doesn't have to be sophisticated and super automated. It can be very simple. I think a lot of dashboards start out as smoke and mirrors where some magic happens between the data warehouses of the world and then the actual thing that's being presented. But over time you close that gap and you keep iterating to figure out what's the best way to get this data from point A to point B, making sure all of your many qualifiers are matched. It's interesting when you talked about just getting data from finance, just choosing the date with which finance looks at something on the legal side of the house. We spend a lot of time on work that was instructed during the year. So you're working by billing periods and things like that. Well, finance thinks about the day I get an invoice. I don't care the date of the invoice or when the work was done. That liability is now in my books. So I care about invoices by date received and date paid. But that doesn't really inform the full picture of when the legal work was done, how much legal work was done in a given point of time. That's one thing I've definitely learned over the years, is know your audience document and clearly state all these different qualifiers because you may have carved out things that are important or material to some areas, but not so much for others, and that there's logical reasons to do both. But that's all knowledge you, you gather during this Harmonization process, where you're for your example is learn from what people want to do. And you're sort of writing a song together and you want all this stuff to exist in harmony. And I have to know what all those notes are and what the rhythm is going to be before I try to assemble it and then actually perform it. So that's a recipe to kind of get people there, I think.

Yeah. And you mentioned dashboards, and I think about this textbook that you've basically helped create. Someone might pick that up and go, these are all good metrics. And they are. I mean, I flipped through it. I'm like, that's useful. That's useful. You could go through and create a dashboard or even a set of manual reports that would be 100 pages long of anything that's. And everything that's happening in your. In your department at every given time. And I actually just wrote a blog about metrics and one of my points was less is more. Having too much is actually overwhelming and doesn't tell the story. How do you balance that? I mean, you want to be able to monitor everything, but what do you actually show? Do you have a GC dashboard, so to speak, and what do you surface there? Is it the same thing every month? Do you actually curate a report that you present? Does that look different every month depending on what you want to highlight? I guess. What's the right approach there?

Yeah, I certainly agree. Less is more. And nobody wants to read hundreds of pages of anything or look at something that's so complex you have to zoom in with a magnifying glass to read it. And that's where I think the principles of just showing things visually really come into play. So using any one of many great books and examples that are out there, and even some of the software products are pretty good at suggesting, you sort of click on some data and it will suggest the best way to visualize that so that it's clear, simple, succinct. So I do agree that's really important. What I've done and what I find useful for finance leader dashboards, legal leadership dashboards, is after, you know, a few, if you're new to the role, new to the company, whatever. After a few meetings, you'll start to understand what's important to them, what they've been receiving in the past and still want to receive and make decisions based on, and then maybe think about how have they been consuming that data in the past to kind of make it better, more informative, make it quicker for them to understand it. But then more importantly, how do I help them understand what they don't know? And can we show that in a way that actually enlightens and hopefully delights people? Start very simple. Start very visual. Sometimes people will knock having a speedometer, gas gauge type of thing on a dashboard because it takes up a lot of space and doesn't show you that much. Maybe that's all you need to start and say, there's our current speed, whatever that is, speed of incoming lawsuits or money going out the door to law firms, whatever it may be. But start very simple. And then you can start to refine that over time and then figure out, oh, I see that you're really interested in what I said earlier about the plan for the year, the plan for the quarter, and how are we doing based on that? Not so much how we did last year is great. That's nice, nice data points. But maybe that's in an appendix and that doesn't show up on the live dashboard. They can get to it if they need it. But I really care about where are we according to plan and if we're not according to plan, is there a place I can double click and get some more information? And either I've shown that in red or I've got another little call out to the side or something that says alert, you should know about this. And it's either a spend issue, a time issue, a compliance issue that we're at risk of not being in compliance with something as a result of this much time gone by. So any way you can think of to illustrate that, I think will mean more to the people consuming it.

So are you sort of sitting down with your leaders on a regular basis and walking them through the metrics and kind of showing there's a trend here? Don't know why, but if we double click, oh, we're starting to see what's happening underneath the surface. So you're walking them through rather than just giving them a dashboard and hoping that they're curious and poke around and can figure out what's happening on their own, Is that correct?

Exactly, yeah. If you can get an audience with your leadership team, at least in my opinion, at least quarterly, if not monthly. I do monthly. And sit down and have our finance partner in the room, have the CLO senior leaders in the room and just walk through it. Sometimes it's a 10 or 15 minute meeting, everything's on track, great, no problem, not a big deal. But then if you've already discovered something that they need to be aware of, then do Your homework, come prepared with an explanation. Sometimes things can be off track and it's not necessarily bad as long as it can be explained. I think that's the worst situation to be in is where something has occurred or is occurring and you don't know why, but you need to report that out to leadership. So do that deep dive early on and figure out what's happening and be able to quantify it in some way. And it may be you have to go all the way back to a law firm or a service provider and say this bill came in late and this is for work you did six months ago. And you know that all of that billing hygiene stuff aside, get to the root cause of this unplanned or unexpected spike in cost, or if something is taking too long. That's another great one is if I'm on a normal cadence and all of a sudden I see a big gap where I'm not getting the right activity. I'm not seeing the same frequency of work coming through, whether that's contracts or whatever it might happen to be. Dig into that and figure out what's going on, that something happened and this is the causal effect of that. And if it requires action, maybe have a recommendation that here's two or three possible things we might do from there. But I try to ask leading questions and prompt people into suggesting where they want to go next and then we'll go there and get the data to do it.

How much are you relying on the modules, reporting modules that are already in, say your E Billing or your CLM system versus building your own reports outside using a tableau or some other BI tool?

Yeah, that's always a tough question because invariably you're dealing with sources of corporate data that doesn't exist or isn't nicely fitting with your legal systems of record. Many legal platforms today have excellent awesome business intelligence capabilities and reporting built in. But rarely can I depend on any one of those kind of point solutions to paint and deliver the entire picture. There are certain things I can know about contracts. I think CLMs in general are getting better at it. Elms have been there for a while. It's certainly not news that you can produce a dashboard out of any ELM product today. But those things probably don't have the input from your corporate finance system, the ERP side of the house, the procure pay side of the house, all these other factors. And then I mentioned earlier just organizational, structural kind of things that any data from workday to properly break down, spend by person, by division, and there may be geographic or other issues at play. And our legal systems of record rarely know all of those different factors. As much as I love would love a silver bullet, I think most of us are still using traditional BI tools and then a data warehouse or data lake to make it happen.

Yeah. So that's really key. And I, in my experience that's it's a big difference between a smaller department, a smaller company that is able to sort of use the out of the box reports that come from your E billing system or your contract system. And those are actually very useful for them when you get to be a larger enterprise. And just as you say you want to bring together the information from workday and your finance Oracle systems and your E billing systems and match that with your litigation systems, it starts to be much more complicated and it almost is. The first question you ask when you are buying software is how easy is it to extract the data into my data lake and be able to report on it myself? I think that is a really big consideration for companies that want to do sort of GC dashboards. That or beyond just one practice area, how do you think about that uniformity of data? I mean that's a big challenge. Right. Because your systems have to not necessarily talk to each other as they're operating, but as you extract the data in order to be able to report consistently, as you say, let's say your, your product map or the way that you're defining different fields or practice groups or accounts in your accounting system versus your billing system. They have to be aligned, I guess. Is that another big project that you have to think about as you sort of embark upon metrics for your department?

Yeah, I think it goes back to that harmonization that the concept of understanding where things kind of fit and there's always a temptation to recreate the entire corporate or organizational structure in some flavor in every little system. The problem with that is every organization goes through change and the minute you get everything perfect, then all of a sudden, oh, we've rebranded, there's a new name for this division or we've acquired this other company or we have something happens at the corporate level multiple times throughout a year. In my experience in most large orgs and if you've over complicated or over designed it in any one solution, then you're constantly playing catch up. You're always reacting to what's happening with the org. So I'd rather see and I think vendors and the legal tech space are getting there where they need to understand that change happens. We live with change in Every organization. And if I can help people understand, give me enough metadata, give me enough fields, but then I need tools when I have to go through some kind of bulk update or change that. Everything that used to be X is now Y. But I don't want to just change it because I lose all that history. So I need a tool to kind of link everything together. And I'm seeing a lot of improvement in a lot of legal tech capabilities that used to be, we would become accustomed to, you know, it scripting exercise or you have to reach out to a consultant to do a bunch of fairness work behind the scenes. Now that's becoming more, more agile, more no code, low code. And a legal Ops team that maybe doesn't have developer resources or consultants on hand could do it themselves, which is great. But if you over over complicate it, you know, that can definitely be a problem.

Another question I wanted to ask you is just about legal ops as a function. One thing that I think is so important for all of us in these roles is managing up and demonstrating your impact, being able to show metrics and impact to the bottom line of the work that we're doing in Legal Ops. And so every initiative that you take on, every sort of process improvement that you do, you've got to think about how do you measure the success of that and how do you report on that? Is that something that you're also thinking about in your day to day and how do you sort of showcase the success of your team and what does it mean to be successful? What metrics are important to show?

Yeah, I mean there's a few ways to unpack that. Many organizations have a goal cascade process where you've got enterprise level goals, divisional goals and so forth, and those cascade down and where corporate functions plug in and support those specific goals, then that's an easy way to recognize and celebrate wins. Because you can directly align something that you've done or that you've supported, which is also great for career growth. I want the most junior person in the legal department to understand how their work translates to this large enterprise goal at the end of the year. And when you get the opportunity to celebrate those successes and maybe share with a broader high visibility audience, that gets people really excited. So it's a pretty easy sell. The harder stuff is where it's not directly obvious, maybe something that you're doing from the corporate side of the house that isn't directly related to its sales or revenue generating activity, but it's something that has to be done due to regulatory Compliance, government issues, just keeping the lights on, running the business. Nobody likes to talk about that work, but it's completely necessary. And the more complicated the world gets to do business in, then the more of that kind of work we see. You know, that's something that I found. I think CLM is a great example. If I can demonstrate that. You may not yet understand how CLM will eventually make you faster, better and ultimately more profitable, but surely you can understand that if my contracts aren't compliant, I'm creating risk so that when this agreement potentially is disputed, then that's X millions of dollars at risk because I wasn't tracking the end date properly, because I didn't understand terms and conditions, or because some group that should have approved some language wasn't looped in at the right time. All the things that caused contracts to be non compliant and therefore disputed. In theory, if you build solid templates and playbooks, then you can enforce that and the value of the non compliant contracts. You know what that is? That's a very compelling business case and particularly because most audit functions report into the cfo. So you make both sides of the house happy by saying, look, I'm going to reduce our audit risk and keep these folks happy while enabling these revenue generating activities so that those kinds of ROI calculations are a little bit harder. Because sometimes it's work avoidance. We're doing this to reduce risk and avoid and it applies to litigation and other things too. If I handle this work better, faster, more efficiently, then I'm protecting the company, reducing risk, but I spent less money doing it. So then how do you illustrate that? So I think being able to tell that story, and particularly if you can build on past, past history, if you know something like this, similar has happened before. How long did it take, how long did it, what did it cost, how did I resource it and what was the outcome and can I do better the next time? So it's all that institutional knowledge that helps really sell this roi, I think.

Yeah. The two hardest things I think to measure are quality and risk. And those are things that I think our teams work really hard to improve quality and to reduce risk. But how do we show that and how do you measure those things is hard.

Yeah, you can do sample size. If I do a thousand things a year and I look at a certain percentage of those and I can demonstrate that these were accurate, they were compliant, they were. Whatever your measure of success is because of the automation, then that makes for a great use case. Because if it's just the wild west and People are emailing stuff over here and doing that and hopefully everything winds up in the right place. If you took a sample of those, your percentage of defects would be much higher. And that kind of data, I think helps answer that question.

Yeah. Are there any emerging, emerging trends, changes in our industry that you're seeing now with respect to metrics, KPIs impact that you find interesting or promising?

Yeah, I think the evolution of tools that will, in a more sort of natural language way, allow you to ask questions of your data. In other words, you've got all this information out there and all of its data nerds are very skilled at writing queries and pulling data and assembling tables and doing all that. What if I could just, hey, computer, tell me how much I spent last year with this firm on labor matters in France. I know how to answer that question, but most people in the department don't. So that's the kind of empowerment that I want. I want true self service. Don't just tell me what you want on the dashboard, let the machine give you the answer. So as we see more and more of those tools that are trained on and understand your data and only your data, and you're keeping your data safe, obviously it'll be able to do that. And I think we're just on the cusp of seeing that in a lot of tools and more general office productivity. Clearly there's some good stuff already out there. So as those get applied to the legal business world, I think that'll be excellent. And then creating, if you think about what people are doing now with AI generated graphics and images, I want that for my data. Right. So create some type of visual reference based on this information or this question you've answered and present it in a way that I would have never thought to present. But something about the data lends itself to this visual. I want it to tell me what that is.

Yeah, these new AI agents that can combine sort of the large language model as well as the ability to use tools like you said, to put things into a Graph or a PowerPoint type presentation and you just being able to, in plain English, instruct it, I think is fascinating. Within Ironclad, you've probably seen some of the AI that we have put out. I don't know if you've had a chance to experiment with what we call kai. It is AI that is sitting on top of basically the entire repository, but also what's in flight in your clm and you can interact with it, just almost talking to a human like and because it is an agent, it can create graphs. So you can ask it things like, what's Mike Russell's contract turnaround time for this type over the last, let's say, 12 months. Put that in a graph and show me if it's been going up or down and it will do that for you. And what I found fascinating. So everyone, this is so new. We don't know how people are going to use it. We don't know all the things that it will be able to do. And one of the reactions of a legal ops person that I was talking to, and this is why I love Community, because that's where all the creativity comes from. She took one quick look and said, wow, so you mean I can stop doing dashboards and reporting? I was like, what do you mean? Because now it's real time. You can actually get exactly what you want. That's an interesting, you know, trend. What happened in June? Show me all the individual contracts or oh, well, show me the ones that were over this amount in value. Showed me which ones took the longest. And then show me which clause was actually taking the longest. And you can start to dig into what's happening. You don't have to spend months and months creating those manual reports like you said that we were doing years ago. So maybe one day we don't have to do dashboards or reports at all. And you just have people interacting and asking questions that they want answers to. And it could do that. I don't know. I mean, I think that's really fun to think about.

Yeah, yeah, absolutely. And the fact that it can learn the Persona of the person asking the question. So the how it answers the question to the clo, the cfo is different than how it answers to the practice group manager or just a line attorney that's talking about one matter, one book of business with a certain law firm, maybe. So it understands that and in theory can automatically apply tranching and just different rules and conditions based on what it knows about the data, who's asking the nature of the question, and things like that. And then of course, you prompt it correctly and it learns and goes from there.

Yeah, but of course you do need a system and you need structured data and you have to have all your ducks in a row, right? All your stuff in one place to make sure it's pulling from everything. So there's a lot of pre work that goes into it, which I don't want to sort of diminish because that's a lot of work. But when you get it set up correctly. What you can do with it is truly very promising.

Yeah, yeah, it really is. And that's even in the early days, I remember experimenting with using some ediscovery tools back when kind of contextual clustering first came out and all this, we spent all this time ocring and we had everything machine readable. So why not put a bunch of new lawsuit complaints in there and have it start to group by, you know, what complaint is what. So back to the complexity factor. That's exactly how we did that, was we took a look at real two or three thousand examples of real world and bucket them based on what we know. And then I just built a little calculator, a simple algorithm that said if this and this, that's high complexity, this and this, that's low complexity. So again, kind of smoke and mirrors, right? So the human capability to use the tool has been there. Now I think we need to relinquish the human capability and let the AI digest and serve us.

Yeah, that's a great point. Even about the complexity rating. That should be one day automated, maybe sooner than we think. So that's really interesting.

Absolutely.

Mike, any words of wisdom to those who are starting their KPIs and metrics journey and don't have a baseline yet? I guess it can be very intimidating how much work there is to do to get to sort of this great clean automated dashboard nirvana, lessons learned, guiding advice, things missteps that you've taken that you want to just make sure folks know how to get started.

Yeah, definitely take advantage of all the many great references out there. So whether that's legalapps, KPI's.com or any others do that get involved with the ecosystem and the community. Obviously there is a ton of wonderful resources and people like us that have been around a while and are more than happy to jump on a call and just show somebody, here's what I'm doing. Does that help you in some way that to me that the spirit of community is really what this is all about. But then take time to go on that listening tour and understand. I think you're in the best position. If you're in a relatively small or less complicated organization that has never had a legal ops function before. So maybe you're a small legal department, you've just come in and you're starting to understand how is this department going to run and function well, then you've got benefit of being able to build that with the ground up, build the relationship with the leaders and the department. Other functional Leaders. If you've stepped into a large, complex organization, which has been my career story over the last 25 years, in the case of corporate, then you have to accept what you can't change and understand that you have to be a good corporate citizen and work within the rules and regulations that are currently in place. But start to make friends. Reach out to your IT counterparts, your finance counterparts, or somebody like you doing operational work in every function of every corporation in the world. So find your internal peers as well as your external peers. And oftentimes our work is a lot about getting work done through someone else. So someone that doesn't work in the legal department, doesn't report to you, doesn't have any particular success factor around doing you a favor. However, if you can demonstrate that by working together, you're bringing value, at the end of the day, it's going to result in better value to the company, better data for both departments, then they'll probably help you out. And that's certainly one of the ways I got started back before we had a lot of the tools and technology and I was just begging for data. You know, give me access to this. Why do you get access to that? Help them understand the why behind your ask and I think they'll be more likely to help you out.

Yeah, it's so true. IT and finance and HR should be great partners in this and they have the same incentives to get their hands on that data and to show their impact as well. Great to know. Mike, always wonderful to chat with you. Thank you for sharing so much wisdom and knowledge on a topic that I think there are so many questions about out there. So hopefully this helped clarify some things and maybe raise more questions for others. But good to get the conversation started. Thank you so much.

Absolutely. Thanks for having me. Find me on LinkedIn. If anybody with questions out there, I'm happy to chat. Thanks so much.