How AI Tools Are Transforming the Legal Profession with Cai GoGwilt and Alex Su at Ironclad

Foreign.

Hello, everyone, and welcome to Pearls on Gloves Off. I'm your host, Mary o', Carroll, and today I thought we'd do something a little bit different. Today I have with me Kai Gogwilt, Ironclad's technical co founder, chief architect, and our resident AI expert. I also have with me Alex Hsu, our legal influencer and expert commentator. When it comes to everything that's happening in the legal industry, in so many of the conversations I've had with people these days, the same topic keeps coming up over and over, and you guessed it, it's generative AI and its impact on the legal profession. In addition, people keep asking how to stay on top of all these developments that seem to be happening at breakneck pace, with news coming out literally every day. And my answer to that is that, sure, I follow some great people on LinkedIn and Twitter. I read a lot, I watch videos, listen to some podcasts, but most importantly, I have a regular check in with Kai, who has really enlightened me and is really on top of what's happened, what's happening, where things are going. And so with that, I thought it might be useful to bring you all into this conversation with both me and Kai, and inviting the always entertaining Alex Hsu to be our host for today. So, guys, it's really fun to have you guys here.

So excited to be here. Mary, I'm so excited to have Kai share a little bit about AI and technology. But, Kai, maybe before we start, tell us about yourself, because I know you were at Palantir and when you met Jason Baymig, when you started Ironclad, you went through Y Combinator with Sam Altman when he was the president there. He has a household name now with his work at OpenAI. But can you share a little bit about your background and how you were able to build a relationship with Sam and OpenAI?

Yeah, thanks for having me on. Super excited to be on this. So, yeah, I'm Kai. I'm the chief architect and co founder. I have a software engineering background. But maybe just to fast forward straight to Y Combinator and first meeting Sam. So Sam was actually the president of YC when we went through YC in summer of 2015, and it was only later that he became the CEO of OpenAI. So we actually started our relationship with Sam because he was the, we'll say, like Paul Graham figure of YC at the time. So we got to meet with him once and he just listened to us talk about what we wanted to do with Ironclad and gave US advice. So it was a very interesting moment for us in the summer of 2015. And then I was actually going through my emails before this to look at all the correspondences I had with Sam. And there is an interesting one from a few years ago that was so GPT2 was great, but we're working on something really cool. You guys should check it out. It's been a great relationship over the past couple of years. OpenAI has been a customer for a long time now. They've been an early adopter of some of the coolest ironclad technology. They're coming out from the workflow designer to the editor, and of course now the AI functionality. It's been a great relationship. Working with Sam and the entire team at OpenAI is fantastic.

I love it. And chatgpt3 when it came out, I mean, everyone in the legal community was talking about it. And so I wanted to ask you because you've been thinking about these challenges and technology for a while. A lot of people seem to be concerned what all of this means for legal work. And I'll just quote you some people on social media before I jumped on this podcast, I told them that I'd be interviewing you and they had some questions. Some of the questions were around how professional life will change for a legal professional. What happens if AI becomes quote, unquote.

Smarter than a person?

And just generally, will it lock out lawyers and legal professionals? So what are your thoughts on that?

Yeah, I think we're quite a ways away from that. Yeah, GPT4 is really fantastic. It is head and shoulders above where GPT3 is in terms of some of the really critical strategic tasks and reasoning and thinking. But if you read the headlines that I believe and that one I read are, there's a Microsoft paper called glimmers of AGI, glimmers of artificial general intelligence. And it was studying GPT4 and basically saying, look at all these amazing things it can do. It can, like, use tools, it can do chain of thought, reasoning, it can do translation, all these like really cool emergent behaviors. But it was called Glimmers of Artificial General Intelligence, not AGI is here. And so I think we're ways away. I think right now we're all learning how to use this technology to make ourselves better. And I think we're a long ways away from maximizing the potential of GPT4. But we're not in a place, we're not at AGI yet. Sam would be the first to say.

That basic question for you. If we back all the Way up. I mean, we've had AI right, and legal for a long time. We've been talking about AI and legal for the better part of the last decade, but obviously there's been a mega shift with GPT. What is so different about it? Why is this so different than the conversations we were having about AI before GPT came out, or generative AI?

Yeah, this technology works differently from anything I've ever interacted with. And I think that's part of what's happening here. The thing that gets me super excited is a lot of the behaviors that we're finding very incredible here are emergent behaviors, behaviors that this AI was not really trained to do. If you think about Ted Sanders talk at Ironclad Live just a little while ago, he was talking about how the way GPT is trained is to predict the next token, the next set of words that come after a previous set of words. And this is why there was someone at Facebook meta who dismissed GPT 4 out of hand by saying, and I'm sure his words were taking out of context, but it's very advanced autocomplete. And that's because it's incredibly advanced autocomplete. But in having incredibly advanced autocomplete, there are all of these behaviors that are so much more than just autocomplete. When we think of autocomplete, we think of maybe if you use Gmail, the thing that just completes your emails. Right, that's great. But what we're seeing it do is like translating from English to Italian and doing it quite well. We're also seeing it through chain of thought reasoning where it actually goes through step by step and actually thinks about its answer. One trick that the community is, the AI community is figuring out is if you ask GPT4 a question or even GPT3 a question, if you ask it for its reasoning, it'll actually do incrementally better in terms of answering that question accurately. So it's very advanced autocomplete. It's incredibly exciting because this stuff shouldn't happen with autocomplete, but it is and it's here and we can use it for immense good.

I really like the point you made about when you ask it for reasoning, it does better. And I think that's part of why the legal community is so fascinated by this, because that's what you do in law school. You read issues and you try to provide reasoning for your conclusions. And I think that when the legal community encountered ChatGPT, everyone got real excited because it was crazy. But Also, there were a lot of people with concerns about ChatGPT making stuff up. And I think that's called hallucinations. Can you speak to that? And how do we think about that in AI?

Yeah, no, you got it. The term art is hallucination, which I love. Yeah. And what I like to say about hallucination and defining it is hallucination is when you ask the AI question and it gives you a very convincing answer that is completely unsubstantiated by the facts. And the thing I like to point out after that is hallucination is actually a very familiar problem I'm sure we all know. Yeah. Specialty hallucination is a broad term. It covers a lot of things. There's not really, like a way to 100% fix it. But what I think we're realizing is, just as with humans, you can't fix, like, humans hallucinating, but you can provide humans, like, the tools and the resources to reduce hallucination. So, Alex, if I asked you a question and you were the expert on this topic, supposedly, and I. Yeah, and I put you on the spot, you might feel pressured to just give a random answer. And I think that's what we're doing with ChatGPT is we're asking it a question, expecting it to be the expert, and it's giving us a very authoritative answer. But if I do the same thing, and Alex, I ask you a question, but I also give you time to research the Internet and maybe like a proprietary library of some sort, and gave you the time to actually give me an answer, I think I would trust that you wouldn't be hallucinating that answer a lot more. And so I think that's what we're starting to see in the AI community of how people are using GPT. They're actually giving it tools, they're giving it browsing access to the Internet, they're giving it finding different ways to give it the right context to give not just a convincing answer, but hopefully the right answer.

I really like when you said giving it context, because I think that makes all the difference. And I know much of what people, when they're responding to AI, they're talking about ChatGPT, the free thing you can sign up for and use for free. Tell us a little bit about how ChatGPT is different than what we're doing at Ironclad with AI.

Yeah, Ironclad AI and ChatGPT are very different, even though they're based on the same fundamental technology. I think of ChatGPT as the raw incarnation of this technology. And it's fascinating that hundreds of millions of people are finding this raw incarnation of the technology really compelling. And I think it's a really promising signal that there's a ton of substance there that, like, this hype that we're experiencing, this hype wave around generative AI is only going to increase because there's just so much substance. What we're doing with Ironclad AI is we're actually taking basically some of the insights of where we found ChatGPT and its underlying technology to be really useful and applying another couple of levels on top of that and pulling in different tools. So the things that you can do with AI assist and AI playbooks, you just can't replicate that in ChatGPT without an immense amount of work. So I think the key part here is actually context. Ironclad, the platform has so much context about your organization's negotiation parameters and the obligations that you've agreed to over the past 10 years, and they can take all of that context and push it into, like, context window of GPT so that you get an answer that's correct, so that you get suggestions that actually make sense. You could do this with ChatGPT, I suppose, data rights be damned. But you'd have to do a lot of manual work to search all that information out, stuff it into the context window, somehow encode your organizational negotiation parameters and stuff that into the context window. And ironclad does all of that for you. And we've also actually tested it with end users so that we know that we're getting something useful there.

I always think of it as ChatGPT is, like me when I was 15, reasonably smart, but have no idea what's happening in the world and have no context, no experience. And then as you get older, I went to law school, I practiced, and so I have a little bit of understanding of how things work. And so ChatGPT, which is what everyone is exposed to, operates a certain way without that context. And then people start generalizing from that. Oh, AI is just going to make up anything. And I think you do have to be cautious about it. Right. Because you don't want to put in confidential information into ChatGPT or you can't fully rely on it if it makes stuff up. But when it's combined with the context of something like ironclad or with all of this legacy contract information, that could be pretty powerful.

Absolutely.

That's really interesting. I actually hadn't thought of. I mean, I think that's how lawyers are actually using AI and GPT right now in their day to day, right. They're dabbling with it, using ChatGPT as an open window and then like probably pasting in stuff from their general work. And that's why Alex, you and I have talked about all these GDCs we talked to are freaking out because they have to put together some acceptable use guidelines around how to use AI in the context of their work. But that makes a lot of sense now when you say context, like having it all in a secure data defined, end to end system where then the AI can play safe, that's really helpful.

I really like context. I've been reading a lot of AI related articles. There was one in the New York Times recently and all it did was talk about, I think the title was AIs coming for lawyers Again. And the entire article is about law firms. And I think what we don't realize is that a huge amount of work is done by corporate legal departments and that's who we help at Ironclad. Kai, I was wondering if you could share what you've seen in terms of how AI can help a corporate legal department versus a lot of what we hear about how AI can potentially help law firms.

There's so many ways AI can help corporate legal departments. And now I'm going to tangent from the question you asked, but I actually think that corporate legal departments, people in corporate legal departments may actually be developing more relevant skills for this next wave of generative AI because they're interfacing with non lawyers. You were talking about like how AI is sort of like you when you were 15. Yeah, I agree with that. And what's really valuable about working with someone who's. You can't work with anyone who's 15, I guess. But when working with someone who doesn't have experience in the subject matter is, it becomes really valuable to be able to explain things to them, to frame questions in a way that is understandable and approachable, to maybe break down the tasks a little bit. And so I actually think that corporate legal departments, people in corporate legal departments have to do this more than associates or partners at law firms. Right. Because the partners of the law firms are probably mostly interacting with like gcs, maybe C suite executives and things like that. But they have this like translation layer of okay, cool, the GC understands what I'm saying now. They can translate into mere moral speech.

Yeah, I would make a joke that a lot of these folks and law firms are disconnected from reality to some extent. Yes, they're very good at the specialized areas of law. But the corporate legal department is so tied to the business, which often you can't live in a fictional world. You have to see what's needed. You have to balance accuracy with speed. And I know, Mary, that's something that you have a lot of experience with too.

Well, we've talked, the three of us, a lot about the fact that in the corporate legal department, everyone is swamped. Everyone is just constantly telling us how buried they are. There's too much work to be done. And you're a cost center that by nature, you can't deny that you're a cost center. So your incentive is always to get more done faster. Like, how can you maintain the quality of work that you do but get through your pile of stuff faster? And that's a very different model than the law firm. So we'll put the law firms aside for this moment. But if we focus on the legal departments, they've always been much more open to technology because it is a lever that they can use to do more and to do more of the stuff that they want to work on and the right things that require their knowledge and their brains and to get rid of some of the routine stuff. And the AI that has emerged in the legal practice, at least so far, is really interesting because it is a lot of the mundane stuff, but it's pieces of it that we didn't really think could be automated or put away. And the AI is showing us how easy it is to do that, how reliable it is to do that. And we've also talked internally about, wow. Like, that wow factor that you have after using it is, oh, this just works. I'm gonna just use it every day because why would I do it the old way? And when that starts to happen, the expectations rise because you start to think, well, then why am I doing this other piece of my job this old way or this other thing? And you start to look for new use cases and different ways that you can leverage AI and technology. That pace is going to accelerate faster and faster. And it's hard to even keep up with all the developments, not only in the technology background, but all the use cases that we're going to hear about. It's just mind boggling.

Yeah, no, I think it's really cool because if we're talking about ChatGPT as the raw technology, I think the way people are using it is something that everyone can learn from. And in the past, AI technology has been siloed in terms of who can actually work with it. It's data scientists, ML, PhDs, not engineers are happy.

Like, yeah, this is something you just log into. I mean chatgpt, like my mom figured it out. Anyone can figure this out.

No, I'm literally learning like from our customers and our users. There was a thread on the community forum just recently where they were talking about will we have to do this? Kind of. They're talking about workflow designer and the launch form. Will we need to do this launch form stuff with generative AI? Actually, yeah. That's the whole thing about being able to communicate clearly with your business stakeholders is actually a really useful skill in terms of communicating clearly with GPT4 and the AI. So doing that actually clarity and being able to explain in the same language as mere mortals is going to be incredibly valuable.

I think that's super interesting because there's so much law that it's almost gatekeeping of content and information and this becomes more open and I think we're talking about workflows and you mentioned how AI.

Could help with workflows.

I think this is a good point to maybe talk about specifically, okay, if you're an in house legal department, how would you tell them where AI can help? Because I think a lot of people read things online like I do and it's very vague and maybe AI could help you summarize this or that. But specifically where have we helped customers when it comes to AI and contracts?

Yeah. So the first application area that we started applying AI to was pulling out structured data from completed contracts is smart import. And then more recently we've applied it to the contract negotiation plays. In both of these contexts, we're applying AI to what I think of as the underbrush of contracting. I think I heard you say that first, Mary. But yeah, if you look at out where AI Playbooks is saving a ton of time, it's by taking these frequently negotiated positions, encoding them into AI positions and then allowing the AI to automatically fix them. Things like governing law, things, cap on liability, et cetera. That leaves more time for the underwater legal team to actually focus on the specifics of the negotiation. Kind of you're negotiating with someone who's currently a partner but maybe is moving into a different territory, might align themselves with one of your competitors or something like that. That's the context that the legal team should be really focusing on when drafting the agreement, not is the governing law in New York or Delaware.

I know, let's think of an example anecdote where you have a new in house or commercial council saying okay, we got to negotiate this New contract. I have no idea what we up to in the past in the company. Then they've got to dig around, they've got to ask around. It sounds like AI is automating, finding what we've done in the past and how should we deal with this so that the commercial council or in house counsel can think at a second level where they're thinking, okay, what implications are there of doing it this way versus that way? What business implications specifically, and where it interacts with legal. So is it something like that where the AI is doing the retrieving? Here's what we've done in the past and here's how we recommend or we should tell you, these are the different ways that you can negotiate the contract.

That's not how AI assistant AI playbooks work today. But that is the great product idea. No, but actually we're in a stage now where you might have told me that idea, like even a year ago. And I would have said, yeah, we could probably do that. It would probably take our AI platform team a couple of months, maybe a year to set up the foundation for that. And then maybe like we have to collect training data and then the end user be able to see something two years down the line. And now it's the kind of thing where you're. Could the AI retrieve past precedent from my existing, like contract negotiations and pull it in and let's try building that this week.

I mean, it's so crazy because I think you showed me a couple of things we were working on and it just. The pace of this is so fast, right, Mary?

Yeah, so fast. But what it can do today, so the smart import piece, essentially, you could take like boxes and boxes of paper or PDFs that you have, shove it into the system. Untrained. I could do this, right? Someone who has no tech background, shove it into the system and it will do its magic and tag it all for you with what, almost 200 different metadata tags. Whereas someone would have had to do that in the past. Like we actually had a job when I was at Google where someone had to scan documents like that was their job, and then tag them to make sure that the data was clean. So that job goes away today. But I'm pretty sure no one in the world is like wishing that they want that job. So that's like one piece of it. And then the lawyer piece, where you're negotiating, it's going to flag the stuff that is in clear violation of what you would normally agree to. And then it's going to Suggest language that you would agree to and you can just plop, plop and say, yes, okay, put that in there and we're done. Again, like that is the mundane part of the contract that you have to think through and put your guardrails in place and create your playbooks in what's acceptable language. But once you're done, that's rinse and repeat type work that I don't think any commercial council is sitting around going, I'm going to miss that. So it shouldn't be too scary. I guess that's my point. There's stuff happening today that you can use, not in 10 years, but right now that's going to help you do your job faster. Is it going to help you do your job better? And it's not scary. It is not taking away jobs. And maybe we can talk about the fact that all this is today creating so many jobs. Like, yes, it's going to take away a lot of the work that we do, all of us. Doesn't matter if you're a lawyer, what job you're in, but my God, the number of regulators and IP lawyers and data professionals, it's exploding as we speak.

Yeah, no, I think it's. We're in a very magical moment right now. This technology, truly, we don't understand the boundaries of it. And in part because of the ChatGPT phenomenon, it's now in the hands of many humans. I almost said every human on earth, but that's clearly not true. But there are so many more people who have access to this technology, so many more people who can find new ways to use it. And with the Internet, just a very fast way to tell the rest of the world, I found this really useful thing you can do. So we're in a really exciting moment. I think I've been watching the IronCloud community working with our beta customers and things because in part, truly, they are just giving us ideas. I'm starting to try to use ChatGPT in my day to day for the Ironclad Live event. I was actually interacting with ChatGPT to kind of give me suggestions on how to improve the hook to the talk that I gave. Because John was on the West Coast, I was in New York at the time. We had only like a small window where we could collaborate. And so I collaborated with ChatGPT the same way I would have collaborated with John. You know, it's like, here's the hook I'm thinking about, how would you improve it? And ChatGPT was, well, for a hook, you probably want it to be a little bit more active, transmit some more excitement, communicate more excitement. Here's. I rewrote it for you. Here you go.

Wow.

And I was like, well, John and I have been talking about these two metaphors. How might you work these metaphors into the hook? It did that. And so that's so funny, because what.

I did last night was I've been working on this presentation for a talk at Clock the conference, and I needed to come up with a clever title and I didn't have very much time. I was like, I don't know how to do this. And so I said, I put in the content of the presentation and said, hey, ChatGPT, give me 10 examples of titles that are witty and clever. And it gave me 10. Not all 10 were witty and clever, but a couple of them were. And it helped me basically figure out the title in a minute or two. And that was super cool. And I'm thinking, my head is spinning about what else we could do with this. But one thing that I was thinking about was some parts of my work are very much it's marketing, it's public facing. One concern that I keep hearing about, and I'd love to get both of your thoughts on this, is legal professionals are not often doing this type of work, but they have to do with dealing with very confidential, very private information, sometimes privileged information between attorney and client. So can you speak to how we want to think about confidentiality in the AI context?

Yeah, great question. I guess the first thing is don't put confidential information in ChatGPT, because the ChatGPT platform that OpenAI provides in, a big part of why they're doing that is because they want to collect that information to improve the AI. And there's no judgment there. I think that's great and I still think it's really useful. I just think we all need to be careful about not putting confidential information in there. But if you use the GPT system within Ironclad, within Ironclad's offering, Ironclad has a Do not train provision with OpenAI. So none of that data that is sent into ironclad AI, that makes it its way to GPT, none of that is going to end up in ChatGPT or in the subsequent version of GPT 6 or 7 or 8 or whatever. I think we need to be very careful about ChatGPT specifically, but in general, for Ironclad and I think for other platforms as well, across the world or in the enterprise SaaS context, we're not going to have to worry about that as much maybe a year from now.

Thanks for bringing that up because I think that's another question that I get a ton or a concern maybe why is AI scary? Is if we use it, it's going to take our data and put it out there in the world. So that clarifies that that doesn't happen. The second thing that we hear that's scary is it's going to take away jobs, which again, not. I think we addressed that. It's going to make a lot more jobs and jobs are all going to look different and that's part of life. And the Internet did that too. And then the third thing that I think we hear is scary, at least in the context of contracts and stuff, is, oh, okay. Well, if you have play and you've defined the language, does this then give too much power to your end users, your business clients within your company to go off and execute contracts? But I think you and I all know that the whole point of having a CLM or contracts lifecycle management system is to have those controls around what the business can and can't negotiate. So you've already put that in place. So I think that fear also goes away.

Would you agree?

I would say so. I think many of us who came up in law firms have this belief that we've got to be very. Have a lot of control over the subject matter, the expertise, so to speak. A lot of technology. And this goes beyond AI. You're empowering others to access that information. I think, Kai, you said earlier about why the corporate legal department is so special is because they're so connected with the business. I think that seems to be what's happening. This technology is really making its way through the corporate legal department. They are providing answers in a more open way. And that's no knock against the firms. I know that the firms are very different, they serve a different role, but I think that seems to be where it's a little bit different for corporate legal.

Yeah, definitely. I mean, this AI, I keep saying this AI technology is going to change everything, but it's not going to change everything inherently. It's going to change everything because we're all figuring out how to use it together. And it's almost like bringing us together, helping us communicate better with each other. So, yeah, I think corporate legal team is epicenter of this. There's all this information flowing back and forth from them to other business stakeholders, maybe between departments through the corporate legal department. So this kind of exchange of information is a key area for AI to be applied.

Yeah, and I totally agree with that. And I do wonder though, at the ironclad live event, had a couple of off the record conversations and I said, hey, I know you're excited about this, but do you sense that there'll be objections from your higher ups, your GCs, your chief legal officers from adopting AI or any type of AI technology? And so some of them kind of confided and said there is a bit of resistance. And I'm curious to hear, what do you think are the consequences of not adopting AI or embracing AI?

Yeah, that's a great question. I don't think there are consequences right now, but we're not many months away from there being consequences. And that fear that you just expressed, that's normal right now. Especially even with our beta customers who. Or the customers who persist, even with the customers who participated in the AI Assist beta, there were almost all of them actually said there were people on their team who really loved it, who were using it every single day, who were finding new ways to use it, and then there were people on the team who categorically refused to use it. And this was like a very common story. I guess the first thing I want to say is it's okay to be afraid of this technology. It's okay to not feel like you're ready to adopt it. But then the second thing I want to say is it's not going to be okay for much longer. So, like sit on the sidelines for a bit. You don't have to throw yourself into this technology if you don't want to, but do pay attention. And certainly if you have a teammate who's throwing themselves into this technology, don't stop them from doing that, because that's someone that you're going to be able.

To learn a ton from this real disruptive technology. I mean, this is going to affect every part of everyone's lives, not just knowledge workers. Children's lives will change, adult lives will change. Every job is going to look different. And sure, you can sit on the sidelines and it's okay to not embrace what's new and change and be a slower person to adopt, but it's coming and it's coming faster than we think. From at least I know a lot of our listeners are legal ops professionals. This is really imperative as part of your job. You need to get educated and to educate your department on what this is. Everything is scary when you don't know what it is and you don't need anything when you don't know what's out. So in all of our experiences, the status quo is fine. We can keep doing legal the way we've always done it. You certainly can. But when the entire world around you changes and everyone's doing it differently and you're still going to the law library with your books and making photocopies, obviously that's not okay anymore. So you do need to adapt. And the sooner that you can show your team, I'm always telling folks, just have a lunch and learn. Bring people to a demo, it's totally unthreatening. But when they see that what it can do for you today is actually really helpful and something you want and not something where you're, oh God, it's a robot lawyer, you'll suddenly see what's possible and be interested. That is work that you do want to go away today. Will it change? Will it get to the point where it really is scary? Yes, probably. Who knows? You don't know yet. But for right now, the game is going to change, at least to the next level.

Yeah, I remember, I mean, on the point of it being okay to be afraid. When we first started working with this technology at Ironclad, this is before ChatGPT came out. I remember being terrified for like the entire week and I was like, oh man, this is going to change everything. Like, we need to rethink our product roadmap. We need to think our strategy, we need to make like our value proposition, everything. And so yeah, even I was very afraid when I first like started interacting with this. And it took me a couple of days of thinking about this basically all day, every day, because that's my job. Because it took me a couple of days to realize, oh wow, this is just a huge opportunity and this technology, I mean, honestly, the GPT3 in my mind became truly usable within structgpt, which was June or July of last year. This technology has not been around very long. And I think for corporate legal departments, for legal ops professionals, for attorneys, I think that could be really like eye opening. Take a growth mindset to this technology because there are no answers here. The reason there's no book on how to use ChatGPT is because no one knows yet they haven't accumulated it. Every week I see something on social media of someone saying, I use ChatGPT in this way. And nine times out of ten I'm that please don't post that. But like, I still read them because one time out every 10, I'm wow, that's a great idea. For instance, I watched a YouTube video the other day of someone coding with GPT. And I started by telling it what I wanted to do and then saying, do you have any questions for me that would help with this task? And then ChatGPT came back, or GPT4 came back with these questions. Person answered the questions and proceeded to, like, help have GPT help write it. That technique of asking the AI for the questions that it needs to help assist you, I've been using all the time. It's great. And I think as a human society, we're discovering these cool tricks that we can use with AI to make it more useful for us.

Yeah. And that's why I love talking to you every week, because I learn so much and things are changing so much. And just with me and Alex, like out in the community talking to people, people bring ideas to us. We come up with our own ideas. And then when I'm talking to you, it's now we've gotten to the point where your answer is, yeah, we could do all five things. How do you prioritize that? Wow, we could do all this now. This is incredible. Yeah, it's a lot of fun.

It really is. And I think, Kai, the thing that I'll remember about AI was how you framed it at the live event when you said AI is there for when it's harder to draft than to edit. As someone who has always struggled with drafting, I love feeding in bullet points. Getting something that's not really there yet and then revising it makes me so far more efficient. And I think many of us in the legal community, you know, I think are the same. We are really great at spotting out errors and editing. We may need some help with the drafting. And I think that's what's super cool about generative AI and all of this.

Yeah, I should say I think that was Ted who said that.

I'm going to give you the credit, guy.

Well, I'm going to start using that because I agree with you. I heard Ted say that and that makes so much sense. It's so it's good right now for tasks that are easier to. Where editing is easier than drafting. I think he said it in a slightly different way, but definitely going to start using that as well.

And so, I mean, the bottom line is that the skill set of the lawyer or the person who needs to do the writing is still required. And this is still your voice. But I think one of you has said the time to first draft just gets reduced.

Actually, I think if you're the only one of us who Went to law school. But yeah, Alex, some of the things that you learned in law school are probably going to become way more useful, not just for like practicing attorneys, but for anyone who's not practicing anymore. Yeah, the Socratic method, the kind of critical analysis, especially the critical analysis of words in an argument, all of those techniques that we use to communicate with each other, to analyze the arguments of other people are suddenly going to be useful and I think will be recognized as new techniques with AI asking the AI to ask back some questions to get the facts right. I'm sure there's analogy in law school somewhere. I like hearing that. And yeah, some of what I hear about people doing this for editorial writing and stuff like that, usually marketing is where it's applied right now. I think some of the things that they're doing are actually like taking what traditional editor might do and applying those techniques, but having the AI do it. So we're in for a really, really interesting year.

Super cool.

What should we talk about next?

I covered money we could do. So have you guys used ChatGPT much? We could trade tips and tricks.

I totally use it so much.

Yeah.

For my. Now we're still recording whatever it's all for. Like I said, I struggle with the writing part, the drafting, the both. The substance I have, the editing I have. It's that middle part. So for example, yesterday I did this preparation for a clock talk and my co presenter and I, we just kicked around ideas and I just took notes, like phrases, words. I said, I'll draft something and then send it out. I didn't really draft it. I just put the bullet points in and asked ChatGPT to give me a title and the speed at which it did that. And then I spent a couple of minutes editing and sending it off. Like everything. The entire exercise went from like, I don't know, like 20 minutes to like two minutes. But I think the key thing wasn't just that it saved the 20 to 2. It was. It didn't weigh on my head. I always had this procrastination thing where I'm like, oh, this thing's writing things can be really hard. Let me just do it later. But now I could just feed it in right after the meeting. I mean, so that's what I'm using it for, that kind of thing. And I've been messing around with it a lot.

Yeah, those are good uses. Yeah. I've been again going back to the ironclad live talk because that was one of those ones where I was, well, this is going to be public information next weekend. That's not. I'll have to check with Nicole on that. But something I did there was I was recording myself on Loom now has this transcript feature and copied the transcript, but it comes out with all these like timestamps, which is pretty annoying. And so then I went into ChatGPT and I was reformat this, get rid of the timestamps, like reformat it. And it did. And then I sent that to John because I know John prefers to like give comments on the text of the document rather than on a video or something. And so formatted it in a way that actually was like easy to read and yeah, some time on the margins. But what I'm really excited about is we're discovering these techniques and I could totally see some video recording technology owner software integrate this technique into their platform at some point just to use this. So, yeah, it's going to be cool.

One question that I legitimately wonder is, I've been thinking about, and this is we're actually getting ready for a couple on stage conversations at clock about knowledge management, about AI. And one thing I've been thinking of is does the idea of knowledge management for a legal department kind of go away or does it not and it actually create more work to prep for it? So here's what I'm thinking, right? Like, so we as legal operations people, as law firms have constantly tried to curate all the knowledge, whether it's a document, whether it is a who worked on it, and the tacit knowledge of were we happy with the work, what was the advice given, so on and so forth, and putting that all into a useful place where people can then find it and refer back to it and have useful reusables. As I always used to say, can we get away with just having a mess of stuff everywhere nowadays, if it's in your email, if it's in your drive, whether it's curated or tagged or not, and AI will be able to answer all the questions or is that possible? But in the meantime, it still needs this context. And so the work that we have to do as legal ops is actually putting in standardized processes and procedures to get things into a place where the AI can do its job, right? Does that make sense? I mean, that's one thing that I've asked myself a lot.

It's a really good question. I think right now my position on that, and this is just my own personal position, the structuring of information is going to be really useful and really critical for maximizing the potential for AI to help you. So I think that knowledge management work, that knowledge structuring work is actually going to pay out in the near to medium term. But three months ago, four months ago, I think I was on the opposite side with Canva. Okay, all the structuring of information, just get rid of it all. We'll just have the AI read everything. And this is to say, the pace of innovation here is insane to the point where my full time job is following it and I'm still changing my mind every couple of months, not every.

Couple of weeks time we do a conversation or podcast. This can only be fresh for life.

Yeah, exactly.

I have to publish this immediately, but. So what you're saying is we can use the AI to structure the data for us, but it is really important that we allow it to structure the data so that then it can use structured data for intelligence.

Yeah, and it goes back to what Alex brought up of hallucination. Right. If you ask a very authoritative figure who's a human something, and you don't give them any tools to actually validate their answer, then they might hallucinate an answer, a very convincing incorrect answer.

That makes a lot of sense.

So yeah, the structuring of this knowledge database, the structuring of knowledge, I think is actually going to assist the AI because it's going to be suddenly this very capable intelligence that has access to a very well structured set of information and will therefore give much better of an answer.

Yeah, I guess that makes sense because I was thinking if the search capabilities are strong enough and you had stuff all over the place, it could find it. But now just thinking out loud, that's not really different than a human.

Right?

You could say, hey, legal assistant, go find me information on X, Y, Z. And if it's all over the place, they can only come back with something. But if it's all very structured and you look through this very nicely structured database and find me the answer, you're going to get a much better. That makes a lot of sense.

But I reserve the right to change.

That is short term. And where the technology is right now, that's our opinion. Right?

Informative releases. Can we rerecord this?

It really is though, changing that fast. I mean, Guy and I have had so many conversations where I'm like, can we do this? And he's like, not yet. And then three days later he goes, oh, we can now. What happened if it changed? He goes, the tech changed. Wow. It's really mind boggling, actually.

Found myself working with a lot of people who like been out on leave for a couple of weeks or something. And every time they're like, whoa, what happened?

I'm sorting you out.

I was out for eight weeks and it was right between when GPT3.5 was released and four and it was just crazy town.

Yeah. You check our first one on one Alex, like, so what on the entire world has changed.

Yeah, yeah. One of our partner, one of the people is a partner of ours at OpenAI was also out for a few weeks. When he got back, he thrown into the defense. I think he was out only right while ChatGPT. He left right before ChatGPT came out and then came back two months later.

I mean, the fact that my parents are talking about ChatGPT, I think that they've never talked about anything I've done for work. They're like, what is the cloud? What is SaaS?

Don't know.

But they're like, hey, what about this AI is it?

What is this?

Exactly the same. And then just last night, just to date this. So I think 60 Minutes put out a segment with Sundar Pichai of Google interviewing him about all the changes that are coming.

I haven't watched this one yet.

I should.

It's only a six minute segment or so. But really interestingly, he says that the advent of AI, the current AI is going to have a bigger impact on humanity than fire and electricity. And I mean, you think that's a pretty profound statement. But if you think about what the Internet did to the world, completely changed everything, everyone's lives. And this is happening and the pace that this is happening, it took the Internet, what, like 10 years to get to how quickly GPT has changed in weeks.

Yeah. And that's whenever someone asks when is this generative AI wave hype wave going to be over? I'm like, it's going to get hypier. But that's only because there's so much substance underneath. We're just scratching the surface right now.

Yeah, I think AI and GPT is more the Internet than it is cryptocurrency or the Metaverse or NFTs where really interesting disruptive technologies that operate kind of on the periphery of the rest of the world, whereas the Internet and AI is every person's life, I think will change.

And I'll add that I think for the legal community it's even more relevant because we deal with words and this technology applies really heavily to words. I do feel the legal community has been paying attention to this more and experiencing more change here than with all the Other technologies that you described.

That's interesting. And all this news and reports and research reports that have come out and said the biggest effect will be on knowledge workers. Why do you think everyone's saying, well, actually connected.

So a different perspective on this is we've all been preparing for this for a decade at least now. Right. We've all said that AI is going to be able to help us in these, or maybe not. We've all, but especially the three of us working at technology companies, we've all believed that AI will get there, and we've all been disappointed that it hasn't been there. As surprising as this feels, as miraculous as this feels, it's also something that we've been preparing for. And that's something that I try to remind myself and others of a lot.

Totally.

It's here. We've been trying to prepare for this for years, and it's finally here. That's awesome.

Well, I keep saying, as someone who's been pushing for disruption and modernization and change and legal for the better part of 20 years now, the fact that this is a dinner table conversation for everyone, I'm just like a kid in the candy store. Can we talk about this more? Can we talk about the disruption being here? I've been telling you, we gotta change. We gotta change for real now. So it's exciting that at least the conversations are happening everywhere you turn.

Yeah. Sorry to hijack your question.

No, no. Except my question was really about is it gonna affect knowledge workers more than other professions? And why do you think everyone is saying that it will? I mean, words. Alex, you mentioned that.

Yeah.

Is it Partially. Knowledge workers are, as you said, the competitive advantage you have or the professional advantage you have is all the knowledge that's in your head and your experience. And now you want to share that and crowdsource it and use it and collaborate. Right.

I mean, the thing that I remember when I was growing up, I was always taught. I can't really point to a person who taught this to me, but they said, go get an office job because it's safer than if you work a manual labor job or a. Some type of. We were working with your hands. And for most of my career, that was probably true. Now we're at this moment where it's not, and it's almost flipped. And I think that's what's getting everyone kind of a deer in the headlights feeling, because we were all taught that when we were growing up. And so I think this is a very interesting time. And you have the hype people. You have the people who are skeptical. But everyone I think agrees that it's super relevant because of that lesson that we were taught.

This is why I always wanted to marry a general contractor. Because useful to have.

Diversifying.

Diversifying, that's right.

Yeah. I don't have a good answer to that. No. I think it's interesting. People are talking about software engineering, obviously, as one of these areas that could just be replaced by GPT4. And that's very close to my heart as some, as a software engineer. And I don't really. I won't say that I definitely can't predict the future, but I will say that this is making me think more and more that kind of finding a passion, finding something that you want to work on is pretty critical. And going forward, I think that's how I would advise other people.

Well, guys, this was an awesome conversation. No different than every conversation I have with you on this topic. A lot of fun. Glad we got to hit a lot of important things with that. Thank you guys so much for your time and let's keep this conversation going. Thanks for listening all.

The.