Ai Agents and customer service and support with James Hotson

Amazing.

Hello, James, how are you? What part of the world are you at?

I'm doing great. I'm in Los Angeles today.

And why the hell did you leave wonderful Seattle and go to, you know, sunny Los Angeles? Why would anybody go from a race that rains 300 days of the year to a place that doesn't rain that much?

I mean, you know, there's a reason why there's a few investors that left beautiful Seattle and started great VCs in LA with beautiful names about the weather. So, you know, I like, I love Seattle a lot, but the 72 and sunny every single day is amazing.

I think it's kind of boring, but, you know, that's weak.

Yeah, yeah, yeah, yeah, I get it.

James, give us a two minute background to yourself. I know that we've talked a lot about AI and how it could change, but give us a two minute background of yourself and tell us a little bit more about what you're doing right now with Katara.

So I got started in AI in 2012. I got really excited about using AI to predict lifetime value of customers, predict what offers or products they might be interested in. And at that time it was still AI. And then I went through the transition of AI becoming big data and then going from big data to being called machine learning. And now we're back to it being called AI again. But I spent about a decade doing AI marketing, tech, predictive marketing in many places and then transferred that over to doing more of AI for customer service and support. And so we, and so we move from a bunch of collaborative filtering over to using GO for the first time and understanding how can we train AIs to answer questions and solve tickets. And so we spent about two years doing that and that company actually got acquired. And so that was my second acquisition that I had gone through as like an executive on the team. And it was really amazing because we were answering 40% of customer service tickets using AI and we were training and this was before OpenAI had been in the market. And then OpenAI came around and launched their product and it was like, oh my gosh, we, not overnight, but in a very short window of time, basically doubled the ability of that AI and that customer service. And so that's a lot what we've been working on recently. I've been, we've been building vector database, graph database for companies, helping them optimize business process. And we're able to solve like 85, 90% of problems now, which is crazy on how short of a time period that's been so I've spent a lot of time in, in the AI space. Most of the time my role is focused on go to market sales, helping customers take full advantage of the tool. So you know, part time therapist for you know, CMOs and CEOs and you know, part time strategists as well.

That's awesome. That's a great story by the way. So one of the things I've got three questions, two of them that came by email and one of them that I usually ask every single person. Right. The first question I'm going to start with what I ask every single person. So since LLMs such as ChatGPT, what has changed for you in the way you work? Start with some examples of how you are using it yourself that you're previously going, man, this I don't know what I did before, but this is the way to do it.

Well, I think the way I think about it is there's a couple different levels and there's a base foundation that I like explaining it on. Before LLMs we had a lot of just memorization. You and I both know two plus two equals four. We can do that in two seconds. Right. But when you get to a more complex problem of like 17 times 37, you, you need to bring in reasoning. And you and I can still get there, but it's going to take us a little bit longer. And LLMs are reasoning and before that we had just memorization. Right. And we could what problems come along with memorization is things break if it's not perfectly linear, if it's not on the happy path, like you can't get the result you want. And so today we have a lot more reasoning, a lot more ability to do things. And so how do you take advantage of that?

Right.

There's and I think of it as a couple levels, like level zero or maybe level five is like you just opening up chat GPT and copying and pasting an email and saying proofread this before I send it. Right. So you're, you're helping it in your day to day doing the exact same things that you're doing all day every day. And you know, it's saving you a couple hours a day or a week individually. So that's like how most people are using it. And I was actually just chatting with one of my buddies at Amazon and it's like I couldn't believe for the last two years if you worked at Amazon you didn't have an internal tool, you know, to do that. And I think they just released Their internal LLM that they go do this thing. I felt very behind for them. So that's level one, now level two, like, or I guess getting to level one is like how do you implement that in the automatic way? And I think LLM's the first time I felt like I was actually using a computer the way a computer was supposed to be used. As a non engineer, someone with not an engineering degree, I could open terminal and I could run process and I could have my computer work for me all the time with very little technical knowledge. And anytime I got stuck I had this person with 140 IQ that I could go talk to and it could answer and unblock me from the problems. And so I think that's how everyone should be using AI. And LLMs today is running really tedious, repetitive tasks that they need to do in their day to day all the time on their computer. Like that seems like table sticks for me. Now if you're running a company or you're an important role and then I think what we're trying to do with Katara is we're trying to help people transition or like adopt agentic AI workflows. And the first version of that is like let's just stitch a couple different AI agents together and then have them do a workflow and you know, do that workflow much faster than you could with humans or do it in a way that can run 24,7 so you can get more output. And so we actually just had a client, they stitched together 45 AI agents into one workflow and those agents are now doing the work of like four full time employees for them, which is amazing. And we came out that that was like about just under 9,000 hours a year, 8,700 hours a year of, of like manual human time savings by like transforming data and getting data in the right places and enriching data and doing all the right things that they needed to do that were just tedious and manual for, for this team previously.

So they use 45 agents together or 45 LLMs together, 45 AI agents.

And if you think of an agent, right? What is an AI agent? It's a piece of, you know, python code or go code, right?

Yeah.

And so you have that agent and that agent's task is to, you know, get data from this source and then the other agent's task is to go get the data from that source and then another agent's task is to understand the intent. And so a lot of that is consolidating getting data into the right place, getting data into the right structure, and then making sure you're using that. With this particular instance, we're creating a vector database, and then they can build a Q and a support agent off of that. And they're getting data from multiple channels, multiple places. And so now they can get to deliver a really consistent, unified customer experience because they have everything linked up together, which is pretty amazing.

What is the difference in your mind between I listen to what you said and back, I'm old. So back in the 90s, this would be just a sequence of functions. So, for example, someone has to. Let's assume a customer came in and said, hey, I'm not able to get my subscription unsubscribed. So first the agent, if you look at the physical agent, the human agent, they go. They look up the customer's information. They will look up when their data is, when their subscription comes up, et cetera, et cetera. There's a series of tasks that they would do. Right. In the case that you mentioned, you actually talk of each of those tasks as agents, as opposed to one agent that does a bunch of different things. So help me understand why these are not just individual tasks. Why do you call them 45 different agents that do that? Is it because all of them can independently operate and they go do their own job and come simultaneously and give you the information? What is it about that that calls each an agent?

Well, what we found is most agents tend to break down today, and this could change next week if OpenAI releases a new model. But they start to break down after seven to nine tasks. They get confused or they get off task and they're not able to give the result you want. And so if you keep it really specific for that particular agent, it can do that task really well. And that's okay. It can run all the time doing that same exact task. And then you can have another agent that also specializes in doing a different task or getting that different result using, you know, maybe the. The information that the first agent's getting. And so a really easy example is like you tell an agent, go read all the latest news on AI Digest, Consolidate and write a report. And then you have another agent that says, go to the database where that report lives, read the report, and now tailor a nice email newsletter to an audience of men 25 to 45 who are in tech. And then it can write that in a really nice way. And so you wouldn't want both of those agents doing the exact same thing, because it just would get muddled so the specialization, I think, is really nice.

Why do agents get muddled?

You know, I don't know. I don't have the right answer for you on what they. On what gets muddled. You know, I'm playing around with them every single day. And sometimes I'll build an agent that I think should be able to do the task and it gets stuck for whatever reason.

Yeah. I'll give you an example of a custom GPT that I wrote. Right. It's a very simple one. I've got a series of system prompts, I've got a series of input prompts, and I've got an output format. So I write a lot about the stock market as well. Just individual stocks in the tech space that I like. So I wanted to do a report on Hinge Health, which just went public a couple of weeks ago. Hinge Health, their digital online Digital Health Company. $3 billion in market cap. Now I've got a format that I like. And so I've taken that format, I've given it to Chad gpd and said I want it in this format. And I've given three examples and I've said this is exactly the format I want it in. And then each of those sections within the format is a prompt. And I've tweaked the prompt, tweak the prompt, tweak the prompt where I've gotten. And that becomes. So the prompts become my input prompt.

Yep.

And then the system prompt is the format I want it in. So I've got an output format, I've got the set of questions, and I'm saying, this is how I want you to do it. Even then the freaking thing barfs nearly five times out of ten.

Yeah. Right. Sometimes there's loose. Right. And sometimes we run like when we do content generation, the same types of prompts through different models. So we'll see how Gemini responds versus Anthropic versus OpenAI. And it's really. And then we'll actually score them. We'll go through a scoring exercise and it's really amazing to see, you know, code or really technical documentation. You know, Sonnet does really well with right now. And. And you know, there's other types of content that it does really poorly with. And so it's just interesting. A how to guide might be better with an open air model right now. It's just. It's crazy to see.

You know, the interesting part is the same. LLM itself is inconsistent an hour later.

Right. It's like there's a personality in there.

Almost it is, right, isn't it?

Yeah. There are times where I'm like, oh, you know, I've been using Gronk a lot lately. Oh, Gronk's not answering at all. You know, earlier last week it like couldn't do anything that I was like, am I imagining things? I really thought that this was working, you know, a few weeks ago. But it comes down to your prompt, like I think prompting and prompt engineering and how concise can you be? How specific can you be? That way your, your agent's not getting confused and how good are your directions at, at, you know, getting this thing into the world. And I'm, I'm constantly watching different YouTube videos and seeing how people are building things. And you know, one takeaway is like people are keeping a note or a log of the problems that they have so they can go back and they can iterate and, and you know, make that better. Which I thought that was a great takeaway. So I've implemented that in my day to day now it's like just a running history of what prompts I've used and how I want to improve them in the future.

Can you actually learn prompt engineering or is, can you teach prompt engineering? Is this a learned skill now? Very much like someone graduates from college and there's a finishing school that teaches them prompt engineering. I'll give you an example. Right. Even till today, 25 years ago, there was an Indian company called Infosys. You heard of them? Infosys Pro tcs. So you finish engineering college. I know a lot of the people that graduated me, my from, from my graduating class, they then had to go to Infosys finishing school for about six months before they could actually write code or they could go in front of a customer. So they had a finishing school of service for a whole bunch of things. Do you think prompt engineering becomes that finishing school for a lot of the kids coming out of, out of. I mean they're using, they're using it every day. Every kid I know from college, I've got four kids and three kids in college, they're all using it, but they're using it in the version one way, as you said.

Right? Well, I'm in the business of helping companies get like 100x benefit and that's either in cost reduction or improvement.

Right.

And I'm sure you read that article on like how much OpenAI is burning on people saying please and thank you. Right?

Yeah.

And so if you extrapolate that, how much could a really concise prompt engineer save a company if OpenAI is charging for input tokens and they're charging for every, every keystroke you put into their model, and you eliminate saying thank you every single time or, you know, could you please, or whatever it is in the prompt. I mean, you could potentially save a company hundreds of thousands of dollars in, in input tokens depending on how much you're prompting. You know, that AI or whatever prompt is going in there because eventually you write a prompt and that prompt gets automated and done, you know, hundreds of thousands of times. So I think that prompt engineering to fully answer your question is both you can learn it, you need, and you can teach it. And I think it's like anything in life where you do the reps, you know, you do your 10,000 prompts and you learn a little bit every single time and you get better and better. And the tough part for me right now is you have to evolve so quickly. You know, I'm GPT 4.5 acts slightly differently than the model that was, you know, available two months ago. And so I'm trying to figure out, okay, how do I get the best out each one.

You mentioned something that was a question that came up as well. You mentioned that certain LLMs seem to be better at certain things. You mentioned that Gemini and then O3 were certainly different at different things. Rock is certainly different at different things. How do people figure that out? Is it just mostly doing reps and doing a lot of again and again and again trying the same thing. And so that's one question. The second part of the question is for someone like whose job is not to be able to go try this multiple different times, they're not sitting over there saying, I want to take the same question, put it in 10 different LLMs, see which one gives me the end chance to figure out how do they not have to go through so much to be able to get the benefit of LLMs?

Yeah, I think consultants are cheat codes. You look at the people who are using the tool and what they're doing in their day to day. And so I tend to follow some of the AI influencers and whatever role they're in and see what they're using and what they're talking about. And I think the perfect example, if you've used cursor quite a bit when you're trying to build something, and if you're trying to build it with Claude, the prompt that you put in the.

Yeah.

Is if you start using cursor with Gemini. I haven't had that experience. I've never Gotten that frustration of like oh, you edited 50 files using the Gemini model within cursor. And so I think I learned that through looking through, you know, different tweets because that frustration of like getting a project 90% done and then cursor deleting the whole thing, it's like oh that like everyone who's played around with it has had that happen to them. And so okay, how do you solve that? Well go check out X and who's posting about building on cursor and what have they tried and see if that works for you. So that's one that's like how you answer it for yourself. And then on the bigger pictures I think consultants are definitely the cheat code. Look at who's living and breathing it every single day. Look at it. Who's getting the results you want to. Who's the best in the world? Prompt engineer. What are they doing? What are they listening to? You know that that's how I kind of got started. I looked at There is a PhD at IBM that was showing how do you know do research and how to use the IBM model using Watson and like what prompts to follow. And it's like just followed that and that got it started. You know that's way out of date now even though it's probably about less than a year old. But that's the world we're living in.

Tell me about Katara. Let's switch gears. Tell me about what was the problem that prompted you to found the company. I know you and your co founder have been working at this. You guys got started a little while ago. But what is it that you are trying to accomplish with Katara? What portion of the problem in the go to market side are you trying to solve?

I think AI is really a sexy hot topic. And then what are the actual business results and how can you actually implement quick like real wins? And we saw an opportunity. I come from the background when we're doing retention science where we would do all these integrations. We did, you know, integrations to tech companies, we did integrations to brands. And what stopped every single integration was one key developer had one small question about how someone little thing was set up. But we had to schedule a call and that call took two weeks to get on the books and then the right people wouldn't be on the call. Then we had to redo another one. So one small question took a month to tackle. And what does every single engineering team hate? They hate writing technical documentation. And so we were like okay, how do we monitor a code base in GitHub automatically enrich and update and get the right tech docs built. And then every single developer, and maybe I might get roasted for this is lazy. They don't want to read the tech docs. People just want to prompt for their problem and get the answer. How do we make it super easy for them to just say, hey, XYZ is my problem, how do I solve it? You know, so now we can come in and we can answer questions like, oh, what's the rate limit on your API? Or you know what, how do I, I'm using this code snippet, why is it not working? And we can start to get answers. The answers have their source cited and it just like for any API centric company, it just accelerates all the development that can be done to that company. So you can get more information built, you can get more tech built. More tech for a lot of these companies means more revenue. So it's a really, it's an amazing revenue acceleration tool. And so to give you an example, you know, we had our, our peak day, our best day so far for one client is we answered 3,800 technical questions for one company in one day. And you think of that in like the terms of outsourcing. A good customer support person can do 100, 110, you know, questions in a day. Yeah, so 40 people and 40, like high level, smart, technical thinking people answering those questions accurately with good responses. And so, you know, we can implement that. Honestly, I can implement it today in about 15 minutes and have that kind of Q and A support agent up and running. And so people can be building and you know, unblocking themselves like instantly within, within, you know, 15, 20 minutes, which is pretty cool.

So who's the ideal customer for Katara? Is it a technology company that builds out products that other people integrate with or is it anybody that is in customer service? Regardless of the type of company, I.

Think any, any business that gets questions right now is a great potential for Katara. We definitely work with, you know, any company that has a marketplace where people are building apps or people are buildings and that's like a no brainer because we can get more development built on, on those things. Any open source type project is also great for us. We're working with a couple of health care companies as well. You know, how do you manage onboarding and talking with potential patients without doing any PII is something we're tackling. It's pretty exciting. So we can apply this AI in a bunch of different respects. And what's really cool is we can get the domain expertise almost immediately. So if you're an aerospace company, we can read through all the documentation really quickly on everything about that aerospace company and get accurate answers right away. And so it's kind of the foundation of how do you ingest the data? What's the AI infrastructure? And I think what makes Katara special is the frameworks that exist, like Crewai or LangChain, they're great, but you run into limitations really quickly. And so we home rolled our own framework so that we could actually do the things that we're doing and deliver the meaningful results that we are. And I think anyone building on those platforms realize they run into those roadblocks really quickly. And so we unblock that infrastructure challenge that a lot of these companies have been having with their data.

That's awesome. Jim, this was terrific. Glad I had you on the podcast. And I think the most interesting part of all this has been how quickly agents are starting to get used. Even in smaller companies, you know, enterprise, you would expect them to start doing some of the POCs and moving that forward. I was talking to a couple of friends. I went to Las Vegas over the weekend. My kids wanted to go to a Bruno Mars concert. And I have a friend there who sells baseball cards. So he's a collector, right? But he's a, he's a retailer. He also sells retail. He's got a retail outlet that sells limited addiction, baseball cards, football cards, basketball cards, things like that. And nearly 60% of his revenue now comes from the website. He deployed an intercom based system, removed that, replaced it with an agent. Now, over the last two months or so, I would not have thought someone like him who's as far away from tech, he's like, yeah, yeah, this AI stuff is great. I've just deployed very quickly, it becomes a lot more easier for me to. Because people have questions, right, well, how old is this car? How true is it? How much? Okay, can you give me. He's so far ahead, James, he's thinking of deploying an agent that helps negotiate.

Yeah, right.

Which blows my mind.

And it'll probably be better than most humans because you can train it on, you know, give it 10 negotiation tactics or 10 negotiation books. And then it can use psychology and game theory and all these things to get the best price for every baseball card. And it can do it 24 hours a day. It won't make any errors, and it will be really good. So, yeah, I mean, I'm, I'm glad your, your friend is taking full advantage of it, and he's getting good results out of it, which is even better.

Awesome. Thank you so much, sir. Talk to you soon.

Thank you so much for me having.