Beyond Hype: The Hard Truth About Building Reliable AI Agents

My learning so far on the agentix side is that it's so early and it's evolving so quickly that it's just really hard to figure out like where the puck is really going to go. Like maybe 90% AI automation and like 10% human involvement. Like I think that's going to be the sort of the big first milestone.

A lot of people quickly think, oh, let's just use OpenAI tool calling and it'll work.

It doesn't.

Most importantly, where you invest your millions of dollars and where you develop a plan for development is probably like the most critical step towards building something of value towards customers.

How did you approach building agents that actually work with the high degree of.

Efficacy and what were some challenges and lessons learned along the way?

The way I think about agentic AI is there's typically. Well currently I think there's three types of AI companies and startups right now. One is in the model AGI research layer. So foundation model companies like magic.dev, openAI and then you have LLM driven algorithms. So typically like the Dev Insacana AIs they've invested heavily on algorithms on top of LLMs and then sort of like where we sit nullify. And a third type of company is integrating AI with existing environments.

As a vertical agentic company, one of the most important notions is that the big value that you'll be able to get to customers over general agents is going to be in the outer tail of distribution of tasks that your agents are doing, right? So if you think about like your agent's performance as think of it like a decay curve, right? Where you have all the way to the left of the graph the most reliable tasks that any general agent can do, right? And as you go further into the tail of distribution of tasks, the more complex the tasks get get right and the harder it is for general agents to do that. So what you have to solve for is that further outer tail of distribution of tasks. So this makes it. So there's two notions of two different ways to do this, right? We can either take enterprise existing workflows and existing tooling like he mentioned and get a bunch of like let's say synthetic data of sequences of tasks that these LLMs are doing and fine tune some model to use the existing software infrastructure that exists. This is typically a lot harder to do because you need a lot of diverse data to be able to fine tune an LLM. Or you can go the complete opposite direction and in a way think of it like representing existing software infrastructure in A way that's in distribution with an LLM's training and what they're good at doing. Right. This is that exact agent OS that he's talking about, some sort of middle layer that makes it very easy for these agents to interface into existing software. That's exactly what we've been very, very focused on and that so far has been very successful. Especially for let's say like building data pipelines. Right. There's a lot of tooling out there, they're very complex to use. But by building a middle layer that makes it very easy for agents to use infrastructure, these agents end up becoming very, very reliable very quickly. So it makes it more of an engineering problem rather than a machine learning problem of trying to capture a wide enough distribution of set tasks.

I think the thing that we found to be super successful is just a constant feedback loop with the customer, with us, with the data set, with eval. We need that constant evaluation. We need that constant feedback from customers. And we found that to be just super critical in building great solutions. Now some of the problem with that is that sometimes you can overfit towards a particular customer set. And so again this just goes back to basic product development. We don't want to overfit. We want to have a, a set of design partners and be in constant back and forth communication with them around making sure that we're building a system that meets their exacting needs.

Quick. I'll do quick thought. I think again maybe contrarian throw humans at the problem is kind of how I see it. It's like if you got a workflow, you can either spend your two to five million dollars sea ground trying to build all of the technology infrastructure and all of the stuff around it to really kind of get the T right or take what's already off the shelf, figure out some automation, some workflow and just like inject humans into it. Because that can do that fine tuning that feedback loop that prompt engineering like and just make that invisible to the customer. Customer doesn't care. Customer, you know, users only care about, you know, to the point we heard earlier. It's just, I just want the outcome. I'm not looking for AI, I'm looking for somebody to solve my problem. So in many ways it just doesn't really matter what the, you know, if you, you can fall into a trap of just making the agents too, like just focusing on efficacy when you can solve the same out, get the same outcome by finding the right balance of agent efficacy and human in the loop. So, so what you guys have done is is packaged in the human in the loop in your go to market offering. Right. And so do you want to expand.

On that a little bit?

And I'm curious if anybody, I don't.

Think any of you guys are packaging.

Humans in the loop in your solution, but I'd be curious if you are.

So maybe I can just answer from our side. So on the one side we work a lot with our clients and we need also the business owner ultimately to own the process and to give us feedback. Because one, there is information about how to solve certain problems, but there's a lot of procedural knowledge as well in these companies that you need to understand in order to really build agents that can do the full task. When we work with clients, we on the one side sell our platform, we sell the agents, but we also do fairly high touch sales. So essentially solution sales where we sell them consulting as well to a certain extent. And we see that it really helps us in the beginning to make sure that we get to these reliability and accuracy levels if we don't. And we have always like people wanting to try out the platform and I want to build a certain case that just barely works. So we really need to focus on very specific problems and then work with our clients. That and to be honest, in the beginning it took us months to get certain agents to work and now we got this down to a couple of hours or days, which is really great. So the effort on our side really is minimized.

Yeah. Just to add on to that, that's exactly what we've seen. While the solution itself is not human in the loop, the whole package, the whole bundle is human in the loop. What we found is that a lot of our clients would be one or two super users and that they would over time expand that knowledge to the rest of their organization. That'd be great for us. But when we added in a learning component, when we added in almost like a consulting component to help that organization understand how to use this usage took off much faster. So that's a critical part, especially in these early days where people don't. Again the point that we've all been making, people don't understand or care about the AI. They care about the solutions. And then showing them what's possible is a big part of it.

As a multiple time founder, I've learned the hard way, like innovation is best described as like make it work small and then go big. And every time I've tried to make it go big, I end up it's kind of like when you touch the stove as a kid, it's like, how many times are you going to do that before you learn? Your hand gets burned. So my experience as a founder has just been like, how can we make it work small and then scale it up and how do we do that as fast as possible with as much interaction and data from our end users as possible, and also how they feel about it. If you have the best quality solution in the world, but there's no trust in it and there's no acceptance, you're not going to get the outcome is we have a mantra in our company, like our number one company value is outcomes over outputs, right? We want outcomes, we want the click, we want the meeting booked, we want the sale. We don't really care about the output. The output is a great leading indicator, but where everybody else is sort of getting high on the novelty of cute outputs, it doesn't really make any difference if you're a small business owner and it doesn't actually move the needle in your business. So the more that we've been able to earn trust on moving the needle on specific outcomes for the business instead of focusing on outputs, the farther and faster we've been able to go and unpack those things. So hopefully that's helpful.