John Rodrigues explains how to design AI agents that reliably do what users want through system prompts, memory, and feedback loops. He built Orbi/Orbee and watched agents forget context and give wrong answers, destroying trust. The fix isn't a smarter model, it's engineering the framework around it.
Chapters
So in this video, I'm going to talk about what is hardness engineering and how it's connected to designing for trust. And as I was started like exploring this topic, and since in the AI space, most people are talking about it. So I started researching it. And then I realized the apps that I'm already building with AI, it's already I'm doing some sort of hardness engineering. And then I was struggling with this pain point, right, where the user's intent and the output that agents give, they don't usually match, and that immediately breaks trust. So I, I, I realized like once you break trust, people don't really adapt to that product, and some, some point the business even can fail. It's a high-stake environment and stuff. So, and the output is also getting like probabilistic, right? You can't really tell what the output is. No, I know traditionally in design we kind of mapped out each and every output and stuff, but now agents are kind of determining the outputs and we're designing the context. So while I was working on the Orbi AI app, I also noticed this pain point, right? I put this, I built this app and put it in hands of the people. And then people like the agent start talking something else and it was not answering the question, lots of latency. And it was forgetting the context. And I realized this happens to some of the best AI products out there as well, right? And they have spent like billions in R&D, but that still products are failing a lot of problems in terms of like the user experience. And the big part is like when you work in a very high stake environment, you can't do mistakes, right, at that stake. And like even a small mistake can be, you know, disastrous revenue-wise, brand-wise, reputation-wise, uh, so even on the impact on the users, um, you know, output and outcomes and things like that. And that's where I found, uh, that's where this hardness engineering started to make sense to me. And I realized while working on this app, I had to like conceptually map out the behind the scene and the AI system. It's basically a harness that I designed for Orbee, and then it realized like it's not really about making the best model, it's about designing the framework and the harness around that so you get the best output. So that was a great realization for me because I feel like if you know how to design and develop better harnesses, you can get better output for the agents. So that means more reliable output, that means increases the trust. So coming back to the definition of what harness engineering is, From what that I've understood is it's basically like you, when you're building an AI-native product, you have AI models, you have like context, you have like retrieval systems, you have like the AI capabilities and APIs and things like that. And you are basically putting all these Lego pieces together. But if you can design that system in such a way that output is more reliable, like managing the context window maybe, or managing the knowledge bases, or like connecting to the right models and things. So even when I was working on Orbee and it started to start derailing from what it's supposed to do, and that's where I started creating like system prompts and things like that. So hardness kind of answers that question, what's the system prompt should be, what's the memory layer that you're using, and what is the retrieval system you're using, and what is the tool calling you're doing, and then how does the feedback loop look like. So it's basically, it creates a hardness and the system And designing that is very important because I think for the longest time we are used to designing deterministic systems where we kind of know what the output should be. It's like a logic-based, but it kind of caps the actual capabilities of AI and even the scalability of it, right? Then why do you even need AI? You can just use like a chatbot, simple answer and go. But then the customer behavior is kind of changing because people are getting used to like softwares being more proactive. You just tell and the software does things for you. And so that you product usually adaptation will usually be low. So as I kind of started understanding this topic, the one thing also I realized is how our thinking shapes the product. Because I feel like more what's happening in like the most of the AI teams and things, the engineers are deep into the tech side of it and they're right and from the place they are. And the designers are more in the design and the surface and the user needs and the problem and things, but there isn't like the bridge between these two. And that's like the growing pain. And then when there is disconnect between how teams think, there is no hardness, right? And the hardness breaks, and then you eventually produce like bad products. So that's why the hardness engineering role, or the people who can think from like engineering point of view and also the user point of view, that kind of bridges this gap between those two. And most businesses are kind of like missing are getting this wrong because how even they are like, you know, having that mindset within the teams and stuff. And that's why that's leading to a lot— that leads to a lot of product failures and product failures and eventually impacting that trust. And I believe in like high-stake environment, designing a better hardness is like the key factor to get the agent output because it's not just about having the best model. You can use the most cheapest model or the most, you know, less parameter model or the most less smartest model, but you can still get a better output. And that's the art there in hardness engineering. Working on this AI native products and putting in the hands of the people, I started to realize like smartness is not equal to trust. You know, people think, oh, my app is the best or the smartest or the model is the smartest that lead to trust, but it's It's not like— it's trust is about matching the users. In this, in this context, it's about matching the user's intent with the output. And if I can get a reliable output and that output that I trust over time in a high-stake environment, this is even not negotiable because you can't really mess up the output because it has some big consequences on the business. And also, I think the role concerns that I have, that there's a middle layer is not there. That bridges the engineering and like design, that gap is only increasing over time. And I think it's very important that people started to like bridge that gap. I believe also like we are kind of not even following— gonna not follow any titles in the future. So I believe like that both mindset is so essential for designing better products with better output, and that everything is tied up through hardness, and that's that's very important. I believe it's not like the, you know, very specialized team that's gonna— can produce better out— can produce better AI products. It's— I believe it's the team that can like have that design and engineering and business mindset together, and then ability to create that hardness that like works effectively to produce a better output and leading to trust is the core part of it. So I kind of spoke about hardness engineering from a very core user experience and business and the team structure point of view. But it's definitely a very deep field. You can understand different capabilities and how to connect with things together. So I did write a Substack article on it. So if you're already watching this on Substack, make sure you subscribe. I'll continue to share about AI-native products and things that I'm exploring and experimenting. So I'll see you in another video.
