The Professional Network for AI Agents, with Agent.ai Engineering Lead Andrei Oprisan

Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Thornberg. Hello and welcome back to The Cognitive Revolution. Today, I'm excited to share my conversation with Andrej Oprezan, engineering lead at Agent.ai, a fast-growing and currently free-to-use AI agent platform that describes itself as the professional network for AI agents, online at agent.ai. Before diving in, I want to take a second to note that this is a sponsored episode, our second sponsored episode out of more than 160 total episodes published over the last year and a half. Our goal with sponsored episodes is to create a win-win-win for the show, for the sponsor, and most importantly, for you, the audience. I consider myself very fortunate that many startup founders are currently interested in doing the show, and as such, we have the luxury of reserving sponsored episodes for companies that I personally find very interesting and genuinely expect to resonate with the audience. I see their sponsorship more as a way to cut to the front of the line so that their appearance on the show aligns to their important company and product announcements more than a go or no-go decision criteria per se. Agent AI is a perfect example of such a company. It is a well-resourced and sophisticated effort backed by HubSpot CTO Dharmesh Shah, with a number of intriguing angles on AI agents and the future of work more broadly. And I think today's episode really exemplifies the win-win-win that I hope to create. I prepared for this conversation with the same depth of exploration that I always do. I tried every last Agent AI product feature that I could find, wrote an outline of more than 1,000 words of questions, and challenged Andrej to go deep on the technical details. In the end, I'm glad to say that he really delivered. Highlights from this episode include Andrej's breakdown of the current limitations of language models when it comes to planning, out-of-domain detection, and error recovery. His analysis, recorded just days before OpenAI's big O1 announcement, foreshadows their release, and suggests that a good chunk of what was previously missing might now be available. Andrej also shared recommendations for how to approach building AI agents, including the importance of narrow, well-defined tasks and robust benchmarking. He shares insights on creating effective prompts, structuring agent workflows, and implementing feedback loops to improve agent performance over time. We also discuss Agent.ai's vision for the platform, including the concept of a professional network for AI agents where agents have their own profiles, And their plan to become a marketplace where developers can build and monetize agents, as well as how this could democratize software creation for everyone, potentially allowing even non-technical users to build sophisticated AI-powered solutions. We also exchange best practices for fine-tuning models, and I was very intrigued to hear Andrej's best practice of using small models locally before going on and scaling up to larger proprietary models in the cloud. We get a detailed explanation for why Agent.ai uses Pinecone over Postgres's pgvector for their vector database, touching on factors including ease of use, scalability, and performance under different workloads. The kind of thing that you can really only hear from someone who has tried a wide range of solutions. We also discussed privacy-preserving techniques for AI. This is something that I really should learn more about. We covered Apple's new approach to encryption of user data and got Andrei's thoughts more generally on the challenges of handling sensitive data in a way that unlocks the power of AI while still maintaining user privacy. Finally, we conclude with an earnest big picture discussion on the future of work and AI's role in it. We explore Andrej's vision for how AI agents will integrate into various industries, the potential impact this could have on jobs, and the skills that will become increasingly valuable in an AI augmented workplace. We even touch on some of the ethical considerations and potential societal impacts of widespread AI agent adoption. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with friends, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, including your thoughts on our experiment with sponsored episodes. You can send us a note via our website, cognitiverevolution.ai, or you can DM me on your favorite social network. Now, I hope you enjoy this discussion, which covers AI agents from all angles. with Andrej Opersan of Agent.ai. Andrej Opersan, engineering lead at Agent.ai, online at agent.ai. Welcome to The Cognitive Revolution.

Thank you, Nathan. Excited to be here.

I'm excited for this conversation. Obviously, AI agents have been a super hot topic since the release of GPT-4, and so much time and energy and thought has gone into them. And so far, I think we're still waiting to see exactly how this is all going to play out. And as we anticipate new models and new capabilities and potentially an agent future coming at us sooner rather than later, you guys have built an entirely new from the ground up platform for AI agents, and I am excited to get into every aspect of it.

Awesome. Excited to share more about it.

So maybe for starters, what is an AI agent? How do you define them? I think there's so much confusion about this. From just post-GPT-4, a year and a half ago almost now, we saw these projects like Baby AGI and AutoGPT. And the idea was like, you just give the agent a high-level goal and it'll go off and make its own plan and find its own way, and hopefully it'll be successful. Fair to say at this point that hasn't quite worked. Probably still will. But in the meantime, the sort of definition of agent has slid around and been fudged by so many different people that I think it's really helpful just to get clarity right now on like, what do you mean when you talk about building AI agents?

Great foundational question. And again, I think it's early. I think this definition will continue to change every 3 to 6 months. And so I think 6 months from now, we'll tweak it slightly. But my definition is semi-autonomous system that can perform tasks or make decisions on behalf of a user. I do think it's extremely important to have these agents do very small tasks, something concrete. Something extremely bounded versus I think the first iterations, very open-ended, very hard to define goals or have some very broad goals. In this scenario, in this future that we envision, we envision having these semi-autonomous systems that do have the human in the loop. There is some work that the agent is going to do. What are your thoughts on the performance? How do we benchmark these things? You know, what success For me versus for you, Nathan, might look very different, even using the same kind of agent, given our expectations, given our background, given the kind of data that we want to feed it to personalize, for example, right? I think the more narrow, the better. The more narrow the definition, the more narrow the set of capabilities an agent has, the more likely it is to succeed. And then we build on that. We'll have, I believe, many, probably dozens and even hundreds of very small specialized agents that will help you do your job on a daily basis. For example, from reading— what do I do as a white-collar sort of worker? I read my emails in the morning. I prioritize them based on what I value. What I value changes day to day, week to week, month to month. There's some strategic, there's some tactical, right? And as you think about even prioritizing something as simple as Emails or reading your calendar and saying, here's a summary of what you should— who are you meeting with today? Nathan for this podcast, you're meeting with Dharmesh for agent.ai strategy session, and so on and so forth. That context is going to change over time. And I think the more narrow these agents are, the better. And over time, we'll introduce more abstraction to be able to decide which agent is going to get which part of that job. But I think we're going to be able to advance the much quicker if we don't try to oversell them and say they're these autonomous systems that can do everything under the sun. I don't think that's the case. I think that's a recipe for disaster, for hallucination, if not for disappointment altogether. And having very concrete, small, benchmarkable agents with clear or broadly well-defined datasets is going to get real work done faster.

The question that I want to maybe follow up on is, like, you spend all day, as you said, building agents, right? So how do you think about what the right scope of action or freedom is right now for them? Because I sort of see that it's not like a binary, right? Like, one of my mantras is AI destroys all binaries. Like, we can never have these clean categorical distinctions anymore, I'm afraid. And this applies to agents too, where the zero point of agency would be like a fully deterministic workflow with no models involved at all. It's just like pure code, right? And then on the other hand, you have a baby AGI type thing where you say, go make money on the internet, and it hopefully comes back with money for you. And a lot of the failure seems to be people not really knowing where they should be on that spectrum. I think there's a lot of success with things that you might just say are workflows with an AI component. I had Zeki Chen from Runway on, and he talked about his AI SDR, and it's lead comes in, we're going to ping this API to get data back, and then we're going to put that into the prompt. And then the final thing is language model creates an email for that person. I think it was like 16 steps or something that he strung together.

Right.

But I believe that they were all deterministic. I don't think at any point in that process was the AI actually trusted to say, I'm going to go this way, or I'm going to go that way. It was like everything is prescribed and the AI is getting tasks within a workflow that I feel very confident can work today. Do you think that is the farthest that we can go today, or can we go further? How far do you think we can productively push this semi-autonomy?

I think we're still missing from a purely algorithmic and LLM evolution standpoint, I think we're a few evolutions away from being able to fully move away from this highly prescriptive approach, which I do believe that's where we're going to live for the next 6 to 24 months. We'll see if the new model coming out in a few weeks from ChatGPT will change that just in terms of the planning. But LLMs are fairly bad at actually planning and actually Explaining, helping a person understand where that uncertainty is, right? So it's very confident in giving you some answers. It doesn't know just how confident it is or how it should doubt itself. I think that's a very big sort of missing piece, having some kind of robust out-of-domain detection as well. So to understand when it's not marketing anymore, it's sales, or it's a different kind of task and a different kind of domain. So again, think through that deep planning is absolutely foundational here for autonomous agents, and we just don't have that yet. There's also, as you think about this reliable self-correction, just how confident we are and what sort of the ground truth looks like, being able to then self-correct will be doable, or at least we'll have some insights into what that looks like. So again, more guided autonomy. And then lastly, I think the part that's missing the most, and I think that this is where we're building towards that future with workflow tools, is building more ground truth, but also building ways to benchmark these outputs, right? And so let's say again, you and I are using the same kind of agent. For my context, what really good looks like or really bad looks like is subjective, but I can tell you, you know what, I want more of this. I might not know exactly why we got there. What of the 10, 20, 50, whether it's concrete sort of steps that were taken or decisions that an LLM or some LLM took. I think it's important to be able to explain those, to be able to reproduce them, to be able to then benchmark them to, to then say, you know what, we did veer a little bit left this way. For the next step, let's tack the other way a little bit and guide closer to Mathematically, what we could describe as the objective right path, or more of the objective right path when you're running this, let's say it's a million different simulations for the same kind of work to be done, which again, if you have structured agents that are essentially workflow tools on steroids, you can get that fairly quickly and a good enough sort of answer. The benchmarking tied with the Out-of-domain detection and some of the long-term goal definition to then do the self-correction. I think those are the 3 big pieces that are needed before we can have anything resembling sort of autonomy. But that doesn't mean that these agents can't do a lot of work very well and really accomplish concrete business goals better, faster, cheaper, and really augmenting those human workflows themselves. Some of our most effective agents on agent.ai. And as we built more and more of them, we've actually learned that the more prescriptive we are, the better they perform, the better the feedback is from both objectively from NPS surveys and other data points like that, but also very concretely in terms of feedback. To give an example from our flagship agent on agent.ai. You can enter any company name or domain name and we'll give you a full company report. Everything from some general information about the company website, founding time, the founders, decision makers, their emails, et cetera, all the way to how does that traffic grow on LinkedIn and X and other platforms down to researching their product offerings and price points and how would I sell to someone at that company, if I'm this other company trying to sell them something, give you enough of that context. That alone, this agent that we talked about has 20 different sections. We've got over 100 different steps, mostly very prescriptive, but also there's some LLM flexibility now that we have structured outputs where we can say we can trust more and more these LLMs to give us their creative answers. But then make sure that it fits within certain data structures. It fits within a list that we can quantify, have certain attributes that we really care about, to then be able to iterate that a number of times, a number of models, this mixture of experts, to then be able to say, okay, roughly what do we think are the top 3 competitors? If you were to try to compete with this company, how would you do it? Et cetera, you can then answer those kinds of questions and support those kinds of use cases much more consistently, much better at scale. And that's part of what we've seen works as we continue to evolve these agents. It takes time. Unfortunately, again, we're early. And so it takes a lot of time, a lot of iterations to get this right. And part of what we're trying to do at agent.ai is provide the platform, provide the WYSIWYG drag-and-drop workflow building toolkit, as well as some of the more creative LLM-based actions that will then fit to certain kinds of outputs, structured outputs. You could drag and drop different kinds of responses and visualize it in, in an interesting way. And all you have to do is just describe the steps if you want to. You can drag and drop and customize for an expert. Or you can just describe and say, I need an agent that will read, I don't know, podcasts for me, summarize the key learnings. Here's the topics that I really care about and focus more on those specifically. What I just said there, those 30 seconds worth of description, maybe some tweaking should give you a full agent that will do that concrete task very well. Now, the broader the sort of tasks are. Or the less structured, the less likely to succeed. But I think that's what we build upon. That those are the MVPs. And then more mature versions of these agents will eventually be able to do even more work across a number of domains if we can tell when we're out of the domain. Let's look at podcasts. Let's not look at YouTube videos. Let's not look at emails because those are different kinds of mediums. The way we craft those prompts look different. The kind of benchmarking data that we have is like likely very different, and how we benchmark those results and then potentially fine-tune a model to really be able to give even 5% or 10% higher accuracy is going to be different. So I think these are the building blocks. I think we're going to get better, but I do still believe we need to live in a sort of workflow-oriented world for some time until we build these other LLM missing capabilities that will then allow for more autonomous execution end to end. Hey, we'll continue our interview in a moment after a word from our sponsors.

On the pieces that are missing from the language models in terms of their ability to succeed today, it was effective planning, task decomposition and planning, out-of-domain detection, so it knows when it's going to get itself in trouble, and The final one was error recovery, basically figuring out when something's not going right, is there a way to come back and overcome that some way or another? That is a pretty good wishlist. It sounds like you don't think that will be on tap with the much anticipated strawberry model, whatever that might be. You're building with the expectation that those things are not immediately going to be solved.

We're building with that in mind. I'd love to be pleasantly surprised to the contrary. Again, we don't really know just how quickly. I mean, there's obviously a lot of research in each of these domains. If interested, I can share what some of that research looks like. I'm fascinated by this. My background in machine learning is actually building mostly the reliable self-correction type of algorithms. And so being able to define what is that ground truth, how do we essentially do surgery on neural networks to then be able to adjust and do that at runtime instead of retrain a whole GPT-4.5 or 5.0? That takes a lot of time, a lot of money, even fine-tuning some very small models that can take time. What would essentially live patching that look like? There are some advancements there, but we're not there yet. We're not there in terms of being able to apply these broadly in this kind of class of LLMs that we have today. I think we're going to get there. I think certainly in the next 5 years or less, we'll likely have all of these and many iterations of these capabilities. But I do think they're critical. And if you can't define what ground truth looks like and you can't tie it to the planning aspects, then I think it cannot really evolve as much as we want it to. It'll be creative, right? I think, and maybe there's this definition of intelligence, definition of creativity. Will evolve. I think it needs to evolve, but it's probably not what we expect it to be in terms of that sort of human-level autonomy and that human-level intelligence. And these are some of the the building blocks. Ian LeCun, the head of AI at Meta and former NYU AI professor, he's been talking about this for years. I remember being a student in his class where that was. the focus of the research of what they were doing. And this is 20-some years ago. And some of the work that he did on convolutional neural nets and aiming to create ways of discerning what that error correction needs to look like. And that some of the boundaries on domains and even being able to quantify within the same domain on the topics, how well you're penetrating the domains and when you're maybe blending different kinds of topics that shouldn't be blended together because it, again, it's mixing the wrong kind of science ingredients in terms of logic. It doesn't build where you think it builds, but it'll very confidently think that it's doable. I think this research is decades in the making and I still don't think we have a good enough, fast enough sort of solution. We do have some solutions in highly specialized domains and highly verticalized. But to be able to do even generic planning across trying to build an agent with a concrete set of steps without turning it into essentially a workflow builder with predefined set of options is still very hard. And I think we need some of those pieces before we can truly trust or quantify how much we should trust some of those outputs.

People say when you're building AI-powered or AI-enabled software, use the AI only for the part that absolutely requires the AI and use code everywhere else where code can possibly work. And I think what I heard from you was largely consistent with that. We're, you know, in the workflow world for now. We're not in the baby AGI. It's not about to turn into grown-up AGI. In the immediate term. But I also did notice one really interesting aspect of the platform to me that I haven't seen done this way very much at all. And I've tried a lot of things and look for this actually, because I feel like it's a little bit of a leading indicator of who is thinking ahead versus who is the sort of filter step that exists within agent AI. And obviously people have had a lot of different experiences with different no-code platforms. In general, there's often a thing where it's like we get to a certain step, we need to advance conditionally, right? Maybe we stop under a certain condition, maybe we advance under a certain other condition, maybe we fork the workflow depending on whether it's A or B or what have you. So that experience is pretty familiar. Typically it is done with hardcoded rules. That of course was the only way we could do it until quite recently. And so you have these exact match or contains match or regular expression match sort of options. What stuck out to me in the agent AI version of this is that it was using a language model to make that sort of distinction. So when I get to the filter or the decision step, instead of needing to work out a regular expression, even better still would be if I could just express, this is the gist of what I'm looking for, and I want to advance under this and not advance under that. And then I could perform that sort of judgment. on top of unstructured data or semi-structured data, as far as I can recall, implemented at one other company, and they were in a different space, but still had that sort of, we're going to trust the language model to make this decision as to whether to continue or how to route. To put that in the form of a question, what kind of guidance would you give people for what to trust the AI to do? How reliable do you find this filter or those sort of routing decisions are? In today's world? And how did you decide to go ahead and take the leap to trust the AI to do that as opposed to just implementing regular expressions?

So I think for some of these conditional logic steps, LLMs are actually very good at understanding complex if-then-else looping. That's coming soon, but the same thing for looping, retry this or give me 50 variations of ad copy or 10 variations of emails and then send them out, A/B test them for me, et cetera. Like some of those repeatable if-then do this, otherwise do that conditional logic to repeat. Human beings are not great at explaining in steps all the different paths that something could take. And actually LLMs are very good at that. There's some really good datasets out there as well, where you can actually fine-tune models based on conditional logic, just hundreds of thousands and millions of these. It can get super complicated with all sorts of nested logic jumping out of a loop if something else were to happen and so on. Even a 2-level nested if-then with continue blocks, with in a while loop, with a for loop, watching for certain system words, et cetera, with a workflow tool. Very hard to even myself being an expert and living this every day, it'd be hard for me to replicate it and make sure that it actually works the way I conceptualize it working. But I can explain that we can have a conversation about what I expect it to do. And LLM does a much better job than I do. And this is actually properly benchmarked. We tested it and it is true. For certain kind of logic like this, they're very good. But when you try to go deeper planning, we're not there yet. But there are some of these steps where I do think we have to push outside of our comfort zone and embrace AI and benchmark it, right? And build more of this ground truth and figure out, okay, can it do a good job, a good enough job, a better job? Deeper, faster? If so, that's a wonderful tool. I love to use wonderful tools that make me faster. I can do more things that I really care about instead with my time, right? That's the way that I would look at these things as well as do mundane and computationally complex things like go and publish different posts on different social media platforms and figure out What are the metrics? What kind of posts got more engagement and why? Be my social media manager, do that job well for me versus I'm going to do the same thing in 5 different platforms or maybe use some kind of platform to aggregate that, but I still need to slice and dice some of the structured data. Wherever we have the structured data, I think these LLMs can be very good, but also translating unstructured but predictable sort of data and logic flows into structured data outputs, we should embrace more and we should always push the envelope. And that's how we think about building these agents and the kinds of capabilities that we continue to add to agent.ai. It's how can we figure out and give you more of those tools so that you are not spending time trying to figure out how do I build this complex workflow? You can talk to it where it makes sense to and have it actually build the logic and give us some flexibility within reason. And the backend, we always translate that to structured data, logic gates, and then so on that you need to be able to run actual code end to end across a number of steps. Hey, we'll continue our interview in a moment after a word from our sponsors.

I think there's so many data points suggesting that we are entering an era when anyone, quote unquote, will be able to build pretty advanced software. I find that to be extremely fascinating and I come at it from a lot of angles. There's of course many coding assistant tools and Claude can code remarkably well. But then there's also this sort of intermediate hybrid form of the workflow with the AI sometimes playing the router. And then there's these questions of like, how does that get built? I think you guys have a really interesting angle on that. as well, where you've talked about having the AI do it. How well does that work? How have you gone about the process of dialing it in? I recently gave a— one of my recent episodes was on AI automation, and I'm curious as to what tips you've learned for automating the process of creating an agent workflow, because that's a pretty advanced use case, and I bet I could pick up some tips. But then you also have The kind of marketplace dynamic, like the form factor of the product is interesting that when I log in at the header of the website, I have agent network and human network. And so I can go browse around and explore different agents that are already there and published and shared. And I can go look at people and what they've created and parse it in different ways. So I'm really interested in like, how well does the AI-generated workflow stuff work and what have you done to dial that in? But also interested in, is that what people want? Or are you finding that people are more gravitating toward like, what has Dharmesh done? And maybe I can just borrow from him. And then there's also an editing layer on top of that too. I mean, with my company Waymark, we do basically full generations of marketing videos for small businesses. One of the frontiers that we're looking at implementing now is, okay, you got a video, but now you maybe want to give some feedback on that video. Maybe you want to have some changes made to it. We have a UI where you can do that. But I certainly imagined a future where I would say, the second scene isn't quite right, make it more like this, and have the AI do all that stuff and never have to touch a UI. And it seems like you're definitely planning for that, as I can see that in the product where there's just this ability to explore, discover. I don't know if yet there's a way to say, tweak this to my circumstance. But anyway, you can unpack that in multiple directions.

It's like you're reading our roadmap, which we'll make public soon. But essentially, yes, right now we have a small number of finely tuned agents, finely curated, every single one being hand-built with workflow tools. Say, get this user input. And then when you get that input, get this data from this data source. And then when you get that result, feed it to an LLM to do something with some kind of prompt. Then you turn that into structured data. Then you use some other source and eventually show user the thing they're looking for. Where we also have a way to build the agents in the same smart action builder that we have by just adding with the builder. And you can say, let's ask the user for the YouTube video and it'll fill in the 4 different parameters that you need for that step. For example, what is the heading? What is the variable name? What do we show the user, et cetera? Maybe how many YouTube videos to crawl? Is it a YouTube video or is it a channel? If so, how many videos deep should we go? What kind of topics should we care about? You can just say that in the chat and it'll fill in all sorts of toggles for you and do a fairly good job. Again, we have some internal benchmarks. It's probably 80% good. That might not be good enough, or that's not yet good enough to make that available more broadly. But we also will make it available over the next few weeks. This idea of templates. So for certain verticals in marketing and sales and customer service, et cetera, we're going to have example agents that users can publish either for users to use as they are. So where all that logic stays closed and you just interact with it and that's it, or make it open so anyone can make their own copy of it, tweak it and say, you know what, what if I want to To customize it for my use cases? What if I want to add 3 more steps and remove 2 over there and then tweak the output to maybe send me emails of the results instead of just showing me the website on some kind of job and, and so on? That is all coming very soon over, over the next few weeks, definitely in September. From there, I think the agent builder becomes more, more useful as just a creative open-ended tool because It will have thousands of these templates to feed upon, to be able to say, just, you can describe what you want and it'll have a larger library of ground truth to, to really use, to be able to say for this version of agents that you'll build on agent.ai, it'll be able to put those steps together much more accurate in the 90% plus stage with some of our testing. And we'll see once users get their hands on it. But I do believe we need these templates. We need people to build their agents and build the sense of community as part of what we're looking to build here with Agent AI. The long-term vision here is for Agent AI to be that platform where all those agents can live. We'll figure out what's in it for the creator. How do we pay them? How do we drive that exchange of goods? Very similar to how the Apple Store. Drives visibility. We'll guarantee you distribution. If you want 100,000 eyeballs a month on your agent, that comes free. And if your agent is also free for everybody to use, then maybe you get the feedback right from the customers and you can improve it that way. Maybe you have a pro version that can do some more things. Maybe you enable certain personalization and customization, right? Just very similar to these app models that we're all familiar with and they seem to work fairly well. I think that model will need to evolve for an AI agent marketplace where, yes, some of the parallels do work, but how do you really charge, right? Is it per app? Is it a flat fee? Is there some rev share of these credits that you need to use? Because one agent does not equal another agent if you're using 10 data sources and 50 LLM steps with very large context versus It's a very simple thing. Do some research for me. That's all part of what we're actually experimenting with. We have a small alpha, I would say, builder group at this point. We're not quite beta yet. We'll have beta over the next few weeks. We'll bring in even more users into our platform and have them start building more and more agents. We're obviously always going to seed the market with dozens of agents just based on feedback, based on what market research we also do. But from there, we really want the market dynamics to take over where we're getting close to tens of thousands of users now. I think we're at over 40,000 as of earlier this week. It's growing very fast. And so the market is now getting enough momentum where we have builders building more useful agents. We have a review process just like App Store has a review process to make sure the apps do what they say they do and they're not siphoning off personal data of customers and, and so on. So you want some guardrails. That's part of what we'll offer through the platform, but also make it easy for anyone to build agents. And, and like you said, what we love about this future is that you shouldn't need to be a developer to build agents. It's all about citizen AI builders. Any person who has a need Just like you can build a form to capture emails, to answer some questions with all the WYSIWYG sort of systems out there. We want to make it as simple as that, even simpler, where you just describe what you need. You have dozens or hundreds of examples to pick from as well. If maybe that's a better path, examples and such, as well as documentation, as well as that community of people who have tried different things. Some things are not as successful as others. We're trying to build that community because it's a space that will continue to evolve, I think, very quickly. You can go from zero to expert in weeks. A lot of it has to do with how willing are you to fail fast, try some different things, get some good examples, right? I think being hands-on, tweaking these things and making it work for your use cases is going to be what's Going to make agent.ai the choice platform for AI agents versus some of the more specialized point solutions, which also have, I think, great uses. I just don't think you're going to have as much adoption long-term because it's going to be harder to acquire those customers, keep them engaged, continue to evolve, right? And as this underlying technology also evolves, these LLMs will hopefully be able to do planning, or at the very least, some of the error correction. pieces, you're going to blend domains very quickly. You're going to have many specialized agents, yes, but also something that can coordinate just across on top. And we're choosing this model. We have this idea of adding agents to your team. I have a team of agents. Maybe this is my social media team of agents. Maybe this is my marketing team of agents. Maybe this is my school research doing my MBA right now, I need different kinds of specialized agents to look up case studies and do some deeper analysis of data and present it in a consistent way. How do we divide and conquer those tasks together in a team? And that's how we're positioning Agent.ai as a professional network of agents. If you believe that work is going to continue to be hybrid, So in-office and not-in-office work, we believe agents are going to be a part of that. We believe that combining agents and having them do professional work means that we need a professional social network for these agents, where you can really understand what is this agent capable of? What are actual employers of those agents? What are they saying about that agent? Is it a good agent? Is it a bad agent? For a lot of these other solutions, it's very hard to tell how good are they. You may have some reviews, but over time, I believe this approach with an actual social network with endorsements and maybe some certifications and so on will be the right solution. It's useful, faster for actual business use cases.

Yeah, certainly. Then there are some echoes of HubSpot where there's very few, if any, ecosystems in terms of Platform and service provider and kind of market dynamics on top of it that have been as successful as HubSpot has been, and so I can see how the DNA there is definitely teeing you guys up for success with that model. I want to just dig in for a second on the performance optimization around getting AI to help build workflows that. Is a challenging use case. And you mentioned, like, you've given multiple tips along the way. One was just be really explicit within your instructions. That applies presumably for every scenario. It's not something that people necessarily are, like, super great at. So it's, again, this is definitely a reason I think that a marketplace dynamic could be really helpful to people. But just more generally, I would assume this is, like, the most core AI you're working on is the AI that makes these workflows. What have you found to be the big performance enhancers for that scenario? Explicit instructions. You mentioned also having a bunch of examples. We know that large language models are few-shot learners, to quote a famous paper title. What examples are you doing RAG on all those examples? If so, is that working? I hear a lot of failures on RAG, more so than I hear successes. Are you fine-tuning it? If so, what is working well there? This is something I think folks who listen to this podcast, many are building their own highly specialized but performance-critical specialist model, and they're looking for nuggets they can use to squeeze out a few more points.

Actually, RAG for us so far has worked surprisingly well. That's how we developed that piece of functionality where we first essentially developed a few hundred examples from very simple. To very complex. That's our sort of template library, if you will. We've then built out a very simplistic benchmarking tool where we can say, here's my goal, here's the kind of agents that we would choose for each of these objectives that we have, how we selected that tool, because this specific ask actually means that I need YouTube data and Google News data, for example. And explaining that, I think making that part of RAG datasets. And then as part of that as well, the challenge with RAG is, I think there's 2 challenges, things that we found useful is number one, making sure your document size and the overlap are tuned, right? And so having too small of a window, having too small of a document with not enough overlap can actually cause a lot of issues. For us, the document size is well-defined because we know just how big these objects are. Our average agent has 6 steps. And so each step looks like this 6-attribute JSON object, and then you can do the math and model out, okay, so how big should these documents be to make sure that we're always— they're just big enough, you know, don't want too much overflow. And then the last bit of advice there is figure out how you benchmark the results. You're going to have some kind of RAG system. You're going to maybe go the other path with fine-tuning. We've tried that too. Actually, you got a lot more false positives in that scenario. And so that, again, there's pros and cons to both approaches. It really depends on the kind of data that you have. And for our use cases, given that everything was fairly predictable set of intents, RAG tends to do a little bit better because it actually— we got worse results by fine-tuning versus by using some of the larger models. So it's interesting research there. But building some level of evaluation tool at the end of the day, I think is critical to be able to say, even if you got the right answer, even if you got the wrong answer, would you know it? And would you be able to quantify it to then be able to do some of this other tuning? That's critical, right? And so whether that's another LLM call where you can say, hey, here's a few shot examples of what really good looks like and really bad, what it looks like. And then you make it give you the answer. In a JSON format where it's one of 3 different enums of like good, bad, and I don't know, or like some kind of numerical scale or whatever kind of works, that is probably good enough to be able to solve for a lot of these complex use cases. But then you have to build your ground truth, right? So now we do have some benchmarking toolkit. We have this RAG system, you know, all of this is like, I don't know, 100 lines of code, maybe 200. It's not complicated to build out, assuming you have it. Now you have to build enough of those examples to then be able to babysit sort of the system to then get a bunch of output, figure out, okay, does this actually make sense? Let's go back. Let's revisit each data point and tune it again. Can we get predictable output? Is the temperature low enough? Right. And how much context should we give it to consistently get good enough answers? And using models that are not versioned, that are not snapshotted. If you're tying yourself to GPT-4 latest, they do make changes every now and then. You might be using an entirely different model at that point, or like a different sort of checkpoint versus try something local or try something hosted where it's a Llama 2 or 3 at that certain point in time, a very small language model. Some of the Define models are phenomenal for this, where you can just run on device for these tests, and we got more consistently good answers that way. Uh, and then we figure out, okay, how do we replicate that using Claude with the larger context? Because that's actually really important for personalization. And we use that now for most of our self-building approach. And we first started with these small language models, and then we worked our way back to the larger ones where we need to be even more explicit with the prompts and then tune our RAG approach a bit to get the same answers. You can't always use the same approach to RAG because how you prompt and the context is a little bit different and you get some kind of different results. And so you have to factor, you know, all those different variables into how you're building this. There's something even that can run on device is likely good enough for 99% of these use cases, make sure you build a way to benchmark things and make sure that you really decide if you need to go fine-tuning or some kind of RAG or just use a larger context window or pare it down and compress it in different ways. You can have the same document take 10 times less space by just asking it to compress it, define the language, and then include a definition with every document, for example. And even like a Claude that has a 200K window, we've gotten 3 million token equivalent documents running at the same speed. And there's papers on this. There's ways of doing this now with much smaller models doing the same thing. I think those are just some of the things that we've learned. We want to include this in our community portal. What are the kinds of things that we found useful? What are some of the libraries that we've developed that we'll share, we'll open source as well? But we're just super focused on Can we get something useful out there for users? This project is 3, 4, 5 months old at this point and close to 50,000 users. And so it's grown tremendously. And then we've been super heads down on just building more of the features that customers want. And now it's more about how do we evangelize? How do we build that community and make it as repeatable for them as it has been For us who do live this every day, who love to geek out with the latest research papers and try 12 different models and everything else, you shouldn't have to do that to build a useful agent. You shouldn't have to do that to personalize. You shouldn't need to drag and drop even 30 different steps to build an agent. You should just talk to the system or describe it in text message format, and then you just get what you need. And it's customized from a massive library. And we believe is the power of this marketplace that we're building, where we're just going to connect the builders, give them the distribution with people with real use cases that they're trying to solve for and make that as easy for them as just describing what they want and you're using it. You don't have to do deeper research.

One of my mantras for AI enablement generally is copy and customize. And I think that is the experience that is by far like the most accessible for people that are not living and breathing AI all the time. I sometimes say clone and customize, although copy and customize rolls off my tongue a little bit better. But when I saw the clone button on the agents that you have there, I was like, okay, this is definitely philosophically similar to what I think is going to really serve people. Let me just highlight a couple of points that you mentioned along the performance maximisation answer there. One that you mentioned was on RAG, you need semantics basically, right?

Right.

If I were to translate your comments to my own language, I would say you can't really take whatever proprietary data format you have and embed that unless you're going to do your own embedding model. Now you've got another— it's like the old regular expression joke. You had one problem, now you have 2 problems. So most people don't want to have to get into the trouble and probably don't have enough data to do their own embedding model. Maybe you could fine tune it, but whatever. Especially when you've got these sort of not super semantic proprietary data structures, that is tough. So what I heard you say is what you want to be searching on is a very semantic representation of that. So I imagine the RAG component being like maybe a paragraph description of what this agent is and does. That paragraph is what gets embedded. You compare the user's request to 1,000 of those paragraphs, not 1,000 structured data things that sort of specify all the tools and the API calls.

Right.

But those paragraphs then point to those things and then you can call in the right ones. For example, do I have that right? Any other nuances there? Yeah, no, that's exactly right.

That's it. And really we describe down to the field, right? Down to some of the prerequisites as well, right? So if you're going to have this, let's say we're going to get YouTube video transcripts based on a URL. Let's make sure we have a YouTube URL to feed it. Or what else can we do? Okay. You could also do something like search the web and you can use Google search capabilities. You can use the channel lookup capabilities. So, it gets more complicated because we do map out some of those dependencies or optional dependencies. There are different ways of getting there. So, there's really, I look at it as almost data preprocessing. When you think about proprietary formats, that's sort of the data scientist way in me is thinking about, okay, how do we preprocess and index and retrieve it efficiently, you need to think through all those different nodes in a tree and how you get there and different potential pathways. Very similar. And I think, again, you can start very simple, but pretty quickly you need a way to manage those dependencies in a structured data format. The embedding part, like you mentioned, is critical, right? If you're going to have domain-specific vocab, then you're going to train it on that. You may need Custom embedding model, etc. That can get complicated quickly. Try to stay generic. But but yes, that'll likely get better results. Is it really worth the technical complexity for two percent better lift? Is that worthwhile? Then what is your indexing system? Is it your likely elastic search or face or something with custom indexing? With proprietary, for your proprietary data, that will likely be highly scalable, good enough for 99.9% of use cases. We use Pinecone for that, different ways of searching and indexing data with additional metadata that you may actually need to provide it, like the entire relevant context. And then the last bit is also the query optimization. If you've got your proprietary jargon, you, you may need to optimize accordingly, right? I don't think you need all these 4 ingredients. They're parts of what we've defined to be able to get to a better state than feeding as much context, feeding all of it if you can in a tailored prompt. And I think for most use cases, that's likely good enough. Once the data footprint is just too large. It just doesn't work. And then you have to invest in each of these pieces that you need to tie together.

Yeah. Increasingly we're spoiled for options, not necessarily spoiled in that they all work super well, but I do think hybrid search is the way it seems to me. If not, maybe depending on the scenario, some sort of graph database, but at a minimum, you want to have the vector component work with a sort of general SQL WHERE clause style capability where you can be like, there are certain things that I know and I want to narrow my search to a certain segment of the overall dataset. And having the hybrid search there is just so fundamental. It's been funny to see how many things have been built on just like a pure RAG approach, pure vector, and it's that you do need a little bit of that complementary structured approach as well.

I was also thinking about Another piece that we experimented with and it wound up not working out for our use cases, but it might work out in other sort of custom proprietary datasets where you could have custom similarity metrics. Just think about the evaluation and adjusting retrieval metrics in a RAG system. Your typical default cosine similarity approach versus something keyword-based versus some other semantic similarity based on your own dictionary and your own kind of structures of, hey, this data maps out to this kind of graph of nodes and topics that work together. Therefore, those examples of agents that wound up actually not working for what we needed versus just creating larger documents that get embedded given The nature of our data, given that they very cleanly translate to embeddings. That worked out better with just having better descriptions of each of the steps and then what the agent would be able to accomplish. So then be able to pick like the right subset of steps with a node view of the dependencies of those actions. But in many cases, especially when you have, let's say you're doing medical research or you're you're trying to come up with some of— some new drug and combine chemicals in a very specific way. Some of the language models that have recently come out in that space use some of these custom similarity metrics against very large datasets of drugs that have been published and so on, where that is the right way to do it, and that will get very accurate results very cheaply. So you got something extremely custom, right? Like, uh, math or science type of language with those dependencies, translating that to a kind of text nodes or topic nodes is cheaper and easier to do than doing the nodes that then don't really give you the right kind of answer. Super fascinating. You can go down these rabbit holes for each of the knobs and just deciding where and to say, okay, that's not, that's likely not it. Or like this other solution that yes. We may lose 0.2% efficiency in certain circumstances, but for the average case, it's actually no better and we're introducing a lot of complexity. Let's walk it back and try to get the simplest possible solution as we generate more data. Let's say we get to 100,000 users or a million users and we've got a billion agents on the platform and have 10 million agent runs on a daily basis with highly complex Data structures and documents that this is mapping to personalization. Maybe that's when some of this custom similarity metric definition and algorithms, like, definitely will come in handy. But for what we're doing today and for the foreseeable future, we modeled that out and we don't believe that to really be worth the complexity. But there could be specific scientific domains where that's actually the only right answer. Like, the only way to get to desired accuracy is to go deep in each of these.

Mm-hmm.

kinds of retrieval and evaluation metrics.

Seems like a very rough rule of thumb. If you can represent your proprietary data structure, or maybe it doesn't necessarily have to be proprietary, but for many of our app builders in the audience, it would be proprietary to their app. If you can map that onto natural language intent in a way that you feel is quite representative, Then you're probably better off doing that mapping and then doing your similarity search in natural language space. If you can't map it, this would be the case with proteins, right? I don't think we have a great way in natural language of talking about how proteins bind with each other shape-wise. There, you're operating in a pretty different space. It's not cleanly mappable onto natural language, so you got to probably work in that space. And That's a whole other world and you do need a lot of data for it. One other big thing that caught my ear was when you talked about using small models locally and working your way up to large models. I'll give you a piece of advice I give, and then I want to hear your kind of compare and contrast. But I always say to people who are trying to build some automation, whether it's an automation for their business or the core thing in their app, in Mark's case, it's make a video. We want that video to be really good. I typically tell people, don't worry about cost at the beginning. Go with the absolute top model, start there, work your way down. First, get it to work, then think about efficiency, performance. You can usually find some gains, but first thing is just, can you get it to work with any available technology and then scale back? So tell me what you think of that advice, and then how would you Give me a little more color on how you're using the small models locally and working up, because I've never actually done that.

Yeah, so I agree with your comment. I think first start with the frontier models, try to get it working within that kind of context. If you can't get good answers out of GPT-4 or Claude or Gemini, you probably have the wrong prompt. You probably don't have the right kind of data that you're bringing into the context window. Et cetera, before trying to do any optimization. I think going and hosting your own models or using something else is only worthwhile if maybe you're living in a regulated space. But even then you can get enterprise agreements for ChatGPT. Hospitals use it. Sending— I know here in Boston, one of my MBA classmates working for a hospital and they're using ChatGPT. To understand things that they might have missed in a diagnosis. GPT on the cloud, obviously there's ways to secure it and then you get all these agreements and you're never training on that kind of data, et cetera. But it works in those kinds of contexts. Think very hard before you try to use a custom model where you really believe that you need to have something very specific versus just use something out of the box. Now cost does become a factor, especially let's say if you're using a lot of tokens consistently, although these models, again, just like AWS is getting cheaper every year. And so I think even ChatGPT has gotten 10 times cheaper and it's going to get 10 times cheaper and so on and so on and so on.

Yeah.

As a concrete example, I'm using a sort of a local embedding model to quite literally index all of my personal data, all my documents, all my images. Automatically categorize it. I don't want all of that to go over the wire. For example, if you got financial records and taxes and all sorts of stuff, right? Even I screw things up sometimes and I'd rather some of that not get leaked. But what's really interesting with some of these small models is you can actually get answers much, much faster in terms of some of the fine-tuning, some of the— there's different ways of now even patching some of these live running models. And you can't really do that in the cloud. Like, you literally don't have access to GPT. Poor weights and data, but some of these models you do. And so what if you want to shrink these models down, 7 billion parameters to 1 billion or half a billion or 100 million and run it on a device that you can take with you? That's the kind of experimentation that I've done that I found useful to then be able to run these models locally and also do some of the fine-tuning much faster.

GPT-4.

Fairly recently, I think it was maybe a month ago when they announced the ability to fine-tune GPT-4. Before that, you could only fine-tune 3.5, for example. And some of the document definition that we needed to tweak, I was able to do locally very fast and then use that comparatively to 3.5. I got better results with a local fine model for our use cases than obviously GPT-4 fine-tuning in the cloud, then beat that out of the water hands down, as different kinds of models will allow for fine-tuning online as well. For example, for Claude, we can't do that yet. There's some of these gaps where the local models can be very useful as well. If you want to fine-tune for highly specialized use cases, maybe financial services where you don't want certain kind of client data or algorithms to get out there. And you just want to build your own language model in that scenario that is just an off-the-shelf language model, but you're fine-tuning it in certain ways. I think those are the real use cases where I would think that would even be necessary in any way. But for 99.9% of cases, there are enterprise agreements. You can ensure that your data never gets used to, in terms of getting trained on. They figure this out and, and it'll continue to evolve. I think we'll get more and more of these controls and we'll probably be sitting here a year from now and they're charging by the 100 million sort of tokens at the same price as— and the context window is in $128K if it's maybe 2 million or 5 million tokens. In that kind of scenario, I think it's going to be increasingly hard to justify running your own models But I also think it's very important that we have a very vibrant open source model community because this is only pushing the closed source model providers to move faster, to provide more of these tools. And frankly, I think to keep some of the larger players honest because I do think in many cases you may not need to pay for ChatGPT. Yes, that's for specialists, that's for engineers, for people who want to set that up and know how to do it and so on. But for my use cases on a daily basis, I get as good results from Phi running locally versus GPT-4. And it's even better in some of the benchmarks in terms of planning. You get like a 50-something score, still abysmally low versus 30 or 20 for Gemini. And so I think in a certain— for certain specialized use cases, you can also fine-tune it to get much better benchmarks because they're smaller models that have less data that can get in the way of what you really wanted to do and really fine-tune it where those lower-level embeddings and the neural nets don't actually come through and override your data for the most part anyways. So again, that's one of the benefits of running some of these smaller models where ChatGPT won't even tell you what the parameter count is, or that's not a thing. It gives you more control in certain use cases, but again, I think for 99% of people, that's not necessary. It's not going to create any value. And it's a fairly complex resource-intensive kind of approach. With machines now even faster and Apple's approach now with Apple Intelligence, great name, by the way, AI, and iOS 18 and some of these more powerful devices, you're likely going to be running local small models like this on the edge. And keep all your data private. There is no cloud. You don't need a cloud. It just runs on device. And increasingly, I think we're going to have this with a blend of what Apple has now pioneered, which is your private AI cloud, right? So you can offload your workload in a way that stays completely private to you. Not even Apple can see what that looks like, even if they wanted to, right? So isomorphic encryption down at the sort of embedding level so that, yes, it's embeddings, but no, it's not embeddings that you can map to anything. It's just the simplistic way to look at it is offsetting by some kind of encryption key. Ultimately, even if anyone were to intercept all that traffic, they can't do anything with it. And then you can trust it to say, now give me unlimited cloud resources, unlimited number of models that just Take a look at this data and give me better answers faster, which again, I think is a super fascinating kind of hybrid approach to take to privacy, right? To scaling the workloads in a way that preserves the private aspects and the sensitive aspects of this data that we're feeding these models. And what does that mean for training them? What does that mean for the next iterations of these models? I think it's very interesting. And I think all of these approaches, the ChatGPT, Claude, Gemini approach, the open source approaches, Apple's approach here. They're all helping us evolve this space very quickly.

One more little follow-up on the fine-tuning. Are you fine-tuning on chain of thought? And do you have any tips around the sort of data to fine-tune on? Personally, I have found huge value in fine-tuning on the chain of thought that demonstrates exactly how I want the system to reason about whatever the problem at hand is. So I just wonder what you've seen in that respect.

I guess there's 2 different approaches here, chain of thought versus chain of retrieval in multi-step tasks. I think it's earlier in terms of progress in chain of retrieval algorithms, but it's been fine-tuning. You're enhancing the reasoning capabilities and its ability to handle problems where you need some of that sequential and step-by-step reasoning. By preparing a lot of data. I think you need to collect a lot of training data. That's not only like the outputs, but the very detailed intermediate reasoning steps. It's almost like you've got an intern and you have to explain every step and make sure your data is highly curated. If you can do that, then chain of thought is really great. Then you have supervised fine-tuning where you annotate all that data, right? That may mean we We have to augment it in different ways. Like we haven't thought about these dependencies that we need to bring in. And again, the last piece that I found useful is being very explicit with what kind of prompt you're using. And I've literally just written prompts like, can you just break this down into steps to solve the problem? And then evolutions of that to then refine the chain of thought examples. And again, math, legal reasoning, et cetera. Chain of thought is very good in terms of improving that accuracy, and it's all measurable. And then for chain of tuning for those inference chains, that's actually super interesting because it's like you're tying it to RAG systems. You have to be retrieving, processing, integrating data from multiple sources, but then you're dealing with the ensuing complexity of dealing with multiple kinds of steps. For retrieval, multiple kinds of steps for processing and resolving some of those. But that's why I think it's harder to get right. You need a lot more infrastructure. There's pipeline fine-tuning and task-specific fine-tuning that you could do depending on the kind of task. Like if it's Q&A versus synthesis, it gets very complicated quickly. And I haven't been able to get nearly as good results with the second approach compared to just Chain of thought. I think just a few good examples, a few synthetic data generation approaches as well to this can go a very long way. It's a little bit meta, but even asking like, how can I break this down further? And what are some of the key potential decision points that we haven't talked about? I found many times, like in some of our agent examples, to be highly illuminating and even gave us thoughts I'm like, oh yeah, we should break this out into 2 different tasks because we're implicitly connecting concepts that like should just be split. And actually we can increase accuracy given certain kinds of intents by mapping each one of them to concrete kind of chains of thought, right? Like dependencies up and down the chain, which again, I think it's a super fascinating thought exercise as well, because then you can map it out and, and get a really good understanding of What are your biases and the biases in that data that need to be cleaned up so that you can get better answers faster? I think it's as much having better data as just asking the right kinds of questions and providing the right kinds of constraints onto the system in the first place.

Yeah, I say 10 gold standard examples, and obviously the exact number varies, but it is surprisingly small. Most people assume that you need a lot more data than you actually need to get started. One thing we've noticed at Waymark, which is really interesting, is we want the AI to be better than our customers were in the absence of AI. So we initially started by just pulling all these videos that had been created in the system out and using that as fine-tuning data. But then as we're looking at them, we're like, actually, we aspire for the AI to be better than this.

Yeah.

So rather than working from real data, we were like, okay, we need our creative team who's the best at this to sit there and grind them out for a little while. We were like, we started with thinking we needed 10,000. We quickly realized that was not necessary. Then we were like, maybe we need 1,000. Then we're, 100 seems to be working pretty well actually. And now what I mostly recommend to people is get to 10 and then you'll use that. Not that you'll necessarily be done then, but that'll be enough of a seed dataset that you can start to backfill. The things that like don't actually exist. The chain of thought is generally not captured. So this is something that is like missing. You know, the internet does not have the chain of thought that typically it's not even verbalized. It's typically totally implicit, maybe even subvocal in the human's brains, but it's definitely not spoken out loud or recorded down and it's not published on the internet. So we see these like shadows of it. We see the inputs and outputs, but what was actually happening in the brain is missing. And have you seen any good tools for capturing that? One thing that I think is a very large opportunity would be a sort of harness, if you will, that a person could load up, strap in, and be like, all right, I'm going to go do my work, but this thing is going to be responsible for coaxing out of me why I'm doing what I'm doing, how am I thinking about it, and then integrating that with the actual discrete actions and clicks and keystrokes and outputs that I had so that we can create this unified dataset of inputs and the process aiming to record it, and then finally outputs. Have you seen anything like that? Because I have not, and I have been looking for it.

I haven't seen specific tools. There's a few libraries that can help you on GitHub. I mean, Frank has some really good resources on that, just in terms of— there's a bunch of libraries. I can share some of those links that are at summarizing some of those documents and using concrete patterns to then define this data. But at the end of the day, they're essentially just data engineering, data preprocessing kinds of tools. And that's basically it, right? It depends on the kind of data that you have and how you're defining what the chain of thought should even look like given the context, which is probably why you don't have Concrete platforms that can do that for you because it's so domain-specific. This may be example-specific, but I mean, that's a really good point. You should be able to define and structure them and automatically load more examples based on annotations on maybe some kind of agent runs and then say, okay, so what was it thinking there? What were the steps? And then validate it and then annotate that to maybe do some actual fine-tuning with. I find that's the closest. There are platforms that will allow you to Load up a lot of data, describe how you want it to be categorized and the kind of entities that you like to be extracted from it, and then maps it to documents. And then from there, you can use some prompts to approximate some chain of thought to then turn that into fine-tuning and like essentially just labeling the data after processing and then human in the loop labeling it. That's a good point.

That feels like a big opportunity to me. Going back to the Question of, I think of it as deep context, but it ties a couple different themes together that we've talked about. One is the customization of agents to a user. That customization has multiple dimensions. We've talked about modifying the workflow because you're using a different system or whatever. Oh, I'm using this task manager instead of that task manager, but I like the rest of this. We need to make that kind of change. Okay, that's fine. That's fairly, not necessarily easy to get an AI to do that on the user's behalf. Yeah. But is at least conceptually something that the platform can be responsible for. Then on this privacy question, I think another huge problem with realising the AI dream right now is we underestimate how idiosyncratic we all are. Or maybe a nicer way to say that is how contextual all of our work is. It's not necessarily that the AI can't do it. But that we're getting generic stuff in many cases because we're not giving it enough to work with. And so this is something that I think is very much an unsolved problem across the board in the space. I'm currently working on a project to try to fine-tune whatever model I can get to work to write as me. And I'm realizing in doing that, that to write as me, you need to have a lot of context. The strategy right now involves fine tuning. Yes, but as far as I can read the research, at least with the sort of low rank LoRA style fine tuning that is available via OpenAI, the fine tuning process doesn't seem to work well for learning facts. So I've tried to fine tune it to answer, what is your name? My name is Nathan Labenz. It can pick up my style reasonably well, but it doesn't learn the facts. So I feel like, okay, well, to get this thing to write as me, I'm going to need it to learn to imitate my style. Fine tuning seems to be able to do that, but I also need to give it just a ton of facts to be able to have something to draw on, to at least have any hope of effectively writing as me in any given situation. And so that sends me down this rabbit hole of, okay, where does that data live? It lives in Gmail, it lives in Slack. In my case, I have a podcast, so I have a lot of things that I've said that are good context there. Some of these data sources are really sensitive, and I'm not a privacy-oriented person. People that know me know that I really don't care that much. But when it gets to the level of every email I've sent for the last 5 years, which is the data that I'm trying to manipulate here, and every Slack message that I've sent for the last 5 years too, that is not the kind of thing that I just want to put over on one random startup that I just found that may or may not work for this. I am willing to trust the OpenAIs, the Anthropics, the Googles of the world, but I'm not willing to put all that data into random new startups. But it is really important, right? If I want to get my AI agents to work well, Right. They need to know quite a bit about me. I think in many cases I'm in the process of working on this. I'm working on prompts to go through literally tens of millions of tokens of my past data and try to put that into 1% compression to give the language model enough context. How should we be thinking about that? Is that something that users are just going to have to continue to do themselves for a while? Or is there any way that you can start to help? Is there maybe a context builder agent that I— maybe that's what I need to do is I need to go On Agent AI, build the context builder agent. You'll review it and verify that I'm not siphoning off people's data. And then folks can have a connect my Gmail and spit out context. Is that the vision? Is that what we're working toward? Or what, what else do we have there?

That's exactly the vision that we're working towards. Privacy is paramount, which is why we haven't released anything yet. We are working on a Google integration. So let's say you do want to connect Google Calendar, Google Drive, and eventually Gmail, to then be able to maybe chat with your emails and chat with your calendar and connect the dots. Given our parent company and mothership HubSpot, we think in relation to customer contact relationship management that at the end of the day, it's all about relationships. What useful information can we extract besides the fact that it's all sensitive information? It's all important. We will do everything that we need to keep that secure, assuming that's already done. Assuming that you, let's say you trust Agent AI as much as you trust OpenAI. OpenAI today, you can upload all of your data and they have essentially a RAG store, right? A vector store. You can upload a whole bunch of documents and there's probably some limits, but probably a decent amount where you can now use that RAG store, maybe multiple RAG stores to then build your own assistant that is then tied to those data stores. And then you can chat with your emails, you can chat with your calendars, et cetera. The part that's missing obviously is doing that real time, right? Like something should just take care of, just always index it, add all the metadata, make sure that maybe we need to expire certain data. Maybe we do need to fine-tune it. Given your use case, if you have a lot of pharma data of some kind, they need some additional algorithms to even retrieve the right kind of data to the RAG. That gets complicated quick. For most white-collar type of work, it's fairly straightforward. We need to recommend some kind of document size and overlap, maybe a few other settings on how we retrieve documents and how many of those documents we should retrieve at a time, given the prompt. And there's different methods of doing that, but that's all that's really needed, plus some data connectors. So you can now say, okay, let's get this data, I don't know, every minute. You get every email, every calendar, and then you could do something useful with that information. I think there are some approaches. None of them that I've seen so far are fully commercially sort of enterprise grade yet in terms of applying isomorphic encryption to vector stores. So that's the kind of encryption where you can encrypt it, you can encrypt, let's say, the vectors, just the vectors themselves or some approximation. And then we query that data and we can then decrypt it with the sort of the private key of the user. And then you are the only one who has a key. You use your private key to, to get the actual results and decrypt them to then give that to the agent to then handle it. We haven't cracked that nut yet. I think that's fairly complicated, obviously, and no one has done it yet for that reason, but the pieces are there. I also think the other solution to this is some kind of proxy answer where maybe Agent AI, or there's other platforms for sure where you can specify the model URL, just like you can specify OpenAI key if you want and use your account directly. Similarly, you can say, Okay, I'll provide my model URL. I'll provide my OpenAI compatible hosted model that might just be on your local machine or on your own private data center or someone that you trust, right? All that data just gets sent through, proxied over, sending for embeddings. If you want to embed millions of documents and financial records, use something local. You don't have to use something in the cloud. And then all that data just gets proxied. more control over exactly how that behaves, what kind of model you're using behind the scenes. And I think the future is like some blend of those things, especially for the power user use cases. I think some of that kind of proxying is necessary or using your own AWS Bedrock sort of model library. And there's different ways of hosting models now that is fairly straightforward at scale. But again, the encrypted data at rest in a way where you could share all your private data. With us, we have no control or any other platform has no way of reading it except when you authorize it for those specific use cases to just show you the results. But to compute, it doesn't need to know what it looks like. It doesn't need to know what the data is because at the end of the day, it's just vectors. It's just math and it just can give you answers based off of that. Then you bring it all back together. Apple, I think, is doing some things there reading from their patent disclosures and some of the research papers that they've put out. They're doing this type of isomorphic encryption to be able to keep all your data in your encrypted enclave on your iPhone, send chunks of that data, maybe it's your email contacts, maybe it's your calendar contacts, and only a very small sliver of that data encrypted in flight and isomorphically at that, to then just give the answers back. And then you're the only one who can see it. If that's the kind of privacy sort of forward solution we need, we're not there yet. I would trust some of these systems like OpenAI, I think I already do, with all of my data. I've done exactly what you just talked about with all of my emails and all my years of data. I did it locally, I did it in the cloud. There are ways of protecting yourself and making sure that your data is kept private. But I think that the biggest takeaway there is make sure you're paying for of products, if you're not paying for them, you are the product, right? If you're not paying the $20, $30, $100 for the API sort of access, which comes by default with all the privacy guarantees, if you're using new startup free product, they're going to monetize what you give them. Your data may wind up all sorts of places. I think there's some good out-of-the-box ways of doing that today. That doesn't require all of these additional advances that I believe will just be industry standard in, I don't know, 6 months to a year. I think you're going to have every vector store have that kind of key-based approach.

Yeah, I need to study that more. I love that you're getting into the Apple patents to start to triangulate on that because I think that is a huge gap right now and a huge opportunity. I mean, just to trace the thought pattern again. It's like the models need a lot of context. They need more than a couple paragraphs. They really, in many cases, need a lot of context if you're going to get them to do work that you're going to be happy with. They need more than just a couple of bits of instruction. But how do we move that data around in a way that is secure? In general, if you're just doing normal embedding, it is easy to reverse that. And so it's not secure, even though you may not be able to read the the embeddings as a human, it is pretty easy for somebody, if they were to come into the possession of your embeddings, to reverse that and get, if not exactly the original, very close, right? Preserve the ability to do semantic work on top of an embedding, but to make it not reversible. That does seem like a huge piece that can make so much more stuff possible without major downside risks. So I'm quite excited about that. Great. Then let's talk about the business aspect of agent AI for a minute. You had said, obviously, make sure you're paying for products or you are the product. You guys are in an in-between phase right now where the product is free for everyone to use, at least up to a point. And you have the backing, as you mentioned, of HubSpot, which allows you to do that without Going personally bankrupt, as some individual app builders have confronted when their things go viral. What is the plan? Maybe you don't know yet, but how do you anticipate monetizing this? Right now it's kind of a simple, you use an agent, it's one credit. Obviously not all agents are created equal, especially if I start to bring in tens of thousands of tokens of context. That's not gonna be all equal. I can imagine you could be like super high margin on some.

Yeah.

1 credit uses and negative on others. What does this look like? Is it usage based? Do you make a margin on usage? I expect we're probably going to see sign in with OpenAI, sign in with Anthropic maybe coming at some point where people will be able to bring their own access to the models to other products. That doesn't exist yet. Well, it does in the form of I could bring my API key, so I could imagine you could have people bringing their own API keys. To the degree that there are fine-tuned models in those clouds, individuals may want to own them as well. What do you think the outlook on the economics and pricing packaging of this over time?

So obviously right now everything is free. We have this concept of credits. Essentially, if you use all your credits, you just get more and there's more ways of getting more. We'll introduce at some point this year, likely some way of buying more tokens when we also introduced the builder side incentives. I think that's when it really matters for us. In the meantime, our goal is really to just get adoption, to figure out what actually works for people. We'll likely be able to sustain hundreds of thousands or millions of users at essentially full unlimited kind of usage for some time. That's one of the benefits of having Dharmesh Shah as the man behind the scenes. Driving the vision forward and helping us democratize this more broadly. Because that is expensive, right? If you're trying to do this yourself and get even 10,000 users, some of those costs add up pretty quickly. I think we're going to see a lot of consolidation in the AI sort of tools and agents and marketplaces kind of space, because frankly, most haven't figured out a business model. Like most, you're getting traction, you're getting more users. There's certain use cases that we're going to really focus on and build a number of agents again for the use cases that we know and understand very well, like marketing and sales and customer service that are very much sort of HubSpot user persona adjacent, if not centric. And we'll expand from there, right? I think the part that we really want to get right is how do We incentivize builders, right? Is it by acquisitions? I think that's part of the equation. Should we go and acquire five, ten, twenty of these companies? We likely will. I think that's part of what we need to do if we want to make agent.ai a marketplace, the platform for developing agents, for getting distribution, right? For being like the app store. I think that is to. Just help seed this marketplace to get more of the right builders on there, to get more of the right agents, give them some kind of guarantees in terms of a likely livelihood, whether that's a flat fee that we'll pay them plus some kind of usage. There's a few models here that we're thinking through right now. What does that pay package need to look like for people that are finding these agents useful? Is it like kind of the 99-cent approach, happy to pay a dollar a month for something that I find useful. Is that $5 a month? Is that $20, like a ChatGPT level price? If you get essentially unlimited ChatGPT usage through the platform with your own data, with a bunch of other agents, is that a decent price point or some, something between 99 cents and that kind of average price? And then I think the, the There needs to be some type of rev share for each of these credits that a user uses to both give to the builder and also give the builders the tools to then charge for a higher tier of an agent. So again, I think there's this concept that we're playing with right now that will have some agents in a closed kind of, you know, beta phase where you can have a free version of an agent, you can have a pro version of an agent that will just do a lot more, that'll connect to maybe more data sources that are going to cost more money. Do you need this version of an agent or do you, do you need this other one? I think it's early. I think we haven't quite figured it out. I think for the foreseeable future, everything is free. But yes, I think if we want to first make sure we take care of the builders, give them real incentives to build agents, to bring their agents, frankly. To agent.ai to make sure that this is the marketplace for all these agents. That is our goal. And then make it super easy for anyone who wants to, who needs to customize their agents, make it super easy to do that. Create again the right incentives for builders to open some of their models to get the right kind of ratings. Maybe get a higher cut essentially of the marketplace sort of transactions and, and so on, all the way to, to some premium. support and so on. So I think that the future needs to look something like that to have a viable market, but also a viable business where again, at the end of the day, we're looking to build this into a business. Again, we have the, one of the benefits of not needing to fundraise where we're just really focused on building the right product and doing what's right for customers. But I think we also need to figure out what does that need to look like for these builders? is to have a healthy marketplace that can actually sustain itself.

Yeah, cool.

All right.

I love the ambition. It definitely sounds like a good spot to be able to give it all away for a while you figure it out. Don't expect it to be free forever, but take advantage while it lasts. I think kind of changing gears, just zooming out a little bit from the product, I've got a few big picture questions. How are you taking advantage of all the AI tools in building this platform? I have just joined the Cursor revolution in the last 2 weeks. I actually downloaded it 6 months ago, wasn't blown away then. If you were in, like me, in that early cohort, definitely come back, I would say. I've been super impressed now with where it is, and I can't believe I ever actually typed code. Like, what a primitive life form I was. What does your tech stack from the development side look like?

Yeah, I love Cursor as well. I probably have 5 Of these tools like Cursor, your Copilots, the custom things that are built as well. Yes, I fully embraced the AI tools for writing code as well. Our tech stack is fairly standard. Python backend, frontend is all Next.js, React, TypeScript with everything on AWS, everything auto-scalable database-wise, relational database Postgres behind the scenes. And that's basically it. Some queue management and those kinds of things. A very simple sort of stack, very standard. Frankly, we love the simplicity of what Python can do. I wish we could just, you could do full stack development on Python versus you can do that on JavaScript, but you can't quite do that on Python. Technically, that's not true. There are some libraries now where you can build some of the frontend using Python, but Yeah, it's just those 2 languages and we're moving super fast. We're shipping dozens of times a day, which is awesome. We have a small team today. I think it's overall, we're just a team of about 12 or 13 people at this point. 3, 4 months ago, it was just me and Dharmesh and another freelancer doing frontend work, mostly on the backend side and some algorithms, DevOps, et cetera. And we're continuing to grow. I think we're always looking for Anyone who's actually interested in working on something like this, we're only going to grow. And yeah, we just love working on this kind of cutting-edge side of what does agentic AI even look like? What it was 6 months ago versus what it's going to be 6 months from now. I think it's more useful every day. It's more concrete and it's more applicable to end users.

One fine point follow-up there. When you use the Amazon scalable Postgres, does that have like pgvector and do you have a sense for Is pgvector all you need, so to speak? Or, because I've been very confused as to, should I be running out to get the latest and greatest vector database, or should I trust that like the embedding model is actually where it's at and Postgres will probably figure it out? Do you have a point of view on that? Yeah.

Yeah. I mean, we've tried all the approaches. I think now performance of the Postgres vector store is as good, in some cases, in some benchmarks, even better than something like Quadrant or Pinecone. We use Pinecone ourselves. We just love the simplicity of it and it scales very nicely. I think that the debugging tools and searching tools are superior, frankly, to some of the things that you can get out of the box with Postgres. Like anything else, if you're willing to spend and have the engineering bandwidth, then just do it all in Postgres. Keep your life simple. If that may also require some tuning, some optimizing data stores. We reloaded a lot of data and had some cascading performance. This was 6 or so months ago. I think it's come a long way, but something like commercial product like Pinecone is essentially infinitely scalable and we'd rather just click one button and have to worry about which replica on our Postgres setup is going to have some issues or get pegged CPU-wise because there's a bug in the specific implementation, therefore I have to failover and et cetera. I would just rather pay another third-party service that is very good at this, that can do everything we need it to. It's cheap enough. I mean, probably millions or tens of millions of records in something like Pinecone is dozens of dollars. I'd rather not have the headache, right? For dozens of dollars or even for hundreds of dollars. At the end of the day, some of those specialized tools tend to do better if you can afford it. But on the other hand, you're always going to pay for it in some fashion. So, let's say you do get the built-in Postgres version, be prepared to spend some time debugging, optimizing. There's always ways of tweaking and you do need to tweak some of those settings to get Optimal results. I've tried it recently. I think it's much faster than Pinecone, but again, for the kinds of use cases we're talking about, whether it's 15 milliseconds or 10, does it really matter? That's not where the wait time is. It's in the language model, right? It's in like some of those round trips. As an engineering purist, technically, yes, that's an improving solution, but our philosophy at agent.ai is. Pick the right tool for the job. Don't overcomplicate it. Ship super fast. Whatever it takes to ship super fast, just do that.

Yeah, it's a good endorsement for Pinecone. And it's an interesting reason that it's not actually some maybe more fundamental technology difference, but it's really just the polish. If I understand you correctly, it's the polish on the product that it like works smoothly. It's not an afterthought. It's not something that they tacked on and you're not going to have problems with it. And that is a pretty appealing value proposition for a lot of technologies.

Yeah, I think it's like 10 lines of code to instantiate your storage, query it, insert, upsert. They have a phenomenal SDK. There are some libraries for Postgres, but nothing quite as simplistic as that. So when you think about code footprint, maintainability, those kinds of issues, you just, I feed it the connection string and everything else is less than a dozen lines of code. And I have all the RAG tooling that I You really need query data, insert it, delete it, create new data stores for new customers, and so on.

Yeah, cool. Zooming out even further, the future of work is obviously a question that is on a lot of people's minds. And a quick perusal through the AI agent network and how the fact that's presented on par with the human network is like 2 different sections of the site across the header. It definitely suggests that, as one of my good friends used to put it, the robots are coming for our jobs. I personally feel like that is true and that we should probably be more real about it sooner in this process. I've actually admired what I've heard from the CEO of Klarna recently, where they put up this thing that said, this is doing the job of 700 people, and we literally have reduced our headcount in this customer service area because we created this chatbot. And he's, I know that's hard to hear, but I think I have a duty as a leader in this space to let people know that this is the reality that we're headed toward. I feel like in an ideal world, people should not have to do work that they don't want to do. Or if you're only doing the work for money because you need to eat, like in a better world, not to do that work and you could do other things. This of course doesn't mean that there's not other worthwhile ways to spend time. I'm like an optimist on our ability to find meaning.

Yeah.

Even in the absence of jobs that we only used to do because we had to get paid to buy food. But what do you think and how do you guys think about your role or your thought leadership in this space? Because obviously it's much bigger than any one product or platform, assuming that we don't hit a plateau like now.

Yeah. So look, the way I look at AI agents is that they're highly effective, specialized, semi-autonomous applications. That's it. I don't think they're coming for anyone's jobs. I think if anything, that's— they're going to help us do certain parts of certain jobs better, faster, cheaper. I think the human will always be in the loop. I think you're going to need someone to decide, okay, what is good? What is bad? What are the selection criteria? What is the broader context that, that we should really care about and, and give that guidance to these AI agents that are going to increasingly become even more useful tools, even more helpful, even more personalized. I think all of that is true. The way we're positioning Agent.ai is we believe, and this is purely because these are the concepts that we're using today, but we, we believe that AI agents are going going to be part of your team on a daily basis. Whether that team for you as a solo entrepreneur is, it's just you or just you and a bookkeeper or someone else, you're going to have this other entity that can help you do some of that work, but it's not going to take it away. It can't take it away. It needs a lot of guidance. It needs a lot of inputs. It needs someone to say, this is what I was looking for. This is not what I I was looking for. It needs your permission to read certain kind of data and really make— extract meaning out of it and create the right kind of voice for the answers. And yes, I think over time, it's— these tools are only going to get better in terms of approximating what you and I may think, what you and I may say, what you and I need to put together a report. Can it put together a similar looking report Or paper or PowerPoint and extract data in a very similar way to then present some analysis to an accountant meeting. Yes, I think these agents are going to be able to do a lot of that type of work on our behalf. Can it complete the whole thing end to end? Highly unlikely anytime soon. But even in that scenario, again, the way we're framing agent that AI is this professional network of agents. Because if you think, and we believe this strongly, that AI agents are going to do work on our behalf and increasingly meaningful work, increasingly complex work and different kinds of tasks, and you'll even have teams or crews of AI agents all kind of swarming together to try to figure out how do I solve this specific problem and They each get a little piece of the problem and eventually it all comes together and you say, you know, good AI agent and bad AI agent and you get some feedback and eventually you get something useful out of it. And over time it gets better. You're going to want to understand, just like you do with people today on LinkedIn and other social platforms, you want to understand what these agents are capable of, which ones are the best ones. At doing some very specific things consistently, repeatedly with reviews, with endorsements, with the level of like education kind of equivalent, right? How much training do they have? What kind of models do they work really good with? What kind of data is this working in a financial sector or is this more marketing or is this more customer service oriented? That type of social network credibility checks and The professional network aspect of it is I want to understand how good these agents are as specialists at certain kinds of things. Help me understand if I'm trying to get maybe not, you know, Kalanar-level customer service AI-enabled reps, but something similar, right? What are the 3 or 5 agents and who are the builders? Who are the people behind those agents? That are building these kinds of agents, right? Who should I reach out to for my use cases, right? How do I start super simple and maybe that's good enough for 80% of the use cases versus how do I go to something even more advanced, right? The kind of agent that is fine-tuned on its own model to be able to accomplish, which is what Klarna did, for example, right? How can I hire that agent? How much would you pay to be able to get that kind of Klarna AI agent? Within your organization, if I could pay twenty bucks a month and get that, I would probably do it. And I'll probably want to understand what that landscape needs to look like, and for me to be able to gain some kind of value, to be able to build these agents and make that part of my company's strategy in terms of getting some of this real work done. Again, I think it's all highly additive. Just like with ATMs and bank tellers, the bank tellers now do more things. It's more customer. Relationship-oriented and so on. I actually think we're going to have a lot more jobs that people are going to need to do, not fewer. Yes, we're going to elevate the kind of work that we all do, but I think we're going to bring in a lot of sophistication, a lot more thinking about the data and the context and real conversations about biases in that data, because all of these AI agents are biased in some fashion, right? How do we understand that? How do we document How do we constrain it so that we can get some really useful work done a lot faster and understand maybe AI agents aren't the solution for all the tasks? Completely agree. I think these are highly specialized, finely tuned weapons that you can use, but you probably wouldn't want that write poetry for you, right? Write a really interesting novel. Again, there are agents that you can use today that can help you do that. But I still think there's a lot of things that people can and will do a lot better. And I think human plus agent in most of these cases is going to win over just agent or just human. And in fact, I think it's gonna make that work itself more enjoyable, having to do less of the mundane. Do you really wanna do data entry all day and copy and paste data from one inefficient enterprise system to another so you can get the thing? You know, use your creativity for just 30 minutes of the day versus all 5 hours, let's say. Maybe that's your day now instead. I think it's super exciting. I think it's super early. I also think we're going to need to figure out more of the regulatory aspects of this, right? As we think about what does this mean, especially within the US elections on everybody's minds and the election season, what does that mean for AI-generated lookalikes of people and the voices, everything. I think we need some of these guardrails. I think technology is moving much faster than any regulation can move. That's obviously a bit scary. And I think also as builders of these platforms, we need to make sure we're thinking about the safety, the bias, the controls, and maybe say, you know what, we're not going to support those kinds of use cases. Make sure that even if that means turning down potential customers, we're going to have some constraints in terms of what we want these platforms to be able to enable. But I also think we need real answers in terms of regulations, real guardrails that don't stifle innovation. And it's a tough balancing act. You want to advance, you want creative solutions, but you don't want to just kill the technology in its infancy and block lobotomize it essentially and maybe let other places be that cradle of AI advancement very quickly. As well as I think you raised an interesting question. I don't know what the answer is, but it's probably something like UBI, right? Like what is that sort of Star Trek? I love that kind of scenario where yes, money doesn't really matter and we're not doing work for just the pursuit of money and sort of capitalistic gains. At the end of the day, I'm a That's a tough pill to swallow, but I do think we need some kind of solution that does take into account that future state where, yes, and maybe for some roles, some of this technology is going to, to more rapidly impact them. And I think we need some real answers from a policy standpoint that can only be done through government intervention, uh, versus what private companies can really achieve more from.

Yeah.

safety and trying to do good for the community, in the communities that we work and live. I think we also need some of these broader guardrails as well.

So last question on how we get ready for the possibility of a near-term, I might call it agent awakening. And my model for this is pretty simple, but it's basically just Obviously, we've seen every 2-ish years, like, a major step up in model capability from GPT-2 in 2019 to GPT-3 to GPT-4 in 2023. And we're kind of pretty close to due for one. The interesting aspect of what's happening now is everybody's out there building all of the scaffolding and structure to make these things go and to give them rails to be on and to try to compensate for their weaknesses. People are chipping away at it. And I sort of expect that from one day to the next, and potentially quite soon, there is going to be a new, better model available that suddenly might, in combination with all this stuff that's being built right now, tip us over from one equilibrium to another or cross a threshold where today highly autonomous things don't quite work. Everything we've talked about, maybe that changes pretty fast where it's actually now Semi to moderately autonomous things do work. And now we're in a world where the AISDR is not only able to respond to inbounds, but can actually go out and do research and build pipeline for me. And everything turns up at once. So I guess my question is, do you have any— first of all, does that seem like a realistic vision? And if it does, and that could all happen quite quickly, Is there anything that we can do in a very practical, tactical way to be ready for that as a society? One kind of half-baked idea I previously proposed was speed limits for AI agents. Obviously, one of the things that AIs are clearly already superhuman on is they're faster than we are. Maybe there could be a speed limit for AI agents to try to make them move more like human speed. I don't know what that should be. I don't know that anyone knows what it should be, Another rule that is commonly proposed is AI should identify itself as AI. There could be protocols, perhaps crypto-enabled protocols of some sort, to indicate that you are engaged with an AI agent, where it's like, who controls it, what its provenance is, et cetera. Can you put any more meat on that bone? Because I feel like that could be coming soon. I feel like we do have a lot of the stuff that could cause us to tip over a threshold. And I think both what that looks like is dramatically under-theorized and like what we can do to make sure it looks good is also largely a blank space right now, unfortunately.

Yeah. So look, I still think we're a long ways away from that being a reality. We've been almost there for decades now. And yes, I think the advent of ChatGPT and making it that much more approachable has been a huge boost. These new architectures and the combination of hardware advancements and making things cheap enough, making things accessible enough for developers really led to this Cambrian-style explosion over the last 2 years with AI. We're hoping to do the same thing with AI agents with Agent.ai in terms of citizen developers. You don't need technical skills, build agents, tweak them for your own use cases. I don't think we're going to, even if let's say we were to have some kind of major technical breakthroughs tomorrow, I still think you're going to need people in the loop in some fashion, right? Unless if you couple that with robots that can actually, we can plant this into and make them cheap enough, in which again, I think we're even a ways away from, I love my Tesla, but we're a ways away away from actual self-driving in any real shape or fashion if it ever rains or there's snow or anything like that. But assume that were the case. Assume tomorrow we have ChatGPT-6, they jump, jump over 5, it's 6 now. It can plan, it can execute, you know, it's way above human intelligence and it's continuing to learn exponentially at this point. The best way to make sure that We as people stay competitive is to embrace these tools, to figure out how can we leverage them to create business value. Because by themselves, as an engineer, I love cool technology. I have every gadget that you could ever get, and I'm always experimenting with the beta stuff before it's ready for everybody else. But part of what's missing is really tying that to, okay, but how will it make money? How is it going to give meaningfully better outcomes for real businesses trying to do mostly predictable structured type of work on a daily basis. That's what a lot of people are doing in a lot of these companies. What would that mean for manufacturing? How can we improve safety with AI? Like actually make people's lives better and safer in, let's say, factory settings with more sensors or more predictive intelligence embedded. I think there are a lot of opportunities there as well. And really, I think we need to be more curious about how can we make our lives better and evolve the way we've been thinking about work. I don't think it needs to be this kind of doom and gloom scenario where now nobody has jobs and what are we going to do? I think we should be thinking about how can we reimagine every single job. If you could outsource 80% of what you do on a daily basis, 80% of those tasks, How can we structure them? How can we reimagine them completely? Maybe they're not even necessary, right? With more system dynamics optimizations across the organization in a way that will enable you to get the same outcomes or better, hopefully better, and do that in a way that is much more efficient by blending human beings and what they're really good at and their creative thinking and, and some of their Defining parameters for success and tying to the broader strategy and some of the political pieces, some of the social aspects of actually getting the job done versus just inputs and outputs out of a computer. Those are the kinds of things that I think we're still very early on in terms of tying that to business contexts, which again is part of what we're trying to figure out what Agent.ai is. How can we bring these agents to companies? We're very much focused on small businesses. Not enterprise, we're talking about solopreneurs, small businesses, really anyone trying to build these agents and use them. If you're an enterprise trying to use agent AI, it's not for you. We're very much focused on that 0 to 100 sort of size companies and figure out how can we make your lives easier with some of these tools. And we're going to be right there bringing these advancements to people, but also making it easy to build them. I still think even if tomorrow we had, fine, we have a new model and now it's more amazing on certain benchmarks in certain scenarios, you're still going to need to integrate it, right? You're still going to need to change workflows and define the problem and define ground truth. And maybe it's going to be that much better at making things up and hallucinating to the point where you can't tell the difference. Okay. How do we solve that? I mean, there's a lot of things that we haven't like really solved yet from an objective reasoning standpoint. And I think we have a lot to do there, even if we have much more intelligent systems, which I have no question that we'll have them. But I think the other pieces are not there yet to even take full advantage of what we have today, of using that to the full potential, of using that in an efficient way. Now, if we have ChatGPT-6 and it has a 100 million token context window, you do all this kind of work real time. Again, those are going to be very different. It's a different world that we're gonna be living in. But at the same time, we started 18 months ago to 2 years ago, if you're following GPT-2, with 1,000 token windows and very low capabilities. We were thousands of times better now in many regards. Even if these models are thousands of times better than what we have today, I don't think it's going to take away jobs in the way that some folks worry about. I think we still need to figure out so much more in terms of the incentive systems and, you know, tying it to real businesses, which is still very early, which is an amazing opportunity for anyone interested in the space, interested in this kind of work. We need people to help shape that, what this future state is going to look like.

Well, that might be a great note for us to close on. The site where people can come build agents and start to contribute to the marketplace is agent.ai. Hard to imagine a better domain for this opportunity than that. So I definitely encourage folks to check it out. Any closing thoughts before we break?

Just want to thank you again. Appreciate, appreciate the conversation today. A lot of thought-provoking questions and back and forth and I'm extremely optimistic in terms of what the future is going to look like, what platform like agent.ai can democratize, can make AI more approachable, easier to use. I have a 3-year-old son, and as I'm thinking about what does the world look when he grows up, right? When, as he's using more of these tools and thinking about, you know, doing schoolwork all the way to what will this like day-to-day job even look like? We want to make tools that are easy for anybody to use, easy for anybody to build themselves, to customize, maybe not 3-year-olds, but close. I think that's, at the very least, interact with them, essentially talking to Alexa every day and asking for things, then videos to play and things like that. I imagine the world that we're building to be much better than the one that we're living in today. And I think AI can really be Part of that solution, I think we can use it to help solve a lot of problems that we have in terms of climate change, in terms of how do we connect people better and ultimately create a community and democratizing through this kind of technology, make it more accessible so that it really can make anybody's lives easier, better. You can focus more on what you're passionate about, and ultimately we want. Want to make it for everybody. We want to make it something that it's not— you don't need to be in a Fortune 500 enterprise to be able to use these tools, to be able to get meaningful outcomes. You can be an entrepreneur, you can be a small business, and that kind of— that backbone of the economy, that's what we're focused on. That's what we're trying to enable. And humbling to read the feedback of, of people using our platform, and both positive and negative. I think there's opportunities to improve every day and kind of seeing, okay, but this has a meaningful impact. Like we're seeing people tell us their stories in terms of, you know, the different kinds of ideas they've gotten that they wouldn't have gotten otherwise and how easy it was for them to build their first agent to come up with recipes to stay healthier and things like you and I may have never thought of that, but, you know, human creativity is kind of unbounded and it's just super reassuring to see that these tools really create more meaningful connections and sort of human experiences. Hopefully get actual work done better and faster and cheaper, and then help us discover more of what we're really passionate about and help us do that. So super exciting. Last note, we're always hiring if you're interested in working in GTM marketing, sales capabilities, or Or engineering-wise, uh, to help build product, we're always looking for top-notch individual contributors. We're a very small team. It's about 12 total of us in, in freelance capacity or another, but we're always looking to grow. So if this sounds interesting to you, if you want to be part of shaping some of this in some way, feel free to, to reach out, andrei@agent.ai or just through the website. feedback and we'll get back to you.

Love it. Andrei Operasan, Agent.ai, thank you for being part of the Cognitive Revolution.

Thank you, Nathan.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.