The World of Vertical AI Agents - Hamza Farooq

Good morning, everyone. Thank you for being here in Atlanta. Thank you for having me, Khalifa and Mark, wherever you are. I can't see you guys, but thank you for making this event happen. So I actually knew Simon too. I met him when I gave a talk at Vector a couple of years ago. So I'm really surprised and really upset about that he's gone. On that note, what I want to do is I want to start about talking about AI agents, vertical AI agents specifically. And the interesting part is I'm going to keep it as non-technical as possible. I know this is a technical conversation, but we get a lot of people from different walks of life who are learning about these technologies as we are. Seeing them. So my idea is that how can I introduce you to them, like a reintroduction or demystification of what AI agents are and then what are vertical AI agents. So without further ado, I hope this clicker works. Sorry, let's try one more time. All right, so agents have taken over the news lately, and then what you've see in today's, in every day, is that there is news about an AI agent every now and then. The world is exploding with all kinds of information and all kinds of memes about agents. There is not a single day or not a single post that you see that comes across with the information about agents. All right. Awesome. Thank you. Hopefully it will work. Yep, it is. All right, we are alive now. The coffee finally kicked in. All right, so this is what everyday looks like when you're talking about AI agents that they're just everywhere. I think I need to be closer. All right, all right. But let's answer this one major question. What on earth agents are? I promise you, every time you ask someone what an agent is, they're going to give you a different definition. They're going to give you an example that what they think is best. I'm going to try to demystify it for you. Let's pick up a complicated explanation that you see. An artificial intelligence agent is a software that can interact with its environment, collect data, and use data to perform self-determined tasks to meet predetermined goals. I almost fall asleep reading that. It's just annoying. Let's try this again. Explain what agents are like M5. We'll start with a very basic example. Now, we all write our emails and everything that we do in our life using ChatGPT these days. What we do is that we prompt ChatGPT and we say, hey, can you just write me this email, make it concise, make it accurate, and make it look like that it's me and make it also look look witty. And what you do is that you look really fancy doing that, right? So that's the basic idea, what we are using or how we use ChatGPT right now. Now what if— and this is another example— usually what we do is that when we as data scientists or engineers, we want to write code, we prompt ChatGPT again and we ask ChatGPT, to generate a bunch of code for us. This code looks, for example, looks like this. What we then do is that we use a Python compiler, Jupyter Notebook, VS Code, something like that. Basically, what we do at the end of the day is that we put that together and we just copy-paste it and execute that code. Now, if you're like me and you're lazy, why should I copy-paste even? What if there was a world where I could prompt a system, it could generate code for me, and execute on the go also? An example that Paige Bailey showed yesterday of how agents are thinking or generating code and self-thinking and reasoning. So what we do at the end of the day is when you are able to generate code and have it automatically executed for you, through a compiler, that is what an LLM agent is. So I'm going to repeat that. Usually what we do is that we use ChatGPT to generate code. Once the code is generated, we copy paste it, put it into our compiler, and then execute this code. Here we're being a little more lazy. We're letting the compiler generate code and execute on our behalf also. That's what an LLM agent are. If you're able to do that, you now have the sexiest job of the 21st century. That's why everyone is an AI engineer and an AI data scientist because you can prompt and you can put together something. That's what an agent is really. Now, if you were to go expand a little more, excuse me, agents are LLMs which have access to tools, memory, and they are tasked with planning and executing goals. This is the most important thing that if you want to see the one that I have put it in yellow, basically is unlike LLMs, today's AI agents are an engineering innovation and not an AI innovation. What that means is that you still have the same LLM running in the background, all you have done is you have connected that LLM to an ecosystem that generates code for you and also executes that. And then you have added more capability to it by connecting different architectures to it. So if you were to look at this diagram over here, what you have is on the left, all the way on my right here, Here you have the user input coming in. The user says, hey, can you make me a presentation on Optimized AI? Or I'm presenting a talk on vertical AI for Optimized AI. Can you please put together a deck for me? What this will do is it will use an LLM again to generate code or to generate instructions. Once we have generated instructions, what we're doing is we're using different tools that we have designed ourselves. Now, please understand all these tools are designed by humans. You have individually, or you have used a machine to individually create these functions. All an agent does for you, or for that instance, an LLM call is made and the LLM routes a call. So usually in the past what we used to do is we used to create individual workflows Here we can put a lot of tools together and call the particular function when it is needed using feedback from the LLM. So an LLM will generate a JSON output that says, "Hey, call the weather app." And then the weather app API would be called. That's all what the agent does in this situation. So I'm going to just want to see a raise of hands. How many of you know Manus AI? How many of you have heard? So quite a few, right? So what is Manasea? Excuse me. Manasea is this amazing tool that has the capability to be connected to every, almost just about every single thing. And what it does is that it has tools, it has actions, it has capabilities. And it just came out of nowhere. It just came out and it just spread like wildfire. And people have been talking nonstop about it. It can just do everything for you. But if you look under the hood, so what happens is that something launches and 2 days later engineers get a hand on it or, you know, they start looking into it and turns out it is literally a wrapper over Claude. So Manus AI, which is supposed to be a multi-billion dollar company, is built on wrappers and tools that are connected to Claude. So if I could read correctly over here, it's Claude Sonnet, it's Claude Sonnet with 29 tools, Sonnet with multi-agents and use browser use. So at the end of the day, an infrastructure is already built out, which looks like this. When you wrap it around, you make it into Manus AI. And this is why after 2 days Manus AI was released, Manus OpenAI, or Manus Open, was released 2 days later because the underlying infrastructure was so easy to replicate that an open source version was available within 2 days. Now, this is the scariest thing that you, you do a startup, you build something, you think you have moat, and then the moat is taken away within 2 days by an open source version of it. This is one of the biggest challenges that you see in the agent work or the agent world because at the end of the day, a lot of people are just putting wrappers over wrappers to use them and selling them as their own product. All right, so I want you to imagine, before I go into this, how many of you actually don't write code? How many people don't write code? So some of them. I mean, don't be shy, it's OK. I still write code, but I wish I didn't have to. So imagine a world where you didn't have to write code. So, you know, great folks from— I know there's some people from Home Depot who write code. And then there's some people from Home Depot who don't write code. And they don't get along well, right? So what if there was a world where we didn't have to write code, and that's what we call Vibe Coding these days. So Vibe Coding is this concept in which you can literally just write a prompt, it will run the prompt, generate code, entire website code, and make the website available for you. And that's what your faces should be like right now. All of the product managers are like, oh my God, this developer said it would take him 5 days, and this guy is showing me a demo. And by the way, this thing takes probably like 20 minutes to run. And like in 20 minutes, of course, this is sped up, but in 20 minutes you have a customer support system. So I know there's some customers facing teams in from Home Depot today. Just want you to know that this could be done in 20 minutes. And a VP is right now going, dude, why do we, why are we paying these engineers so much money to build these? Because wipe coding is here and we don't have to worry about their Jira deadlines and we don't have to worry about them blocking us. We can just write code ourselves. In fact, we don't have to write code. We can write text, it will generate code for us on the go. So what does that give you? It gives you great power. But I just want to say something. With great power comes horcrux that you never intended. And this is what it looks like when you as a product manager, when you as a product manager are putting together something and you use a Vibe code, like you use ChatGPT to create code for you and you're sending it to your engineers, that's what your engineer is looking like right now. Right? Because they don't know how the hell and what framework did you follow and what were you thinking or what were you on to build that, but that's what happens. So anytime when you're putting together a demo, wipe coding is great for that. These agent wipe coding agents which generate code and execute on behalf of you, they do a good job for demo purposes, but please understand they're not built built for production. That's one of the main points that I wanna deliver. I wanna use all familiar examples so that you all can relate to it, what that means. Now, what is the recommended solutions to use agents in today's world? Like how do you actually go ahead and use agents in your ecosystem? We have something what we call vertical AI agents. And what are vertical AI agents? So instead of building one broad spectrum tool, instead of saying, hey, I'm gonna build a tool that can do every single thing on earth, like minus AI. We are gonna build a tool that does two things and it does it really well. So jack of all trades, master of none. You wanna think about how can you build an architecture of something that is focused on solving basic, like one or two, not basic, maybe it could be complicated, but you are adding depth and not breadth. A lot of times what we see in today's world is we add a lot of breadth to things because we want to cover every single use case that is available. Instead of thinking about that, think about solving 2 problems or solving 3 problems with one vertical AI stack. And that is what we as a company have been trying to pursue. So what's an example of a vertical AI agent? Or what can, can do. So instead of being all-knowing, all-great, all-amazing, what we have is a vertical AI agent that is focused on solving one problem with precision, accuracy, and repeatability on what you want. And this is an example that I've put together. This is a data analyst agent. Now, a lot of, a lot of, Leaders, they need insights to data right away. Currently what we have are tools like Tableau or Power BI. They're still limiting in what they can do. What if you could just talk to a system and it'll give you answers about your data? I'm not saying it'll run prediction for you, but what if it can give you insights about your data through natural language? And while some of you might be thinking that SQL is dead, Guys, SQL and JavaScript are two things after a Holocaust, they're the cockroaches of the world. They will never die. So if anybody thinks that SQL is gonna die, think again. If you think Excel is gonna die, think again, right? What's gonna happen is we just using them behind the scenes. So for a data analyst architecture, you have to think about 5 steps. You have data ingestion, You have preprocessing and query understanding. You have RAG for contextual insights. You have guardrails and report and generation and the visualization. So basically what we are trying to do over here is when we start thinking about a vertical AI agent, and I'll just show some pictures instead of showing the text, what we do is, and this is the most serious part of the conversation, is that we have to add guardrails. At this point, guardrail is not considered a very important step in a lot of demos. But when you come to enterprise AI, the one thing that you need to think about is guardrails of how you're predicting what's coming, what query is being asked, how it's being asked, and who is asking it. The guardrails help protecting that. And this is a very basic concept in which you introduce, you ask a question, you authenticate the user, and you build a router which decides, is this going to be a SQL query or is this going to be a RAG query? And this is very basic to implement, but those of you who has worked on RAG know that how complicated RAGs can be. Similarly, those of you who are data analysts, you write, 2,000 lines of SQL code for one query because SQL queries can get very, very complicated because they have many, many complex joins. This is a similar thing that you can put together in an architecture that builds a router that understands the user query, expands on it, and then routes on based on what is the question. And you can put two layers on top of that. You can put the one layer on top, which is your unstructured data for RAG purposes, The second layer is that you can put structured data behind the scenes. What you get at the end of the day is when you ask a question, it is able to generate a response for you based on a router. It's not all solving, it's solving two things for you. It's solving a RAG enterprise question which you can build separately, and it's asking you or it's helping you answer the SQL queries that you have in your system that you are looking for and based on that you are able to generate a response. And that's the crux of what you wanna build. So why do you want to use agents in the first place or why you wanna want vertical AI agents? When we have worked with our customers, so what kind of customers is going to need this or what kind of customer is gonna need a data science agent? It's become clear that not every single company in the world can afford to have a data science team. A data science team is expensive. They require a lot of research work. They require you to hire a whole team. So there needs to be a VP of data science, then senior director of data science, director of data science, then people who actually work. Sorry. Sorry. I've been in corporate for too long. But if you were to imagine that if you don't have a wing of data science and you have a supply chain ecosystem where you rely on forecasting, when you rely on inventory optimization, when you rely on assortment strategy to know that, and that's the backbone of US industries. Any company that you pick up, you pick up Coke, you pick up Best Buy, you pick up Home Depot, they have inventory optimization, they have forecasting, they need to know which item to put in which store and what quantity. That is a problem that these agents can solve for you when you deliver them and you pass on that workflow to them that, hey, this is the only thing I want you to do. We're not going to give you something which is insanely big. We're not going to give you an open ocean world that, hey, I want you to reduce my cost by 5%. Please do not use LLMs for that. It's going to give you 5% cutdown, which will not make sense to you at all. It's probably what's going to be, it's going to say you should cut your top 5% items or something. The idea is you make, you use AI agents to automate a workflow. What you want to think of agents right now is bad interns. Anyone has a bad experience of a bad intern or was a bad intern? I was one, right? So think of agents as bad interns who come in and the first time you will run an agent, it will probably produce something that you don't want. But over time, if you improve your workflow, you identify, if you improve your workflow, you identify the right use cases, you identify the pattern that you're looking for in your data, you can make them do the workflow. And this is not automation, just running a shell script. This is deciding and decision-making as we go. One of the biggest problems of automation is that we can't think of all the use cases. This thing can help understand a multitude of use cases. How do you determine that these vertical AI agents work? We build benchmarks. We as a company, a traversal.ai, we're building data science benchmarks where we are understanding the complication of— we have basically created simple, medium, complicated, God mode SQL queries. What we do is that we test our agents and what is their proficiency in getting us answers to those individual questions. Once we understand those benchmarks, we know that they're performing. Because one thing is very clear, every one of you can make a really good demo of an AI agent, But I, what I'm pushing for is when you build enterprise, it has to be fast, it has to be scalable, it has to be accurate. And those are the things that we don't measure anymore. Because what we want to do is we want to push a demo out and we think it's a great demo. And then when the engineer actually uses it, remember the, the picture of the Horcrux? That's what happens at the end of the day for all of us. Because We have not done benchmarking. So our use case has been use particular AI agents for specific use cases, benchmark the heck out of it. And once you understand the benchmarking and you know that what is the performance, we have seen that with the customers that we've implemented these agents, we have seen 80% in data analysis time, we have seen 20% lower operation cost and 30 to 40% reduction in investment, inventory waste because they did not have a data science team at that point and they needed direction on what to keep in what store or how much to produce for a given week. That's what we have been, that's what we see measurable growth. So identify a pain point, solve that, and once you're able to solve that, your customer's gonna pay you money to get you there. And we are not the only company who's building vertical AI agents. There are a ton of companies. This is an ever-growing slide. Every week, 5 companies, 5 or 10 more companies are added to it. But these are the organizations that are building vertical AI agents in different use cases. There's customer support, there's insurance, there's legal, there's financial services, healthcare. All of these companies are building agents that can help you put to work. Eventually, there'll be one that will be unified, that will unify them all. So that it's the same problem that you see right now is that an organization have 50 applications running. You have a Slack, you have a GitHub, you have Power BI, you have Tableau, you have so many different tools running. Similarly, you will see a huge ton of vertical AI agents being introduced and eventually Salesforce company will come and they'll unify them into working together. And this is the last example which I have about our company in which, What we do is that we pass a request, a single request as a user would speak to a system. And once the user is able to speak to a system, it determines in that realm, in that vertical, what are the objectives and how to run. And it sort of works, goes like a workflow when identifying step 1, step 2, step 3, step 4. And not only it's able to run particular model on, you know, on demand, it is able to generate response and a feedback that, hey, this is how you should go by and you should be able to build something. This is the response which you're looking for. Why is this useful? Because this is immediate on-demand information available to you, which you probably haven't run before. It allows you to run different workflows and use cases which were not available previously. That's the idea that we want to push forward is there is hope for agents beyond the demos that we have seen on what they deliver to you and how they work. What we have to identify is specific use cases. Once we identify specific use cases, we can build agents that can solve that particular problem Do not build horizontal tools that can solve everything. That is one of the biggest problems that we have seen with a lot of customers is that they start out with something and they're like, hey, we want one agent can do everything. Instead, build vertical agents that go deeper in 3 use cases. Once you get that, you can build a really nice tool that solves the particular problem and then you can expand on that. With that, that's the end of my speech. Thank you, everyone. We're doing— we have a workshop about mastering AI agents, how you build from scratch tomorrow. So if anyone is here tomorrow, please feel free to join our workshop. It will be a full-day workshop where we teach you how to build your own agents from scratch. Thank you. Thank you.