Introduction to Creating AI Agents in Python (Part 2: Hands on)

Hi there, data scamps and data champs. Welcome back. I'm Richie and today we are continuing to talk about AI agents. So if you joined us yesterday, you know what it's all about. If you didn't, don't worry, you can check out the the recording for yesterday, LinkedIn resources and you should still be able to code along with us without too much trouble. So I'm hoping some of you have a clue what's going on with AI Agent. It was all explained yesterday. So without further ado, I want to welcome our guest, Richmond Dalake. He's a developer advocate@mongodb and since I introduced him fully yesterday, I think we can just jump right into coding. So please do take it away, Richmond.

Excellent. Thank you guys for joining me today. So yesterday we started off really talking about AI agents, high level concepts and really, really just kind of connecting aspect of memory within humans to agentic system. So, so it was good stuff. Today we're going to get our hands dirty and actually look at what building an agentix system looks like and implementing different forms of memory within the agentic systems. So we've got an hour and there's a lot to get packed in. But even though if everything is not covered, the notebook will be shared with you and the GitHub will also be shared with you. So on that note, also you can always ask ask questions in the comment section as we go along and someone has requested, would it be possible to have little screen with Richmond Alarke in part of the presentation where it doesn't cover the text of the presentation? I don't know. Can we make that happen backstage folks? Yep, that is gonna get sorted out for y'. All. Okay, I'm gonna share my screen and let's get that started.

So today I'm actually going to set the scene for what we're going to be covering and let me know Reese, if we can see my screen.

Yep, Ron. Okay, excellent. So normally when I do this, so when people do this they tell you a bunch of stories and whatnot, but I like to do things differently. I'm gonna skip to what we're going to work to and show you what we're going to create.

We're going to create an agentic system. It's going to be one agent that has access to a bunch of tools. These tools are essentially different ways to modify data and also do some form of real time search. What I'm going to be doing now is just showing you guys what we're going to work towards in fact to put Myself in a deep end. I'm actually going to run this again and we can then work backwards. Then I can explain what's going on, what we're doing. Let's run this. I'm just going to say hi hider, it's calling an agent node. We're building an Agentix system and there's an error that happened and it looks like I've overloaded.

My.

My token limit which is fine. So let's pause it there, let's see if it does it again. Okay, it looks like we're back. So this typically happens if you have rate limiting on some of the API keys we're going to be, we're going to be using today so run Agentix system can be expensive in terms of token and some providers like we're using today, Anthropic have rate limit. So this is something I'm just gonna mention very earlier on and I guess for, for Anthropic they have maybe a free tier system for rate limit and you're limited in terms of the tokens you can pass as input and if it exceeds a certain amount of tokens you will be limited to, you will get errors like this. But I have a retrial retry system in my agentic system so whenever it fails it tries again for a couple of times then it works. But basically today we're going to build an AI powered logistics assistant and it's going to be focused on shipping use cases, mainly supply chain. And the reason why I've chosen supply chain is because it's Black Friday very soon, I think next week. And Black Friday is usually a time where people buy a lot of goods, there's a lot of sales going on. But what's happening behind the scene is there's a bunch of cargoes and goods moving from one part of the world to another. It's always good to maybe automate some of the manual process and maybe make things a bit agentic. I want to show you guys a real life use case. In MongoDB we worked with a partner called Gravity9 and they actually solved this problem for one of their customers and built an agentic solution. And what we're doing today is showing you a skeleton version of that just to give you real world practical use cases and so you can understand what happens. So if we're talking about real world, yes, errors happening within agentic system is a common thing. But error and in as well is something that you should always consider when you're building and I guess when you're implementing Your Agentix system you actually have to build some error handling mechanism. Sometimes you might want the Agentix system to actually not do anything and report the error. But in this case I've just, I've just set a retrial mechanism. But let's see. So we're building an Agentix system and it can help us with shipping inventors in our supply chain. I'm going to show you the data set, I'm going to show you exactly what we're talking about here. Now I'm nervous to like ask it questions because I don't want to hit the token limit for the real demo but let's see, let's ask it. What can I you do? I've given it. So the rate limiting is usually limited to a certain number of tokens per minute. So I think I've given it a minute so which is good. So I've asked it what can you do? And it said hey, as a, as an AI powered logistic assistant for an international shipping company, it can retrieve information, give us shipping status updates, give us information in the supply chain and provides operational support. So this can do. The AI agent can do a lot of things. So I'm going to stop it there. Just giving you an example of the demo that we're going to be seeing later on. I'm going to be very cautious of time. There's a lot to cover. So if I don't get to the end I will still show you the end result of this demo actually getting the Agentix system to do a couple things. On that note, I'm going to go back to the beginning. We're going to leave the Agentix system running while we go through some of the stuff that is happening within this notebook. You all have the notebook or have some version of the notebook anyway and you can understand the objective of this Agentix system, of the solution we're building. But also I also explain what we're doing, how we're actually building the system and why we are doing it. There are also use case examples that we've actually placed in here to help guide the execution or the implementation of tools for agency system. So yesterday, if you watched the session yesterday, one thing, a question I was asked was how do you implement AI applications or think about it? And one way that I think about it is to break down some of the steps and I think Richie, I'm Richie but I'll call him Richie too specified that it's always good to think about what you want your user to use. The AI application for and you can work backwards. This is what we're doing here. We're thinking what would an operational manager ask of this Agentix system? And then we're working backwards to build this AI application. That's enough background. As I said, this is a real world use case. And MongoDB, we have a partner within our program called Gravity9. And this is a use case of an Agentix system that they built for one of their customers. And this is how they went about it or a very simple version of how they went about it. One thing I'm going to show you is this notebook is a two parter. So part one we touch on semantic search and RAG implementation. Then in part two, Part two and I'm showing free, but in part two we make things agentic. I wanted this notebook to really show learners really what we've been talking about for the past year, right? So we've been talking about semantic search, we're talking about vector databases, we've been talking about rag, we've been talking about AI agents. You get all of that in one notebook with real world application. So let's. On that note, we can get started. For those of you who were here yesterday, I touched on this process which is a RAG pipeline essentially. I don't want to spend too much time going over this. I'm going to go straight into the code. The coding section within this code within part one is going to outline what's in this diagram here. This is what part one really shows. What we're doing in this diagram, the first thing we're going to do is we're going to install all our libraries. We're going to be using data sets. This is a library by hugging face that allows you to access an external repository of data sets that are publicly available. We are going to provide you the dataset. For this particular use case, we're going to be using Pandas, which is a Python library that enables efficient data operation and processing. Within Python we're using Pymonger, which allows us to connect to our python to our MongoDB database and allows us to actually access our cluster and perform some operation. Then we're using Pydantic. I don't think we're using Pydantic too heavily in this notebook, but Pedantic is a data validation library focused on Python. But for LLM and if you remember yesterday when we spoke about Agentic system, you need LLMs. For agentic system, the brain, we're going to be using a model from Anthropic and For our embedding model, we're going to be using Voyage AI and I'm going to tell you why I chose Voyage AI for this when we get to the embedded model generation. So on that note, we'll move on to the next step. Let me zoom in so it's a bit clearer.

And if you have any questions, feel free to ask in the comment section and I will leave some time to get to them. So the first thing, this function just allows you to set your environment variables. There's going to be a few environment variables that we're going to need to up set, set. I'll go through them and I'll talk about where you can get them. We're going to go through this data loading stage. If you remember in our diagram, first we have to get our data. We're going to go through this data loading and we're going to be obtaining our data from the mongodb. Org on hugging face. And we're going to be accessing the supply chain contracts data set. Small yet it's a small data set with about 100 data points or I think 200 data points representing some inventory information that is relevant to the supply chain industry.

So we're going to convert this data set into a data frame and then we're going to visualize it and here is what it looks like we have. Each data point basically represents a shipment from one location to another. But whenever we're shipping cargoes from one location to another, there are, there are a massive amount of information that goes into, that goes into the, maybe a contract or any document regarding that cargo, such as the description of it, the quantity of the cargo, the weight of the cargo, the value of it as well. But also things like the inventory status, is it in transit in terms of is it in transit in custom or is it delayed in actual transit when it's moving from A to B? But this, this inventory status is referring to a con. When, when you have a cargo that is in custom, you can actually track the status of this, of your cargo within the custom process as well. And sometimes you can get delayed. We're also going to be tracking some penalties for late fees and liability. So this is really relevant information for the supply chain industry. But everything we're going to be showing you is in this session can be applied to pretty much any industry. Right. And I mean the techniques I'm showing you today, semantic search rag agents, you can apply that to any use case with any data sets. We're just choosing supply chain for today because It's Black Friday next week. So the first we're going to do for step two is we're going to set our embedding, we're going to generate some embeddings. And I'm using Voyage AI. Now, Voyage AI is an embedding model provider. But one thing that they do that is very different is they actually create custom embeddings for particular use cases. So in terms of this particular use case, I am using the Voyage Finance two.

And this is an embedded model that is essentially trained to work well for financial data set or data set that contain a lot of numerical values and a bunch of financial terms. So this is what I've ch specific embedding model. The embedding dimension is going to be 1024. Whenever you pass in a data object into this embedding model, the dimension of the vector embeddings, that is output that comes out as output, has a size of 124 dimensions. This function is a very simple function that gets. That basically takes a text as input and we have the input type, which means that we're specifying the type of embedding method that we want to use for the particular data set. This case, we're going to be setting it a document, which means that we want to be embedding. We want to set the embedding to actually handle documents that we want to transform into numerical representation. This is a simple function called get embedding. We pass in a text and then we pass the text into the voyage embedding and we call the embed function. And then we get our embedding, which is a numerical representation of the text of parsed in. But the real thing is we want to specify what fields we want to take into our data point. We want to take in the fields that are relevant and we can extract semantics from. We're going to take in the shipper, the receiver, the description and other attributes of a data point that allows us to extract some semantics. What we're going to do is combine this in a new attribute called combine text. Now, this combined text, we're going to combine essentially all of these attributes in a new field called combine text. This function, this apply function, essentially just does a combination for all of the data points within our dataset. Next, we can generate our embedding. We're just going to use the apply. We're going to use the function get embedding against all data points within the data set, and we're going to use the combined text as our text argument that the get embedding function is expecting. We're going to store the results in this new attribute called embedding. All right? If you have any questions, put it in the comments. But one, and what we're doing there was just. This is a very simple RAG pipeline. There's nothing complicated there. So we're not looking into any chunking strategies. We're just. This is naive. We're just gonna pass in the full text and not conduct any chunking strategies. We're going to keep things relatively simple, but we're also going to go a bit complex in terms of the form factor of our application. So I'm going to remove this combined text. I don't need it anymore, so I'm going to remove it from the dataset. But let's see what a new dataset looks like, the one with the embedding attribute. And you can see we have the numerical representation of all the attributes that we selected to pass into the embedding model. Step three, we're going to connect to our MongoDB database. And one thing is MongoDB is going to be acting as both as our operational and vector databases. What this means is you will store your metadata, all of this numerical text data in MongoDB, but also we'll be able to store our vector data in MongoDB as well. Rather than having two separate databases that can handle different forms of data, you have one database that could actually hold your operational data, your metadata and your vector embeddings. And you can do different search retrieval mechanisms on other data types. We're going to look at how we can do vector search and also hybrid search with mongodb in the notebook. You can follow these instructions on how to register for Atlas account and also you the steps to get your connection string, your URI that connects you to the that represents a connection to your MongoDB cluster. And with MongoDB you can actually get a free tier as well, which will allow you to go through this, through this notebook. So I look forward to you guys creating your MongoDB account and going through the steps to get your connection uri.

But once you get your connection URI or your connection string, you're going to pass it into your environment. But if you're using the Data Lab environment, you can actually put in the environment variables in Data Labs. But in Google Colab, we're just going to use this with the function and we can import our environment when the input box prompt us to all of this is the next step is we're going to establish, we're going to create a function that establish a connection to our MongoDB cluster. We're going to specify we're going to pass in our URI to create a Mongo client that will give us access to retrieve from the Mongo client. We'll be able to create a database called supply chain, because we are actually the use case here is for the supply chain industry, but we're going to create a collection called shipping data here we're going to get an object of our database once we create it, and also an object of our collection from the database object. What we're going to do is we're going to delete any existing results. Because this is an experimental notebook, you might want to do this several times. Normally when I'm doing my presentations or when I'm experimenting, I always want to start with a clean slate of a collection. I usually delete any existing records. I can just ingest some new data. But if you're doing this for the first time, this would just be zero and it'll be all fine. Then we're going to ingest our data, this process, and some. For those of you who've gone through this process as well, you might realize that when you're ingesting data from within a RAC pipeline, sometimes you might go through some transformation to get data in the right format. You might do some serialization to get data in the right data types to be ingested. But with MongoDB, because MongoDB is built on essentially the document data model, which essentially replicates storing data within that JSON format, usually it's BSON format, but JSON format, we can actually do something that is very simple. We can ingest our data directly into MongoDB in just two lines because we just convert our data into our data set into a dictionary, which is basically a data structure that follows that key value pair similar to JSON and just ingest it directly into MongoDB. Ingestion process with MongoDB in languages like JavaScript or Python or any languages where you can replicate data in JSON is very quick. Here is just two lines and with some Python wizardry, you can make this just one line. But once we ingest our data into MongoDB, we have all our data in MongoDB. I'm going to show you what that looks like. We have it over here. We have the shipping data and I will show you essentially what that actually looks like once the data has been ingested. So let me zoom out. Looks like we're too zoomed in. So once the data is ingested, if you remember, we have all this is a document and this is one data point, and you have several documents. So we have 202. We have 202 because there is again, we have a working memory, which we spoke about yesterday, but I'll explain it later. So here we have our data set ingested into our MongoDB database. And you can see we also have our embedding sitting right next to our metadata from the data set we have, as opposed to having this within a separate database. Looks like I wasn't sharing my.

MongoDB tab. I'm doing that now. Let me say that one more time. Here we can see the MongoDB database, and here we can see the collection, the documents that we've ingested into MongoDB. We have 202 documents and we can see what one document looks like. This is all the metadata that we saw earlier that describes a cargo. But we can also see the embedding over here. This is essentially we can see all the embedding alongside the metadata. So we don't need to have a separate database to hold our embedding. We just have it in one database, our metadata and our embedding data. So I'm going to switch back to code and we're going to share this tab instead. Okay, looks like we are on the right tab. So we've ingested our data, but for us to actually conduct vector search, we need to create a vector search index. And this allows us to actually conduct vector search and actually do it in an efficient manner as well. All of this code here is just going to set our vector index to conduct vector search. And it's a very simple function that takes in the collection that you want to create the vector index for and the index definition and your index name. You need to go through the step to make sure vector search actually works because of time. I'm just going to skip to the part you follow this notebook, you'll be able to understand what's going on there. And it's nothing too complicated. The key thing is your vector index should have the path of the path of where your embedding data is located and the dimension of the embeddings. The dimension in this case is 1024. And this is the dimensions of the embeddings outputs from the VOYAGE AI embedded model we use. And it's that simple.

Then this function creates it. And if you run this notebook several times, you will get this error where it states you have a Duplicate index. But no worries, it will still work. You run it for the first time, you're going to get your vector index created. All right, if you have any questions, please put in the comments and I will get to them. When we get through the entirety of the notebook, the next thing we're going to do is actually create a vector search function. I'm going to quickly tell you the key parts of this function. This is a function that allows us to basically search for documents in our collection using semantics, essentially using meaning. We're going to take in a user query and we're going to convert the user query into numerical representation. Just going to zoom in a little bit here so we can follow. We're going to convert the user query into a numerical representation, pass the numerical representation into something we call a vector search operator. In MongoDB, the way things work is we have something called an aggregation pipeline. And if you're from a machine learning background or data engineering background, you would know what a pipeline is. A pipeline can be a sequence of process where the processes within a pipeline execute sequentially or in parallel. In this case, in this pipeline, we're going to be executing a process sequentially and the output of one process goes into the next and then the operation continues. Just to show you, let's work backwards from the code here we create a pipeline and we can see we have this array of some pipelines. We have a vector search stage or vector search process. We have a stage called onset. I'm going to explain what that is later. Then we have a stage called project stage. Now, what this is is different processes within our pipeline that can operate on our data. If you're from a machine learning background or data engineering background, or even a software engineering background, this is easy to understand. And that's what MongoDB is doing, right? We're making developers have a familiar and easy way to understand how to work with data and construct queries that operate on data. Another key thing that you observe and that observant people would have noticed, is the way you construct queries in MongoDB is it follows the JSON format, right? We have that key value pair format where you specify what type of process you want to run. When I run a vector search process, then we specify the fields that goes into the process. And you can think about this as arguments for a function, but these are arguments for the process we're trying to do. The key thing is we're going to run a vector search process and specify where we expect we're going to Pass in the vector the query embedding and actually specify the path of the embeddings in our documents that we want to conduct a vector search against and the number of candidates we want to consider for the vector search operation and the number we actually want to return from this stage. Once this stage is done, it passes the data that is extracted, that is passed out as output, into the next stage, which is the onset stage. In the onset stage, what I'm doing is very simple. I'm just removing the embedding from this stage. This process removes the embedding data point from the documents that have been returned because I don't need the embedding in downstream processes. That's fine, I could just remove it. Then I pass the data into another stage, which is called a project stage. In this stage, I'm just including, or not including some of the attributes that I want as well. I'm just showing you different stages as to what can be done with MongoDB and how it feels familiar to developers across different programming languages. Any questions you have, leave it in the comments and I'll get to it. Once we execute our pipeline, we just have to say, we just have to state quarter collection and we call the aggregate framework, then we're passing our pipeline. What happens is all of this process act upon our data and we get our results and we can return the results as a list. Now we can get into the semantic search aspect. We've heard about semantic search, which is essentially retrieving results from a collection or any data source, but retrieving it based on semantics or meaning. We actually. Or searching by meaning. So here I'm searching. What are the custom requirement. I spelled custom wrong.

What are the custom requirements for toys shipped to Canada?

That's our query. If you give me one second, I'm going to go get some water because I feel myself getting a coughing fit.

All right, I guess I'm just improvising. Richmond's away. Okay, so trying to recap what we've done so far. So basically it seems like a lot of the stuff that's going on here is just about getting the data, putting it in a form where the LLM can understand it. So if you've not come across vector databases before, we do have courses on that. We actually have a series of AI code alongs as well. So definitely learn all about vector databases and how you break down the contents in a form that LLMs can and can consume. So that's, that's, that's the big thing that you should have been concentrating on. I'm gonna go away now. I'm gonna let Richmond continue.

Oh, sorry, Richie. One second.

Okay, looks like we're back. Excellent.

So sorry, guys.

I. I was. I found myself having a coughing fit. I want to keep going, so I didn't want to. Want to have a coffin fit on a live stream. I think we're good to go back to the code. I'll get to the questions once we get to the main part of the demo, I'm being cautious of time. What we're going to do is we're going to pass in a query. We pass in a query, we pass it into our vector search function that we specified earlier. Remember, this query is going to get transformed into a numerical representation and it's going to be. Then we actually then get the results of this vector search operation, which will be the list of documents that match the semantically similar to this query. What you can see is once we actually get our results, we get five data that are semantically similar to the query. That's semantic search. But we could take it one step further. We could take it one step further.

And implement rag, where you actually use some data that you've extracted from a knowledge source and parse that and supplement it with the input query, which could be a prompt from your user and pass that into an LLM. Then you're able to get a response that is grounded in relevant data. We're going to see that in a second. But first things first, we're going to be using Anthropic as our brain. In this case, we're using Anthropic as a model provider for the RAG pipeline. You could go over to Onpropic and get your API keys. They have a very good documentation on how to get your API key. But like I showed at the start, if you're just joining, there are rate limits in the API keys and Agentix systems can be expensive in terms of token consumption and also generation as well. There are some rate limits on Anthropic, but it should be good enough for you to. For your POCs. So we're going to get Anthropic API key and I'm going to be using. I'm going to be. Then we import our Anthropic module and then we're going to actually set our Anthropic client and this will allow us to access the LLM. Essentially, this function here is a very simple function that handles a user query. Let's say imagine this function is like a chatbot and it gets a Chatbot, then it gets a prompt from the user and what happens is we convert the prompt. Remember this vector search operation, this is the one that gives us the semantically similar documents. But once we get those document, this is some, some formatting of extracting information from each of the data point they were able to then pass, if we assign them to the search result variable, they were able to pass that variable search results as input alongside a query to the LLM, which is in this case claude Free Opus is an old version of the anthropic model, which is still fine because this is what we can use to actually synthesize or generate a response from our LLM that is grounded in domain specific data.

We're going to ask it the same question. What are the custom requirements for toys shipped to Canada? Before when we did semantic search, we just got a list of documents. But now what we can do is we can pass it to the LLM, pass it into the function we just explained in the previous cell and we can actually get a response that feels like natural language. So what it said is based on the shipping contract details, provided the most relevant information for answering the questions about customs requirement for toy shipping is in contract. And it gives us a contract number, right? And it gives us some details about the contract. Now that, that is amazing because now we have this system that can based on a query can look into a knowledge base and extract some information and respond to us in a manner that feels like almost human. Like this is where you can create a chatbot for maybe enterprise search or maybe for your own users where if a user has a question about certain items within your databases, they don't have to go to the data science team or go to any team responsible for data. They just have this interface where they could just pass in natural language query and just then request some specific information about the knowledge base or the knowledge source. That is semantic search. And that's rag. Now we can make things agentic. We saw this diagram yesterday. For those of you who are joining today that weren't here yesterday, please watch the replay. That's, that's important because I've explained what goes into an agentic system and what makes an AI agent an AI agent, but very long story short is an AI agent essentially is an LLM which we, we've seen already. We're using CLAUDE in this case and, and what it does is it uses the LLMs for reflection, reasoning and breaking down some of the prompt has been given and then it has a bunch of tools, it has access to we're going to be looking at that later, but it's also backed by memory. In this case, we're using MongoDB as our memory provider for agentic system. We're going to see three forms of memory today, which is very good. What are we building? I showed this yesterday as well. We're building this very lightweight agentic system which can search for real time data but modify records and read records within a knowledge store. We're going to see working memory, we're going to see the execution of all the outputs of each of the process within our Agentix System logged into MongoDB as well. Three different types of memory. Again, I covered more types of memory in the session yesterday because we're doing.

A lot in this notebook and because we don't have all the time in the world, I'm going to talk about the most important bits, but let me break down the steps of creating this agentic system.

I'm using Lang Graph, which is like an agent orchestrator or an agentic framework built by the team over at LangChain. So mongodb works very closely with the team over at LangChain and we have a MongoDB integration called LangChain MongoDB which allows you to connect, to actually connect to a MongoDB collection easily and actually access all the retrieval mechanism we've spoken about. We're going to see all of that today. I'm also going to use a LangChain anthropic module as well to access the chat anthropic class.

For those of you that are interested in monitoring and observability, which I showed is an important part of the AI stack In yesterday's session, there's an optional cell that shows you how you can use langsmith or set up langsmith for monitoring the execution of your Agentix system. This is a very robust notebook where we're covering a bunch of things in one notebook. MongoDB has a checkpointer built in with Langgraph that allows you to store the state within langgraph within mongodb as well. So we're going to talk about this, but the first thing we're going to do is we're going to create some tools. We're going to create some tools. These are just Python function. We're going to define the brain of our AI agent, then we're going to create the AI agent node and we're going to create the tool node with Land Graph. The way we view an agentic execution process is as a graph and within a graph you have nodes and. And each node can represent a process essentially. And the process has an output and the process can feed into another process. Here we have agent and we have two node and the agent can execute and pass its execution into the node, into the two node. And the two nodes can then pass its results as output into the agent node. That's essentially how things are working. We're going to see a diagram that shows this. In fact, let's see that. Now in this space we have an agent node and we have a 2 node. And here you can see that they can go back and forth in a cyclical manner. But then there's another process where that could end the entire execution. Once we have. Once the objective has been achieved.

All of that is how we're building a Gentex system. Very simple. But one thing I want to focus is how do we create tools?

So the tools essentially define how our Agentix system operates. So if you remember, if you look at this, these are particular use cases or scenarios that we want our Agentix system to accommodate. And they're based off the scenario and the problems we're trying to solve. You can read this to understand what the agency system can actually do. So I've specified scenario 1, 2, 3 can be achieved with a tool that searches the collection using hybrid search. The keyword is hybrid search. What is hybrid search? Hybrid search is a combination of different search mechanism. In this case we're going to be combining vector search and text search and using that to retrieve information from our MongoDB database. We're using MongoDB as our vector database. That's how we could conduct vector search. But MongoDB is also storing our metadata and we can use text search for that. This is why we're going to be using a combination of both to create a hybrid search mechanism. How do we create a tool this steps? I'm actually showing you how you create your index for the text search mechanism. It's very straightforward and we have lots of documentation on mongodb.com that explains the purpose of this indexes and how you create them. I'm going to go and skip ahead into how we actually use LangChain to implement hybrid search. First thing we're going to do is we're going to initialize our vector store.

We're going to get the MongoDB Atlas vector search from the LangChain MongoDB integration and we're going to connect to our connection string over here. We're going to connect to our database by passing in the connection string and specifying the database and the embedded model and the index, that name as well. This is our vector store. Once we have a vector store, we can actually pass that as arguments into our hybrid search class or hybrid search retrieval class. Now we have a vector store in our database, but we can also conduct text search. When we pass in our text index here, we can see that we're passing in the same query we did earlier on when we looked at the RAG use case. But now we're using hybrid search and we can get a list of results that is more robust because we're using two different search mechanisms. On that note, we can talk about how we convert this hybrid search, this ability to search our database. How do we convert it into a tool that our agent has access to. The way we do that is we're going to create a function called hybrid search and we're going to pass in the hybrid search functionality, which I explained earlier on, we're going to call the method get relevant documents. And this is all abstractions from the LangChain library, which saves you the time from writing all the query. We did earlier on, where we created our vector search functionality with the MongoDB query language and we saw the pipeline and the stages within the pipeline. Here we could just say with LangChain we have this abstraction and we could just call the method get relevant document and pass in the query. That will give us our results. Over here we have our results. I'm removing the embedding from the result, and then we can convert the result into a string and that could be passed. And that is essentially our tool. But to make this into something that the LLM or the agent can understand, we're going to use the Decorator tool, which is from the LangChain library. This essentially just converts this into a JSON definition, that JSON definition schema that is passed into the LLM. And the LLM can understand that and can know when to call this particular tool. The key important bit is the name of the tool and also the Google Doc string of the tool as well. You should make this as descriptive as possible, because this is what the LLM we used to determine when to actually search, when to actually utilize the tool, depending on input query. That's one tool that we've just seen. But there's another tool. We have a tool to update status within our knowledge base, and this is just a simple MongoDB query language statement that just updates a particular field within a document. So we're updating the current transit status with a new status and we're just going to identify the contract or the field or the document to update by the shipment id. We're going to use the set operator to basically update a particular field. As simple as that. Again, we call it the right function and we wrap it. We specify the actual decorator on top of of this. That's how you create tools. And once you're done with this, just because of time, I'm going to move on to how you can then parcel of this into an array and into the LLM. What we have here is you have a toolbox and we create an array of all our tools and there's an optional stager that creates the ability to search real time information from the Internet as a tool as well. So we're using a tool called tavily. I explained all of this yesterday. So you can look at the webinar from yesterday to explain what we're to understand what we're doing here. We're going to use tavily to access the Internet. But long story short, we're going to convert, we're going to actually get the process just going to scroll down. We're going to get the process of getting real time news on shipment and we're going to make that a tool as well. We just create a function that specifies all of this and returns the results of any query that we've searched. Using hybrid rag, again explained what that was yesterday. We're going to pass that we define into a function and we specify user to operator to convert into a JSON definition JSON schema for the LLM. How do we pass it all into the the LLM? Here you can see the tool with this particular tool working and it's able to get some. I told it to get some real time news on this particular shipment contract and I was able to do that. And it showed us that there was a congestion in Brazil and that could affect our shipment. And this is real time news. So the function I created earlier basically constructs a query, constructs a query based on the shipment id. Then we pass that query into tavily, which is enabled to give us real time news. And then we can use that to inform any information or any user of our tool. We then append this tool into our toolbox.

Next thing we do is we define our LLM very quick and very simple. We're using Claude and we're going to use the Chatanthropic class from LangChain and this will allow us to construct an nlm. We specify the model and other arguments as well. Next we're going to define the state. Now a state is a data structure that captures different outputs that is occurring within the nodes within our agentic system. The rest of this is more land graph specific process and and LangChain, AKA Langgraph have extensive tutorials on their platform on how they think about state and agentic execution as well. Very high level. I've explained that within Langgraph we see things as nodes. So we have a chat node which is a node that has our LLM, a Chatbot node which is a node that has a LLM and we have a 2 node which holds all our tools. What that looks like is this. We have this node, I have all our tools in this node and the agent, the chat, the LLM can basically access tools to execute on an input. Now we can go and see this in action. I'm going to use a different thread and this has been running for a while just because of time. I just want to show you guys maybe the full extent of the execution of this agentic system. Just gonna quit it and I'm gonna run it again. Hopefully we don't get hit any rate limit. So let's go. Hi there.

Let's see.

It looks like I've made a mistake over here. Human X message. Okay, it just looks. I just did a typo by mistake. Let's run that again. Unmatch. I'm making errors here.

Why?

Okay, let's go. Five times the charm. Hi there. So calling the agent node. Fingers crossed we don't hit our rate limit. Excellent. So he's responded. Hey, welcome to our AI powered logistic assistant. Is there a specific area you like information on? So it's told us what it can do and it's basically here to help us. So let's pick from our database. So let's pick the first one. If you guys can see the shipment id. No, let me share this tab. So we're gonna go to our database and we're gonna look at the shipment id. I'm gonna ask it to get me some information of the shipment id. And one thing to note is the current status of the shipment ID is delayed. So remember that guys.

So I'm gonna ask it share this tab instead.

What is the status of. So I've just asked it where's the status of this shipment? So let's see that in action. Okay, so what you can. So what we can do, let's see. It's told us it has an extra zero, which is. Which is interesting. But let's see it's making a mistake with, with our id. But very quickly what you're seeing here is it's accessing all the tools that we've specified for it to do to use. So it's using the supply chain hybrid search tool and it's passing in the right arguments for the parameter. It looks like this is storing the history. It's passing in the right arguments as well. It's using all the tools that we've actually specified for it to use. But it's gone through some errors, which is fine. It's telling us to double check our id. I don't think we made any mistake in our id. So let me see what's going on here. So looks like it has a history of the previous execution. So I'm going to do something here which is we're gonna kill it. Even though I'm just going to show you guys what it looks like when it's executed. And let's start another thread. I another Fred hair. So here we're starting another thread which is like a new, new conversation. Let's run it. And let me just go get me shipment. I don't think there should be an extra zero there. And if it makes any errors here then I would just go straight and show you guys the output. So it looks like it's still. It's still things. We're having some outputs here, which is fine. So I'll show you guys this working. It's saying there's some other shipment id which is interesting. Let's see. Let's just try this one then. If you want us to use that, get me.

That's not the initial shipments we wanted.

It's using the tool that we specified, the hybrid search tool. And it's gotten us some information on this shipment id. I'm very confused as to why it didn't like the one that I gave earlier. So here is the shipment ID over here. And you can see it's giving us some information that we have in our database about the ID and it's giving us some additional information. Like this shipment is currently delayed. Okay, it's delayed, which is fine. And it's told us that we're gonna get a penalty and the insurance value and whatnot. This is good information or within a chat interface. Now let me actually do something where we can actually get that specific ID so we can actually look for that specific shipment. Let's see, that needs to be here.

Okay, here we go. So this is that shipment it didn't like the one I was using, which is fine and we can see that it is indeed delayed. Okay, let's update it. Right, so we're going to use the Gentic tool to update it to not delayed.

So.

Let'S go just while doing that, since we're coming up time we are going to run over by a few minutes. There's a couple of great questions from the audience already. For everyone else in the audience, please do add your questions. Now we're probably going to run till I guess about 5 past how much more there is for this.

Let's just get it to update the. The. The tool then the shipment. Then I can go for the question. So let's go. So update to not delayed however it is. So let's see. It's going to use another tool. It's going to use another two that we've defined for it to use for now. It's already successfully updated it to on schedule. So we can go back to our database and see this in action. Just going to quickly refresh the database.

We could just access it. Richie, you can ask me the questions and I could just answer it as we're going through this, right?

Yeah, sure. Okay. So there's one question. This comes from Krishna saying how do you cleverly manage rate limits while accessing APIs? So I know this is a bit of a challenge throughout the session. So. Yeah. What's your strategy for managing rate limits?

One thing is managing rate limits is you can try techniques like prompt compression. So reducing the techniques where you can reduce the amount of tokens that you're sending in by using an another smaller model to reduce the, the, the output that you want to send into the input you want to send into the LLM. Another thing you can do is contact with, with Anthropic you can actually contact the sales team to give you more higher token limit, I guess.

Yeah, like if you pay more money, the problem just disappears.

Exactly. But the, but again you usually there are different techniques you can use and the reason why we went into a rate limit is because I'm probably have a bunch of people abusing their APIs and, and whatnot. So they've implemented a rate limit for probably higher paying customers or customers that are really like serious about what they're building. I don't think open a open AI doesn't have any rate limit but. Or maybe they do. Maybe I've just.

I think they do. Yeah, yeah, yeah.

So again a good way to one reduce the amount of tokens and it would be a good by using some prompt compression techniques or Some summary techniques, there's loads of this on the Internet.

I guess the other thing is making your code run slower. So I've seen cases where people, if you add the sleep function, just make your code wait a few seconds before calling the API again so you don't just end up spamming it so very.

Quickly just to end the demo. We can see that for the Agentix system we're able to upload update record within our, our actual database. On that note, I'm going to stop sharing so we can go back to the Q and A properly.

All right, Super. Yeah. So we've got one more audience question then. I. I actually have one question for you as well. So Tisti asks, why have you chosen MongoDB over other vector bases such as we VV8 for vector search? I suppose the obvious answer is because you work for MongoDB, but I guess do you want to give your pitch for MongoDB and then I can maybe talk about.

Well, the obvious answer is because I work for MongoDB but I'm also a developer, right. So one thing when I'm building products is this, right? I want, I want to create as less friction as possible. So imagine having two databases because we or those folks, they store your vector embeddings but you still have to get another database to store your metadata, right. Or store other aspects of your transactional data. Now you have two databases. You have to try manage and then synchronize data across. I'm a developer, I'm not a database admin. I'm not trying to be one. If I can do the work in one database solution, I only have to manage one database. That's fine for me. And I could do what I do as a developer, which is actually build products as opposed to managing different databases. That's one, one of the one that stands out as a developer.

Okay. Yeah. So I think in general the, the solution for like, how do you pick the right tool? It's. It's a mix of like, what, what scale do you need?

Yeah.

How much you're willing to spend, what licenses do you care about? What are the features you care about? How does it fit into the rest of your system?

I always tell my customers and developers you need to. When it comes to AI, you think about the two things, right? And I call it the two Ps, not PP, but it sounds weird when I say that. But the two P's performance and productivity. So AI workload performance is crucial and developer productivity is very important in the era of Genai with MongoDB we actually have features where you can dedicate a compute to your search mechanism. If you see more vector search utilization, you can actually dedicate compute for the vector search operation and your developers are more productive because you don't need to learn another query language like SQL or you're using one that is more familiar. Like we saw it just look like JSON and it follows a paradigm that people are familiar with and that is as a developer where your, your VPs or your CEOs are like hey, we need to do some gen AI stuff now Productivity is very important. So performance productivity, absolutely.

And I guess so the ventured neutral answer for like there are a million, well, not many, but there's certainly dozens of different vector bases to care about. I think like if you're doing enterprise stuff, it's a choice between MongoDB and Pinecone and then you've also got a couple of open source equivalents. So we've mentioned the question. Milvus is another one and then for small scale projects you've got ChromaDB or there's a Postgres plugin called PG Vector and it's like so basically they're the only six you probably need to care.

About, but really, yeah, it's all about maturity. Right. So MongoDB has been in the data game for over a decade and we've worked with customers building different types of application and we know the problems that we run into, that they run into and we've solved some of this. So we all we talk about maturity and if you're building enterprise application, you really want to be thinking about availability, reliability, scalability. We know we've done that for over a decade. There are a few databases you mentioned, but MongoDB has been at this game for a while.

Absolutely. If you're making web applications, then it's very much a standard. Wonderful. Final question is going to be a quick answer because we're running past time we had some problems with problems with the output at the end. Can you just do a quick discussion of how do you go about testing agents, how do you go about debugging them? Because that seems like that's half the challenge.

That's a very good question because in.

A notebook as well, I did put some observability and monitoring with Lang Smith and to answer your question, you should be using some monitoring tool to see the execution and see what's causing problems for you. Right. So in a notebook there's an easy setup that you can follow.

When you use Langsmith as well. That's a straight answer. Right. So to be honest, for my. For what just happened. Right. It couldn't recognize the id. And it was like, I don't know what this ID is. But you. You have, like, these are some examples.

Of IDs you should be using. And we just use that just for the demo. But in. One thing I would do after this webinar is look into the logs and see why. What is missing there? What was the difference? And sometimes it's something very simple.

Absolutely. I feel like just tracking stuff and making sure that you have examples and seeing whether they work or not. That goes a long way. All right, wonderful. Thank you so much. That was a very, very cool thing you built. So, yeah, great stuff. Yeah. All right. So, yeah. Thank you, Richmond. Thank you for recent moderating. Thank you to everyone who asked a question. Thank you to everyone who showed up today. Two days in a row. Well done. Uh, yeah, you can take a rest now. At least until tomorrow, uh, when there is, uh, the YouTube only session. That's with, uh, Adele and, uh, and Reese. And then we're taking a short break from code alongs. Uh, so. But next week we have a session on. We got four different professors talking about how, uh, AI is used in education. So please do come back for that. And then back with some magic focus sessions after Thanksgiving.