Vectoring in on Pinecone

Foreign.

Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at Fly IO, the home of changelog.com Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents so you can launch your app near your users. Learn more@fly IO.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am the CEO and founder at Prediction Guard where we're enabling AI accuracy at scale and I am joined as always by my co host Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing Chris?

Doing great today, Daniel. How's it going? I know we're recording leading into a holiday weekend.

Here we are and so many exciting things. Last week I got the chance to briefly attend the AI Engineer World's Fair which is sort of prompted in, in certain ways by our friends over at the Latent Space podcast. And that was awesome to see. And of course a big topic there was all things having to do with vector databases, rag, all sorts of, you know, retrieval search sorts of topics. And to dig into a little bit of that with us today, we have Rohi Schwaber Cohen, who is a developer advocate at Pinecone. Welcome.

Hi guys. Thanks for having me today. Really excited to be on the show.

Yeah, well, I mean we were talking a little bit before the show. Pinecone is from my perspective, one of the OGs out there in terms of coming to the vector search, semantic search, embeddings type of stuff. Not that that concept wasn't there before Pinecone, but certainly when I started hearing about vector search and retrieval and these sorts of things, Pinecone was already a name that people were saying. So could you give us a little bit of background on Pinecone and kind of how it came about and what it is position wise in terms of the AI stack.

So Pinecone was started about four years ago, give or take, and our founder, IDO Liberty was one of the people who were instrumental in founding SageMaker over at Amazon and had a lot of experience in his work at Yahoo. And I think that one of the fundamental kind of insights that he had was that the future of pulling insights out of data was going to be found not exclusively, but predominantly in our capability to construct vectors out of that data. And that representation that was produced by neural networks was very, very useful and was going to be useful moving forward. I think he had that insight way before tools like ChatGPT became popular. And so that really gave Pinecone a great edge at being kind of the first mover in this space. And we've seen the repercussions of that ever since. With the rise of LLMs, I think people very quickly came to recognize the limitations that LLMs may have. And it was clear that there needed to be a layer that sort of bridged the gap between the semantic world and the structured world in a way that would allow LLMs to rely on structured data, but also leverage their capabilities as they are. And that is one of the places where vector databases play a very strong role. Vector databases are distinct from vector indices in the sense that they are databases and not indices. So an index may have basically is limited by the memory capacity, right, that the machine that it's running on allows it to have. Whereas vector databases behave in the way that traditional databases behave and the way that they scale. Of course, there's a completely different set of challenges, algorithmic challenges that come with the territory of dealing with vectors and high dimensional vectors that don't exist in the world of just simple textual indexing and columnar data. And that's where like the secret sauce of Pinecone lies, Right. It's its ability to handle vector data at scale, but maintain, you know, the speed and maintainability and resiliency of a database.

As you're kind of comparing vector databases to indices and then kind of bringing that compared to that. One of the things that I run across still a lot are people, you know, vector databases are, are really, you know, incredibly helpful now, but there's still a lot of people out there who don't really understand how they fit in. You know, they don't really get it versus the NoSQL versus relational databases or fine tuning. Yeah. And so, and they, they hear you say it does vectors and stuff like that. Could you take a moment since we, since we have you as an expert in this thing and kind of like lay out the groundwork a little bit before we dive deeper into the conversation about what's different about a vector database that is storing vectors versus storing the same vectors in something else. Like, why go that way for somebody who just isn't quite. Hasn't really ramped up on that yet.

So the basic premise is you want to use the right tool for the job. Right. And the basic difference, right, between a relational database, a graph database, and a vector database, or a document database for that matter. Right. Is the type of content that they are optimized to index, meaning a Relational database is meant to index a specific column and create an index that would be easily traversable and in scale. It would be able to traverse it across different machines and do it effectively. Graph database does the same thing, only its world is nodes and edges. And it's supposed to be able to build a optimized representation of the graph such that it could do traversals on that graph efficiently. In vector databases. Vector databases are meant to deal with vectors which are essentially long high dimensional set of numbers, meaning you can think of an array with a lot of real numbers inside of that array. And you can think of this collection of vectors as being points in a high dimensional space. And the vector database is building effective representations to find similarities or geometric similarities between those vectors in high dimensional space. And that means that basically it would be very effective at given a vector, finding a vector that is very close, quote unquote to that vector in a very large space. Right. So to do that like, you need to use a very specific set of algorithms that index the data in the first place and then query that data to retrieve that similar set of vectors to the query vector at a small amount of time. And also being able to update or make modification to that high dimensional vector space in a way that is not cost prohibitive. Right. Or time prohibitive. Right. And that's like the crux of the difference between a vector database and other types of databases.

Just to draw that out a little bit more. So from your perspective, like what would be if you were to kind of explain to someone, hey, here I've got one piece of text and I'm wanting to match to some close piece of text in this vector space what might be advantageous about using this vector based search approach and these embeddings in terms of what they mean and what they represent versus doing like, you know, TF IDF has been around for a long time. I can search based on keywords. I can do a full text search. There's, there's lots of ways to search text. You know, that concept isn't new, but this vector search is, seems to be powerful in a certain way. From your perspective, how would you describe that?

Yeah, I think that the linchpin here is the word embedding. Right. The vector search capability itself is a pretty straightforward mathematical operation that in and of itself it doesn't necessarily have value. Right. It basically, it's like other mathematical operations, it's a tool. Right. The question is like, where does the value come from? And I would argue that the value comes from the embeddings and we'll talk about what exactly they are, would just point a flag and say embeddings are represented as vectors, which is why the vector database is so critical in this scenario. But why are embeddings helpful in the first place? Right, right. So embeddings come from a different set, right? Like a very wide set of neural networks that have been trained on textual data and they create within them representations of different terms, different surface forms, sentences, paragraphs, et cetera, that map onto a certain location and vector space. The cool thing about embeddings is that it just so happens, and we can talk about why it just so happens that terms that have semantic similarity have a closeness in vector space. And that means that if I search for the word queen and I have the word king embedded as well in my vector database, and I also have the word dog, right, because the word king is more semantically similar to the word queen, I will get that as a result, and not the word dog. Right. And that allows me to basically leverage the quote unquote understanding of the world that machine learning models and specifically neural networks have, large language models have of the world in a way that I can't quite leverage from other modalities like TF, IDF and other BM25, et cetera, that look at a more lexical kind of perspective on the world. And so when we talk about practical use cases, RAG comes up very, very frequently. And the reason for that is because we are in semantic space. A user interacts with the system in semantic space, so that means that they ask the system a question in natural language. We can take that natural language and basically again, quote unquote, understand the user's intent, right, and map it again into our high dimensional vector space and find content that we've embedded that has some similarity to that intent. Right. And so we're not looking for an exact lexical match, but we're actually able to take a step back, right, and look at the more ambigu ambiguous intention and meaning of the query itself and match to it things that are semantically similar, if that makes sense.

Would it be fair to say that you're essentially, because the output of those structures being embeddings and those are vectors, and therefore you're essentially storing it and operating on it in a closer representation to how they naturally would be? And so you're not doing a bunch of translation just to fit it into a storage medium and, and to operate on it, therefore, it's going to be quite a bit faster since you're is that fair? Is that a fair way of thinking about it?

Perhaps we're, in a way we're compressing the representation into something very small in a sense, right? So you can think of an image, for example, right? An image that could be like a megabyte big. Right? We can get a representation that in terms of like its actual size in terms of a vector is order of magnitudes smaller. Right. And we can use that representation instead of using the entire image, right. To do our search. Now it just so happens that again, when we're doing embeddings for images, right, we get like that same quality where we're not looking at, you know, we're not looking at an exact match or like pixel matching pixel to pixel with, with images that we have, we can actually look at the semantic layer, meaning what is actually in that picture. So if it's a picture of a cat, we would get as a result other pictures of cat that we've embedded and saved in the database. Right? And that will come out of the representation itself of the embeddings that were a result of like, say like a clip model that we use to embed our image. So I don't know if it necessarily means that like it simplifies things in a lot of ways. It actually adds a lot of more oomph to the representation, right? So you can actually match on things that you wouldn't necessarily expect. Right? And that's kind of like what the beauty of semantic search in that sense, right, is that users can write something and then get back results that don't even contain, right. Anything remotely similar in terms of the surface form to their query. But semantically, right, it would be relevant foreign.

Hey friends. This episode of practical AI is brought to you by our new friends over at Plum. Plum is a low code AI pipeline builder that helps you to build complex AI pipelines super fast. You can easily create AI pipelines using their node based editor, iterate and deploy faster and more reliably than coding by hand without sacrificing control. Deployment is easy. Pipelines are live API endpoints eliminate the need for constant code redeployment and debugging by deploying complex AI pipelines as API endpoints. Team collaboration is easy too. Plum's declarative node based editor enables you to build quickly while empowering non technical roles to iterate on what you've done without breaking it. You can build advanced AI features, get structured output every time, transform data and leverage validated JSON schema to create reliable high Quality structured output. So Plum is built for builders earliest stage. Product teams are using Plum to go from idea to validation in record time. To get started, go to useplum.com that's plum with a B as in plumber to request access today that's use P l u m b.com again useplumb.com.

Well Rohi, I really appreciate also the statement about adding oomph to your representations. I think that would be. There's some type of good T shirt that could be derived out of that. I see for listeners who are just listening on audio, Rowie's wearing a shirt that says love thy nearest neighbor, which is definitely applicable to today's conversation. Well, this is great. So we've kind of got a baseline in a sense from your perspective, what a vector database is, why it's useful in terms of what it represents in these embeddings and allows you to search through. You mentioned rag. We've talked a lot about RAG on the show over time, but maybe for listeners, this is the first episode that they've listened to what would be the kind of 30 seconds or some type of quick sort of. Remember, RAG is X from Rory.

Right? So I love quoting Andrej Karpathy with his observation on LLMs and hallucinations. So people, when usually people talk about rag, they say, oh, Rags sometimes hallucinate and that's really bad, right? And Andre Kpathi says, actually no, they always hallucinate, right? They do nothing but hallucinate. Right. And that's really true, right? Because LLMs don't have any kind of tethering to real knowledge in a way that we can trust. Right? We don't have a way to say, hey, I can prove to you that what the LLM said is correct or incorrect based on the LLM itself, right? We need to go out and look in search, right? And RAG, to me is that opportunity where we can take the user's intent. We can tie it using a. For example, right, A semantic similarity search to structured data that we can point to and say this is the data that is actually trusted, and then feed that back to the LLM to produce a more reliable and truthful answer. Now, that's not to say that RAG is going to solve all of your problems, but it's definitely going to give you at least a handle on what's real and what's not, what's trusted and what's not, and where the data is coming from, where those responses are coming from. And it shifts the role of the LLM from being your source of truth to basically being a thin natural language wrapper that takes the response and makes it palatable and easy to consume to a human being.

Great. Yeah. I think a lot of people have done a sort of. Maybe they've done even their own demo with sort of a naive rag, maybe pulling in a chunk from a document that they've loaded into some vector database. They inject it into a prompt and they get some useful output. Put one of the things that I think we haven't really talked about a lot on this show, we've talked about advanced rag methods to one degree or another. But I know Pinecone, along with other vector database providers, you know, offer more than a simple, just like search. That's the only function you can do. There's a lot more to it that can make things useful in particular, like having, you know, you mentioned, Pinecone mentions kind of namespaces that can be used, metadata filters, sort of hybridized ways of doing these searches. Could you kind of help our listeners understand a little bit? So they may understand, here's my user statement. I can search that against the database and get maybe a matched document. But for an actual application, like an application in my company that I'm building on top of this, what are some of these other key pieces of functionality that may be needed for enterprise application or for a production application that go beyond just the sort of naive search functionality in a vector database?

Yeah, for sure. So we can take these one by one. So metadata is definitely like one of those capabilities that vector databases have that are above and beyond what a vector index would provide you. Right. And basically what they are is again, the ability to perform a filtering operation after your vector search is completed. And so you could basically limit the result set to things that are applicable in the application context. Right. So you can imagine different controls and selection boxes, et cetera, that come from the application that are more set in stone, so to speak. They're not just like natural language. Their categorical data, for example, and you can use those to limit the result set. Right. So that you hit only what you want. That is something that is very common to see in a lot of different production scenarios.

And could you give maybe an example of that? Like in a particular use case that kind of. You've run across what might be those categories or what, just to give people something concrete in their mind.

Yeah. For example, like, you can imagine a case where I'm not going to name the customer, but like, you can imagine the case where. Where you want to perform a RAG operation, but you want to do it on a corpus of documents, but not on the entire corpus, but rather on a particular project within that corpus. So imagine that you have multiple projects that your product is handling, like finance and HR and whatever engineering, and you want to perform that search and then limit it only to a particular project. And in that case you would use the categorical data that is associated with the vectors that you've embedded and saved in Pinecone to only get the data for that particular project. That is a kind of super simple example, but it can go beyond that and move into the logic of your application. So you can imagine a case where you're looking at a movie data set and you want to search through different plot lines of movies, but you want to limit the results only to a particular genre, right? That's another case, right? Like we could just leverage metadata. You can think of wanting to limit the results to a time span, right. A start and end date. Right. Things of that sort that kind of like have to do more with the nature of when and how in what category the vector belongs into and not specifically the contents of the vector. Right? So that's one thing. Namespaces are another feature that we've seen as being like, incredibly important for multi tenant kind of situation. And multi tenant rag has become kind of like a very strong use case for us. And that's where you know, you, you see a customer, that customer has customers of their own, and not one or two, but many, many, many. And in that case, you definitely don't want to have all of the documents that these, that all of the sub customers have to be co located in one index. And in that case you basically break them apart, right? So they're still on one index. So management of the index overall is maintained under one roof. But the actual voice content and the vectors themselves are separated out physically from one another in namespaces. They're sort of sub indexes to that super index. And that's another feature that we've seen as being super important to our enterprise customers.

As you're looking at these enterprise customers and with maybe most enterprises, you know, getting into RAG at this point at some level and trying to find use cases for their business to do that. I know, you know, my company and lots of other companies are doing this, what are some of the ways that they should be thinking about these different use cases when we're talking about RAG and semantic search and multimodal things that Pinecone does? What are good entry pathways for them to be thinking about how to do this because you know, they may have come up with their own, their kind of own internal platform. It might have some open source, it might have some products already in play, but maybe they don't have a vector database in play yet. And so, you know, how do they think about where they're at when you guys are talking to them and you're saying, let me, you know, we've, because we've been talking in the show so far about kind of the value of the vector database and the kind of these use cases, but not necessarily kind of the, an easy pathway. So how do you onboard enterprise people to take advantage of the goodness on this?

Yeah, that's an excellent question. And in fact it's like a quite, quite a big of a challenge because it ends up being a straightforward pipelining challenge that has existed from the beginning of the big data era. How do I leverage all the insight that is locked in my data in a beneficial way? The sad part about this story is that it always depends on the specific use case and it's hard to give a silver bullet. A sort of light at the end of the tunnel is that we've recently published a tool called the RAG Planner and its purpose is to basically help you figure out what do you need to do to get from where you are to an actual RAG application and follow through all of the different steps that are required in between, right? And sort of like, understand, like, from an understanding of like where your data is stored, how frequently it updates, like what the scale of your data is, etc. Etc. To the point where it could give you some recommendation as to, like, what are like the steps that you have to do. Like in terms of do you build a batch pipeline, do you build a streaming pipeline, what tools should you be using to do those things, what kind of data cleaning are you going to need to do? What embedding models are you going to want to use to do this? Right? Like, how are you going to evaluate the results of your RAG pipeline? So all of these questions are pretty complex. So what I would say is a general rule of thumb. First of all, you have to evaluate whether or not RAG is for you, right? So, for example, there are a lot of situations where RAG may be the wrong choice because the data that you have and the actual capability of answering the end user's questions based on that data does not match up. And that's how you get to see cases where chatbots spit out results that may seem ridiculous, but nobody catches it. And companies get into a lot of hot water because. Because of it. Right. There are a lot of scenarios where it's much easier to start that journey and to sort of develop the muscle memory that's required in order to set these things up. In a lot of these use cases you see a lot more internal processes definitely in bigger companies where there's a very big team that just needs access to its internal knowledge base in an efficient way. But it's not a system that is going to be mission critical in any way. So like if a person gets a wrong answer, it's not going to be the end of the world. Nobody's going to get sued. Right. And so what I would say is there's definitely a learning curve here for big organizations for sure. It's usually recommended to develop again that internal knowledge of what the expectation versus the realities on the ground is going to be to have like a really good idea of how you assess risk in those situations and most importantly, how to evaluate the results that are produced by those systems. Right. Because a lot of people are like, okay, you build the rag system great. And now produces answers. I'm done. Right. Like we're, we're. Everybody's happy. That's farthest from the truth that you could possibly be. Right. Like these systems need to be continuously monitored and feedback needs to be continuously collected to the point where you can understand. Right. Like how changes in your data and the way that you're interacting with it, changes in large language models that you're implying are actually affecting the end result. Right. Are going to be. And how your users are actually interacting with the system overall. Right. How all of these things kind of coexist and happen together and are they working in the way that you want them to and of course you want to do that, you know, in a quantitative and not qualitative way. Right. So like there's a lot of instrumentation that has to go into it.

I'm curious as a. Is a little follow up to that and obviously leaving specific customers out of it, are you tending to see more internal use cases of rag deployment to internal groups of employees and stuff? Maybe from a risk reduction, are you seeing more of an external. I'm going to get this right out to my customers and try to beat my competition to it. Where do you think the balance is as of today?

I think that there's a widespread and I think that it's a journey. Right. I think that the more tech native companies that we see that are more, I would say, forward looking or technologically adept to Kind of do these things quickly, are more ready to not only take risks, but take educated risks in this space with the evaluation that comes with it, right? So like these are not just like let's set and forget, but they actually know what they're doing. In those cases, you see them going out to production with very big deployments. That is our bread and butter, I would say, at the moment. Right. With companies that are more traditional, that have been like, that are not necessarily getting tech native, you see a more cautious sort of progression, which is only to be expected. Right. Like I think that's kind of like natural to see.

Well, Rohi, I have something that I saw on your website which was new to my knowledge, which I think is also really interesting. One of the things that I've really liked in experimenting with vector database rag type of systems as an AI developer is having the ability to run something without a lot of compute infrastructure, maybe in an embedded way or an on disk index, something that I can spin up quickly, something that I don't have to deploy a Kubernetes cluster or something, or set up a bunch of kind of client server architecture to set up and test out maybe a prototype that I'm doing. And I see Pinecone is talking about Pinecone Serverless now, which is really, really intriguing to me, just based on my experience in working with people. These sort of serverless sort of implementations of this vector search, I think can be really powerful. So could you tell us a little bit about that and how that kind of evolved and what it is, what's the current state and how Pinecone thinks about the serverless side of this?

So Serverless came about after we realized that tying compute and storage together is to limit the growth factor that our bigger customers are expecting to see. And it basically makes growth kind of prohibitive in a space. Right. And so we had to find a way to break apart these two considerations while maintaining the performance characteristics that our customers are expecting and are used to having from our previous architecture. So essentially like serverless has been a pretty big undertaking on our side to ensure that the quality of the database is maintained. But at the same time we can reduce cost dramatically for customers. To just give you like an idea for the same cost of storing about, I don't know, around 500,000 vectors before you can now store 10 million. And that's a humongous difference, right? Like it's an order of magnitude difference. I think that like, to accomplish that, right? Like there was like a lot of very clever engineering that had to happen, because again, now having compute and storage separated apart means that storage can become very cheap. But on the other hand, it requires you to handle the storage strategy and retrieval in a lot cleverer way. We have a lot of content on the website that kind of delves deeper into how exactly technically that was achieved. And we won't be able to cover that given the time that we have. But like, the basic premise is that you can now grow your vectors index to theoretically infinity, but practically to tens of billions and hundreds of billions of vectors without the cost of the expense becoming prohibitive, which is the main drive for us with our bigger customers and also with smaller customers, like, you can start experimenting. We have like an incredibly generous free tier that allows you to start, you know, like you said. Right. Like, if I'm just a developer on my own, testing things and trying to understand how vector database works in my world, it's very unlikely that I'll be able to tap the entire free tier plan even several months in with many, many vectors stored. Right. And it will work the same way that our Pro serverless tiers work in terms of its performance. So it's not like a reduced capacity or performance in any way. So you get to feel exactly what it would feel like. And the effort that's required to stand that up is minimal to negligible. Right. You just set up an account and the SDK is super, super easy to use.

Yeah. And am I understanding or sort of representing things right, like in terms of the, you know, massive. So there's a massive engineering effort, I'm sure, to, as you mentioned, to achieve this, because it's not a trivial thing. But in terms of the user perspective, like if people use Pinecone before and they're using Pinecone now, you already mentioned the performance. Is the, is the interaction similar? It's just this sort of scaling and sort of from the user perspective, scaling and pricing and maybe also you could touch on. So Pinecone is. People might be searching for different options out there and some of them would require you to have your own infrastructure or some of them are hosted solutions. Pinecone, at least in its kind of most typical form, would be hosted by you and yeah, could you just talk a little bit about the user experience pre post serverless and then also kind of the infrastructure side, like what do people need to know and what are the options around that in terms of.

What happened pre and post? So before serverless, there was a lot of possible configuration choice that you could do. Right? There was in Fact, a lot of confusion with our users. What exactly is the best configuration for me? Should I use this performance configuration? Should I use the throughput, optimized configuration? What exactly am I supposed to use? And the pricing mechanism was a little bit convoluted. And I think that serverless, the attempt there was to simplify as much as possible and to make it really, really dead simple, right, for people to start and use but also grow with us. Right? So again, like I said, the bottom line is the external view into what Pinecone offers may have looked pretty similar, right? So, like, if you're just a user, you may say, like, hey, I got like a cheaper Pinecone bill this month and I can store a lot more.

Always a good thing, right?

Always a good thing, but not super amazing. But the end result is the question is, what happens when you can actually store a lot more vectors? What does that unlock for you? And again, I think that the end of the day, the way that we see Pinecone, and this may help us talk about what's next for Pinecone is a place where your knowledge lives, right? And it allows you to build knowledgeable AI applications, right? And having more knowledge is always net positive, right, in that context, right? So the assumption is that, like, you know, as AI applications grow, they accumulate more and more and more knowledge and they become that more powerful with any additional knowledge that you can stuff into them. And so there's like actual value beyond the fact that you can store more, right? Like, and it's cool, right? Your application actually becomes more powerful because it can handle more types of use cases. It has a better ability to be more accurate and respond truthfully to a user when they are interacting with it. And so I think that, like, in general, like, there's like this blatant kind of value that is only going to be apparent once people really experience what it means to have, you know, a million documents that are stored in Pinecone versus 10 million documents that are stored in Pinecone. And that effect is going to be very powerful. I think that's the majority of the. Of the benefit that I see.

Maybe that gets to the next thing which I was going to ask about, which I also see the announcement around Pinecone assistance, and I'd love to hear more about that. Like, what is from. Of course, sometimes maybe that can be loaded language also for people in the AI space. But in terms of this assistance functionality for Pinecone, what are you trying to enable and where do you see it headed?

So that has to do with the question that Chris had before, which is like, what is the journey for customers? Right. And I think that like, as a. The general purpose that we had around Assistant was to reduce the friction between me having a bunch of documents that I want to interact with, with an LLM or an AI in some form and capacity to the point where that actually works. Right. There are a bunch of ways of going about it. Right. I think Pinecone wants to bring on top of our very robust vector database a very smooth experience that lets users really do very little and get all the value out of Pinecone without having to think too much about it. So for that purpose, we don't only have the ability to take your documents and then embed them and do the end to end process of creating that completion endpoint for you. Right. We're also the ones providing the actual inference layers as well. Right. And so it's not going to be. Again, if you asked me this question, how do you build a rag pipeline like a year ago even I'd have to tell you, hey, you have to go to some embedding provider. You have to find someone who would do your PDF extraction or take the data and chunk it and do all this stuff. No more. The reality here is you can take a set of documents, throw them at this Knowledge Assistant, and the rest is kind of quote unquote magic. Right. It just happens for you behind the scenes while maintaining the quality that you want to get and at the scale that Pinecone can deliver. Right. Which is again, another differentiator. So like I said before, Pinecone is built to withstand hundreds of billions of documents. Right. That of vectors that you would store with us and still be able to produce responses in a, in a reasonable amount of time. And that's true for Knowledge Assistant because Assistant sits on top of the vector database.

So it sounds like that may be a really good way, especially for small organizations. You know, we talked about enterprise and they have a certain infrastructure and teams to go with that, but there's so many more small organizations out there that have very little in terms of, of trained people necessarily to do that. And they, you know, they don't have all the infrastructure in place and they're looking, you know, with assistance and serverless. They're looking for simple ways to onboard and get utility out of it. Would you say that the combination of serverless and assistance and then maybe whatever they might have in AWS or whatever platform that they're using is kind of just made to gel easily for them so they can get to something working pretty quick.

Yeah. I mean, at the end of the day, like, if you think about it, right, like, the process shouldn't be as complicated as it is. Right. It's just that there are many parts to it and nobody picked up the gauntlet of saying, like, hey, we'll just do it all, you know what I mean? Because all of it is quite complicated to do. Right, Right. And so, yeah, like, I think that like, initially we'll see smaller organizations kind of, you know, picking that up because they don't have the resources. But as time moves along, you know, you're going to have to ask yourself, even as a bigger organization, do I want to own this pipeline? Right. Is it something that I need to own? Right. And what value am I getting from actually owning all this? Right. And so, yeah, it would be interesting to see. So this is a very, very new product still in public beta and it will be interesting to see how the market kind of reacts to it and sort of experiments with it. But my bet is that as time progresses and knowledge assistants themselves become more capable doing things maybe beyond rag or beyond simple rag, quote unquote, that, you know, like more, more and more sophisticated organizations might want to actually give it a try.

And that really brings us maybe to a good way that we like to end episodes, which is asking our guests to sort of look, look into the future a little bit and not necessarily predict it because that's always hard, but to look into the future and kind of what are you excited about? It could be related to vector databases, specifically your pine, specifically, but maybe it's more generally in terms of how the AI industry is developing, the sorts of things that you're seeing customers do that are encouraging, whatever that is, what sort of keeps you excited about where things are headed going into the rest of this year?

I'm excited about the fact that we're seeing sort of like a resurgence of what you would call traditional AI kind of come back into the fold in the form of, for example, graph rag. I think that the notion here is that for the longest time, and I think it's been since GPT 3.5, you basically saw this, I think, over indexing on LLMs for good reasons. They're super exciting, they're very powerful and they can do really, really cool things. But with that said, it's as if every other technology that has ever existed before just dropped off the face of the earth and nobody has ever talked about, okay, wait, so what can we do with those things? And LLMs. Where do LLMs fit in the bigger picture? I think that vector databases kind of put LLMs in their place a little bit in the sense that, you know what I mean, you're not thinking of the LLM as being the end all, be all. This is the only tool that we need. I'm very excited to think of LLMs as these operators or agents that can tap into the capabilities that exist in other systems. And I think that what we're going to see more and more and more is that people are going to figure out in what subset of the ecosystem does each tool belong, so what set of problems that each tool solve. For example, a vector database solves the problem of bridging the gap between the semantic world and the structured world. A graph database can solve problems like formal reasoning over well structured data. Relational databases can solve a whole set of different problems that they used to be solving, aggregation, etc. Etc. And then you can imagine that LLMs and agents can sit as sort of like an orchestrating mechanism and a natural language interface mechanism on top of all of those things together. And that's what I'm excited to see. Like, it's kind of when the community as a whole is going to wake up from its LLM fever dream and sort of realize that there's other things out there and realize that it has so many more powers that it could yield to make really exciting applications.

That's awesome. Well, thanks for painting that picture for us, Rohi, and for taking time to dig into so many amazing insights about vector databases and embeddings and knowledge management in general. So yeah, appreciate what you all are doing at Pinecone and hope to have you on the show again to update us on all those things.

Thank you so much. Thanks for having me.

All right, that is Practical AI for this week. Subscribe now if you haven't already. Head to PracticalAI FM for all the ways and join our Free Slack team where you can hang out with Daniel, Chris and the entire Changelog community. Sign up today at PracticalAI FM. Community. Thanks again to our partners at Fly IO, to our beat freaking Residents Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.

Sat.