Building LlamaIndex: Jerry Liu on Scaling Retrieval-Augmented AI

Welcome, Jerry Liu. You are the co-founder and CEO of Lama Index. Lama Index is a data framework for your large language model application, basically an interface between LLMs and, and your data. And in just a few months, LlamaIndex has become a very popular open source project and community. So as I was prepping for this, I was looking up some stats and it's really quite impressive. So almost 19,000 GitHub stars, 227 contributors, 25,000 Twitter followers, 8,000 Discord members. So the kind of metrics that successful projects and new communities, fast-growing communities tend to have. So congrats on all of that. You also just announced just last month an $8.5 million seed round led by our friends at Greylock. I'd love to start with your personal and professional journey that led you to start LlamaIndex.

Definitely. Yeah. And first off, thanks, Matt, for having me. It's great to be here and really appreciate the invite. So I started off, you know, I've been in AI for most of my career. I started off as a machine learning engineer at Quora after graduation. Actually around like junior or senior year of college, I started getting into GANs, the initial version of generative models. And if you're familiar with what GANs were, or generative adversarial networks, they were just initial deep learning models that could generate very small pixelated images of different things. So for instance, like faces or bedroom images or a lot of other stuff. And so when I first saw that, I thought it was magic. And that's, I remember that was really the spark that got me interested in AI and deep learning in general, because I wanted to work in a field where I could also generate cool pictures. So I started diving a little bit more into my machine learning journey as a machine learning engineer at Quora. After a year, then spent some time as a research scientist at Uber ATG. Where one, I met my current co-founder, but two, I spent a lot of time deeping, diving into the foundations of machine learning and really learning a little bit more about model architectures and how things work and applying them in a self-driving computer vision setting. Afterwards, I spent 2 years at Robust Intelligence, which was a Series B startup focused on testing and monitoring of machine learning models, pre-production and also post-production. And so there I learned about the intersection of how do you actually, you know, take all this cool AI technology, apply it in production and make sure that they're functioning correctly. And also in the process, you know, Robust was like, uh, uh, like I joined as like 8th or 9th hire and it grew to 40 or 50 people by the time I left. And so, uh, be a part of that startup experience and really understand what it means to bridge like industry and AI. And so I think like all that kind of culminated in, um, in a variety of ways. And the way I discovered, like, how LLMs operate and how to build apps on top of LLMs really occurred in a spontaneous manner, right? Because, like, in the end, I think this company is really a mixture of all my previous interests. But the way I actually stumbled upon this project was relatively spontaneous. I started off playing around a bit more with large language models and DALL·E. You know, I've kind of been keeping track of the developments of transformers and all these models over the past few years. But really started just like trying to hack on some apps back in November. And I think I remember as I was trying to build applications, I really thought like, wow, like, you know, these large language models are not just capable of knowledge, but they're also capable of reasoning. And how can we adequately leverage this reasoning, these reasoning capabilities to really kind of operate over, you know, data that it doesn't necessarily inherently know about, but over like your own personal data or over your enterprise data. For instance, like, you know, ChatGPT, how do you get it to know about, like, you as a person or, you know, you as an organization? And so I started working on an initial iteration of GPT-Index at the time, which is basically designed around how do you organize, you know, data that the language model doesn't know, but in some sort of structure that can access and retrieve. And that led to the initial iteration of GPT-Index, which at the time was more of a fun design project to solve a pain point. But it became clear as more people became interested in the project and it got a little bit of traction on like Twitter that, you know, there really was an opportunity here that more and more people were starting to build large language model-powered applications. And that pretty much for the majority of these use cases, everybody was trying to figure out the best practices for how do you organize, retrieve, and store data. And so that really led to the evolution of this toolkit. From an initial project designed to kind of augment language model context windows to this overall comprehensive data framework for building LLM applications.

Great, great. So fast forward to today. What does LlamaIndex, that data framework that you mentioned, what are the different components and what do they do?

Yeah, so the high-level mission is really to connect your language models with your data and basically unlock the capabilities of language models. Whether it's like reasoning, agent-like planning, or also like question answering, insight extraction on top of your data. And so to that end, we offer a lot of tools to both like get your data in the right format, so kind of the ingest and storage part, but also the query side. On the ingest and storage part, you know, we offer a lot of modules to allow you to connect to your various data sources, for instance, your PDFs, your APIs, your databases, and ingest unstructured and also structured data in the right format that you can then pass to your downstream application and also define additional stuff like metadata, like annotations, those types of things. We then have—

While we are on that topic of ingestion, so what is the right format? What does that mean?

Yeah, so I could probably spend an hour talking about this, but at a high level, the trick is, you know, there is a— I can talk about the kind of like naive thing that everybody does these days and then the kind of like consideration that you might need to think about if you're trying to build something more production quality. So just a quick overview of retrieval augmented generation as a concept. You ingest some source documents. Let's say they're a set of PDFs. Now, the current stack is basically you take these documents and then you split it up into a bunch of text chunks and then you store the text chunks in like a vector database. And these days, you know, there's a bunch of different vector databases from Pinecone to Chroma to Weaviate to a bunch of others. And these act as like a storage layer, right? And so then when you actually, say you're building a question answering system, you ask a question, what's gonna happen is it's, you're first gonna do retrieval from your storage layer. You're gonna hit the query interface of the vector database and do embedding-based retrieval from the vector database 'cause that's what it's specialized at. Then after you get a set of retrieved contexts, then you put these contexts into the language model. To actually synthesize an answer. So that's the overall stack of Retrieval Augmented Generation as most people understand it today. And the kind of, like, naive way to do a lot of these things is you just do some very basic text splitting. Let's say you split after every, like, 1,000 words or so, or every, like, 1,000 characters, and then you just throw everything into a single collection in the vector database. So what we're really invested in is we allow you to do the easy stuff out of the box pretty easily. But we also invest a lot in thinking about what are the additional data considerations that you need to take on when you actually build a more production-quality LLM application around your data. And so the ingest parsing side actually is quite important because one, we have more sophisticated text splitters than, for instance, just splitting every 1,000 characters. Second is being able to inject metadata actually is quite important. To actually improve retrieval performance of the downstream application. Because let's say you're splitting up an SEC 10-K filing into a bunch of chunks. Within a single chunk, it might lack the context of how does this actually relate to the rest of the document. And so being able to explicitly define that, whether in terms of relationships or additional metadata, can help a lot. At a high level, we provide a lot of these abstractions and structures so that you can define these advanced annotations on top of your data to allow you to define the right state. The metadata is just one piece of that, kind of like the address side. Now the next few steps is, if you think about the overall pipeline, is you have this set of chunks that you get from a set of documents. Now the question is, how do you actually store this in some downstream storage abstraction? You know, we're not building our own vector database, but we have a rich set of integrations with, you know, like 12 to like 20 different vector databases out there these days.

Yes, I saw that, which I thought was amazing. If one, because you built so fast, but two, that there would be so many vector databases in the first place.

Yeah. And so, you know, we really do try to be like a framework that provides the right toolkits, right? To allow users to kind of pick and choose the storage solution that makes sense for them. But also, you know, we provide kind of these advanced capabilities for orchestration and, you know, additional indexing on top of the additional storage abstractions. And so we offer support for both vector databases as well as kind of existing like object storage providers, like for instance S3 or MongoDB. And we allow users to configure the storage provider that makes sense. But the high-level idea is like, you know, you take in all this data, You split it up, you've maybe added a bunch of annotations. Now you need to store it somewhere and you need, need to be a little bit mindful about how you actually store it. Like what are, for instance, like the collections that you wanna store this data in? How do you want to define indexes on top of this data? These are all tools that we provide. And so, you know, our ingest and indexing stage really is about getting a lot of this data in the right format so it can end up in the storage system. In a way that can fit in well with the rest of your LLM application. And so that's probably one half of our toolkit, right, is really getting your data in that right format. The second half of that toolkit is, OK, now that you've defined the right state over your LLM application, how do you define the compute side, the query side? How do you actually leverage the language model as this reasoning engine on top of this data that you've now gotten into a certain format? And so, uh, from the basic retrieval augmented stack, that's typically what happens when, um, like typically what you do is you do some sort of like, uh, top-k retrieval from a vector database, and then you take in all this context and you synthesize a response from that context. But what we allow you to do is we allow you to do that part pretty easily, but also define more sophisticated interactions, uh, between your language model and your data as well to satisfy kind of more advanced query use cases.

Okay, very, very good. And you started alluding to this a little bit, but to help people who may listen to this make sense of where LlamaIndex fits in that emerging LLM or generative AI infrastructure stack, people may have heard of names like LangChain or Fixie or Dust, or are those competitors? Are they partners? Is that all sort of still, you know, moving pieces that everybody's trying to figure out?

Yeah, to some extent it's all moving pieces. I think there are definitely overlaps with certain frameworks, but there's also key differences. And so, you know, let's talk about, for instance, like LangChain. LangChain is a great application framework for you to just like get a set of building blocks for a lot of different components. For instance, from like the LLM modules to prompts to from basic retrieval and vector database abstractions to also agent frameworks. We are, almost from the beginning, have been very focused around the data. And so just like, you know, really what we think about is how do you get your data in the right format so that you can use it with the LLM? And then also how do you get the LLM to effectively query your data? And so there are some overlaps between that and LangChain, but we're very hyper-focused on developing deep tech around that and making that really good. And so not just doing, again, the basic naive retrieval augmented generation stack of text splitting your data into some form, dumping it into a vector database in a single collection and doing top-k retrieval. We offer that, but also a rich set of advanced functionality to do additional capabilities, combining LLMs on top of your data. And so we've almost intentionally made it so that we want to provide deep tech around this space. And really, we, we think we provide a very nice toolkit with a nice set of abstractions for both beginner and advanced users. But we also understand that if you want to build, for instance, like kind of things that are not explicitly within our framework, at least at the moment, for instance, stuff like kind of auto AGI or agent simulations. But for those, like, you know, maybe data is a component of that, we make it pretty easy to integrate with other frameworks like LangChain. We also, for instance, have an integration with Microsoft Guidance, the framework that allows you to have token-level control of interleaving prompting and generation. We're looking into integrations with other toolkits like Semantic Kernel. And yeah, also in conversations with things here as well. And so I think the high level is there's a lot of pieces still moving. And for us, what we really care about is really just being thought leaders and providing a good toolkit in the data and LLM space. Great.

And as I was looking through your materials and sites, I saw that there are just a couple of things. It's Lama Hub and Lama Lab. I think you, you already covered some pieces here, but just like to, to, to bring it home. So what are those?

Yeah, it's a good question. So we have an entire ecosystem of different projects that are extensions of the core repo. So the idea of Llama Hub as it is right now is to create a central toolkit for just community-driven data loading abstractions. And so what it really is, is it's supposed to be an extension of the core repo. The core repo consists of some basic abstractions around data loading, indexing, and querying. And what we really did with Llama Hub is let's make certain parts of this purely community-driven and just encourage everybody to just submit a bunch of modules for certain components. And so data loading made a lot of sense because there's a long tail of different data loaders that you might want to have. You might want to load data from your Slack, your Salesforce, your Notion, your different file formats that you have. And we wanted to open that up to community contribution. And so these days, like, Llama Hub is a very rich repository of like 100+ different data loaders from all different services and formats, and it's growing like every day. And the next thing we're thinking about, and we'll have a release very soon, is expanding LlamaHub into becoming a repository, a community-driven repository, not just of like data loaders, but other aspects of it, which we'll get to.

And what about LlamaLab?

LlamaLab is almost just like a fun repo for kind of experimental projects. And so we launched it a few months ago, mostly as like, I think this is when BabyAGI and AutoGPT came out, and it was a way for us to just like, you know, play around with, here's like if you want to have some sample implementations of agents, here's ways that you can use, like kind of both like build that with LlamaIndex as well as kind of like use different components of LlamaIndex within that ecosystem. It's more experimental and more of like a fun showcase of what we can do. And which is why, you know, the kind of contributions there are probably a little bit more sporadic, but it's something that as like new fun project ideas come in, we'll certainly add to.

Going back for a second to Llama Hub, thinking of parallels with the world of data infrastructure. As you know, there's this whole world of ETL or ELT companies on the one hand and orchestration companies. And all of those are— you know, the entire companies, $100 million plus, there are companies that are being built solely on building those connectors. Do you anticipate that it's the same level of complexity over time? And do you believe that long term you can do it all, both all the connectors and the orchestration and the compute side?

Yeah, I mean, that's a good question. I mean, I think like, um, like there's the overall mission statement that we're solving. Um, there's what the open source project does, and there's also the, uh, kind of eventual enterprise version. And, you know, all three things are kind of like, um, slightly separate, but like overall tie back to our core mission. Um, if you think about what the open source project is right now, like what Llama Hub really represents is an easy way for people to get started building their LLM application because now they just have this open source repository of loaders that represent basically boilerplate that they don't really want to write themselves. No one wants to look through the API docs, like 50 different services just to try to build an application. If there's something that works a little bit out of the box, they would just use that and plug it into their toolkit. And so it's a way of, it's an entry point for people building LLM apps on top of their data. And we wanted to make that entry point as low friction as possible. Of course, there's like this existing world of like data loaders, of like ETL, ELT tools, like kind of in probably a more like production setting, for instance, like, you know, being able to ingest data from different sources, sync it to your data warehouse, like transform it in some way, get it in the right format. There's like different variants of these companies. I do think high level conceptually, If we're building this new age of LLM-powered applications, the kind of requirements for the type of data that you want to load as well as how you want to extract information from that data will be a little bit different than the existing ETL stack. The reason is LLMs have this inherent capability of just understanding unstructured information along with structured information. And you do want to be mindful about how you actually leverage the context window of LLMs At the same time, if you just think about this idea of just splitting stuff into chunks and throwing it into a vector database, that's already very different than, for instance, how you think about ETL before. And so then the question is, how do you make this new ETL pipeline very effective to build LLM-powered applications on top of? And how can we provide the tools for people to do that? So our open-source toolkit is going to be focused a lot on enabling users to get some initial value there. I do think there are going to be similarities with existing data connectors for kind of more production-grade workloads. And this may have— this is something of consideration as we're building an enterprise product. So for instance, whenever you're building data connectors, being able to, for instance, deal with updates to the source data is a big component of that, right? What happens if the source documents change And then all of a sudden you have to propagate updates from your source data to your destination. In this case, for instance, like a vector database. And then how do you, for instance, do batch refreshes? How do you update on stuff in real time in a streaming setting? These are all— there are lessons there that you could probably take from existing connectors. But at the same time, I think the actual form factor of how you do ETL on this data will be a little different.

Great. What are some of the most recent stuff that you've released? I saw you had a 0.70 release. What does that do, for example?

Yeah, so if I had to paint a picture of the overall toolkit, LLMIndex started off relatively high level. I think what people really liked about it was the fact that it provided really nice kind of out out-of-the-box tools for people to get up and running really quickly of building any sort of like question answering system over their data. Because in about 3 or 4 lines of code, you can load data, ingest it, index it, and then query it. And then, you know, you would immediately get back a response. And so I think just from pure user experience standpoint, a lot of people did appreciate that because they didn't have to dig super deeply just to get something basic working. Of course, for beginner users or users that just want something out of the box, that works quite well. But in general, for more advanced users, people want to define custom workflows over their prompts, orchestrate different flows over their data, and actually write the core business logic, which is really the the prompts, like modifying, like deciding what LLM to pick and defining the wiring themselves. And so really for the past few versions, even from like 0.6.0 to 0.7.0, what we've been really focusing on is taking this, like the power of LlamaIndex as a high-level framework, but really exposing, rewriting the underlying abstractions to make it way more modular so that, you know, you could peel back the layers and really customize as deep as you want to. And what 0.7.0 really is, is like, it's a step in that direction. Because, you know, we now have our own kind of like very powerful like LLM modules as well as like response synthesis modules that users can just use completely on their own and independently of the rest of our abstractions. And it's only if they want to kind of like plug this into a more powerful query engine system and they don't want to write it themselves that they can actually use some of our out-of-the-box data retrieval and query abstractions as well. And what this overall enables is kind of a trend towards enabling bottoms-up LLM application development over your data. And it's a little bit different than top-down development where you just get something out of the box and it basically works. What bottoms-up development allows is for more advanced users to fiddle around with different modules over their data, be able to really like start off with the very core building block modules like LLMs, prompts, see what's working, see what's not, and then eventually compose them into more powerful systems. And so just as a framework level, it's very nice to be able to have the properties of both like stuff that's working nice out of the box, but also modular enough that you can just plug in and swap in different pieces.

And yeah, that's, that's an important point, right? Like, so you're trying to be a solution for two kinds of users or users with the two kinds of appetites. So both the people that want to do the simple stuff, but also the people that go very, very deep. How are you thinking about the tension between those? Are there trade-offs? And then as a related question, where do you feel the most pull in the current state of the market? Do people want simple stuff, or is there a growing community of people that want to go really deep?

Yeah, so I think a key philosophy that we launched in like 0.6 Auto that we're continuing to build towards is this idea that was inspired by Keras, which is this idea of like progressive disclosure of complexity, where in the beginning things are very high level and work well out of the box. But the more advanced you are and the, like, you know, if you have a higher need for customization, you can peel back the layers of the toolkit and really customize things at a very, very low level. So at a certain point, if you think about what a framework should allow, it should offer some degree of opinion because it provides a certain degree of abstractions that are nice, that are like a core set of abstractions that most users generally wouldn't want to touch because otherwise they would just build everything themselves. There always is going to be some segment of users that might potentially want to literally write everything themselves, right? And at a certain point, maybe they just wouldn't use any framework at all. But it is important for us to capture the entry users that might want to just understand things at a very high level and get started up and running, but really capture those core advanced use cases. And so I do think our framework has made pretty big strides in being able to capture that. These days, I honestly think that whatever workflow that you have of your data, you can basically do within LlamaIndex, even if it's a very custom workflow, because we provide just very light base-level abstractions that you can just plug in. And then if there are individual components that you want to use to basically abstract away the need to write some boilerplate, you're free to use that as well.

What are some use cases that you've observed in your few months doing this, both maybe the ones you predicted and some that surprised you?

Yeah, so I think this actually probably relates to your earlier question too as to what how are people using maybe the high-level stuff versus the lower-level stuff. I think in the beginning, there is and there still are a decent amount of, for instance, chatbots over your data and question-answering systems over your data system. These are all systems that are built on top of LlamaIndex. And the reason for that is we do make it very easy for people to get up and running just building these systems over their data. And so We've seen people build these workflows at different settings from, for instance, like hackathon projects at startups, building, for instance, like a ChatGPT-like plugin over like your Slack or your Notion, all the way to kind of like bigger companies, for instance, like Instabase or like other companies later stage like Uber as well, using the toolkit to kind of build more maybe like enterprise-level applications over their data too. And so a lot of our toolkit, for what it's worth, is centered around like search and retrieval and insight extraction from your data. So it's not surprising that a lot of the applications are chatbots. These days, I think it's very interesting because I think there's starting to become like a broader range of different apps that we're seeing within LlamaIndex as well. For instance, I'm having a webinar this Thursday with CEO of OpenBB, which is an open-source financial analysis tool. And they recently— incorporated LlamaIndex as a natural language layer to help power their basically open-source Bloomberg bot, right? But not, you know, but OpenBB style. And then, you know, we've talked to some other companies. They're actually using some of the more modular components of LlamaIndex, for instance, for structured data extraction to define their own custom workflows. So for instance, like just convert unstructured data into structured information. And so And then the last category that I think is actually also very, very interesting is using certain Llama Index constructs to not just like kind of deliver short answers over your data, but actually generate entire long-form pieces of content. And I do think like, you know, generating something that's like a paragraph is pretty easy for ChatGPT to do. Generating like an entire blog post or essay, especially over your data, is a pretty challenging problem. And I've talked to a few users that are using components of Llama Index for that. And so I think like in general, we, this very much is in our wheelhouse. Like we want to offer the best tools that build these types of experiences over your data. And we're also starting to see a bit more application developers build things potentially in like the agents category too, right? Where you not only do like search and retrieval over your data, but can actually act upon your data in different ways too. And I think that's something that we are actively keeping an eye on and are, should have a release very soon for that.

Speaking of upcoming releases, so what's on the roadmap, both short-term and I guess more medium-term? You talked about an enterprise version. Is that something that you're actively working on just yet, or is that more in the future?

Yeah, so we're actually building out and scoping out the initial version of what, like, kind of production-grade LlamaIndex would look like. The way we're thinking about it right now is that the open source tool is a Python package that provides a really nice orchestration framework for you to, again, ingest, index, and query data in different ways. What we really want to do is we've talked to a decent number of enterprise users that have used our toolkit, and they are interested in potentially— and we've asked them, what are the additional pain points or gaps that you would still want? Out of like a service that wasn't just an open source tool. And so what we really want to do is we want to develop like a complementary set of services around the open source tool that really helps augment the capabilities, especially in an enterprise setting. And so there's a few initial things that we're basically building right now. One is around like scalability, like being able to handle like larger volumes of data, especially that's something that like a hosted service that could provide that maybe like a Python package itself on your laptop might not have the same guarantees. Another is, especially in an organization setting, there's a big need for kind of multi-tenancy access control, those things, right, over your data. And then how do you actually provide this as like a service as opposed to, again, just like something that they build on your computer? And then third is like going back to this point of data connectors. We have a rich set of community-built data connectors that make it very easy for users to load in data. But what about data connectors with enterprise-grade guarantees? So that thing is that allow you to do loading from custom data sources with reliability guarantees, scalability guarantees, and also, for instance, also have access control and those things baked in.

How do you almost personally as a founder keep track of everything that's happening in this space, uh, where, you know, things seem to be changing every day, sometimes several times a day, uh, you know, both in terms of like your personal information regimen and then, um, how you translate that into the roadmap for LlamaIndex?

Yeah, yeah, I think that's a great question because, I mean, I think there's a question of where do I get my information, which is just— I, I mean, it's just like Twitter and Hacker News. And, and, you know, I downloaded Threads as well. And so, you There's a small but growing community of AI people on Threads. But then there's also a very valid question of, in a space where everything's changing very rapidly, how can you make, like, plan out product roadmaps effectively? How can you actually make sure that, you know, that you have, like, maintain, like, team morale and kind of maintain focus and vision of about what's going on, but still be adaptable to the changes that are happening. I have told pretty much the team, like, you know, there's going to be cases where there's gonna be things that come out where like, you know, this really is gonna be like a drop everything moment. Like, you know, just stop what you're doing. Like, this actually is like P0. We have to like figure out our strategy for this thing. But at the same time, you know, that's probably maybe like 10 to 15% of like the actual day-to-day work. Like we still have a concrete vision of this thing that we're gonna build and we're going to make steps towards that, cuz that really is a north star. So for instance, like a drop everything thing is just this past weekend, we like, like Wamindex ended up as a post on Hacker News and we were going through the comments and a lot of people said that like our docs were not great because they couldn't figure out actually, you know, going back to our previous points, like they were able to figure out the high level of what's going on. They couldn't figure out how to customize things and how to actually, you know, like play around with the low-level modules. So we basically, like, you know, that was very important to us because if new users are coming in and they're just churning because they can't actually figure out how to do things they want, then, then, you know, it meant that we wouldn't be able to have like real user growth and adoption of the framework. So we basically just like stopped what we were doing, spent the past 3 days just completely rewriting the documentation. And then we just launched that yesterday. Um, but at the same time, you know, there are these longer-term feature releases that are going on. That we have planned out and that, you know, we're launching in the next few weeks. And these things are really just bets that like, hey, you know, like this would be good to have. It's good to spend like at least a few weeks working on this and let's get this out, develop it a bit more, make it a bit more kind of usable and production ready, and let's iterate on user feedback. So in the end, I don't think the roadmap cycles can be super long, but you should have a north star of what you roughly aim to get within like 6 months to a year, but kind of iterate quickly and probably like a week, 2-week increments.

It sounds like you have a wonderful, very, and very responsive team. On that note, how do you think about building a team and who to recruit for a company like Llama Index? Because arguably the generative AI space, depending on how you define it, is, is new. And there's not that many people that have experimented with those things. How do you, how do you, how do you recruit and who do you look for?

Yeah, so I think it's a great question because it's something that, you know, is very relevant to what we're thinking about. You want to hire people that in general for like startups are passionate about what you're doing. I mean, it sounds pretty basic, but like the people that are just passionate or just willing to, you know, put in more hours, do more things and just really like do whatever it takes to help the company get off the ground, especially in those early days. Is that's so important to just like make sure that your company is successful. And it's something that you kind of like really know it when you see it, right? Like some people are just like very energized about what you're doing and willing to do whatever it takes to help the company. The next part is like relative to like this whole fast-moving trend of AI space. Yeah, you have to be like both like technically savvy, but also very adaptable, right? And it's not gonna be for everybody 'cause like, you know, when you kind of, sit down at the beginning of every day and you check Twitter and you check Slack and you check everything that's coming in, it feels a little bit like a fire hose. There's a lot of information to parse through. And you have to be willing to just both aggressively prioritize what's important and try to ignore the rest of the noise, but also not overly fixate on something for a few weeks if it means that something more urgent has come up. And so we do really try to look for those qualities. Right. And along with, you know, we have a high bar for hiring. We want to hire like smart technical AI people, like the backend people, really from like a values perspective. That's what we optimize for right now. And again, you know, it's, it's like, it's a high bar. I think there's a lot of people that are very interested in like working on a project like this, but we want to make sure that, you know, these are the qualities that's going to take the company forward.

Great. So maybe as a last theme to, to close, uh, zooming back out, uh, any thoughts on where you see things going, the space going, uh, whether that's, you know, in the next few months or in a few years from now? What, what gets you excited? Um, what do you think is overrated on the flip side? And yeah, how do you think about all of this?

What do I think is over— it's a good question. I guess I could drop my spicy takes in all of this. Let's see. I think what gets me excited is this overall vision of having knowledge workers that can do automated decision reasoning and just understand data that's given to it, but also act over that data. I think that's just almost like a kind of like AutoGPT, but focused around data orchestration management, right? And just really thinking about what that new software stack is going to be composed of. Because on the one hand, you're going to want to define a new data stack to enable these kind of LLMs and agents to operate. But these LLMs and agents can also be core components within that data stack too, right? They can help really automate a lot of the pieces of ETL, insight extraction, all these different things. And so just like defining that vision is very exciting, 'cause I think like if you're actually able to achieve that, you unlock like a tremendous amount of value for basically anybody that just like, that is untapped with like the current state of software. Just like the ability to just like understand and act upon information just becomes like much better, right? Than what existing software allows. So I think as we build towards that vision, we want to make sure that we're constantly providing these tools that are allowing people to build this new world of software to build these types of knowledge workers over your data in a manner that's very reliable and performant. And I think the other part that's exciting here is that no one really understands what exactly does that mean because there's so many different choices. Out there these days in terms of like what model to pick, what embedding model to pick, how do you actually define different types of prompts, how do you like wire different agents together? And then also like, you know, the field moves very fast every day. And so really just like getting a handle on like the best practices here and also helping to define this new software stack is something that's very exciting, right? Just like from a very conceptual level, whether it is on the open source side or kind of more on the enterprise value side.

Great. And well, that feels like a wonderful place to leave it. Sounds like you are building a really exciting company in a very exciting time. So congratulations on all the success so far, and hope to catch up again, you know, maybe a year from now, or as, as you continue to progress and, and build a Llama Index into an industry giant.

Awesome. Thanks so much for your time.

Thanks, Jerry Liu.