In this NYSE Floor Talk interview, host Judy Shaw speaks with Jerry Liu, CEO and co‑founder of LlamaIndex, about the company’s mission, origin, differentiation, and future roadmap. Liu explains that LlamaIndex is a data framework designed to enable large language model (LLM) applications to reason over private, enterprise‑specific data such as documents, spreadsheets, and APIs. The idea originated when Liu built a personal sales‑assistant bot that needed to ingest and remember past customer conversations, revealing engineering challenges around LLM input limits and data ingestion. He positions LlamaIndex as distinct from other orchestration tools like LangChain by emphasizing deep data management, robust ingestion pipelines, and extensive integrations with vector databases and major LLM providers (OpenAI, Anthropic). Looking ahead, Liu outlines a product roadmap that moves from a developer‑friendly prototyping toolkit toward a production‑ready, enterprise‑grade platform focused on data ingestion, indexing, retrieval, and reliable deployment of LLM‑powered agents and chatbots.
I'm Judy Shaw for NYC Floor Talk. Joining me today is Jerry Liu. He is CEO and co-founder at Llama Index. Jerry, it's wonderful to have you here. Thanks for joining me today.
Thanks for having me.
My pleasure. So now, Jerry, tell me about your company and its mission.
Of course. So Llama Index is a data framework for building large language model applications. And so just to give a little bit of background, large language models like, you know, GPT-3, Anthropic Claude, GPT-4 by OpenAI, have demonstrated a tremendous amount of value in not just being able to generate knowledge, but actually being able to reason over things in kind of a magic manner. If you've ever used like ChatGPT, you know that it basically can just know the answers to things once you tell, once you ask it to do something. And so one of the key challenges that we thought was this is an amazing interface that could really automate knowledge workflows, but how do you actually combine the power of LLMs with your own private data? Let's say private to an enterprise or an individual. If you have your own databank of notes or documents or PowerPoints or spreadsheets or even like conversations or APIs that you have in a workplace setting, how do you feed all this information into a large language model so they can not only just reason over what it already knows, but what it doesn't know and stuff that's private to you? And so we set out to build a toolkit about 7 to 8 months ago to really capture the essence of that and to really make an open source project that was easy for people to use. And so since then, it's gotten a ton of usage and really grown. And I've been very fortunate, or consider myself very lucky, to be kind of in this position where a lot of people are getting interested in building large language model applications, and a lot of them are using LlamaIndex to power these types of applications over their data.
All right, so Jerry Liu, you're co-founder of the company. What was the spark? What inspired you to start this company?
That's a great question. So back in October or November, I was playing around with GPT-3. I think a lot of people were playing around with OpenAI models and ChatGPT actually hadn't come out yet, but I was playing around with the API. And so as I was playing around with it, I wanted to build a sales bot for my previous company where I was in some customer conversations, but they were sporadic. They were, for instance, once every 2 weeks. And every time the second week rolled around, I would forget what was going on from the previous meetings. And I wanted some AI assistant to help me capture those thoughts and synthesize a to-do list for me. For the next meeting. And so I set out to build a bot that could just ingest volumes of previous customer conversations about this current company or client, and I wanted it to be able to memorize that information and generate a to-do list for me. And so actually by building that, and if you think about the concepts, it's basically large language models over some data that was private to the company, which is customer conversations. I quickly ran into some engineering challenges of how do you actually do that efficiently? For instance, large language models have an input limitation like ChatGPT of how much data you can feed it at one time. And so what I wanted to do was build some initial toolkit that can abstract away some of that complexity. And so that quickly caught on within the developer community and I was doing it just for fun initially. And then I think about a month in, I saw there was a real opportunity here to build a transformational business because LLM is kind of like LLMs are eating software right now and they're going to be a critical part of software development in the future.
All right. So now tell me, how are you different from others that are in this space?
Yeah. So just to give you a sense of like the landscape, there's other orchestration tools. And so some of you might have heard of like LangChain. There's also some other tools in this space. I would say ever since the beginning, we've been very, very focused about the connection with your data and we've been trying to build a very quality toolkit to really solve the problems of both data management and ingestion, as well as being able to combine data with your LLMs when you actually ask a question over it or give it an input task. And so we've really curated this really great set of tools that anybody can use, and we've been very focused about that. And so I would say, you know, compared to orchestration tools, we're a lot more focused about the data. And we also integrate very deeply with other providers in this space. So for instance, like vector databases and other storage providers we have extensive integrations with. We have integrations with all the major large language model providers like OpenAI and Anthropic as well.
Okay. And finally, tell me, as you look to the future, what are you focused on? What's the priority for you?
Yeah, so our product roadmap is roughly kind of in sync with how people are developing LLM applications these days. A lot of people started off prototyping and testing these initial features without actually putting it in production. And so to that end, we made a toolkit that allowed basically anybody to create an LLM application over their data, regardless of their kind of technical level of sophistication. As time has gone on, more and more people are putting these type of application features like chatbots, search bars, and agents into a production setting. And so we've started to really double down on improving both the quality of the open source tool to make sure that it's robust in a production setting, as well as developing an enterprise platform centered around data ingestion, indexing, management, as well as around the retrieval querying part in order to provide a robust platform for enterprise needs.
All right. Well, Jerry Liu, wonderful to talk with you. Thanks for joining me on Floor Talk today.
