Document AI Workflows: Automating Real-World Document Work with Jerry Liu

Those are basically the two patterns of use cases I see related to documents. As people move from very simple chatbots to more advanced feature automations, one general category related to workflows I think is going to be very high value is this full industry of process automation related to documents. And this captures all the work that humans are doing in a relatively repetitive, kind of low-thinking type environment where you're basically just like reading a document and trying to put that data somewhere else. Whether it's invoices, whether it's onboarding forms, whether it's claims, there's a team of humans that try to do this in a very manual manner. For a lot of these workflows, there's a huge opportunity to encode these workflows as high-accuracy, agentic automation over your documents. LLMs have this amazing ability to consume and understand unstructured text tokens, but they can't natively read a lot of these file formats. That's very prevalent. Throughout pretty much every organization.

Welcome to ODSC's AIX Podcast, your gateway to the latest in everything AI. Whether you are a seasoned AI expert or exploring how to bring AI into your work or career, we bring you insights from the best minds in the field. We host researchers, practitioners, and industry leaders shaping the future of AI and how it's being applied across research, product and industry. This episode is sponsored by ODSC AI East, the premier AI builders conference coming up this April 28th to 30th, 2026 in Boston and virtually. Learn more at odsc.ai/east and use the code PODCAST for an exclusive 10% discount.

Today we're talking about document AI workflows, automating real-world document work in a way that actually holds up in production. My guest today is Jerry Liu, CEO of Llama Index. And one of the big takeaways is that Llama Index is now focused on document processing and workflows. Because for most companies, the hard part isn't the model. It's getting accurate structure out of PDFs, Word docs, and spreadsheets so downstream AI workflows can actually run. Now that shift from chat with your data to systems that execute is exactly the thread we're going to pull on today. Jerry Liu, welcome back to the podcast.

Thanks for having me.

Jerry, I don't have the exact date, but I think you were on the podcast a couple of years ago. So much has happened in the AI space. It's been quite incredible. So delighted to have you back on the podcast. And we've got a lot to talk about, but before we get into that, for the people who do not know you or do not know Llama Index, I'm sure everyone knows Llama Index, but maybe they don't know about you. Jerry, if you want to give us a little bit about your background and some pivotal moments in your career.

Everyone, as you might know, I'm the co-founder and CEO of Llama Index and more on the company itself. In a bit because the company itself maybe shifted its focus a little bit since the early days as an open source framework. We started the company back in 2023. Before then, you know, I spent a little bit of time as an ML lead at a startup called Robust Intelligence focused on security and monitoring for machine learning models. Before that, I led a little bit of research projects at Uber ATG, kind of focused on deep learning for self-driving cars, and then also spent some time as a machine learning engineer at Quora. So I spent most of my career in machine learning. I would say the most pivotal moment was getting into machine learning in the first place. Just in 2017, my senior year of college, when I really didn't know that much about deep learning, but I came across these things called GANs, generative adversarial networks, which could start generating pictures of things like faces, bedrooms, scenery, and all that. I thought it was super cool. Obviously now, like, you know, 8 and 9 years later, you can generate entire, like, motion videos. Actually start building AI-generated movies. And so it's crazy to see how the space has come. But that was really the impetus to get into machine learning. I would say that was basically the starting point for my career.

Jerry, that's quite an impressive career. Some great gigs you've had over the years. Those of us who know Llama Index, Llama Index was synonymous with RAG, Retrieval Augmented Generation, right? And as you mentioned, you've had a few changes and I saw that you recently posted, maybe it was a month ago or a couple of months ago. You said LlamaIndex is no longer a RAG framework. I was surprised to hear that. Could I say that was kind of like McDonald's saying they're not making burgers anymore? Because you are very well known in the RAG tooling industry. A lot of instructors who came to ODSC and gave workshops on RAG, LlamaIndex was one of their reference frameworks. A lot of people were saying RAG is dead, but you were more succinct in your statement. I think you said RAG 1.0 is dead. Yeah. So what actually died? What's going to replace it?

Today, the mission of LlamaIndex is document processing workflows using state-of-the-art LLMs, VLMs, and AI agents. We decided to go all in on document understanding and really being able to unlock context from unstructured document-based data. And by documents, I mean like PDFs, PowerPoints, Word docs, Excel sheets specifically to really digitalize all the context that exists in these unstructured file containers. So it's a little bit more of a narrow focus, but an important one compared to, say, our early days as a broad framework. If you've been following our evolution over the past 2 to 3 years, it probably has made sense if you do a gradient descent from kind of the early days over into where we are right now. You kind of tracked in like with us 3 years ago and now you're following where we are now. It's a little bit of a shift. We still very much value the open source community and being able to equip developers with the right tools to be able to build various types of LLM-powered applications that make sense over their data. It's just, they were focused more on the data aspect. Like, rewind a little. We started off as a very popular open source framework focused on, you know, this is basically the early days of ChatGPT when people were still trying to figure out various patterns for building AI applications in the first place. We started creating some initial patterns around RAG that were open source abstractions in the library to enable people to connect various types of data. The space itself has changed quite a bit over the course of the past 2 to 3 years. Besides different RAG patterns, there's also been advances in, in agent reasoning, tool use, memory, and all that. Nowadays, like, the ways that agents interact with data has fundamentally changed. But even 2 years ago, we realized that even as people were starting to build some of these simple applications, like RAG applications over their data, we noticed a good chunk of this data was basically unstructured PDFs. LLMs have this amazing ability to consume and understand unstructured text tokens, but they can't natively read a lot of these file formats that's very prevalent throughout pretty much every organization and individual's hard drive. This plethora of PDFs, like Microsoft Office documents within a person's computer or even a server somewhere. Existing OCR tools at the time to enable the translation of this document format into text tokens were really terrible. I tried out a bunch of them and, you know, if you try to feed a financial report, into one of these parsers so you could get text tokens, it would corrupt all the data in the tables. It would mess up the charts and the valleys. And at that point, it didn't matter how good the LLM was because you just weren't able to actually do anything with this data. So that actually motivated us to really, as we thought about building deeper tech to one, make money in the space because we were a VC-backed company, but two, also figure out how to build enduring, long-lasting tech to invest more heavily and making sure we have really good technology for the parsing layer. Basically making sure that we're able to create foundational infrastructure to translate unstructured documents into these interpretable text tokens with extremely high accuracy and low cost so that whatever upstream or downstream agent pattern they're trying to implement, whether it's a simple chatbot, a chatbot workflow, a generalized coding agent, that they're able to make use of this data in a high accuracy manner.

Yeah, Jerry, I think that's so fascinating. Over the years, I've worked a lot with documents. We were trying to look for signals in the noise. So it's a fascinating field and it's very interesting what you say about OCR. When I was doing some research on this podcast yesterday, it became more apparent to me that naive RAG has failed to a certain degree. And that's why, as you said, we've got new RAG patterns, but legacy OCR has also failed. And anyone who's ever done this, I think you mentioned intelligent document processing, IDP, as they call it. It's very painful. And if I can layer onto that point, everyone was saying that 2025 was the year of demos and 2023. '26 is the year we actually get AI to work. I love this shift because this is when we can actually dive deep into what companies are doing because they want to automate all this document grunge work. Anyone who's worked with OCR, you've done work with OCR, you can see that it has this coordinate-based fragility, right? Like, um, it expects the total amount field to be in a certain place, and if that moves a little bit, it'll mess up everything. And then naive RAG didn't work when we tried this either because It chunked everything, it broke up context, and then lost much of the meaning of the whole documents. Tell me how NuFocus solves the problem of OCR and naive RAG.

Maybe to piggyback off your point about trends, which I totally agree with, add some more context to the audience. It's right that both existing OCR solutions are not very good, and the initial form factors of AI applications were relatively naive. As we've progressed from 2023 now to the beginning of 2026, You see both of those advancing in parallel. And so obviously from the AI applications perspective, we still very much value the open source tooling that we have to enable you to build applications. It turns out we also have really good tech around the document processing piece so that you can use it as a foundational layer in your apps. The applications have evolved from simple RAG form factors to agentic workflows that started to take off in 2024, 2025. People are starting to compose these multi-step patterns that actually do things versus just something that that you can chat with. 2025 was basically all around the coding agents, and that carries over into this year. Just generalized problem reasoning with a code interpreter, file system, full use skills, MCT, whatever. You can start doing a bunch of things to automate various types of knowledge work besides writing code. So, you know, there's kind of some agent advances in terms of enabling knowledge work automation. And at the same time, the OCR space has also evolved, you know, and, and I would like to think we've contributed to it too, because we started this tech back in early 2024. It requires either, like, as we said, very template-based or rule-based extraction that fits a specific template. Even custom-trained ML models need a sample of data to fit. In this new age of LLMs, the LLMs, and, and kind of figuring out how you can prompt them too, they provide a much higher layer of generalized baseline accuracy, which means you no longer need to actually train some specific model. For a specific type of document. You can just use one thing and hope that it works well across every document. Now, the kind of baseline everyone goes to is what if you just screenshot the page, feed it to ChatGPT, um, and then let it parse the text? And that's actually not a bad baseline. It's just one, pretty expensive, and then two, still got a lot of hallucinations. A lot of what we're building these days is being able to make use of this step change in the OCR landscape to enable you to do one-shot parsing. Of any type of document, but do it with much higher accuracy and lower costs than a lot of these baseline approaches. So we interleave Frontier VLMs, fine-tune VLMs along with our own heuristics, parse any type of document with like super high accuracy and low cost. And we see that kind of powering two classes of applications. On the AI agent side, there is this evolving pattern of like chatbots, deep research agents, and you know, coding agents where you want to provide a layer to digitalize all these unstructured documents to provide a knowledge base for these types of agents. For instance, if you have a lot of files in your SharePoint, Dropbox, Box folders, you need some layer to be able to convert these all into Markdown so these coding agents can understand it. Now, as I said, there's also this existing industry of intelligent document processing, um, and that's been around for 20 years, way before AI. A lot of humans manually review invoices, process documents, onboard users, do KYC, you know, review insurance claims. You're just scanning through packets of these documents and trying to do manual data entry into some system. And thanks to a lot of the advances in OCR technology, and also due to our stuff, you're able to automate this processing rates of doing a lot of this with way less human in the loop. It's still there, but the accuracy levels are way higher and the setup time is a lot lower.

Yeah, so much to unpack there, Jerry Liu. That's all massively insightful. I guess my first question there is, is again, looking at traditional OCR and IDP, the pass-through rates were one of the key metrics for figuring out if this is working or not. And I remember it's all over the place, different industry, different tools, but a poor pass-through rate was as low as 70%. Sometimes they got up to 80, 85% without a human to check. What kind of pass-through rates do you see? And when I say pass-through rates, I mean that percentage of documents that flow through without human intervention.

Yeah, I think the baseline password rates vary quite a bit depending on the complexity of the document. Some of our customers see password rates as like 10% because it's actually just, you know, everything is kind of terrible. There's a lot of density in the pages. Existing solutions just fail massively. And then sometimes baseline password rates for like a certain type of document that conforms to a certain type of template is actually decently high, like 80, 90%. Typically when we come in through our core capabilities, around the LLM-enabled document OCR, we are able to lift the passer rates of even the most complex documents to like 40-50%. For the ones where there's already a high baseline, lift it to like 95% plus. The one thing that I think people also would want to consider, especially if you're considering like a document processing solution, is the amount of human time it takes to onboard a new use case. Because I think with existing OCR, to get to that passer rate of 80-90%, or just like in terms of accuracy, you typically would have needed like a few folks on the IT team to configure this using one of these like C3 or a hyperscaler solution to train a custom model over your data type for a few days at minimum before you can actually onboard it for the use case. These days you can get started with any type of document almost immediately on our service. It provides an accuracy lift in terms of the end-to-end quality of the outputs. We are able to both increase accuracy, lower costs relative to existing and then there's also minimizing the time to value in terms of onboarding in the use case. Yeah.

And just go back to Naive RAG again, because so many companies have spent a lot of time implementing that. And just in case they're wondering why we're ragging on RAG for a little second, it is very useful. We're using it in workflows and finding still a lot of use out of it. But one of the drawbacks that I see and people mention is the loss of structure. If you're looking at a financial report or a research report with multiple columns, converting all that into text embeddings. The structural context is gone. That's kind of huge. Another problem we see, especially with these bundled documents, like mortgage approvals, massive bundles of documents. It could be financial statements, their investments, their bank accounts, you name it, all this stuff. Does this kind of AI, if I can call it documentation AI, I'm not sure what you guys call it.

Documentation AI.

Does it address those problems of loss of structure and that bundled document problem?

Yeah. So if you're looking at the end-to-end stack, the end use case you're trying to solve is you have, let's say, a million documents. Contains the Microsoft SharePoint, contains your entire enterprise repository of knowledge, and then you're really trying to provide that as context to an AI agent. The nice thing about what we're focusing on is you absolutely need the OCR layer. You need something to accurately convert your documents into well-structured text with right metadata, headers, citations, confidence scores, bounding boxes, so you have all the information you need in a very accurate manner. So you need OCR. Now, if you're following the naive RAG approach, Or just a general retrieval pattern, you need some way of actually putting it into some storage system in a way that's indexable and searchable. Naive RAG, you chunk it, you embed it, you put it into a vector database so that you surface it for search later on. There's other patterns now, thanks to some of the evolving capabilities of agents that make some of those existing patterns either obsolete or not worth it for small and medium-sized documents. To give people a sense. If you were to architect the system, what type of stack should you use? These days, for small to medium-sized documents, and when I say that, I mean like a few hundred to a thousand documents, you're just trying to provide basis context to your agent. It's not actually clear you need to do indexing at all. You know, the baseline these days is you just give it as a file directory or file system to some coding agent, and the coding agent with access to CLI, and able to, able to use like ls and grep and a bunch of these different bash commands, maybe with some lightweight keyword search too, so that it can search across these files. It does a good job of actually finding these documents and it's able to like surface the relevant results for you. And the advantage of basically the advances of agent reasoning is now instead of just getting like fixed chunks of the document, once they find a file, they can use tools that scroll up and down the file. They're burning compute while making the retrieval interface simpler. but now agents can dynamically figure out what context within the file that actually matters in order to give itself the right information to make a decision. So it's nice by advancing agent reasoning these days, you can create relatively simple patterns similar to how a human would actually use, you know, your file system. Like I, you know, as Jerry Liu, I don't have a semantic index of my file system built in. I just use the tools on my computer. And these days, agents can also use the tools on your computer to figure out information there. Small to medium collections is quite interesting. I mean, a lot of people are interested in this idea of just like file systems and using a coding agent for search instead of needing to build an index at all. You still need OCR. You still need to parse the doc somehow for machine understanding, but you might not need the retrieval index. I have maybe 70% confidence in this statement. That's something that people are exploring. If you look at coding agencies like Cursory and Quadcode, they probably have some retrieval built over their codebase. Yeah.

Yeah, that's spot on. Go ahead.

The other piece now, I think if you go beyond a few hundred to a thousand docs and you're kind of going to this world where you have like millions of documents, you need to maintain access control across all the docs. You actually need the search to be pretty fast, like Google levels of fast, and you need to make sure you always maintain high relevance to the user query. Then yeah, you totally need some sort of index. You need some sort of retrieval. You need some database. And in fact, that's actually an extreme hard problem. Once you get to that territory, you need to invest effort in making sure your retrieval pipeline is performant. And can serve user queries with low latency and high accuracy. We work with a variety of customers at that level of scale. We have some of the tooling to enable that to be possible. But it's almost like a little bit of a different ballgame. If you're just trying to build a prototype over a small set of docs, you can get pretty far with just a file system and a coding agent. Once you start to really get into retrieval, you need to get into it. You need to spend some brain cycles making sure this is an entire system that can scale.

No, absolutely. Because one of the biggest issues with RAG is they could do a decent job of answering questions, but they actually don't do anything. So building in that agentic workflow, as you mentioned, so key. Your last statement there about latency and speed made me think it must be quite an interesting stack because you said at the very start, these multimodal reasoning models are so key to this. All the advances we've had in 2025 around these multimodal models has made a lot of this possible. But now, as you know, model routing has been a big theme of 2025 because sometimes you want that fast mode, right? You've got low complexity bundles of documents and you've got a high volume of them. But other ones you've got more complex documents. And then I'm sure that there must be a fear that involves this genetic reasoning. And, and I'm sure with these documents as well, you would have to be doing multiple passes because that seems to be a design pattern that came out of 2025, not just try zero-shot it, but do multiple passes.

I think in the OC layer, when you're trying to convert a document format into like structured markdown text, you use the VLLive and under that you use a draft thing too, so you can have the VLLM do a little bit of self-reflection as a simple example to make sure that's actually parsing the text and getting the output right in a fairly accurate manner. And so, you know, even with these advances in multimodal models, you know, you can figure out a few kind of like core ML tricks to make sure that you actually get both like highly accurate and cost-optimized. So doing age validation is one technique to achieve higher accuracy. You can also do some auto-routing to use a smaller lightweight model for Chica parsing and a heavier weight model for more complex parsing. On the OCR side, there's certainly a lot of DLM, AI, secret sauce happening there. Once you start using this parsed output for retrieval, you probably still want some sort of agentic validation on the markdown so that the agent itself, in terms of solving the user task, can actually figure out, is this the right context that I want to load within my context window, or do I need to continue searching to make sure I'm actually solving the user task? And that is kind of captured by this pattern of coding agents being able to navigate a file with tools versus say naive RAG, which just returns fixed chunks and then the agent doesn't have a way of actually going across the file.

Yeah, it's interesting too, because just explaining a little more in OCR and you can correct me if I'm misstating this, but OCR basically traditional OCR, you're looking at pixels, right? With VLM, they look at an actual image. And even though my image is a collection of pixels, the difference is the VLM can identify those visual cues like the bold text, the header hierarchy, segment separation. I think that's so key to it. So when you say the OCR is getting better, at what point does OCR stop?

So part of the technology shift in OCR is instead of going from rule-based or specialized model-based or template-based document extraction, you can feed in an image encapsulating a document, an LLM, and have it do some general visual reasoning over the document to give you back parse out. If you think about it at a very simple level, that's basically what we do on the OCR side. Now, as you said, like the baseline that you could always do is screenshot the page and feed it to your favorite LLM vendor and then don't do anything outside of that. Yeah. Feed it to Opus, you could feed it to Gemini and just ask it to parse this image into Markdown. If that worked 100% of the time, we wouldn't have a company or a business. And so I think basically like it, one, it doesn't work 100% of the time. But actually there's like a ton of these failure cases and documents where a one-shot VLM call isn't enough. Oftentimes you get way higher accuracy levels if you blend in a few different techniques and approaches. We use both frontier models as well as specialized models and heuristics to boost general accuracy. People want metadata and citations. They want the ability to draw a layout over various elements. If you want the users to be able to see where this answer or something came from in the document, be able to highlight the bounding box and attribute. Where that came from, but it's super, super important user experience and you need to be able to have that metadata. And then the last piece is cost. For simple text-only pages, you don't really need to screenshot the thing. You know, that text oftentimes just exists in the PDF binary, so it's kind of a waste of tokens if you screenshot that and feed it, especially at scale, it's super expensive.

Yeah. And then just lifting it back up a bit here, because I assume you're talking to a lot of people in industry about this. And of course, everyone I talk to in industry, they're very excited about AI, but they're more excited about workflow. How do I build a workflow to do financial financial analysis? How do I build a workflow to improve invoices and reconcile purchase orders? How do I trigger payments? When I was researching last night, it became very apparent to me, like, for the last couple of years, we've been very much focused on search. I've seen that mode kind of fail because when I've built search interfaces, like, okay, what do I do? But enterprises do want to move from search to workflows, and you're advocating document-heavy workflows, which for most enterprises, that's a lot of what they do, especially large enterprises. Do you think people are recognizing that shift in industry, moving from search to workflows? And do you think teams are underestimating how big a transition that is and how hard that transition is if you try and build your own or roll your own?

Yeah, there's a lot of downstream use cases related to documents that take a lot of human time. So we're very excited to provide some end-to-end tooling. Well, both those CRPs, but also beyond that to enable different types of developers to actually build these types of use cases. As you said, some downstream document-based use cases include deterministic workflows like invoice processing, claims processing, reviewing contracts, that type of thing. It also includes more general search and chat interfaces where the inputs can be less deterministic, but you want to interact with the agent to do all their research and surface information as an assistant to the human. You still need the core components like document OCR. The layer on top of that is some sort of orchestration layer that encodes business logic into rules that can execute. Allow the human to validate, but still execute somewhat deterministically and also autonomously so that you can achieve the outcome that you want. So we've been thinking about various types of document workflows that we can automate. We have a layer called Llama Agents on top of our core platform, Llama Cloud, a document OCR layer that enables users to build and encode these document workflows and run them as a traffic automation with human in the loop.

Jerry Liu, Llama Agents sounds very interesting. Can you give us a broader overview of what Llama Agents are? What they do?

Maybe just to give a product overview of our overall ecosystem. We have a cloud, which is the core set of modules for document understanding. That includes document parsing, converting a document into Markdown, document extraction, converting it into structured outputs, document splitting, classification. So you take a document, you can chunk it into a bunch of different pieces, and then you can also classify and route it to a downstream system. So a lot of fundamental modules to convert your document into something On top of that document processing layer, we now have this feature called Llama Agents. Llama Agents is the ability to orchestrate workflows on top of your documents and allows you to encode various types of workflows. For instance, define an invoice processing workflow or review this packet of resumes that I sent in. And you can do that both via code, but also natural language because we built like a live coding builder within the platform. Translate your request and compile it into some sort of codebase workflow that can run autonomously on a trigger, then also give you back some results with you. It's a seriously cool feature that we released this week in beta. It's built on top of a lot of the open source tooling we've provided over the past 2 to 3 years, what kind of relied on our expertise around orchestration and allowing developers to build various types of apps. That's over their docs. It's built on this workflow layer that's open source and allows us to have this coherent story of you need to solve the use case over your docs. First, you need to process it. Then you want to use some of our tooling to help build workflow over your docs. Llama Agents serves that workflow layer. If you haven't checked it out, feel free to. And we just released the builder in Veda. And so that's a natural language layer. You just type the workflows you want through English. You spend effort. Making sure it compiles into a code-based workflow. So even if you don't know how to code, that's okay. You define what you want in English and then you have a process automated set up in minutes.

Yeah, that's very cool. What I'll do is I'll find a link to that. Seems very new, Jerry Liu, but I'm sure I'll be able to find it. So can you tell us some of the use cases you think that are going to be popular?

There is a category of document extraction use cases that we're very focused on right now. This gives us the opportunity to expand it to broader use cases. General knowledge work off of your docs later on. In terms of single document extraction, this includes basically everything in the IDP industry. Invoice processing, claims, contracts, anything where you want to have a document and then make sense of it somehow. You don't just have one document, you have like a million similar types of documents and you need a codified workflow that uses agents to repeatedly extract the relevant information from document at scale. The neat thing here is it's not really like a chatbot, right? You define your task. In natural language, it'll convert that task into a code-based workflow, and then you can run this workflow over a million documents. For the TrapBot, you'd have to type in a million trap prompts to get the agent to do what you want, and it'll do it in a bit of a non-deterministic manner. Here, you define the workflow, and then you can just run the workflow. You can define multiple workflows relatively easily in minutes. Yet, you know, you can focus on single document extraction use cases. That's the next generation of what IDP looks like instead of intelligent document processing. This is agentic document processing and agentic document workflows. After that, there's a ton of opportunity. I mean, the, this market expands way beyond like the traditional sphere of like just a document workflow over a single document. Imagine like you're basically building like an AI employee. Like let's say you are a kind of private equity analyst reviewing a deal room of docs. You need to do some sort of cross-document analysis over this data. Come up with financial modeling and then generate a recommendation report at the end. That is possible within the broader sphere of use cases. We're not there yet within our own product, but having a lot of this infrastructure does allow us to expand to the overall space of what humans are doing in a way that's repetitive or could be automated over documents. Yeah.

And you dropped a key phrase there, Jerry Liu, that I'm expecting to hear a lot more of this year, the AI employee. But going back, I do like your chat analogy, because if we think about that, it really is a dimension of time, right? As you said, a chat session can last minutes, right? Business workflows can last days, weeks. My last mortgage approval lasted months. I've seen how you guys emphasize this, we call it durable execution, which is an interesting concept, this idea of durable execution for long-running workflows, because if you look at some of the benchmarks out there, there's a couple of benchmarks out there for looking at how well AI does long-running human tasks. And I think with Claude's Opus 4.5, it's getting close to 5 hours. 5 hours, you know, 4.5 or 5 hours where it breaks down. Last year it was 1.5 hours, so we've come a long way. But back to my point, we know a mortgage approval takes days, not minutes. So what happens when the workflow encounters a discrepancy and needs to pause for human review? But that could be 1 AM. Like, does it go to sleep? How are you guys solving that pattern?

Yeah, I think solving the problem of having a human-in-the-loop review is actually not bad. We kind of have that already within our workflow framework. Other frameworks have this too, but it's basically like within a workflow, you can pause the events and then wait for human events.

That's the problem. I don't know if you've been using Llama, for example, and every 4 minutes I'm telling it, do not ask me any more questions, just go do it. It keeps asking it. There's too much human-in-the-loop, not enough AI as a paradigm.

Yeah. So that's why I think there is a little bit of a difference between a workflow, which is deterministic, and workflows could be, honestly, it could be short running or it could be long running. But if it's a mortgage approval, I'm sure, you know, a lot of the reasons it's long running was because you need a lot of back and forth with a human. For invoice processing, it's usually a few minutes per human. The real problem is being able to process a million invoices at scale. But the actual processing of one thing, not too long. I agree with you in that agents are moving towards longer horizon tasks. And that's awesome. We've been thinking a lot about what it means to run these long horizon agents in the background. Right now, we think there is a ton of opportunity within some of the short horizon tasks of just like being able to enable humans to define these automations that can still do it much faster and help automate some of the straight-through processing rates of docs. But yeah, even within our own product, we're not quite there yet. But like, obviously, there's a ton of document work where if you have an employee that just like deals with docs all day. This is something that's going to be super valuable.

Yeah, I think that's spot on. I think we're going to see that. Just to talk a little bit about your approach here, because we're talking about kind of like in a very abstract level here on workflow, which is interesting. You mentioned event-based, which we should explain to people. So my background coming from finance, everything was event-based. You only did X, Y happened, you know, short gold if things go crazy or go long gold if things go crazy, things like that. Landgraf is very popular. And then of course, We've got a lot of tools coming out. There are graph-based multi-agent orchestrations. We've had quite a few discussions around graph-based multi-agent orchestrations on the show, and any engineer can see the limitations of DAGs. For example, we use Airflow. It tends to work well when it's linear, which you would think would work well for workflow, but it's also very fragile when going back to a simple agentic RAG or even something similar like Airflow. If it fails on step 3, What happened in step A and B is sometimes lost depending on how you build it out. So are you guys relying on DAGs or are you emphasizing more on the event-based side of the workflow or a combination of both? How do you approach that space? Because I've had some interesting conversations with people around that.

Yeah, for sure. Maybe during the days when we were purely focused on the open-source framework, I would have had more of a dog in this fight, but these days we're not really trying to build a company around workflow engine execution. If you think about frameworks that are just temporal, temporal is Probably our default engine of choice for workflow orchestration. And then there's a system for actually executing the task. And then there's how you let humans define the various events to actually build an agent workflow. I think for us, we've built this workflow layer almost more as a syntax or semantic layer to enable users to build various types of agent workflows in a flexible manner. It is very event-based. You pay attention to input events within a step, and then you can emit output events. And this could kind of go in cycles, like you could pass events back and forth between two, two steps, almost as like two agents passing events back and forth, just to make sure it actually got things right before emitting like another event. It could also run in parallel and async, like you'd have two branches that just like pass buffer events, make sure that things work well. It's a very lightweight semantic layer to help you define various types of agent workloads. What I care about most is making sure that we're actually solving document use cases. Workflows, they're super simple and you don't really need a lot of the fancy stuff that LangGraph or even our stuff provides. It's just deterministic, like invoice processing is step 1, step 2, step 3. You execute it in sequence and then you get it out at the end. Some more complicated, let's say you want to build some agent system that can execute step 1, search some tool and reflect and validate it's a big thing a little bit, kind of go back to step 1, redo a little bit of research. Go to step 2. That's where I think some of the event-driven orchestration approach we've built could be helpful. We basically version checkpoint every state. So if it fails, we can always restart from a given step. It's async. So we have a little bit of efficiency because steps will pass events and listen to events when they need to. And you don't need to figure out how to orchestrate between different steps as an overall system. And, you know, it gives users a relatively flexible way to define these agent automations. No matter how simple or complex, in a lightweight manner. So that's our workflow layer. It's fully open source. Some of our customers are using it to build various applications. And so, like I said, like, I think the open source layer has more tooling to enable users to build applications. But at the same time, you know, there's other frameworks too. You know, we're also not building a workflow engine. I think for us, it's mostly in service of helping us solve our end client use cases around docs.

Yeah, I'll add it to the show notes, but some of the blogs that you and your team have put out have been very insightful. I still talk to friends in finance a lot and they're quite frustrated with trying to build systems around financial data because years ago we used to joke that the number one platform in finance was Excel. I think it still is. Financial analysts, people in financial engineering, data analysts, they often ask me to explain why these LLM models and agents don't work well for their use cases. I said, I don't even look at it because The issues you have around spreadsheet data, one is tokenization. LLMs don't see a numerical sequence. They see just a sequence of tokens. As we talked about before, you lose that relationship. Sometimes I say the LLM math is kind of an approximation. It's not computed or calculated, which people seem surprised at. But the main thing is we've been focusing a lot around context engineering this year and trying to add that context to these numerical spreadsheets. One of the issues someone showed me actually last year, the LLM didn't understand that the negative number in parentheses means the same as a minus sign, right? I think some of the other models have solved that problem. But my question there is, you and others have described this as dark data. Spreadsheets have been very hostile to RAG approaches. How does your approach work with spreadsheets and numerical data, which is a problem?

Yeah, I think the space is quite interesting. I agree that you shouldn't do RAG over Excel. I think the space has advanced quite a bit since the last year. These days, I actually think these models are doing pretty well over Excel. The Claude in Excel release like a week ago has been quite popular.

What's the name of that?

Claude in Excel. They basically released a Claude plugin in Microsoft Excel that you can install. I mean, I'm just like vouching for another product. Like, you know, it's pretty good. There's other startups in the space too that I've seen, respect, and admire. Basically, it's like chatting over your Excel file. To give you a sense of the architecture of how it works, it's basically a coding agent. Like a code interpreter over your Excel file. You just have an agent write code to interpret various cells in an Excel file to help you reason over it, edit the spreadsheet, rearrange the tables, generate new ones, do analysis, blah, blah, blah. I think for us, we see ourselves more as the data layer to back parse Excel sheets into presentable formats instead of a human-assisted version where you need to chat with an Excel sheet and the chatbot version of Excel. We see ourselves as providing core components to help with Excel automation. We have the APIs to let you batch parse a ton of Excel sheets into very structured 2D tables, particularly at that scale. And you could use this with a lot of these deterministic workflows that are super high accuracy and doesn't require a ton of human in the loop. And so I think it's a bit of a similar but different class of use cases that we want to solve with Excel, whether you're trying to use these tables for deterministic automations or for CirC. We want to give developers the flexibility to build these applications. Of course, you're seeing this rise of horizontal apps like Claude and Mixo or some of these other YC startups that are building this horizontal agent over itself. And I think they're going to complement each other, similar to how like Chatbots like ChatGPT complement a lot of the agent automations people are building.

Do you see OpenAI and Tropic and Google, as you just mentioned, they're adding that native document handling. Do you see them creeping into your territory a little bit? And what's your plan there? Kind of be the neutral Switzerland of this space?

Yeah, I mean, all these models already have benchmarks over like OmniDoc Venture and whatever. And yeah, they natively have those language capabilities. I don't know. I just feel like our entire company is focused on this.

Yeah.

And they're already doing so many things at once. They're, you know, ChatGPT has like consumer products, enterprise, like they have to, they also have this problem, like their model needs to be not overfit to visual understanding. It needs to be general across coding, math, reasoning tasks. Writing, a bunch of things. For us, there's a ton of alpha in building specialized things that optimize cost, get similar levels of accuracy at like 1/10th the price, or just like push the frontiers on applications. So I'm not too worried about that.

Yeah, I noticed in 2025, a lot of AI products came out. I won't mention them, but it's kind of my job to be trying these. Will they work? How well do they work? And all that stuff. Not to rag on them, but you know, some of them I feel like they kind of got abandoned. To a case in point, speaking of things that were popular in 2025, MCP is obviously very important. And one of the big criticisms of that is security and enterprise readiness. Is MCP something that you guys are doing? And also with your agent framework, how are you dealing with security and making sure it's enterprise ready, which means security and compliance and governance and all that kind of stuff like that? I guess that's a big question, but a very important one.

Yeah, for sure. I mean, I totally agree. It's a big problem. There's actually companies in the space that are focused on MCP and Auth. I think those are probably good people to talk to if you're interested in tracking some products that help you centralize the governance layer around employees. We don't, have native products in the space because, you know, our stuff is like an OCR engine. You can plug it in wherever, but I agree, like, especially when you got to the enterprise, being able to maintain who has access to what and having like some central service that can make sure that you, you know, have the right level of permissions to access the right layer of service and almost have some layer that can map some ID to all the different services that you use. It's going to be super important. And I think there's companies in the space doing it.

Yeah, it's very interesting. And then did you see over the weekend, I tried it myself, but I think it was built on Claude's coworker, Claude Code, but it was mapped blowing up her Twitter.

It's called OpenClaw now, by the way. They changed their name.

Okay, so it went from Claude to MoBot to OpenClaw. And for people who haven't, not so active on social media, it's basically an open source agent you can download from GitHub. It runs your machine, controls your apps and accounts instead of living purely in the cloud and tries to integrate into real life channels. But the big story was Cloudflare stock jumped because of that and the security concerns and people trying to build a VPN around that. But do you see like in 2026, it's not surprising me for the enthusiasm around this because everyone wants to build these workflows, whether it's from your desktop to your, uh, GenAI browser to long-running workflows. And it seems like a messy space because think about how most of us work. We work on our desktop, our workstations, then we do a lot of stuff browser, and there's backend tasks running. When you talk to business about workflows, they're thinking like, how can I have an AI employee? Or as I like to say is a better term, AI is a teammate. If it can't open this application, go into another application, move data across, run some process, log into another system. So yeah, I think that was an interesting development.

I think computer uses text, so you can code it. It's ClaudeBot, Claude Coworker, and Claude Code are identical instances. Being able to use your computer. Will Open Claude, that's the name these days, be the main interface for computers? I haven't actually had a chance to play with it.

And I've been using Coworker a little bit and it works and doesn't work. Just to put this in context. For what we've been talking about. Do you guys have a waitlist for this, for what you're building out?

No, anyone can sign up. You got 10,000 in credits and—

Yeah, 10,000 credits. Yeah. And what do 1 credit get you?

As I'm sure you know right now, help process your docs. There's a certain credit price, according to— Yeah. Per doc. And the cheapest mode is 3 credits per page, or default mode is 10 credits per page. And then the most complex mode is 45 credits per page. Yeah. At different layers of cost.

So, yeah, that credits will get you quite far. That's very cool. Anyone can sign up. Yeah, I'll include a link for that. Maybe a two-part question, but some practical advice and vision. So I think there's two segments of our audience are going to be very interested in this. The data scientists and the engineers building these document workflows. And then on the business side, because I know we have some business side listeners, they say, okay, I just listened to the ODS AI podcast. Document AI is a thing now. Let's think about document AI and how can this solve some of our problems? So. I guess first to the AI engineers and data scientists out there, what should they be thinking about? Some practical advice for building these document AI workflows.

Yeah, for sure. I would probably first scope out what you're trying to build. Is it actually something for business? I guess, yeah. Is it something that's deterministic or do you actually just want to build a chatbot? In terms of building a chatbot these days, everybody is just doing like Claude code for everything. For all, all of it, whether it's cursed or Claude code, some coding, Devin. If you're trying to build a general agent, just use one of these Open-code, log-code testing case or something. Build some general prototype, keep it simple, and then see if you can, you know, equip the agent with harness, like the MCPN skills to actually make this thing pretty good. If it relates to documents, you need some OCR tool, which we do provide, and there's also like open-source other vendors out there. If you're trying to build a deterministic workflow, that has a lot of value. I would try to pick the lowest-hanging fruit and the highest-value business processes to try to automate. First, every company cares about process automation these days with AI agents and coding agents. Process automation is still super important. Carefully think about the target accuracy level you're trying to achieve and how do you have like human input at the end to review these outputs.

Very good advice. I'm not sure if you can share some KPIs, but what kind of numbers are you guys processing at this page? Can you see that data? Obviously you can't tell people using the open source stuff, but I see a lot of downloads. Are you seeing a lot of pages being processed? How widespread is the usage at this point?

Yeah, for sure. We have over half a billion pages processed, 30 million monthly downloads on our client SDKs for Long Climate Month. We're growing quite rapidly in terms of overall usage. Some of these metrics that we have in terms of our customers, as we mentioned, are able to raise the accuracy quite a bit from whatever solution you're using. If it's like 50%, we oftentimes bring it to 80 to 90%, which directly translates into straight-through rates. For a lot of these use cases, it helps generate new revenue opportunities because you're just able to do the repetitive apart faster. Well, yeah, there's a lot of demand for better document understanding.

That's incredible. 30 million monthly downloads and a half a billion page processed already. I guess because you're seeing so much, what mistake patterns are you seeing emerging that you could tell people to look out for? Is it better documents? Is it metadata? What can help in this process?

I think oftentimes it's actually kind of reasonable. I can share a few things. Oftentimes there's kind of some good feedback for our service because there's an infinite long tail of edge cases where, you know, maybe we have a bit of an error in the parsed output. They give us feedback to help improve the parsed representation of the doc. In terms of the application that they're trying to build on top of the data, it's probably something related to the extraction schema or the complexity of the end-to-end workflow. I think in terms of the extraction schema, if they're trying to extract structured outputs from these sets of docs, oftentimes it's like, you know, complicated, it's underspecified. They didn't actually very carefully encode what is the process they want to automate and then, you know, be able to translate that into some like config that actually reliably works well. And I think that highlights the general trend of being able to build effective agent patterns in prod, which kind of relates to that MIT report last year that everyone was talking about, which is— I mean, part of the reason AI projects fail is you, you do need to have a really good translational layer from a specific business logic that you are trying to encode into the agent workflow and have realistic expectations of accuracy. I think when way too complicated and your expectations of accuracy are 100%, it is oftentimes impossible to deliver ROI. You're just setting yourselves up for failure. Being able to scope out the actual process and workflow you're trying to build in the first place is probably one of the most important pieces so you figure out a net right. Otherwise, it's not even a technical implementation thing. It's a project scope thing. You're just setting yourself up for failure.

Yeah, that very much rings true from a lot of conversations we've had with our executive roundtables at ODAC. That's a point they bring up again and again. People are led to believe that this is very simple, build these workflows, drag and drop, but it's not. That's quite sage advice, Jerry Liu. And then just to wrap up with maybe a fun question, which I know is a very hard one and who the heck knows, but it'll be interesting to see because the mission that you guys have automating that knowledge work over document workflows, I see great progress in that, but it's hard for I need to see a timeline. So if you were to say between 2 and 5 years, if I'm working with documents and I'm working in a bank, a healthcare system, what does my interaction with this document workflow look like? And what work is going to disappear entirely? Where do you think this is going in a couple of years?

Oh, I think a lot of it. I mean, honestly, if there weren't like regulations, there aren't, you know, like change management, that type of stuff. Yeah. You probably start automating 30, 50% of the entire human side. If you think about it from software engineering. Yeah. Probably 80% of our code is generated by some AI assistant. Claude is the top contributor on GitHub repo. Basically everyone has the expectation of being like 2 to 3x more productive this year than they were last year. If people's hesitations fade and they actually have the resources, you could start doing that today. Yeah. The OCR stuff, you have the tools, you have the skills. But price change, that's hard to predict. I think that people are just going to be understanding it has to actually adopt.

Yeah, I think it requires rewiring because if you study history, when electricity came about, there were factories well before electricity. Electricity took decades to optimize because the nature of work in factories had the factory floors have to be reconfigured. I think you're right. Like so much of workflow now is built around the human at the center of it and all those compliance and for good reasons. So it'll be interesting to see when we get to 90%, but I do think you're right. Culture and organization structure is going to be the bottleneck here and not necessarily the code given the pace we're going at. So Jerry Liu, last question, how can people follow your work? Because as I mentioned, I'm going to I'll give a link in the show notes to the blog post, but how can people follow your work and what you guys are doing? I will definitely include a link to the credits as well for the platform.

We're pretty active on social, so you can follow my Twitter @JerryLiu0 or the Llama Index Twitter @LlamaIndex, or our LinkedIn page. If you want to sign up to Llama Cloud, it's cloud.llamaindex.ai. And then, you know, we have 10,000 free credits. If you're trying to scale up your workloads, explore volume discounts and figure out how to actually run this at scale in prod, you know, you're always Thank you so much, Jerry Liu.

This was fantastic. I really enjoyed getting deep in the weeds of document AI. I feel like the space has progressed a lot more than I thought it did. I think what you guys are building out, well, it's already very useful to thousands of companies with half a billion pages processed already. So congratulations on your success. Very interesting development moving from RAG to this, but I think you guys are on a great trajectory. And 2026, a lot of hype around AI, and what you guys are doing is helping dispel that hype and bring real results to companies. So I think that's important for the ecosystem and industry.

Well, thanks so much for having me.

Yeah. Thank you, Jerry. We'll talk to you again.

This episode is sponsored by ODSC AI East, the premier AI builders conference coming up this April 28th to 30th, 2026 in Boston and virtually. Learn more at odsc.ai/east and don't forget to use the code PODCAST for an exclusive 10% discount. Supporting our podcast by Subscribing not only helps our small dedicated team at ODSC AI, but also unlocks exclusive benefits for you. Your engagement directly contributes to the growth and success of our community. Please subscribe now and don't miss out on the next enlightening episode.