Will the New GEMINI PDF Feature Replace RAG?

This might be controversial, but if you are building rag systems for PDF files, you first want to check out Gemini 1.5 Flash, especially after the latest update. Google might have just killed the need of RAG for small PDF files. They had some major announcement around it, so now there is a substantial price drop. You can also fine tune Gemini 1.5 flash and there are significant updates to the Google Gemini API as well as Google AI Studio. There was a price reduction of more than 70% for Gemini Flash. It went from 35 cents to just 7 cents per million tokens of input if you're using less than 128,000 tokens. They also substantially reduced the pricing if you're cashing your tokens, which is pretty great because this enables you to save money and the price per output tokens has seen similar drop. But I think this is only limited to the Flash version because for Pro version it lists the POS pricing and as a developer you can continue using this for free. The second major announcement was around the ability to fine tune the Flash model with your own data, but the one that I'm personally excited about is this PDF vision and text understanding. Now you can send PDF files without any parsing directly to the Gemini API and do retrieval on top of it. And since it's a multimodal model, if you have graphs, images and other non textual information, Gemini will use their multimodal capabilities to process PDF files. You don't really need to do any type of pre processing in terms of parsing it. You just upload the PDF file directly through their API and it will work right away. Now if you have built any sort of RAG systems, this is one of the most critical and error prone tasks. There are several libraries like unstructured I.O. or Lama Parse llama that you can use for parsing PDF files, but in case of Gemini you can just do that directly on the backend. Next I'll show you how to do retrieval on PDF files both using the Google AI Studio as well as the API. But before that, just a comment around Rag. I still think there is a place for rag, especially if you have a large number of PDF files, because economics is not going to make sense if you have millions of PDF files. But if you have a small number of PDF files and you want to process them directly through the Gemini API, I think it's still a viable option to test this new feature this video is going to be divided into two sections. In the first section we're going to test out this new PDF understanding feature within the AI Studio. And in the second section, I'm going to show you how to use the same feature through their API. I will be also directly comparing this with GPT4O and its capabilities in terms of understanding the contents of a PDF file. In order to upload a PDF file, you can click on this plus button and then we're going to upload a PDF file or set of PDF files to Google Drive and that's going to be loaded in here. For this video, I'm going to be using the Call Poly Efficient Document Retrieval with Vision Language model paper that I previously covered in my video. This contain images, text and tables. And we're going to test the ability of this new updated Gemini Flash in its visual understanding of images and tables along with the text. Okay, so I just uploaded the PDF file. You can see that it's already using about 30,000 tokens. The paper itself is about 20 pages long, so that seems about right. My first prompt was what is the title of this paper? And is able to correctly extract the title. Now, this is a relatively easier task because the model can see that the title is at the top. But if you're doing text extraction using some of the packages, you can actually run into issues because during parsing they are not going to have any contextual understanding of where this text is coming from. Now, ChatGPT has no problem whatsoever either. But when I ask it what is the total number of figures and tables in the paper? ChatGPT or GPT4O to be more precise, came up with seven figures and five tables. But the Gemini flash says there are eight figures and five tables. And in fact there are eight figures which are captioned with figure and there are five tables. So Gemini Flash was able to correctly tell us how many tables and figures are there. Now, there are also some auxiliary images at the end, but they are not captioned as figures. So that's why I think the Gemini Flash didn't identify these. But we are going to look at these images later in the video. Next, I asked it extract captions of all the figures word for word in the paper and create a table with these captions. And here are the captions in the form of a table. Now these seems to be correct. Also, GPT4O doesn't really have a problem extracting that information. So this is pretty good. In normal rag, it really depends on how you chunk it. So for example, for this longer caption, if this was chunk right in the middle into two chunks, then you're not going to get this whole caption probably the RAG model will return a part of it. Another issue that you're going to run into with traditional RAG is that if there is any mention of figure in the text, not in the caption, then it could confuse that as well and count it towards a figure. Let's see if it actually can retrieve all the references from this paper. So if you see there are no numbers. So if I ask these models to tell me the number of references they actually have to go and not only extract them but count the actual number of references in there, let's see if these models will be able to do that. Alright, so when I said how many references in the are there in the paper? So GPT4O came up with 39 references. In Gemini Flash came up with 40 references. Now both of them are wrong because in reality there are about 50 references. I said are you sure about it? Can you count them again? So with the updated count, Gemini Flash comes up with 47 references where GPT4O came up with 71 references but not even close to what is there in reality. But then I said I think there are 50 references. So it says after you a thorough review of the paper I can confirm There are 50 references in total. And Gemini Flash also confirmed there are 50 references. Now both of them may be making up this number or just telling me because I told it that there are 50 references, let's see if they can actually retrieve those. So I said can you extract the information from each reference and create a table that has the these columns. So it's supposed to have authors title, journal confluence and dates and then add a column for the reference number. Now you can see out of the box that GPT4O is hallucinating because the first one that it came up with is this infographic vqa but the first reference is getting vit which is visual transformer in shape scaling loss for a compute optimal model design. The second reference in the list is Doc Visual QA but this is much later in the list. Now this is where I think the Gemini Flash model is pretty accurate. So it came up with the first paper accurately. The second was in one is flamingo paper and you can see that is at the second spot. The third one is the cloth three model family and is able to keep the order correctly. Actually pretty impressive. The last one is towards complex document understanding by discrete reasoning and you can see that it is able to keep the order correct which is pretty impressive. So 45 number is this and this is actually the correct paper in the correct order. GPT4 was hallucinating, but it only gave us 19 papers or 19 references. So I think for extraction tasks from PDF file, Gemini Flash is definitely the winner here. This video is sponsored by me. Now, although we are talking about long context models, RAG still has its usability, especially if you're dealing with millions of documents. Something like Gemini is not going to be helpful in those cases. If you want to understand RAG beyond Basics and learn the advanced techniques, check out my course titled RAG Beyond Basics. We start off by Basic Concepts where you learn about RAG and then we discuss more advanced topics. And the course also will give you a complete Python package that you can start using right away in your own applications. So if that interests you, check out the video description. Now back to the video. Now since the whole paper is in context of the nnm, you can ask pretty broad questions which a normal RAC pipeline will not be able to answer. So for example, I asked it what are the major contributions of the paper? In the paper they talks about two major contributions. One is this new data set and the second one is the Poly architecture. Gemini Flash just picks up both of these and list those rather than adding any further information. On the other hand, GPT4.O also does a pretty good job. It talks about this new benchmark data set, then the poly model. But then it will also add some further information in terms of the limitation of the current system performance improvement. Open release of project artifacts Now, I think even a normal traditional RAC system will be able to give you these two, but probably not the rest because it's not going to be in the context. One thing which I really wanted to test out was the multimodal capabilities. So we are going to ask both these models questions related to this specific figure which shows a comparison between the standard retrieval architecture and the called Pauli architecture. It also describes different processes that are happening in these two approaches. Can you explain what does Figure 2 shows? How does the proposed system compare with existing approaches? So this is able to figure out that there is a simplified retrieval architecture. It talks about the need for doing OCR and parsing on the PDF files. Then you pass that on to the LLM. Now, Kali Poly directly processes the PDF file as images, right? So this part is correct, but I think a lot of details are missing in here. Now in my opinion, Gemini Flash does a better job if you look at the image. So the standard retrieval is divided into offline and online. Similarly the call poly architecture is divided into offline and online part and it's able to provide information about both those parts and differentiate them based on those two different steps. Now, for performance comparison, it actually mentioned what metric is being used in the paper. GPT4O doesn't really give us that information. Okay, so I specifically asked, what are the NDGC at 5 scores for both approaches based on the figure. Now, GPT4O says that it doesn't directly provide these numbers in the figure itself, but you. If you look at the figure, these numbers are actually on the figure. On the other hand, surprisingly, GPT, the Gemini Flash has no problem at all, and the numbers that it retrieves are correct. I don't know quite exactly. They did it like, okay, here, done. But this seems to be really good at multimodal understanding. So here's another figure which shows hourly average hourly generation by fuel type and also kind of shows where the model is paying more attention. Okay, so I asked what is the title of this image? And I was hoping that the model will be able to pick this up, but gpt4o return the caption. On the other hand, Gemini Flash actually returned the title that is present in the image. This is pretty impressive, and I think I might be actually switching to Gemini Flash, especially for PDF files. Now, next, let's look at how both these models are at extracting information from tables. Now, there are two different types of tables that are present. One is this, which is pretty complex in terms of the visual information that it contains. And the other are more flat tables like this, where a normal parser should be able to extract information. Now, extracting information from tables is one of the most difficult tasks, especially for the models that are fed text directly without any structure. But you can use markdowns to represent tables. But in this case, I think it's going to be looking at these tables visually. So let's see if they can extract information directly from the tables. Okay, so I'm going to ask both of these models to reproduce Table 45 row by row in markdown format. Okay, so both of them actually did a pretty good job at it. The information in the table extracted is accurate and here is able to do the same thing pretty accurately. Now, table extraction is a very important task, especially there are companies around it which are trying to extract tables from financial documents, and there are whole businesses around it. And both of these model models are able to just visually look at the tables and parse valid JSON files. Let's see what happens if we ask it to parse this more complex table. So I'm going to ask it to parse the Table number four. So table number four is a bit tricky because there are missing values in here. And let's see what both these models do with these missing values. So GPT4 I think was looking at the wrong place. And the reason is since it's a relatively complex table, it has three different sections. So I think the GPT4 model started looking at this section and completely ignored the first part. You can see that it's just looking at the last two parts and completely ignored the first part. Even in here it ignored the call poly paper, I think it ignored this one. So there is a row missing. Now on the other hand, Gemini Flash has all the information in terms of the rows and columns. However, it did mess up the ordering. So if you look here, there are no values for archive Q for BM25 and BGE and M3. But this model is somehow saying that for BM25 the value is 26, which I think it's ignoring or confusing this value. For the second part of the table we where none of the values are missing. It does a pretty decent job. But for the first time part it does have some issues. Okay, so not great at extracting information from complex tables, but it does work for simpler tables. I'll highly recommend to interact and test this model. It's available for free at least in the US and I think they are also opening up to other regions. It's a pretty smart and highly effective model in my opinion. Now we can run other test, but let me show you how you can interact with the model and use the same features through the API, which I think is going to be the main appeal for developers. So if you go and click on this get code, it's going to give you the code that you can use right away and it also gives you the code for replicating everything that you have done so far. So we're going to copy this code and I'm going to just show you how you can start using this model. So I'm not going to copy all these previous conversations. So here is a Google Colab notebook that I put together based on that code. First you need to install the Google Generative AI package. Then we need to provide our Google API key that you can get from your Google AI studio. Next we'll need to write Python function that is going to upload file to Gemini. And the way you do it is you just call the Gemini AI client. Then there is a function called Upload file. You provide the path of the file and then the type of the file now, they also recommend this function which actually waits for files to become active. So it has to upload the file and then process it, does all the parsing and then will give you a signal that it's ready and you can start playing around with the file. Now you can say that you don't really need any type of Python parser or any other packages. You just directly load the file and send it to the Gemini API, which makes it extremely effective and very efficient. Next, we're going to set up some configurations so the temperature is set to 1. Currently, let me set it to 0. I think in all my experiments this was set to one, so you definitely want to set it to a lower value. If you're doing retrieval, we're not going to touch the other values and the max output tokens are going to be set to 8,000 tokens. Okay. Then we're going to create a model. We're going to be using the Gemini 1.5 flash. We provide our configurations. You can also set the safety settings. You can either do them here or directly in the code. I have a couple of previous videos which shows you how to do that. So I will refer you to those videos. Next, we need to provide the path of the files that we want to upload. It can be a single file or multiple files. In my case, I uploaded the same paper to Google Colab and here I'm providing the name of the paper. Then we're telling it that this is going to be a PDF file. So we wait for the PDF files to be processed. Once we get the signal that everything is ready, then you can start interacting with the model. So, for example, my query was how many figures are in the paper? And the response is the paper contains eight figures, which is correct. Then who are the authors? And it was able to correctly extract all the seven authors that are present in the paper. And then I asked it about the main contributions of the paper, according to the author. So it talks about those two main contributions that we saw previously. Now, I'm not sure in this case when you are interacting with the file through the API, whether it's actually sending the whole context every time, like the whole paper every time, or it's actually caching the context or the paper and then interacting with that. But if you directly want to control the context caching in Gemini, I have a previous video. I'm going to put a link to that. It's a very powerful feature that will not only help you reduce the number of tokens that you're using, but will also help you reduce the latency that you're going to encounter when you are sending a large amount of text. So do check that video out link is going to be in the description. You can also fine tune Gemini Flash. I'm going to create a separate video on that but I think if you're looking for very specialized applications and want a model that is good enough Gemini Flash with its long contacts and the ability to read PDF files directly and with its crazy cheap price is going to be a very good option for developers. In most of the cases you don't need this the smartest model especially if you're building something like chat with PDF as a SaaS they also substantially improved their API documentation so I'll highly recommend to check them out. I think along with OpenAI Claude Gemini is becoming a very good option for developers. Anyways I hope you found this video useful. Thanks for watching and as always see you in the next one.