Local Multimodal RAG on the NVIDIA DGX Spark | Part 4 - Publishing our demo to Hugging Face

Testing, ladies and gentlemen. It looks like we might be live. 1, 2, 3, put your hands up in the air. As always, if you're joining the chat, say hello, say where you're from. And let's get started. Today's stream goal is to do part 3 here. I only have a couple of hours because I had some things on this morning, but I gotta cook dinner for the family later, so probably go to about 5 PM, or if we can get our demo built before that. So this is where we're up to. It's actually Thursday, so Thursday is today deploy our multimodal RAG system as a demo. So we want to get our demo onto Hugging Face. We've got it working locally. Aditya, yo, from India. Hello from Australia. So we've got a fair few files here. But I've got make multimodal RAG demo, I just want to show you how this works. So if we run this cell, let me zoom out, I want to turn this into a file that we can just run. So text query, say for example, I wanted to cook steak with potatoes tonight. We can either generate a summary or re-rank the results. But if we do this, notice how quick that is. So the query and embed match time is 0.0— about 3 milliseconds across 10,000 recipes. So the first, steak and mustard cream potatoes, steak bake, and steak supper. But then if we want to re-rank, what do we get back? So re-ranking takes a fair bit of time. Now I believe re-ranking is because— oh wow, the original rank. Now see down here, I don't know if you can see that. Maybe we have to push this to Chrome so we can see it. But the original rank was 8, and now it's moved up to rank 0. So 0 obviously is the best. Steak with caramelized onions, steak supper. So what's this say? I can barely read that. Let me download this. Steak with potatoes, does this work? Steaks Butter, shallots, red wine, demi-glace, parsley, black pepper. I don't think that's as good as the original. So that's the thing with re-ranking, right? You're not necessarily guaranteed the best result. Neither— well, that's true with embedding, right? But then we get our text back. And we've got a bunch of examples here. So we can also do image matching. And then we get a recipe summary. Tomatoes roasted with pesto, roasted tomato salad, scalloped tomatoes barefoot contessa. So that's pretty good. So we can do natural image to recipe. And we can do text to recipe. And then we can generate a summary. So let's get this running on Hugging Face. Multi with NemoTron Embed VL and ReRank VL. So we'll put a input, an image or text about food, and we'll go here. Dataset used. Let's update this with our dataset, and then we want to go embedding model. So the embedding model that we're using is NVIDIA Embed VL. So the important, or the crucial part, is that it embeds vision and language into the same space. Re-rank model used, and we'll put the re-rank model there. I did have to do some pull requests to the NVIDIA Hugging Face page so that, that way we can, um, generation model used, we can load the models without Flash Attention 2 because I think, I don't know if Hugging Face Spaces has Flash Attention 2. And I've used a small model, Quen VL, uh, Quen 3 VL 2B Instruct, as the generation model because I wanted to keep it small and lightweight. We could use a larger model such as— where did Where'd NemoTron go? I have so many open tabs. NemoTron V3. I'm gonna just put this in here. Beautiful. ET Videos, what's up, my friend? Thoughts on the 4K 32-inch monitors versus ultrawides for software devs? Um, personally I just like one monitor. I've, I had an ultrawide for a bit, but I, I, I don't know, I haven't tried an ultrawide in a while. I think my personal preference is a non-ultrawide monitor. Maybe I need to revisit them. I'm not gonna lie, I just like— oh, almost 11 PM, wow. I just like the really high quality of Apple screens. I'm biased towards that. I like the Retina a lot, and I sort of justify it to myself because I basically look at a screen all day. And so in my eyes, I'm like, I want to maintain eye health, so I want to keep it, keep it pretty good. Chris JM, how are you? Hello from San Diego, California, US. Hello from Brisbane, Australia. I've been to San Diego, it's a really nice place. Okay, so let's figure out how to push our, um, our demo to Hugging Face. You love my PyTorch class? Oh, thank you very much. I appreciate that. I'm playing around with a lot more different models now, so we're gonna, we're gonna really upgrade the channel with some 2026 style AI videos, and one of them is multimodal RAG. So let's get this to Hugging Face. I'm going to interrupt that. What do we need? I'm going to write this down. So add in text retrieve results and display them. We've done that. We've implemented re-ranking. There's a little tidbit on re-ranking. We might in the future, we might have to optimize that. But for now it works. Timing optional. We've added in generation. Yes. Update the description so people know what models we're using. Yes. And update the description so people know why. Okay, so for the demo, we're going to need— on Hugging Face, we'll need the following: a single app.py file with all of our helper functions as well as dataset loading and our embeddings.safe tensors. Because now I want to put a note here, because we only have— this is a fairly toy problem, but we only have 10K. Even though 10K is quite a large dataset, it's embeddings, this is only about 40 megabytes. So if you had— let's say you had a million embeddings, what's that? 100 times 10,000 equals 40 megabytes times 100. Now what's that? Equals 400 megabytes, 4 gigabytes. So 4 gigabytes, because the NVIDIA NeMoTron models Output a size, a vector of 248 in float16. And what else do we need? We need a README and requirements.txt. So let's create— wonder if I can just pass it to Claude and go create me an app.py file. Let's just create the app.py really manually, and we'll just get the app.py to run locally first. So as you can see, our notebook's quite big, but that's because I've made it verbose. But you'll see this, this will all fit. So Multimodal RAG, I'm going to create a folder called Demo. And in that we will have— hello from Sydney. Hello. That is amazing. Costa Rica as well. I'm a bit biased, but I'm always happy to see someone from Australia. App.py. Who wants to see how well Claude can one-shot turning this into an app.py? This is where I think AI is really helpful, right? Turn this into a single app.py file I can upload to my Hugging Face space. Notice all the excess display code and remove it. Focus only on what's required to get the Gradio demo running. All right, we're just going to let that roll. But in the meantime, we'll see what we can do on our side. So app.py, I'm just going to start literally just copying in stuff here. We'll put our imports at the top. Yeah, that's what I was saying about AI, right? Like, we've already done it once, so we kind of know what we need to do. Now we just get it to clean up. I don't actually need all the image paths because I've got the dataset. I don't need the recipes as markdown because they are in the dataset as well. So now we're going to load the models. An important thing to note here is that I've— for the NVIDIA models, I don't have flash attention on my DJX Spark at the moment. Flash Attention 2. I'm probably going to install it over the weekend because I think it takes a few hours. Um, I really should have done that before this, but so while we're doing the embed model and the re-rank model commit hash, so if we go to— ah, there we go, nice! Uh, if that runs, I'll be super stoked. So if we go to NemoTron Embed VL. So NVIDIA Oliver Hallworthy has added this PR to the model which enables us to use SDPA, so scaled dot product attention. Now ideally we would be using flash attention too, but I don't have it as I said. So this PR enables us to use SDPA. So, or eager, but we're probably going to use SDPA because that's generally quicker. And then so I just replicated this PR on the re-rank model. So this is my code here. But if we have a look, basically the same thing. So before the intention implementation, so we're kind of waiting for NVIDIA, someone from NVIDIA to accept this PR. I've tested on my end that it works, but before the attention implementation was hardcoded to flash attention to— now all we've done is we've just added options here for SDPA and eager. So when we go to the commit on Hugging Face, we get this little commit history here. This one, this commit number, that's the short version of it. And so I want to use on my demo this file because I've found— now don't quote me on this if you know better than me— that FlashAttention 2 is harder to load on Hugging Face Spaces. And if we want to deploy it, we need to be able to use SDPA at the sacrifice of speed, but at least then it will work, which is also pending. So I need to load these models from a specific commit. So usually we could just use that, but until these commits are merged into this, we have to put the commit hash. And then when we load the model, we just pass it to from_pretrained and we put the revision in there. So load the embedding model, load the re-rank model, and then we go down here. What do we need next? I'm just here working on my FastAPI skills to use them in my projects. It's almost bedtime here in Costa Rica. Well, thank you for tuning in. Uh, we've got a little bit of a lullaby happening, so we need to make sure we upload, load in the embeddings. How delicious is coconut water, by the way? We got coconut water. But I will never forget that one of my friends Every time I drink coconut water, I think of him because he said coconut water tastes like shoes, and I found that funny. So we'll set that to device. So we've stored our embeddings in SafeTensors. So the important thing is, uh, I keep saying that, don't I? Note, we are loading the image text plus text embeddings by default. This is because these— excuse me— because these were found to work best in practice. So if we go to the original blog post that we've been working on— where is it? I've linked it in one of these. Do we have it here? I think it was small but mighty. Hugging Face, uh, NemoTron VL Embed. There we go. Small yet mighty. So if we look at— these are the new models by NVIDIA, right? Llama, Nemotron, Re-Rank. So they are VL models. So the important thing is they can combine image and text embeddings into the same feature space. And so if we look at all the benchmarks, the image and text performs the best. So if you have images and text— so in our case, our dataset is what? If we go to here our dataset is images of recipes and text of recipes. And so the images are synthetically generated to PDFs because that's— if you're a big business, you probably have millions of PDFs laying around with figures and graphs and tables and whatnot. So we created our own toy dataset to replicate something like that. But of course, we have the ground truth text as well. So what we've done is, in practice, we could just create image-only embeddings. Right. Or we could just create text-only embeddings. But because we have the option to embed them both in the same feature space, which is generally going to see a boost, that's, that's a repeated pattern in terms of multimodality across many different problem sets is generally the more modalities you can stack, the better representation you can get. So that's like SigLIP and clip and whatnot, but these are specifically for documents. And then if we add in re-ranking by benchmarks, we generally get better results as well. So by default we're using these image and text embeddings, but we could also use our text-only or image-only embeddings. So if I go back to here, I've got image-text embeddings 10K, uh, I've got image-only embeddings 10K, and I've got text-only embeddings $10K, but by default we'll use the $10K version. So where's my number 4? Let's update our— how's Claude churning away over here? Nice. This is incredible. Well done, Cloud. app.py. See, few— what, 150 lines of code? Cool. Where's the path to embeddings? Let's just see if this runs. I might run out of space here though. Yeah. Just gonna add a note to our markdown. So let's free up some memory. There we go, beautiful. And let's see if Claude enabled our app to work. Loading dataset. Wonderful. So there's our embedding size. We have 10,000 examples and each of them is a 2048 vector. That's what's representing our images and text of recipes. So this is multimodal RAG. Now we're loading in the, uh, embedding model. I might put the name in here. Loading the generation model. So again, I've specified that we want to load from a specific commit. Oh, our app is loaded. Beautiful. Aha, Claude didn't update our preprocessors. That's all right, we'll just see what errors we get. Nice! Okay, who's got a favorite food they want to check? So for example, let's try steak and potatoes. What do we get back? Steak and mustard cream potatoes, Old Bay steak fries, steak supper. See, that was nice and quick. How do I get out of this? Oh no. Oh, escape. And then we get the text back as well. So generation turned off, no summary, summary created. And then we can search for eggplant dip, and we should get a summary back and a timeline. How many parameter in that model? So we have 3 different models running at the moment. We have a 1B, or it's actually 1.7 billion parameter model in the embedding model, and then another 1B model which is also, I believe, 1.7B in the re-rank model. And then Quen, Quen 3 VL 2B, so 2B is running the summary, but we could of course use a larger model for the summary. So we might have an error here. What's it saying there? Oh, there we go. So yeah, see, the re-rank took quite a while there, but the generation also took quite a while. So that would be something that we'd want to speed up in the future. But this is not using Flash Attention 2, by the way. There we go. So what did we search for? Eggplant dip. Sinful eggplant aubergine. Awesome bacon tomato dip. I don't know if the, the re-rank— let's turn re-rank off for that. What do we get back? Greek layered dip with Peter Crisp. Again, we gotta beware that some samples might not have, or some queries might not have samples in the dataset because we're only using 10K, 10K recipes. 1 to 4B parameter, is that enough for running RAG? Well, it just depends. It depends on what you need done with RAG. I'm, I'm finding these results to be pretty good. Like, let's get a summary of— what do we want? Chicken tikka masala. All right, if we go chicken tikka Right, we want a recipe for that. We're going to generate a recipe summary. There we go. What do we get back? Restaurant-style chicken tikka masala. Chicken Tikka Masala Vegan Style. Chicken Tikka Masala by Jamie Oliver. I love Jamie Oliver, he's a great cook. So we get a recipe summary here. Restaurant-style chicken tikka masala, tandoori-inspired marinade featuring chicken thighs, liquid smoke, and a rich smoky sauce. It's ideal for a quick weeknight dinner. Best for vegetarian. Best for quick weeknight dinner with flavorful homestyle dish. That is cool. Now let's get this up to Hugging Face. Hey, I'm happy with that. We might put the extra note in there. So I want to update So if we look at where are these keyword args coming from, they are coming from here. Yeah, there we go. And then if we go to the re-rank, yeah, that's where we're getting that from. So let's put this in our demo. We can get rid of our app.py because Claude's done a good job of putting that. So there's the demo, and what else do we need in there? And then we should have a demo image. Let me just add this to the demo. And then we also need a README. I'm going to use Where's my latest demo? Title, and we're going to call this Multimodal RAG with NemoTron embed and VL— oh, and re-rank emoji. And we're going to go I document. Can we do green to green app file? And now let's write a little summary. I know it's not fun, but you got to get on Claude Code or OpenCode. I have used Claude.ai, but I haven't— do I need the Claude app? Is it really that much different than just a chat window? Uh, Chris JM, are we using a vector DB? No, we're not. So we could use a vector DB, but for now, because we only have 10,000 embeddings, we're just using a SafeTensors file, and then we just load that into memory. So this is only 40 megabytes, 10,000 embeddings at 2048 embedding size of float16. So, um, yeah, if we had upwards of like 100,000 or even a million embeddings, we would probably put that into a vector DB. Um, and in that case, I probably use Postgres or something like that— for Postgres or pgvector. So multimodal RAG with NemoTron Embed and ReRank VL models. So, um, search for recipes using input queries of text or images. Dataset is 10,000 recipes. From where it is at Kaggle, from foo.com, turned into images and markdown and embedded into the same feature space. Default is to retrieve most similar recipes via visual or text search, and then And then optional as well. We want Let's just put in our same. Jaden W says, trust me, it's super productive once you get used to it. Opus 4.5 plus Cordco is an amazing dev experience. Yeah, I'd be happy to use it now, but for the initial start, I wanted to do it all from scratch. So now we could definitely automate it, a lot of it, with Claude Code. So now we understand it, we've done it once, we could make it better with Claude Code. That's my sort of take on it. Let's put our YouTube link in there, hey. Sweet, I'm happy with that. So do we have what we need for our demo? Demo app.py. Let's see if we can get this on Hugging Face, hey. Multi-modal mag with rag with Nemotron. I think that's all I need to do. Howdy y'all, I recently got Cursor and that's been a lot of fun to play with. It's an incredible piece of tech. So, uh, Jaden also says, totally agree with setting up the project myself and not letting it write too much code, but it feels like having an extra set of hands because you can use speech-to-text. That's cool. So does Claude Code have speech-to-text built in. Let's see if we've got a space. Or do you just use— there we go, nice. So that's building, but we want to update our app.py to import Spaces, and then we want to find our retrieve function. And we want to use Spaces GPU and then generate. Where's our generate function? I'm not sure if I need What's us? I'm not sure if I need it with every function or just the main one I want to run. So turns out I only need it on the retrieve function. Sweet. And now let's put a GPU on our backend. We get a H200. Thank you. Zero GPU. No, no speech-to-text, but there are so many Whisper wrappers that work perfectly with it. Okay, sweet. I will give it a shot one day. I'm definitely not adverse to it. As you can see, I've got 3 tabs open of Claude. You're using the Mac Mini? I am using— I am coding on the Mac Mini, but I am— do I have my setup here? This is my setup. Oh, I am coding here but running code here. So I'm connected to the Spark, uh, there's a cable you can't see that goes from the Mac Mini to the DGX Spark, and then we are running via SSH. So I code everything on the Mac Mini, but then I execute it with the GPU on the DGX Spark. So let's have a look. How's our space going? It's building. What's everyone working on? Who's working on a cool project? While we wait for this. Yeah, Hugging Face Spaces has free. Um, now if you're— I'm Hugging Face Premium, which is $9 a month, and arguably it's one of the best deals. I really like it. I mean, you get so much value from Hugging Face, and I kind of want to support them as a company, so We're just waiting for our app to build. There is two— nearly 2.5 million models on Hugging Face now. GLM 4.7 Flash, I've heard some good things about that. Vibe Voice ASR can do 60-minute long-form audio. That's incredible. Is this new? Oh, 14 hours ago. Beautiful. 60-minute single-pass processing. Unlike conventional ASR models, that slice audio into short chunks, often losing global context. Vibe Voice ASR accept up to 60 minutes of continuous audio input. That's cool. Wow, so better than Gemini 2.5 Pro and Gemini. The model jointly performs ASR diarization and timestamping, producing a structured output that indicates who said what and when. That's incredible. I'm going to put this in my Machine Learning Monthly. Is the app working? Joining in a bit late. Hey, Drusi, how you going? James Kensington, what do you think about Claude Code? I think it's awesome. All the new models are awesome. I'm using it right here. Claude, Claude, Claude. So yeah, Jaden, the Spaces Hugging Face Spaces GPU. You've been abusing your 5090. That's awesome. I want to get a 5090. So you can upgrade your space to use a GPU accelerator. So this is not necessarily for training, more so for inference. So the GPU hardware available on there is— where's zero GPU? Oh, here we go. Free GPU access enables cost-effective GPU usage for Spaces. Multi-GPU support, so pro users get 7x more daily— oh no, what do we get wrong here? Embeddings loaded. Okay, all we need to do is Torch Vision. Zero GPU spaces are available to use for free to all users, but if you're a pro, you get 7x more daily usage quota. So GPU type, NVIDIA H200 slice. So they have a big cluster of NVIDIA H200 GPUs and they must give you a little piece of it, right? And you get 7 gigabytes per workload. So 70 gigabytes of VRAM for your space, that's pretty cool. Getting started with the zero GPU, and here's what you do. You import spaces and you decorate the function you want to run on the GPU with spaces.gpu. So pro account, you get 25 minutes of GPU usage per day. That's pretty good. Like on your demo spaces, right? So what's that over a month? 25 times 30. So about 600. Well, 750 minutes, that's pretty good. So that's what, about 12 hours of GPU usage for $9 a month? The trickiest part is when people talk over each other. Yeah, you're so right. So we're loading our model again. Building. Lovely. Kolipri, zero-shot image classification. What do we got here? Is a 3D vision language transformer designed to encode chest CT scans and reports. What other zero-shot image classification models are out there? MedSig, CLIP. The CLIP models are still booming. So have we got our space? Oh, we haven't got any errors, but having a 5090 is, is really good to have locally. All right, we're an hour in. We're gonna do some squats. I'm gonna do 10 squats. I do push-ups, but my shoulder's a bit sore from jiu-jitsu. There we go, we did 20 squats. So all of our stuff's loading, this is great. Uh-oh. Well done, Chris. 10 push-ups, 10 squats, that's the rules. Once every hour. So I'm going to update my counts, plus 20. So the push-ups, I'm pausing for me. Okay, so we just got rid of the duplicate. Trust remote code. Where is your video telling you how you started your machine learning journey? Not the pathway, the story. Story video link. Can anyone share this guy's story video link? Maybe you want to go Mr. D Burke. How I got started with machine learning. Here's a good blog post, right? There's a video with that. And this is another good one. So try out all those. Okay, we're rebuilding our app again. See, that's what you gotta— that's a lot of what machine learning is, is waiting for your GPUs to run. And waiting for your app to build and waiting for your training to go, do we have any new zero-shot object detections? Now, are these actually YOLOv26 or someone just uploaded them? Hey, you are— you're welcome, User Mac. Do you like the stories? Do you? I could make more day-in-the-life type videos as well on YouTube. People seem to like them. Lm debt. Was very good. I've tried that out before. How have I not liked that? Wow, this is cool. These are the type of models that just slip through the cracks. I have already start this one. How's our demo going? Another error. What? Oh. Flex Wiz, what's up? Akhil, what's up? I'm facing issues setting up the model in DGX Spark. The model inference is too slow. What issues? I've personally faced similar issues. Hi from Ukraine, bro. Teach us how to create AI agents. Torchvision, there we go. Hi from Ukraine. Hello from Brisbane. AI agents, yeah, we want to work on one of them soon. What would you like the agent to do? Now let's have a look at this paper. Is this WeChat? That's cool. The proposed data engine. We first use an objectness detector to detect all regions of interest along with masks produced by SAM. I want to try the model. That's cool. Runtime error again. Oh no, what happened? Liquid AI slash Apollo is my favorite thing lately. Feels like GPT-3.5, closer to 4 intelligence, running on my iPhone. What's Apollo? Is Apollo an app? What are you building? Senpo is asking. I'm building this. Multimodal RAG pipeline. I've seen agents and orchestrator work along and perform autonomous tasks like writing a mail or performing some action, but I never really understood how it works. Please look up Expo, for example. What is Expo? Okay, we keep running into some errors, but we're going to get this working, don't you worry about that. We first use an objectness detector to detect all regions of interest along with masks produced by SAM. Then an LLM is used to generate multi-granularity and instance-specific labels for each object. The LLM is fine-tuned by us to ensure high-quality labeling. And structural output. Expo. Scale your offensive security in hours. Is this what you're talking about? I've personally never seen this, and I'm not sure how their product would work, but you also said Agents and orchestrator work along perform autonomous tasks like writing a mail. So are you talking about using your computer? Oh, we need to update. Is it possible to create agents like Expo using PyTorch? Uh, yeah, but you probably want an LLM, and the LLM is creatable using PyTorch. But for that kind of agent, you probably want a big and powerful LLM. I actually don't know how their system would work, but yes, you can definitely create agents using PyTorch. But these days you'll probably use something more like Transformers, and you'll call the agent— the agent will be a model, and then you'll call the model, and then the model will perform some task. So your agent can literally be like you design an app and a workflow, but then you delegate choices to the model and that's what's called an agent. As in someone comes to your task, what you would usually do is say, for example, you would program every step. But now models are getting good enough to go, okay, I'm going to execute the step. LM Studio Mobile. Is there a mobile app? I need to learn more about Unslough, I think. How's my demo? Okay, we're getting the data. We've got the model. Hello there, Asim. Daniel, an off-topic question. I want to buy a new laptop for upscaling into ML/DL. I have a 5-year-old M1 A256, but the RMA is too small to do anything. Any suggestions on what to buy? Um, if you want a laptop, I love, um, the Mac M5. Actually just had Apple ML blog, M5 Mac. The M5 GPU just had a significant upgrade, so I check out this. If you've already got an M1, upgrading to an M5, you'll probably see a fairly significant upgrade. But yeah, as you can see, we got good upgrades here, like 4 times faster mostly. Uh, if you go, you could go on M4 Pro, or you could wait for the M5 Pro. I don't know when that will be though. There we go. Okay, so after a bunch of trial and error, is this gonna work? Let's try out, um, what is some favorite food? I want— maybe we go egg and lettuce sandwich. All right, let's see how fast this is. Oh no, oh no, what's the error? Oh no, what have we got wrong here? Okay, this might be a significant error. Hi, saw your videos coming up after looking for some workloads to run on my DGX Sparks. I'm going to check your videos out. Oh, thank Thank you very much. Enjoy, bro. Bro Hizzle. This is zero GPU device mismatch with zero GPU on Hugging Face. Faces— the GPU is only available inside the GPU function. Your models and embeddings are loaded at startup, but there's no GPU. Load embeddings to CPU. Okay, so we're trying to rebuild now. Let's go for another round of rebuilding. Let's see how this goes. I'll be right back. Okay, we're back online, so let's go egg and salad. Another error, of course. Okay, I think it's a versioning issue. Yesterday I got my new laptop. I installed Ollama 5.4 Mini. But it wasn't using the GPU. I checked CUDA was installed plus Python, but now Python can't get the GPU too. So, Asim, Macs don't have CUDA, they have MPS. Yeah, pip freeze will do the job more faster, but I just want the exact versions. Like, if we do pip freeze— yeah, too many. All right, we're rebuilding. All right, we're trying to load again. Let's see what we get. Is it working locally? Egg and salad. There we go, it works locally. And let's go. Incredibly fast locally. Now let's do re-rank and generate. Hey, it's working! There we go. Beautiful. Egg tomato sandwich, egg salad sandwich. Let's go pork burrito. Macho burritos, wet burritos, Thai style pork burritos. Lovely. Now let's turn on summary and re-rank. And we'll go beef burrito. Now you should be able to see this now. And let's try this. Let's try it again. Beef burrito. Try it locally. And what do we get? Oh, lovely! Creamed— creamy smothered ground beef burritos. Creamy burrito casserole. Beef burritos here. There we go, we've got it working. Multimodal RAG with NemoTrom Embed VL and ReRank VL. And so is it the same as a local Local's going to take a bit longer because we've got a, a DGX Spark, which is a little baby, versus an H200 on Hugging Face. So the embedding match is much faster than the re-rank, but we can also do image query. Let's do this. Search for this image generated with Gemini. And so we get tomato with, with tomatoes roasted with pesto, roasted tomato salad, and then we get a summary out here. And we get a very similar result for our local version. So I'm going to upload the local app as well to Hugging Face. So if we check here now, we should have app_local. Beautiful. Okay. So I think it's time for me to go for a walk. But that has been this week's streams. Now I would keep going tomorrow, except I'm going away. My friends are going to a wedding and where my wife and I are going to look after their kids. So I'm offline all tomorrow. But I'm going to turn this over the next couple of days into a YouTube video. So I've learned a lot building this system. We can now officially put this in green because we have built a multimodal RAG system completely from scratch, from dataset. We didn't build the models, we used them, but we built a dataset. We embedding the data, embedded the dataset. We saved that to a database, which is just an embeddings.safe tensors. Um, and I'll put a little star here for later on. If your dataset is larger than, say, I don't know, a few hundred thousand embeddings, which is definitely, uh, entirely plausible, you might want an actual database. But for our use case, um, we kept it small for a reason because we wanted to actually work with these models. And then we re-rank the top samples we return those re-rank results. And then optional, we generate answer based on those results. So optional re-ranking as well. So few extensions as well. I've got these noted down in here. Extensions, Code, and then Data Creation, and then App Extensions, um, integrate Lexical Based Search alongside embedding search. This would allow simple text matching as well as semantic matching. So lexical, lexical base search, e.g., search for egg sandwich, get back all results with actual text, egg sandwich, in there. Because embedding search is great, but you can still use, uh, lexical search for— you want a very, very specific dish type, you might want to get that exact dish type. But all the code is publicly available, the dataset is publicly about available. And over the next couple of days, I'll clean it all up and make a dedicated YouTube video, which basically puts everything together. Because right now, as you can see, we did code this live. So we've gone from zero, create a multimodal RAG dataset, image generation, we didn't actually use that much, generate multimodal RAG pipeline, that was version 1. Then we tested the re-rank model, and then we created into a demo. And then finally we uploaded it to Hugging Face and troubleshooted it a fair bit on Hugging Face while doing a pull request to Transformers as well as— where is it— NeMoTron? Nope, we did it to, uh, the ReRank model. But that's okay. Actually, let's see our PR. We'll finish with this. Rag. There we go. There's our PR. So this enables you to use it in Hugging Face Spaces. Because I don't believe Hugging Face Spaces has Flash attention too. But I may be wrong in the future, or I may be wrong right now. I don't know. But we've got our demo working. So ladies and gentlemen, thank you for tuning into the series of the stream. I'm going to make a playlist of multimodal RAG. But as always, keep learning, keep creating, keep dancing. It's time for me to go for a walk and chill out. But stay tuned to the channel for a video of— yeah, we'll discuss this a little more succinctly than what we have here. So I'll see you all next time, eh?