How to create the ULTIMATE Ollama UI app with Streamlit

So you just downloaded Ollama to your system and you're ready to run your first model. You also pulled your first model and you're ready to run it. Maybe it's Mistral, for example, right? And you're wondering, okay, I'm going to run it on the terminal. I'm just going to do ollama run mistral and then I'm going to chat with it right there. It's pretty cool. Looks really nerdy doing it on the terminal for sure. But then maybe you're wondering, okay, what other options are there that I can use as a UI that is easy to use? Well, then you look at the options, maybe you find something like Ollama web UI, but then with that you have to set up Docker containers and maybe you're not familiar with Docker containers in this case, or you don't want to do it. You know, sometimes you just don't want to do it. So I'm here to show you an easy way to do this using Streamlit as the UI for your application. And you can chat with Ollama models directly using the web UI. And so let's get to it. Before we get started, I just want to preface by saying that for this tutorial, I'm just not going to really do like real life coding where you see me like putting code and then typing it all out and running it. That's probably not going to happen because I want to spend more time explaining to you what the code actually does and also show you the end product while explaining it so that way you're able to understand my thought process and choices that I made when building this UI so that way you can replicate it or when you're building yours as well the same way. And I'm going to share the code at the end as well. So when you're using this code, on your own and trying to modify it, you also understand why I made some specific choices and this. So I don't want to be able to— I don't want to be spending more time coding and trying to figure out, okay, which code comes up next and trying to put stuff on the screen because I've already done that. Right. So I don't need to show you that. All I need to show you is how it works, why I did it, and how you can do it. And with that said, I think that's enough. So let's just jump to the code itself. So I'll start with the directory structure on the side first so that way you know how it looks like. And so the directory structure, I have this Ollama Streamer demos, which is the main folder. And I think it's on my desktop if I'm not wrong. And yes, it is on my desktop and I have it there. All these folders are within it and files as well. So the first one here I have is .rough cache. So this is for rough. And if you're not familiar with rough, and I'm gonna say, mention some stuff and just for you to look it up, I'm not gonna go into too much detail, but I'm gonna mention it. So Ruff is a Python linter. And what I mean by linter is if you write your code and you wanna make it like in a specific structured format, you can use Ruff. And there's so many other ones that you can use. There's pylint as well. But I like Ruff because it's made using Rust and it runs really fast. And I might be able to demonstrate that at the end as well. So you can see, so you might have some errors in your code. It's gonna check through it and show you exactly where the error is and try to fix it. And just makes it easy for you instead of you kind of getting some breaking changes without knowing where it's coming from. The next one here is a virtual environment. I have some videos on virtual environment on my channel and if you can check those out. But simple way to explain it is it's like a container. So you have a little container where this project that I'm building right now, this Ollama app, everything lives within that container. So all the packages especially, that I'm gonna use to build this application, which those packages, talking about those packages here, these are the packages that I'm gonna need. Ollama, Streamlit version 1.33, and then OpenAI package. So when I do pip install on those, instead of installing them on my own personal system, the entire system, I wanna install those just within this project file so that way it doesn't pollute or conflict with other projects that I'm already building that are using maybe a different version. So that's a short explanation of that. And then there's this VS Code is auto-generated from the code. And then the assets I have here, this is where I kind of put all the GIFs, pictures that I'm going to use on my README. And the README is just explaining what this project is about. I'm still building that a little bit for you, but essentially that's where I put all the assets there. And then the next one is pages. And this is a Streamlit feature actually. So Streamlit allows you to have multiple pages in your application. So instead of having one page and having all these apps in one page, I can have several pages. And if you go on the sidebar, you're able to click and navigate to different pages and you have content in different pages, which is really cool. And I'm going to show you a little bit of this. So in this case, I have multimodal and I have settings pages. So these are 2 pages that I have. And then the main page is chat.py. And for the main page, you want to have that outside of the pages folder. And it should be in the root directory because this is the one that you'll be doing Streamlit run. And then you reference that app, the name of that file, and it's going to open the app as the homepage. But here the labeling, the naming as well. So 02 is basically going to put it in order on the sidebar. You'll see like multimodal and then 03 and then settings as the third option. Also, you can add emojis as well on it, which is really cool. Then it makes your app look really cool. All right. And that is that for the pages. And then we go to utilities. So utility file, if you're not familiar with it, this is where you can put your extra Python functions that maybe you don't want to repeat building the same thing in each different page, for example, because you'll have that and you just call it as a module in your application. So in this case, I'm calling icon. I built a page icon and basically it's a Notion style emoji page icon. So I like those. Notion, if you're in Notion, you can see those emoji icons. They're really cool. So I want to add that within my Streamlit app just to make it nice. And here's some code, a little bit, some CSS to be able to create that icon here as well. Cool. So we're done with utility folder. And now let's go to— so .gitignore, basically I'm just putting all the files that I don't want to commit to my GitHub repo. And if you put everything here and their path as well, it will just kind of ignore them when it's time to push your commit or your code to GitHub. So that way you don't want to have some stuff in there as well. So this is cool. And then the next one is the main application. And this is the code for the main app. This is the homepage app. And this is the code for it. And we'll go through this in a sec. And the other one is README. README is pretty clear. This is where you put all the explanation of what your application does and how people can get started with it. And then the final one I have is requirements.txt. This is where you list all the dependencies or modules that your app will require to be able to run. So pretty straightforward, right? That's my folder structure. It could be more modular, but that's how it's going, uh, for now. All right, cool. So let's go back to the main app. So the main application is the chat application, um, and I have 3 different, uh, pages in this application. I have the chat app, and then I have the multimodal, uh, portion, and then I have the settings where you can We can do stuff, really cool stuff in this page, and I'll show you once we get there. But the chat one is basically the way you work with Ollama right now. When you downloaded Ollama on your system, you're either running it on the terminal, you're just doing ollama run something like llama 2 or Mistral, and then you're just chatting with it. You're like, hey, why is the sky blue? Something like that, which is very popular. And then maybe some random questions you can ask it, and then you chat with it on the terminal, which is not bad. It looks really nerdy. It's cool. But this is just another way of doing it on a UI and building a UI that you might be able to customize and make it easier. So this is essentially what's happening here. I'm trying to recreate a good UI. And let's start from the top. So first here, I'm importing Ollama, which we'll need in this case. And I'm also importing Streamlit. We'll need Streamlit. And if you're not familiar with Streamlit already, Streamlit is a Python framework. It's an open source Python framework that you can use to build data apps. very easily and very fast and they look really amazing and you'll see them and you see this application here as we get started. And then you have to import also OpenAI because Ollama uses, has that OpenAI compatibility and we now use that to call the API for most of the models that we'll be using here today. And we import the utility file that we function that we clear that we created under the icons utility folder. And so that way we can display that Notion page look and icon thing on our application. So the first thing we want to do is use st setPage. So it's a Streamlit setPage function, and it configures kind of default settings of the page. And you can have page title, page icon, layout, and you can have layout as layout. You can think of it as your content might be centered, and then you can also do not centered, which is wide, and it's going to be more broader. And you'll see I put mine here as wide for all of them just for consistency. And also looks pretty good as well. And then you have initial state, sidebar state, and the sidebar, as I mentioned, that's where your app pages are going to be listed at. And I want that expanded when it starts on the main page. That way I can see all the other pages as well. And we'll be able to go through that as well here in a sec. And then the next thing here is extract model names. And this function specifically, what I'm doing with this is I'm extracting the model names from the model information and getting all the models that I have on my system that I've downloaded. That way I'm able to use it within the application. And for Ollama, the good thing we can do with this is Ollama just not so long ago, they released this Python way to be able to chat with Ollama, which is really cool. And this is what I'm taking advantage of and doing a lot of what I'm building with this application today using that feature they released. And so for this, think of it as I'm extracting all the names that I have on my system and specifically taking out just the names of the models that I have pulled on my system. And to the main function, what I'm doing here in the main function, I'm setting that page icon, which I mentioned. In this case, I'm just going to put the chat bubble here and it's going to appear on my page. And then the subheader, I'm going to put as Ollama Playground and I'm going to put a divider underneath it. It's going to be color red and anchor is going to be false. So the anchor is going to be the kind of like the chain thing that happens or shows up on the side of that title. So you can click and reference it. So I'm putting that as false because I don't want that. I just want a clean layout. And then the next thing we want to do is now set up the client and And the client, we're using the OpenAI compatibility. And in this case, you're setting up the base URL, which is localhost. And we're doing 11434. That's provided by Ollama. And you just put in whatever API key name you can put here. You don't need an API key. So you can put Ollama, you can put whatever name you can put. This doesn't matter. Just put something there. And then the next thing we want to do is call the model files and instantiate that. And what we're doing here is just listing all the models that are within our system that we've pulled. So we are doing ollama.list, and this will call all the models that are, or show you all the models that are within, that you've already pulled into your system. And then available models, I'm gonna do extract and then extract the model info. So I'm gonna pull and see all the model information that I have on my system. And this is basically this function. And after that, what I'm doing is if there's models in my system, what I wanna do is create a select box, which you can click. And then it opens and I'm gonna show you in a second. If you click that, it's gonna show you models. And here I'm asking you, pick a model that's available locally on your system so that way you can use it. And it's gonna show you from the options you have, click one, then you can use it. And then if you don't have a model on your system, I'm just giving you a warning here. Hey, you haven't pulled any model from Ollama yet. So I'll give you an option here to tell you, hey, Go to the settings to download a model. And I'm going to direct you to the settings page, which we have that page here. And we'll get to that later. We'll talk about the settings page where you can actually have set up in a way where you can actually download those models there, delete some, and also create the model file there, which is really cool. All right. So the next thing here I'm doing is I'm setting the message container, which is basically setting a container. It's basically, you can think of it as a container. And I'm going to show you here in a second. It's a container and I'm setting a height as 500 and I'm creating a border around it as well. And that's just a variable that I'm going to use down here as well. And then here I'm instantiating the session state. Session state is more of like temporary memory that I'm going to use within the application. So when I refresh it, it still maintains the memory of what we just chatted about just within that session. If I get out of that session, it kind of clears the session state there. All right. So I'm instantiating that. And then also I'm saying for message in session state messages, I'm setting the avatar for The model, I'm going to put this robot and it's going to show it as the, when you chat, you see the face usually. Sometimes it's like somebody's face, but in this case, I'm just going to use avatars. And so you have this, the avatar, you have this emoji icon for robot, that's for the assistant. And then it's going to be else somebody's smiley face with, or the cool emoji, that's going to be assigned to you as the user. And you can interchange this. You can put whatever you want on that side. And then Here I'm just saying, hey, message container, which I instantiated up here. So within that container, what I want to do is display the message and the avatar that's related to that message. So I'm saying, hey, this is, you know, this is a robot. Show the content that the robot responded with that we've saved already or that's already producing. So that's pretty cool. And then if prompt, I'm using this walrus sign in Python. I'm just saying, hey, if There's a prompt, and I'm using the chat input here. And the chat window is just a display chat widget, although we don't call it widget anymore. It's just more of like element that we can use here to enter a prompt. And so once you enter a prompt, it will try to put that in session state, and it will append that prompt that you just typed into session state. That way, when you type something, maybe say, why is the sky blue? It's gonna pop up straight up on that chat application, like on ChatGPT. It's gonna show up, hey, that's your question. It's gonna show your face and the chat that you put there. And then after that, it's just gonna assign your avatar and show your prompt there. And it's gonna put it in session state so that it remembers. So as you run the app, it still remains there. And so the next portion here is saying with that message container still that we have, we wanna assign the assistant here and we are doing a model spinning and the spinner is just gonna show, hey, the model is working. And in this case it's just saying, hey, The agent is generating a response, not the agent, but the model is generating a response. And then it's going to go to— I'm going to set up a streaming here and it's going to stream the completion. I'm using the clients to create and I'm going to use a model. And the selected model is the model that we've selected up there. When you clicked on the dropdown, you selected a model. That's the model we said we're passing in a model here for it. And this chat completion is just like the one for OpenAI actually. So it has the OpenAI compatibility and that's why you're able to do this. And so we're passing in the model that you selected up top. And then for the messages, when the messages come out, we're just saving that to session state. That's essentially what that is doing. And then we set stream as true. So basically it's gonna stream the content instead of waiting for the entire response to generate before it shows you, it's gonna show you as it generates as it goes. So very trying to make it really real time as possible. And then the stream response here is just gonna write Stream. This is Streamlit's generator, has a generator and iterable. And it's going to use a generator to just basically every content that's coming in, it's going to yield and it's going to show on the screen. So that is pretty cool. And then it's going to put it on session state as well at the end for that response that responded with. All right. And that's it really for the first page. And if there's an error, of course it's going to throw an error. It's going to show you the error. And there's an icon there that's going to show up before the error prints out. And then I'm just having this main main to run the application. Cool. So that's the first page and first page is done. And to the second page, which is multimodal. And this is pretty cool. So with Ollama, there's other 2 specific models, Baklava and Lava models. And these models are multimodal. So basically what you can do with them is you can pass an image to it and chat with that image. So maybe you can put a random image and tell it, hey, what is this in this image? Can you describe to me what this image does? So I played around with both a little bit. They tend to get things wrong a little bit as well. Sometimes they get it right, but maybe the resolution of my images or maybe the images are not blown up enough, so they're not too accurate. But they do still do a pretty good job, especially for simple images. They do a really good job as well. So let's get started with this multimodal page. And this is another page that was also very exciting for me to build. And we're also importing Streamlit, we're importing requests, and we're importing base64 and Pillow here for image and bytes.io. And those 3 will mostly handle our images that we will be uploading, try to render that pretty well on Streamlit as well. And then we import JSON, Ollama, and then of course the page icon that we've done before. And in this case for the set page config, kind of setting the default setting page of the page, I'm going to put as Lava Playground as the name that's going to appear at the top of the page itself. And the logo that's— or the image, the page icon that's going to appear is going to be like a mountain lava spewing out of the mountain, which is pretty cool volcano. And then the layout is wide as well. And then the initial state, I'm putting the sidebar as expanded so I can see all the other pages as well. So let's start with converting the image to base64 format. So you upload an image, the first step it does is convert that image to base64. And that's where it's able to be used downstream when we are chatting with this model. And so this is what it does with that conversion there. And it returns a base64 encoded image. And then The next function is we are getting allowed model names. And in this case, the allowed model names to be used for this multimodal portion is Baklava and LLaVA. And in this case, the reason why I restricted to only these 2 is because the other ones can't really handle multimodal, at least the ones that are on Ollama at the moment. I think it's Baklava and LLaVA. Correct me if I'm wrong, but if in future more get added, we'll just add it to the list and We all of them get handled pretty well. So that's the good thing about being modular a little bit. And so what it's doing here is just basically returning a tuple and for model, and you know it returning it's returning a model for models in allowed models. So allowed models is just returning each and every one of them. Emoji icon for this page to be the volcano erupting volcano, and the subheader we want it to be lava 1.6 playground and. Maybe I could have just put this as just, I don't know, multimodal playground instead of being specific with Llama because there's then, but then there's back Llama. I didn't even account for that. So anyway, you can change that. Titles don't matter as much, but they do at the same time. So the next step here is model info like we did on the chat page. And we're doing Ollama list to list all the models that we have on our system. And then the next page is we're doing available models and we're just kind of sifting through to see what models we have. And then The missing models is we take a set of these 2 minus the set of available models. So basically to get the difference, the missing models that we don't have. If we don't have baklava, it's going to say you don't have baklava. If we're missing lava or missing both, both of them just not going to show up. So that's a quick way to handle that. The next step here now we do is create 2 columns in Streamlit. It's pretty cool for the UI. You can split the page into 2 halfway. So you have this side and left side and right side, and you can put different content there the way you want it. And for me, my thought process for this portion specifically was for me to have 2 sides where when you upload the image, it's, of course, there's gonna be the chat box, right? Where you're chatting with that image. But I also wanna show that image to you on that same page. So when you upload the image, it shows you the image on the right side, and then it shows you the chat interface on the left side. That way you're able to look at the image, and if you have specific questions about it, You look at it and then you're able to ask it on the chat and be able to chat with it instead of just uploading and not being able to see the image that you uploaded, which is really hard to chat with. But in this case here, what I'm doing is on the first column, I'm using a popover, which is a new Streamlit element that got released. And basically it's inside within a container. And what I'm doing is within that and with popover, if you click on it, it throws a page at you like one of those login pages that throws a new login. But in this case, I'm using it for model management and I could have been more modular in this case and just have this entire functionality within the settings page, which I'm going to show you in a second. And actually, if you use the settings page, you don't really need to use this portion, but I just put it there just in case and it's not bad anyway. And so in this popover, when that popover opens, first of all, we're doing if there's no models available. So that means if you don't have Backlava and you don't have Lava installed, What you— what it's going to tell you is just going to show you an error. No more— no allowed models are available on your system. So basically it's just telling you, hey, you don't have any models, and it's just not going to send you away. It's just going to give you an option right there and then to either download or the model or delete maybe a model that you have that you don't want at the same time. So in this case, you have to select the model to download. You pick one, either Baklava or Lava. And then you download the model itself and it's going to show you all this fancy toasts, be like, hey yo, we're done, and rerun the app again so that way it registers the new model that you just downloaded. And if somehow nothing here works, it's going to throw you an exception and an error and it's going to say, hey, failed to download the model because such and such a reason. Maybe you don't have enough memory or just something broke really. And it's going to gracefully give you an error without just breaking and leaving you in the dark. And if there's missing models, it's going to ask you to download the models and you're going to download the model here. And the models, at the same time, we're handling the errors here very gracefully. If there's an error, it's going to stop. Here's the error, try to fix it kind of type of thing. So for this selected model here is for deletion. So if you want to delete a model, maybe you want to delete Baklava or you want to delete Uh, Lava in this case, uh, one of them or both really. Uh, you have a select box where you want to delete that. It shows you the available models that you, that you can delete, and it will give you a button and you click on it and it's going to delete that model and it's going to rerun the page again just to make sure that it's refreshed to reflect the changes within the application. Cool. And then it's going to return if not. All right, so And in this section here, we're selecting models, and then we have a box again where we're showing you all the available models that you have. And now we get to chat. So we also instantiate session state here as well to store our chats. And we have here the file as well, and we want to save that in session state as well. And then we upload the image itself here. In this case, it can be a PNG, can be a JPEG, or yeah, JPG, JPEG. You can have it in Those formats, those are the types that are allowed to be uploaded in this case. And then at the same time, we also have the columns, just 2 columns in this case as well. But you can interchange. So with this one, we're saying with column 2, we're creating the first container as well. The container height is 500, which is like the recommended really. If you're building a Streamlit app, you use the containers. Height 500 is really good because then it renders pretty well on the phone. Anything more than that might not render very well on the phone. So just a side note. Because there's going to be users, especially if you deploy an app. But in this case, it's local. You won't deploy nothing. But if you end up deploying a Streamlit app for this container, you want to set it to this because your mobile users will struggle if you set that higher than that. All right. So with container 1, which we just set up here as well, with that container, if the file is uploaded, we want to show that file within that. Like what I was mentioning earlier, you split the page into 2, you show the image and then the chat container. So this first one is showing the image. And you can see here st_image, and we're passing that uploaded image there. And then with container 2, this is where we're doing the chat, and we're setting again the container to 500, and we show the border as well. And then if uploaded file is not None, just making sure that we have the file already, and we start the chat. And we're saving here the avatar for our model is the volcano because lava, lava, cool. And then for us, it's a melting face as a user. Like it's amazing being able to get the model to be able to tell you what's in a picture, right? You can switch this one as well. I'm just being too cheeky here. And then for that container as well, what we're doing there, we're saving those in session state as well. And we're also printing out each content for each user here, the assistant and also the user as well. So we're printing the content as it goes for what we've done in history and also what we are doing going forward as well. And then here, if user, we're doing again the water sign, user input, we're just taking in the user input and basically ask a question about this image that you just uploaded, right? Because why not? And then we're appending that to session state like we did before. And then the image base URL, we're passing that image in this case here to the model itself. And then we're able to get the responses. And here we just kind of Passing that information that the model is running. We're just saying, hey, the model is running when it's running. And then the once if the response from the from the model is good, which is this status code 200. Status code 200 is good to go. We got it. We responded to it. If it's good to go, we got a response from the model. We parse through that JSON that we received and we are splitting it just to get a cleaner output. from the model to be able to display on our page. And then we go through it, we skip empty lines within the responses that we got from the model. And also we check if any invalid JSON, we just skip those lines to make it easy. And we then display that content from the LLM that we got from that, which is pretty cool. And if there's an error, it's just going to tell you, hey, there's an error, which is pretty straightforward. And then at the end, we append that message from the LLM to session state. That way we can have a continuous conversation. Super cool so far, right? So with that, let's go to the final piece before we actually run the application so you can see it in real time. Cool. So the final piece is the settings page, which I find really cool. And I really enjoyed working on this page, to be honest. And so it's nothing revolutionary, but it was pretty cool. So we also import Streamlit, we import Ollama, and then we import time. Here, and then page icon. Of course, we do the setup here. I'm using a gear icon on this one, expanded, and then the title, blah blah blah. And then now to the main page, we do that same exact same thing. We set the title, we set the the Notion page style icon there, and then we have the first subheader, and we have three subheaders. So the first subheader is download models. So this is the first settings within our settings page, right? So download models. So the first thing here is enter a name of a model to download. So, and I put a placeholder here for Mistral. It's just going to tell you, hey, this is maybe how you would name a model that you want to download. So you have a Mistral, you can put Llama 2, you can put whatever, Vicuña, whatever that's already on Ollama. And here we're just setting, hey, if you push the button and say, hey, you type the model that you want, you click on download the model, it's going to go ahead and download the model. It's going to pull the model. And here is Ollama pull. And we're going to pull the name of the model that you just put. Here in the text box, and we're going to pull that. And once that model is already pulled, because we can see what's happening in the background in terms of like pulling, and we can see because on the terminal you can usually see that, right? But in this app, I haven't set it up in a way that that is shown on the application. I'm sure there's a way, but I haven't gotten that far. But basically what will show you that it's done, I've set up here stream had st success has st success, and it's basically going to show like a Hey, it's done. It's gonna pop up and it's gonna disappear in a second. And then also, if you don't see that, if it happens too fast, I also set some balloons there. So st.balloons, and it's gonna show some balloons going up and it's gonna say, that's gonna be a signal like it's good, it's done. And then I put the app to sleep for 1 minute and then it'll rerun the app just to register the new download or the model that you just pulled. And if there's an error, it's gonna fail gracefully as we should. Or as it should in this case. And then if there's no model name, it's just going to tell you, hey, please enter a model to download. Cool. Now we're going to put a divider so that way we have a separation between that's the model downloading and then the next section of the settings, which is create the model. And this is using the model file. And we're also using here the Ollama Python SDK as well. We have the text area. We're going to create a text area where you're going to need to put in your from and system. So this from, you're just going to mention a model that you have on your system already. And this is for creating a model file. And in this case, I've used Mistral as an example. If you have Llama 2 or whatever model, you put the name of that model there. And the system message is basically you can tell it whatever you want. So you can tell it, hey, you are Mario from Super Mario Bros. Or hey, you're, I don't know, Warren Buffett. The famous investor, blah, blah, blah. And then the model is gonna act like that person that you just put there. It's gonna respond to you in the style of that person, which is amazing. And we'll see that shortly. And then the model name of this model that you just created now out of that model is you're gonna put it here. And I put a text box where you can put it, and I put a placeholder here for you as Mario. So if you just, so way you can remember, hey, this is the model I created. And if you go to the model options that you'll be given next, you will see that pop up there and you're able to use it. And so once you've inputted those options there, then you will submit it. And this is the button to do that. You'll create the model. It's going to go through the same thing again. If it's successful, it's going to show balloons. You're going to sleep for a minute and it's going to rerun. If it fails, it'll fail as gracefully as it should. And then again, we put another divider and we get to our final setting, which is to Deleting models. This is also important. Instead of going to Ollama terminal and deleting it from there, we can just delete it from the app. So there's no need to just move around. You can just leave within this application and do everything that you need to do there. Cool. So in this case, we're doing Ollama list again to find all the models that we have. And then the next thing is if available models, if there's models available on your system, what we wanna do is put them in a multi-select box. And that multi-select box means you can select more than one. model within, you can basically select all of them and put them and just do like a batch delete of all the models if you want. And that's very possible with this. Or you can select one and delete 3, whatever. Delete. So that's basically what it does. Once you select them, you hit the button delete and there it is. They poof, they're gone. And same thing, we are deleting them, doing the balloons, sleeping 1 second. And rerunning the app to make sure it registers that. And it's complete. All right. And now we just need to run the app and for it to render on our screen. And the way you do this for Streamlit is you do streamlit run and then the name of the main page of the application, which is 01. And you can just type 01 and then hit tab and it's going to autocomplete for you and you hit enter and voila, the app just popped up now on the screen. And let me just do some zoom in a little bit so you can maybe see this a little bit more clearly. But this is the application itself now. So like I've mentioned earlier, we have the pages here on the side. We have the chat and we have the multimodal. And then we have the settings page. And you can see the settings page here. And this is our settings to be able to fiddle around with our model management here. So let's start with the chat, right? Let's see what models we have. So if you come here, you pick a model locally on your system. So right now I have 2 models locally, which is Gemma and Llama 2. So let's maybe chat with Gemma a little bit, right? Let's just ask it some random questions and see what it says. So Gemma, why is the sky blue? Just something to be aware of as well. These models might take a minute to kind of kick off when you start it the first time, because then it's loading it to memory before it starts using it. And they do time out after a while, but there's a way you can set that so that way it doesn't time out as frequently as maybe as it should, especially if you're using it for a long extended period of time. But here it is. So you can see it's printing out stuff for us and it's doing it in real time. And this is pretty cool. And the other cool thing about this StreamIt container as well, as it goes, instead of like filling up the application, then you have to scroll on the side to be able to track, okay, what's the next information being printed? In this case, all of this is just going to be within a container. And so once you have this content passed down here, you'll see that it keeps scrolling up. So that way you can scroll up and down and see where the other content is, which is— I find it pretty neat. So this is going to take a minute to do that. So while that is printing, the other option we have is Llama 2. And you can just easily switch back and forth between them and use whichever one you want. And for this one as well, you can just ask it, why is the sky blue? And let's see what it does. I personally prefer Gemma if I need structured output. By structured output, I mean really nice visually looking explanations or breaking down of bullets and things. I find Gemma does that a lot and it does a really good job of organizing. The other one's just going to spew out stuff in paragraphs for you. And most of the time, really, reading paragraphs can be a pain. So I appreciate sometimes what Gemma does with making it easy to be able to just kind of follow through. All right. That's that for the chat portion. Nothing too fancy. You've seen that already. Now let's go to the LLaVA kind of family group here for multimodal models. And the first thing you see here, this is what I was talking about when I mentioned popover. So if you click on this, it pops over this page, right? And it tells you, hey, no allowed models are available. And like I mentioned, on my system right now, I don't have any of these 2 models, which is Baklava and LLaVA, but Guess what? I put this interface for you, and what you can do is you can pick either Backlava or Lava, and you can click on it, and you can download it to your system. So let's do one with Lava, right? Let's download and see. So it's running, and you can see that by seeing this running mount up top here. And this usually takes a moment because even if you're doing it on the terminal, it usually takes some, you know, it takes a little bit for it to run and finish. But when that's done and it's already downloaded on our system, It's going to do like the refresh page so that it registers the new model. And then it's going to give us the option here now to be able to upload a file and be able to chat with it, which is pretty sweet. And then it's going to split it into 2. Like I mentioned earlier, it's going to split the page into 2. So one side is going to show the image that we will upload and one side is going to show the chat interface. All right. And now it's done. And we have the latest model here of Lava. So pretty cool, right? So we did what we went and did. Now you can see it changed, right? When you come here to model management, it shows you, hey, download model. So it only gives us now the option to download back LLaVA here because we already have LLaVA. And you also have the option to delete LLaVA just because you have LLaVA on the system. So you can delete it here as well, which is pretty cool, right? But let's leave that alone and let's go ahead and chat with this, right? And test it out. So we'll see here. Let's pick up a simple file, right? An image. And we've loaded up a Python image right there. And so let's chat with it. Let's just ask it, what is this? And so like I mentioned earlier, you'll see it preprocessing. And this is our image to just kind of show like this is us chatting with whatever the model. And the model is the volcano, erupting volcano. And this will also take a moment here as well as it does. It kind of takes a minute to really process the image and then spew out the results like it did. So this is a— this is an image of Python programming language logo. The logo features the snake from the game Snake, which is part of the interpreter itself. The snake is biting its own tail, which symbolizes recursion and looping in programming languages. The image also contains a visual representation of Python code that defines the snake behavior, with lines, boxes representing the code statements, which is pretty cool. But as I mentioned, there's limitations to that. It's not like these models are not the best at figuring out maybe depending on resolution. So if your picture's not that great, it might not even recognize it. So that's cool. And you can see how to do that. And basically if you want, you can just do Baklava and test it out as well and see, or you can download both and just switch between them here, switch whichever model you want and run it. Maybe another thing you could do by improving this app, maybe you can create a side-by-side instead of the image. You just upload an image. And then maybe do a side-by-side of the model where you feed both Backlava and Lava, and then you get both of them to chat or to ask them one question and you see how both of them respond separately, which is very interesting to do that. But for these purposes, we're not doing that. And so let's get to the last piece, which is the settings really. And so the settings, this is where all the magic happens, especially with model management. And so we have the download models. You can just put whatever model you want to download there and then hit download and it's going to download it. And we're going to do a Test here as well, and then we have the create model, which I think is pretty cool, and I want to test this one out first. And then we also have delete model, so you can delete. So right here you can see that we now have lava as part of our models that we have option of deleting in our system. So we have Llama 2 and Gemma and then lava. Cool. So if you come up here, we don't have Mistral. So if you come up here, let's install Mistral, right? So let's write Mistral. Just to test this out, and now it says download Mistral. Let's say yes, download Mistral. It's going to take a minute as well, so I'm going to forward it to the point where it's done. All right, cool. You saw the balloons. So now we have Mistral under our models now, and we can confirm that by clicking here, and you'll see here Mistral latest. It's already in our system, so that's pretty cool. And so the next step here is create a model. And we are going to create it from model file. So in this case, let's do from, and maybe let's use Mistral because it's the latest one we've downloaded. And then for system, and this is what you need to put for system. I should spell system properly. So system, and then let's tell it you are maybe Warren Buffett, right? Warren Buffett, the famous. investor. Cool. So we want it to be Warren Buffett and we're going to tell it, hey, you're that famous investor. And that's our model file. And here we're just going to call it Buffett, right? Easy. We just call it Buffett. So create model Buffett named Buffett. So yeah, cool. And it already did. That was pretty quick. And so if you come here to deleted models, if you click on it, we have Buffett as an option for us to use. So, which is pretty cool, right? So let's go on over and use Buffett first before we come and do some deletions here. So let's go here, pick Buffett. We have Buffett and it's already pre-picked as the first one and just ask Buffett, hey. So let's wait for Buffett to load to memory and yeah, there you go. Whoa, Buffett. All right. I'm Warren Buffett. The investor, not an expert at atmospheric physics. However, I can provide you with a simple explanation of this phenomenon. The sky is blue due to a process called Rayleigh scattering. The sunlight enters the Earth. All right, all right, all right. Cool. Let's ask him, why are you called the Oracle of Omaha? All right. I'm often referred to as the Oracle of Omaha because I was born and raised in Omaha, Nebraska, and I have built a reputation of making successful investments and business decisions based on thorough analysis and long-term perspective. This reputation has earned me the title of an investment guru or oracle in the financial world. My approach to investing is simple: buy quality businesses at reasonable prices and hold them for the long term. This strategy, along with my dedication to learning and understanding markets, has led me— has led to consistently strong returns for Berkshire Hathaway, the company I lead. All right, cool. So it was really nice chatting with Buffett. And basically you can make whatever model you want to make just using that. And you can come here as easy as saying, hey, from whatever model you want to use to create that model out of, it could be Mistral, it could be Llama 2, whatever one you want to create. And then for system message, this is where the magic happens. You just tell it, hey, I want you to be, I don't know, Mario. I want you to be LeBron, Snoop Dogg, whoever really that is famous and potentially the model knows. And you give it some character there and you give it a name and create it. And as you can see, it did really fast. So the next phase now is cleaning up your models that you don't need to use. So in this case, let's start by deleting Buffett. Sorry, Buffett. And you can pick several of them actually, like I mentioned, to delete. So you can pick Buffett. And let's pick Mistral in this case and just delete them and see how it does. So delete, boom, gone. And if you click now, you only have, oh, it's still there. Let's do this again. Delete. Boom, it's gone. It probably needed a page refresh, but you can see now we don't have that as an option. And even if we go up here and do Page, click up here, you can see it's not an option anymore. So it's deleted. And you can confirm this several times, right? And you can even open up your terminal here. Just if you're doubtful that the app is not doing exactly what it's supposed to do, you can do Ollama list. And you can see we have just 3, like the ones we have on the app as well. So we have just 3, these 3 models. Cool. So it's doing everything as it should. which is pretty cool. And so that kind of wraps up this demo that I wanted to show you. But another cool thing that I wanted to show you is that I'll be sharing this code with you so that way you're able to go ahead and play around with this code, with this app as well. If you want to modify it or there's some parts that you were looking at me like going through it and you're like, I don't like that, I want to change it up a little bit, you can come to this repo and I'll share the repo as well in the description section. And you can come to this, you can clone it. And the first thing you want to do is just click up here. And then there's this option here to clone. You click or copy that. Then you come to your terminal. If you're not familiar with this, if you're familiar with this, just ignore what I'm doing right now and skip forward. And from here, you're just going to do git clone and put in the model, put in whatever you copied from GitHub right here. And you'll be done. It just downloads that to your system. And then All you have to do is fire up maybe VS Code if you have VS Code and go to that specific location where you just downloaded or cloned that file to and open it up. Just basically just do, if you have already Ollama installed, what you need to do is just do Streamlit run the main folder of that application, the main file of that application that's in the root, which is 01. You're done really. And then just run. And once you open it, it's gonna open this and it's gonna feed everything off whatever you have on Ollama downloaded already in your system. So it's pretty easy to get set up. to get started. And it's also easy to modify. So if you go to the code itself again, you can just modify a bunch of things on this code base. And if you want to study up a little bit more Streamlit so that way you can maybe effectively change some stuff that you didn't like about there or add some new stuff, feel free to do that. If you have more questions, please put it in the chat comments or come here to the issue section of this repo and add your issues here. Welcome to chat. More about that in there and figure out ways to improve this. Another thing that I'm working on, I have a RAG page that I'm working on that will probably go somewhere here, but it still has some bugs that I wasn't really comfortable releasing and putting it out here. So I wanna have it in a way that it's a little bit more cleaner. And so that way you can play around with it. It's gonna be really basic, so you can maybe be able to play with that as well. And I'll create a separate video for that just because it's a little bit more special. in terms of dedication of time and actually getting it to work correctly. No, it can never be perfect, but correctly. But that's all really I have to share for this video. And also don't forget to subscribe so that way you won't miss future content that I'll be producing as well. So with that, I wanna say thank you and see you next time. Bye-bye. Bye.