How to build the FASTEST AI chatbot with Groq and Streamlit
In this tutorial Tony Kipkemboi demonstrates how to create a high‑performance AI chatbot using Groq’s ultra‑fast inference API and Streamlit for the front‑end. He begins by introducing Groq’s claim of being the fastest LLM provider and briefly explains Streamlit as a Python‑only framework for building interactive data apps. The walkthrough covers obtaining a free Groq API key, storing it securely with Streamlit’s secrets.toml, and setting up a project structure that includes requirements.txt, .gitignore, README, and a virtual environment. He then walks through the core app code: configuring the page, loading the API key, defining a model dictionary (Mixtral, Llama 2, Gemma) with per‑model max‑token limits, using session_state to preserve chat history, and implementing a streaming generator to display token‑by‑token responses. Error handling, avatar UI elements, and model switching logic are explained. The demo runs the app locally, showing rapid responses from the three models and a side‑by‑side speed comparison with ChatGPT, highlighting Groq’s low latency. Finally, Tony shows how to deploy the app to Streamlit Cloud, configure the repository, and share the live URL, GitHub repo, and social links, encouraging viewers to fork, contribute, and tag him on their own builds. The video ends with a call‑to‑action for likes, subscriptions, and suggestions for future content.
Chapters
Have you heard about Groq? Not this one, but this one with a Q. Yes, this Groq has the fastest inference speed of all the other cloud providers at the moment in the market. And to demonstrate the speed of Groq, today we will build a Streamlit application using the Groq API. And in this process as well, you'll be able to build this application for yourself and share with everybody else or use it for your own purposes. So let's get started. Before we jump into the demo itself, if you've never heard of Streamlit, 'cause this is the platform we'll be using or the framework that we'll be using to build this application, just go to streamlit.io and you'll see it here. So Streamlit is a fast way of building data apps using Python. So if you're not familiar with things like React or JavaScript for building frontends and you just know Python or you just prefer Python, you can just build Python apps that you can share and interact with just with Python using Streamlit. And here's an example. As you can see on the screen, you just do import streamlit and you write your first You know, write hello and it shows you your application right there. So that's just a simple explanation of Streamlit. If you if you don't know about Streamlit, just do that: pip install Streamlit and you get Streamlit right there. And it's very flexible and very customizable to your needs, really. And it's really cool. I like using it. So that's out the way. So let's go to Grok. So go navigate to console.grok.com. Slash docs quick start, and as you can see here from the quick start, we'll need a few things to get started with our demo here. So we'll need an API key from Grok, and right now, as far as I can tell, the APIs are free. Jump on there; it's free access. Use it. I'm sure they're gonna close it down and let us pay after a while, but at the moment, it's free. So please use it. And you want to go to the API key section here, and the first thing you want to do is. Create an API key. And I have a few here. I'm going to use this one, but I'll just show you how to create one. So you can do— just give it a name. So in my case, I'll do demo and copy it. And just make sure that once you copy it, put it somewhere because it's going to get— if you lose it, you have to create a new one because you can't come back and see it. As you can see, you can't do anything. Even this edit, you just edit the name but not the API key itself. So in this case, I'll just delete that one. And I'll leave this test one, which I'll be using. And this one is for the deployed application that I already have deployed. So let's go back to the documentation and continue that process. So the first things we want to do is there's export. You need to export your API key and install Groq. And then here's just a boilerplate example of how you need to structure the call to the application so that it has the OpenAI compatibility, which basically means kind of OpenAI kind of set the standards of how you call the LLM to respond to you. And this exactly what they're using. So it's pretty standardized really at the moment. So this is really cool. But the easiest, I'll show you the easiest way that I usually try to get if they have a playground and they do have a playground. This is the page where I usually just customize the model and then I see the code of how the model looks like. So if you click up here at view code, you'll see the code that kind of is running that application right there. And as you can see, we just have this Groq client and client chat application create mixed raw model and all that. So this is pretty cool. It kind of gives me a rough boilerplate that I want to start with. So I usually copy that and put it in my IDE, which is VS Code in this case. All right, so now we're in VS Code. So the first thing you want to do is since we got the API key from Groq, you want to store that somewhere first. And with Streamlit, we're going to use Streamlit secrets to manage that secret so that way it's easy to manage. So the first thing you want to do is create a folder, .streamlit, and Within that folder .streamit, create another file called secrets.toml in this case. And within that, save your API key that you created on the Groq website and put it here. And under, so groq_api_key equals to, and then do the quotation marks and put the API key between there. So we'll need that later. And then create streamit _app, or just name it whatever you want to name it for your application.py. And in this case, this will be the code, just the boilerplate that we got from, uh, Groq's website. Copy that, put it here, or you can follow along and use mine in the case because I've customized mine a bit here. So I'm not going to do real live coding here, I'm just going to show you what I've already built. I'll walk you through my code. And you need to create a requirements file, which is Groq and Streamlit. When you deploy this application later You'll use the Grunt and Streamlit to be able to identify this file in your GitHub and be able to pull these modules and install them in your application that you deploy on Streamlit. And I have a README, which is just, it's always good to have that to guide people when they come to your repository to see what or how they can use your repository code. And one of the important things, you should have a .gitignore. So basically when you're committing this code to GitHub, It will ignore anything that you put within the gitignore. So these are important stuff that you don't want them to go to your GitHub so that everybody can see them. So my first one here, I have a streamlit.secret. So definitely you don't want to send your API key to GitHub. You don't want anybody to see your API key. They will end up using it, rack up charges on your account. It's always not a nice thing to do. And there's a virtual environment that I created here so that I can have all my modules within one Sandbox, and I have a lot of videos on creating virtual environments. So please check some of them on my channel. And once you have that ready, go back to Streamlit application. So the first thing walking you down through this code base, I imported Streamlit here as Streamlit, and then I imported generator, which we'll need later for the streaming process so that you can see the responses from the model coming in in a streaming fashion. And we imported Grok. From that, and then this first line here set page config. Basically, what we're doing here, we're just setting the page name, the title at the top of the page when you see on the bar or on the tab itself, you see the name of the application. I named it as Grok Goes Brr because of the speed, duh. But you can name it whatever you want to name. I have the page icon layout. You understand that. So the next one is an icon, which is basically what will be look like one of those Notion style. Page icons on the application just for styling purposes, and then I'm calling that function right here and putting a race car as my icon, which is pretty cool. I think the next thing is a subheader. I'm giving it the header for the application and giving a divider, and then from here I'm setting the API key for Grok. So the way you call it on Streamit is we're using Grok, we're calling Grok, and then we're passing the API key for Grok. And with Streamit used. St_secrets, and that will call the secret API key here from this file, and we'll call it and use it in this application. So it's pretty nice. And I'm also initializing the chat history, and this will save the model and will save the messages. That way you're able to chat with the same model and you'll see the messages as they come as well. So that way it doesn't disappear after each run. The next thing I'm defining the models and where I usually go to get this, and this is for Your knowledge for future, whenever you have this code and you want to update it, you can come to their. They have a Grok Discord channel, and if you come to the Grok Discord channel and go to models, this is where they update the models that they're getting integrated within Grok. So right now they have Llama Two, they have Mixtral, and they have the Gemma from Google. And if they add some more, they'll add it down here. So cool, nifty thing that I use there. But I'm putting it as a. In a dictionary so that way I'm able to call it down here. And as you can see, I have 2 columns. The first column, I'm just adding the model option. I'm just giving the user option for them to be able to choose the option for the model they want to use from these 3. And then I'm also putting the max tokens for that model that they pick so that way it's automatic and you don't have to worry about that. You can just reduce it, but it gives you the max. And then I'm saving that in session state. I'm able to clear the session state for each of those models, when you switch to a different model, you not see the chats from the previous model. So that cleans that up. And also the next column, now I have a slider which is the max tokens as I mentioned before. And then I'm displaying the chat from, or the messages from the history of the application when you rerun the application. So that's a pretty nifty thing to do. And then like I mentioned, this is the generate chat response. This will using the generator that we imported at the top. And this basically just goes over or iterates over the chunks that's coming in from the responses from the model and it's lining them up so that way they are streaming within the application and you're not waiting for the entire thing to generate before it shows you the full output, which is pretty nice. And then so here we're just asking the user to enter prompts and they'll type whatever the question is and then they'll submit. And once they submit, and it also has an avatar, which is pretty cool. Then you can able to tell this is the user, this is the model. And for the model, it's like a bot thing emoji. And so once you type in, it shows you, it runs the model. And this is where the model is fetching response from the Groq API. And so I put the try and exceptions here and it's always nice. So that way if there's an error when we do send the question To the API, then we're able to get a full response that we're able to understand and debug and fix our application. So that's always nice. And also I'm passing those messages to the session state so that way it stays in session state. And let's see, and almost— and you have to make sure that you put stream as true so that way it streams and it's not just waiting for the full response. And then as we get to the end here, we have the generator, we're passing the response that we get from the model, we're putting it through the generator using the app The function that we created above, we're putting it to chat response, and then we're getting the full response and we're writing it to the screen so that the user can see the streaming response of what they asked the question for. And if there's an error, it'll show us the error and give us a quick explanation of what the error is about. And then here we're just appending the message, the final message that we got from the model to the session state, so that way When we print it out, we can see the previous conversations that we've had with the model. And then, and this is just within that session, just to be clear. And then if not, if maybe the output is not a string, we're gonna handle it down here with a different way to join it and still pass it to the— we try to basically put it in session state if it's not even a string. Now that we have our code ready, let's run it and see the application in action. So what you usually do is Streamlit, you do streamlit run. And then the name of the application or the name of the file that you named where you put your code. And as you can see here, we have our application running the 3 different types of models that we already had in our code. So we have Mixtral, we have Llama 2, and then we have Gemma from Google. And then we also have this max tokens. And one thing you should note is as you change the model, as you can see, this one for Mixtral is 32,768 tokens, max tokens. And as we switch to 70B, it definitely changes based on the model, which is really neat, which is part of our code. So that's pretty nifty. So let's start with MixDraw and let's ask it a question. As you can come here, let's just ask it, what is life? Let's see how fast it answers. And as you can see, it's really fast and you can see it's streaming. It's no waiting for us to— there's no waiting to finish the full response before it responds to us. So let's ask it again, create a table of comparison for all super superheroes. All right, let's see. All right, so it did that, and that's pretty fast, right? Um, compared to ChatGPT, which I think would be much slower. Let's just do ChatGPT. Let's give it the same question that we gave it. Um, create a table of all superheroes. And you can see that did really fast. So let's come to ChatGPT. See, that's, I mean, it's taking forever, right? So he's talking about all this stuff before he even creates a table. So that's already taken a long time. We're still rolling anyway. We don't want to wait for that for too long. So as you can see, it did pretty good. It did answer a lot of good questions here. Let's ask it more questions, right? So the raddest Jokes. All right, so you got some jokes here. Why did the scarecrow win? Why did the scarecrow win an award? Because he was outstanding in the field. All right, all right. Why did the bicycle fall over? Because it was too tired. Huh, you're actually funny, Grok. All right, not Grok, really, Mixedwell in this case. Why did the tomato— tomato, tomato, whatever— turn red? Because it's so salad dressing. All right, cheeky. Why did the hipster burn his tongue? He drank his coffee before he goes to school. All right, why did the chicken go to the séance? What is even see? What is séance? Uh, to talk to the other side. English is not my first language, so I'm gonna have to look this up. What is? All right. A mediumistic practice in which people gather to receive messages from spirits. Okay, something like that. Makes sense. All right, to the other side. Okay, makes sense. Why did the golfer bring 2 pair of pants? In case he got a hole in one. Huh. All right. Why couldn't the bicycle stand up by itself? It was too tired. Okay, I think it repeated itself a little bit, right? Yeah. Um, why did the computer go to the doctor? It had a virus. All right. Why did the chicken go to therapy? It had some poultry issues with To work out. Hmm. All right. All right. Yeah, it did make me smile. Thank you. So you can see it's really fast and it's really cool to make this stuff. And it's, you know, if you want to take it even to the next level, you can deploy this application and share with people to use. And I've already deployed one, but if you're building this and you're wondering how to deploy it, as you can see here on the top right corner, you can just click deploy and it gives you 2 options, right? So you can do custom deployment. You can deploy it on your own hardware or cloud service. And then you also have just deploying on Streamlit Cloud. And Streamlit Cloud is for the community. It's free. And, you know, you can deploy unlimited public applications for free. You only have one private application that you can deploy so that, you know, you want to keep it closed. And apps are going to be discoverable. They also get indexed by Google. So that's another really cool thing. So if you click Deploy Now, it'll take you to this page where you can log in with your Google account. From here, it's just taking in from the repository. It's gonna create a repository, have the main file and app URL. You can put the app URL, you can put your domain name that you wanna put here. So you can put something funny, but I don't wanna do this again cuz I've already done before. And I'm, so I'm just gonna show you my app that I already created and I call, I named it Groq Demo Streamlit App. And as you can see, it's already deployed and you can go to this application link and I can share this with anybody. And they'll be able to interact with this application wherever they are and play with it on their phones, really anywhere they have the connection. So this is really cool. You can build this and share with other folks. And as you can see, they share here. You can share on socials. You can click on socials. And you can embed it also within your application if you have another application or you have your blog. You can embed it in there as well. So I also want to share my GitHub repository here so where you can find all the code that we just went over. And you can fork it. And play around with it, add more features, make it more better, or pull do some pull requests to give me some suggestions with what you want me to build next. Feel free to do that. But if you go to Tony Kipkemboi, this is my profile, and click on Grok Grok Streamer Demo, and you'll be able to see this. And you can follow the instructions down here, and also come back to this video for reference. If you end up building something with Grok API and Streamer, please tag me on it and. I'll be happy to retweet, reshare, comment on it, so, uh, give more visibility to it. All right, that's a wrap for this video. Please like and subscribe if you enjoy this content, and also please give me your suggestions, your comments, uh, your feedback on this video. And also, if you want me to cover something else, please let me know. And, uh, stay tuned for the next videos to come. See ya!
More from Tony
Summarize Videos and Research Companies with CrewAI + Gemini Models [Quickstart Guide]
How to Make Your AI Agents More Reliable with CrewAI Task Guardrails (Step-by-Step Tutorial)