Building a Custom Telegram News Bot Is Easier Than You Think!

Hello everyone. Welcome back to 2 side AI. Last week we shared a video about building a telegram news bot that can get us news feeds through telegram interface and without any ads. And you can tailor your newsbot to your preference and receiving news from multiple news sources as well. This week we're going to dive into the coding session.

Here. I have. So what you need to do is to set up some of the environment variables, telegram token, webhook, URL, whatever LLM you want to use. I have defined both OpenAI and Gemini here articles per digest. So meaning how many articles you want to show to the user and things like that. So I set it to 5 news update interval. So how often it has to go and get the news from the website is 60 minutes. So I have defined some of these things, but I haven't really implemented every single one of them. And here you can see all different services. So there is new service, there is a recommendation which is in charge of recommend. There is a scheduler and there is a telegram handler. So if I go to the main file which is when the fastAPI application starts, then here I have this line. So when the web server starts at the beginning then it's going to run this start scheduler. And right now it's commented out. I comment out this because I didn't want this to run every time. Basically this is a. You can even just separate this from the main file and run it separately once. And then so what it does, it's is the module that is going to go and based on all of the URLs of the news websites, just grab all the news items from them. And if I just uncomment but I'm not going to run it and come to this file which is this scheduler over here. You can see I have different functions run scheduler and start scheduler. So I have implemented the code. It's going to get. You know how often it has to do that, right? Every 60 minutes. It then has to go and get the news. So it automatically does that. Users don't need to mention that every one hour it's gonna just go. Then it's going to exactly get the news, new news items from the sources and then it comes to this update news which is going to update the news. But if there is no news then it's going to get the news. And then there is a fetch news here you can see when you click on it it comes to the news service and here it says based on the news sources and the news sources are defined here you can see in my config file. So I have defined this news sources and here is a list of URLs that bot is scheduler is going to go and get the news from. I have already defined the cnn but you can have a list of them. Ideally users can have their own lists, right. Right now this CNN is going to be for every user. But if someone you know has their own preference or URL, they can. They have to change the logic of the code here. But this config file is going to load the environment variables and also have a couple of extra things. You can see the new category news categories. I have defined these categories and these are the categories that are displayed to the user when they run run that command slash categories. Again this could be dynamic or a set of like initial categories and users can add their own categories into that articles per digest. Again I have defined that in my environment as my environment variables. Anyway, so coming back here. So if I go through this scheduler here, the fetch news is going to go and get the new sources and it's going to just inside a for loop it's going to this. For each news website it is going to just call a particular library that will show you it's a fantastic library specifically designed for grabbing news from websites. And then it is going to iterate over all of the article like URLs that it gets. And here I restricted to 300 again because otherwise if there are 10 different news websites like CNN, Washington Post, so on and so forth and then for each one of them there will be hundreds of new article URL then it will be like a huge database. But for this demo I restricted to 300 URLs news URLs. So it's going to get the. It's going to download the news article behind the scene and it's going to parse. It is actually using NLTK library. So that's one of the dependencies that users need to define. And it's gonna clean the text and all that. And it is going to store the article and some metadata, even the category, because it's gonna categorize the article. So that's again the nlp, not no LLM and all that. This is more traditional NLP like nltk. Yes. So here it's gonna just find the keywords from the article. So I'm using NLTK library as part of that library. So the thing is you don't need LLMs for everything, right Here I'm using NLTK so it's gonna just from the news article, it's gonna identify the keywords and it's gonna store them as the categories and things like that here.

But you can find that the keywords are only from the title.

From the article, right? Yeah, from the article, not the title. The title is actually a separate thing. So I get the title, the text and keywords. Keywords come from the content of the article and the keywords come from the title.

And the keywords are for the news categories definition you could have.

Yeah. So here, depending on. Because I need to assign specific categories to this news. So here I have this basic kind of condition that if the category or is somehow inside the keywords, then you can also assign this specific category to this news. Right. It's basic condition. But of course users can add more complex logic. If none of the categories are in the keywords, then it's a general category, It's a general mean.

So basically you're the one who come up with those categories. That's why you're doing it this way, right?

Yes, because the categories are predefined. I showed you politics, things like that. Yes, makes sense. And then I also store them inside the database. The database is a SQL Lite database. Here you can see the News DigestDB. It's been created. And then here inside this database you can essentially take a look at the tables that I have. So I have a user categories table where it's gonna like one too many. Each user can have many different categories. There is user interaction right now. So again, this one is going to capture the user preferences and later on useful recommendations. But I'm not using it. I haven't implemented the logic for that. There is a users table where they're going to capture different metadata about the user, username and things like that. There is also a category table and articles tables just in conversation. So you can see it's fairly comprehensive. But I haven't used. Right. All of these tables. I'm using just a few of them. So that's another thing. If I go to Telegram handler here, typically this is the file which is in charge of handling the messages coming from the users and generating, let's say, the response. Right. Using AI services that I have, I guess I have that here. But this, all of the logic here is how to handle the commands, or if there is no command, just simply have a natural conversation. But again, it's not fully implemented. But you can see the code for handling Telegram messages. I'll also have a conversation here, which is the part that is going to. You can see different functions for getting user preferences and things like that. Getting recent articles. So it has actually that. Right. You can see that I even stored that based on the time the most recent ones. So you can see I've implemented the logic, but not 100% of the logic, but it's actually there. In my opinion the code is like 80% there. Users just need to add a little bit extra and then they can use this part for. For the. For themselves. Right. As RSS feeder and things like that. You can see the logs here and here the prompts. So I am here is actually see I'm using the light LLM to generate the response based on the user input message. So it's there, it's implemented. I am actually even having the history of the conversation. So I am actually capturing the history. But it is still not 100% complete in terms of handling every single scenario when you have a conversation. It does keep the conversation natural. However, in terms of checking and going to this particular table and check different things or go to user preference and all that, the best way to do it is to implement a bunch of tools, right? And then do a function calling with the tool. So like more agentic. But I haven't implemented that for this particular demo here. Overall, this is the code here. You can see I essentially explained more or less everything here and I'll also demo that. So any questions?

Yeah, and the recommendation I think you didn't cover Recommendation. Recommendation.

Oh, recommendation recommend news.

That that's similar to maybe what I'm interested in. So you can. That functionality can be added as well.

Yes. If you look here, I have say generate digest for the user and this is when user types slash digest. However they could just simply type it, right? It doesn't have to be a slash digest, but when you type that that condition needs to be improved. So it actually does the very same thing. And then. But I have implemented. If the user doesn't have any categories, then let's say do this. If then for the digest go and get the news articles and then do that. So the conversation also is there that get the conversation that happened between the user. So this is the like a history of the conversation and I pass that. So generate personalized digestion with LLM. So this is the function that I have here. This function still needs a lot of improvement. You can see I have written something then I have commented out and I have the prompt here. So I say create a personalized news digest for the user with the following preferences. So I am getting the categories from the user categories, right? So I am doing it and then I'll pass the history of the conversation, all the articles that are already captured and recorded in the database for this particular user and then I give it a prompt and I say based on this prompt, I call the LLM and I get that. And then I essentially generate the response and send it back to the user. It's there, but I tested that. It still needs some improvement. So it's not that say fully, yeah, implemented, although that I have added no everything here. But I think it still is not 100% complete format digest. So I have some utility functions essentially to format the digest the news so it can better pass them to the LLM. That's a recommendation. Recommendation doesn't capture the explicit feedback from the user. So that's one thing, right? The users, if they click on different categories, it's going to capture them. But if later on they. They want to change or update or things like that via conversation, then it doesn't handle that. That's why I'm saying that it's not 100% complete. They have to do it via that command. So they type categories so it shows the categories, they can add new categories. But if user want to remove something, let's say I have already picked politics, but I'm not interested anymore if I want to give that feedback. It doesn't recommend that anymore. Now it's not implemented because it still shows all the categories. So that's why it needs some improvement regarding recommendation. But besides that it's a fully functional code. It works, but not considering every single scenario. I think it's a very good practice for users who are interested in using Telegram with LLMs AI agents and then doing a project and a practical project if they want to get personalized news, which is something all of us need to look at. My own news and all that.

Yeah. I have another question. So from the builder perspective, right? It sounds like we're building this for let's say one single person. So if you're building it for one audience like me, then I tell the bot that I want to see CNN news and you will fetch the latest news and curate them and send it to me. But how should we think about it when we want to scale to many users? Because you and me might be interested in both politics forth from cnn, right? But you don't want to fetch every time one user asks for that.

Right now it is not for a single user. Still it's a bot that when you run it different users can interact with it. However, in terms of efficiency, like how.

Do we factor in efficiency when we want to scale.

So obviously if you want to scale that as a product so everybody can use it or sell it as a production ready, then instead of doing let's say some of those predefined categories or at least we can define them, but somehow implement the logic. So users, different users can give their preferences and then you get them and based on each user preference, essentially just you go and grab different news. Right now for instance, it's gonna go and only do cnn. But if you don't want to get your news from CNN right now, there is no way unless you change the cnn, right? Or delete all the records, right? If you do it for your own, then it works. But if you want to let other users use that, then you need to implement mechanism where users dynamically can change their preferences or add new sources and things like that to make it more efficient. Right now for single user is okay, but not for multi users if they have different, completely different preferences and things like that.

The database has to be more efficient when multiple users are asking for the same thing.

If you look at it for hundreds of thousands of users, then yes, it has to be a little more efficient. Meaning that let's say it can cache some of the results, right? It can do that kind of stuff so it doesn't query the database every time. Let's say if it's. If I ask the same, if I'm interested in business from CNN and so are you, then and the news are essentially the very same news, then it has to just show the same news articles to us. However, because I have defined this user interaction and user categories, right? And based on the time of the digest and things like that, even both of us are interested in the same category of the same website, we may end up with different types of news, right? It doesn't. That's why it has to just go and get it from the database. Doesn't necessarily. Just has to, I don't know, cache some of the news and things like that. But yes, for production we can add more efficient techniques into what we currently have to make it work for at scale.

Yeah, that makes sense. I think the benefit is very clear, right? First of all, I have something that can fetch news from multiple sources so I don't have to go to different sites one by one. And there's no ads for sure.

Yes.

Then the last one is I have that digest sent to me at a set time as a scheduled Time as I want it. Basically I can curate my own news feed with ads free and send to me like when I'm. When I get up in early in the morning. Something like that.

Yeah, exactly, exactly. This sending part at the exact same time that you prefer and all that is not 100% implemented here, but again it's there. Users can implement that part. As you said, there is no ad. It essentially gets the news articles and even summarizes them. You can see these are all the libraries that I use for the dependencies. This library is called newspaper 3K. So if I come here. Yeah, let me just close these two. This is the library that I'm using. The library is fairly old and it hasn't been updated for a long time. They have updated the readme a couple of months ago, but you can see that's 10 years ago, five years ago and all that. I guess they have updated the library to work with Python 3 and all that. It's a fantastic library. It does this job. It specifically designed for scraping news articles. Right. And curation and it, as I said, one of the things that it does behind the scene is using NLTK and more traditional machine learning libraries to get the news, to clean it, to come up with keywords, things like that. But it's a fantastic library if you want only to just grab news articles like things like that. This library is working very well.

Awesome. This is a really interesting product to work on and it has a lot of potential as well. Research has shown that average person spend more than two and a half hours per day on your phone, on your tablet, on social media. If that's your news source, you're spending a lot of time probably doom scrolling and you inevitably going to see a lot of the ads which can be very annoying and there's a lot of push on like different news that you're not interested in as well. So this way you can reclaim your attention back in a sense. Right. If you really want to use something like this and you only get what you care about, that will be very useful I think.

Personally I agree. If you think about it, it's going to be your personalized assistant, in this case only for the news. Ideally you would have some kind of personal assistant that works for you and doing a lot of different tasks. But here for the news, which is something everybody does, I go and first thing that I do in the morning when I wake up is look at my phone and go maybe to Twitter or this website or that website to see what's going on. Right out there in the world. And potentially this could be a very good personal assistant, especially if personalize it for your own use, because it's with the Telegram. Telegram has a lot of, like, rich features that you can use, we can use. And then I think this is a fantastic project. I'm going to build this further on later for my personalized news assistant. That would be. Yeah, yeah. But the code is available on GitHub. We're gonna just share the link later on in the video description and people can go and take a look and just use the code however they want.

Yeah, sounds good. Thank you, Meady, for another great session for new AI projects. And let us know if you have other ideas you want us to implement or try it out. We're happy to experiment together with you. Remember to subscribe and sign up for our community.

Absolutely. I'm thinking of creating a series of kind of educational projects that combine AI assistants. Right. Or AI agents. With Telegram specifically, I am very interested in using some of these messaging platforms like Slack, WhatsApp, Telegram, even Microsoft Teams and all that, depending on the use case. But Telegram is pretty much one of my favorite messaging apps that I use. And so integrating these AI agents with Telegram is a fantastic, in my opinion, way of using AI.

I agree. Yeah.

All right.

Okay. See you next time.

All right, take care. Bye.

Bye.