PDF Files to FLASH CARDS - Automate YOUR Learning with ChatGPT - Tutorial

When I'm learning something new, I like to create flashcards so that I can memorize the information. The idea is very simple. You take a concept and then create a flashcard based on that. And you simply review those flashcards to revise the information. And I use this software called Anki, which is really helpful in creating flashcards. It uses spaced repetition for reviewing material at systematic intervals. However, the biggest trouble is creating these flashcards. Not only you have to read and understand things, but you have to spend extra time to create these flashcards. But today I'm going to show you how you can automate this whole process. So in this video, I'm going to show you that based on the document that you're studying, how you can automatically generate questions and answers and then export them in a format so that you can import them back in Anki. For demonstration purposes, I'm going to be using the Constitution of the United States, but this will work on any PDF file. So first we need to create a virtual environment. So I'm going to say I'm using CONDA for this, but you can use any element manager. So let's call it Unkey, let's click enter. All right, so my assumption is that you already have Python installed on your system. All right, so we simply created a virtual environment called Anki Underscore cards. Okay, next we will just activate the virtual environment. So that's going to be done using conda activate Anki cards and you see we are in the virtual environment now. Okay, next we need to install all the required Next we need to install all the required dependencies. So we just need two packages. One is the PyPDF 2 and OpenAI. In order to do this, we're going to do PIV install requirements TXT and so this is going to be GitHub repo. Okay, so the requirements are already met. So let's start creating our code base. I have this file called Anki underscore flashcards, underscore creator py and this is the one that we're going to be here our code. So first I'm importing all the required packages. As I said, we need a PDF file reader. We're using ChatGPT. That's why we're importing OpenAI and then we just need to import the operating system module. Okay, next we are Simply defining our OpenAI API key. So usually or the best practice is that you want to create a virtual environment variable and, and assign it to that and then you just read it here, but just scrappy here okay, next I am simply defining the root directory. So basically this will store the address of the directory where this Python file is. So everything is going to be relative to this path, and that's why it's going to be a lot easier to work with. Next, we want to write this function where we're going to be reading a PDF file. So you simply provide the path of the file and then it uses this PDF Py PDF 2 library or package to read the file. It creates this reader object, and then it goes through each and every page, extract text from that page, and put this into this text object. Okay, so here I'm simply printing the text from the file. So let's see if it actually works or not. So I'm going to say Python and then, okay, so yeah, so you see that it actually printed all the text that's contained in the file. So this step works. Okay, next we need to divide our text into smaller sections, because the idea is that we're going to be providing text to the ChatGPT or any other LLM, right? And then we will let ChatGPT or the LLM decide to divide into question and answer. So it will, based on the text it will generate, automatically generate a question and then also the corresponding answer. Now, there is one small issue. All of these LLMs have something called context window. So it's the amount of text it can process or the amount of tokens it can process, right? So in this case I was looking at this specific file, right? And it has around 55,000 tokens, right? So you cannot give this to a ChatGPT because ChatGPT has a context length of only, I think, 4,098 tokens. Now, if you have a bigger file that you're working with. So a simple way would be to divide into smaller chunks that you can feed into ChatGPT for creating these question answers for your flashcards. So for that I came up with this divide underscore text function, which simply takes your text and then divided into smaller predefined chunks, right? So in this case I'm just playing around with 1,000 tokens, but it can be anything. You can set this parameter. Okay, so next is this Create Anki cards. And this is the main function where all the magic is happening. Now, so we are reading the PDF file and then we are passing on all the text to this function. Now, for this specific example case, I moved my text division function here. So we're creating those chunks here, and you can do the chunking process using lagchain as well. But for this specific application, that is going to be an overkill. All right, one more thing that I'm simply taking the first chunk because I just want to show this for demonstration purposes. But let's say if you want to go over all of them, then you probably want to have a for loop that goes through all the chunks and does this process for each one of them. Now, in terms of our prompt to ChatGPT, we are using this system message in which we are assigning a role to the system. So in this case we are telling it that you are a helpful assistant. Now, in terms of the actual prompt, here's what's happening. So we are asking it create Anki flashcards with the provided text using the format. So it's going to have question, then semicolon, answer, the next line, question, semicolon, answer. Right. Then I'm adding keep questions and the corresponding answers on the same line. I found that if I don't include this, then it puts question and then the answer is on the next line. But we don't want that because for Anki, we want the question answer to be separated by a simple semicolon. And next we are simply providing the text that we are reading. Now, we take that message or prompt and then pass it on to the OpenAI API. So for this specific case, we are using the chat completion API. Right. The model that I'm selecting is a GPT 3.5 Turbo. So it's the ChatGPT model. It's one of the cheapest model in terms of the cost. Right. Then you pass it on the message that we created along with the system message. I'm setting the temperature to 0.7 to give it some flexibility and freedom of creativity. Right. And then you can set the maximum, the max tokens as well. Now you definitely want to play around with these parameters for your own applications. Then from calling this function we'll get a response. And in the response we want to separate the content. Next I'm printing those, the response from the API and then also writing the response to a text file. And in this case I'm calling the text file flashcards Txt. Okay, so with all this updated code, let's run the code and see what happens. So first, if you see here, it actually printed everything. I think I had a print statement here. So I'm going to actually just comment this. And now here's the output from ChatGPT. So you have a question then followed by semicolon and then the corresponding answer. So here's the file. And you can see that it is actually following the format that we defined. Okay, so next we're going to go to Anki and then click on Import files and here is the file that we just generated. So let's import this. And when you import it, you can actually see it's in the correct format. So you see you have the question, then the corresponding answer, and they are separated by semicolon. You can, in your system prompt, you can tell it to do it by tab or colon or comma. Right? But I think semicolon works best. So this is how it's going to look like. You're going to have the front that is going to be the question and then the back is going to be the answer. So let's simply import this. It's asking me notes found in the file. So there are 11 different questions answered. I'm going to close this. And here we have our deck. Okay, so I simply renamed that. If you want to rename a deck, simply go here to this wheel, click on rename or you can assign it whatever name you want. Right? And now you can actually start studying. So for example, let's say what is the constitution of the United States? Right, here is the answer. And then if you are not familiar with the Anki, so you simply select whether it's easy, good, hard, and again. Right? So these are basically the time interval after which it's going to show you. For example, if I say it's a very easy question, so it's going to show me within four days, right? So here's another question. Then you can just look at the answer and select. Right? So it's basically space, time, repetition. Now this video is very different from the one that I you usually see on my channel. But I just wanted to show you how you can automate different things with the help of ChatGPT. Now want to highlight one thing. You can actually do this in ChatGPT environment, especially with the plugins, if you have the paid version. So there are plugins which can read PDF files and you can use the same prompt to create flashcards for you. However, if you do it through the code, it gives you the flexibility of automating everything and you can even give it multiple PDF files at the same time and it will go through the process and create those for you. I'll put this code on GitHub. A link to the repo is going to be in the description of the video. If you found this video useful, consider liking the video. Give it a thumbs up and subscribe to the channel if you are not already subscribed. We also have a Discord server now for the channel. There are some great folks who are helping each other out. So come join us if you want to consult with something, somebody to automate your own business processes, you can reach out to me. Details are in the description of the video. As always, thanks for watching and see you in the next one.