EP 336: A Complete Guide to Tokens Inside of ChatGPT

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

This small little detail about large language models is something that most people don't understand and, and it's probably impacting your work. So we're going to be talking today about a complete guide to tokens inside of ChatGPT well, and other models, and helping you all understand what tokenization is, why it's important. And this might explain why maybe you're working with a large language model and things are going great and then all of a sudden they go off the rails. It's not because the model stinks, it's because you have to understand how they work. And that's what we're all about here@everyday AI. What's going on, y'? All? My name is Jordan and I'm the host of Everyday AI and this thing is for you. It is your daily livestream podcast and free daily news, free daily newsletter helping us all learn and leverage generative AI to grow our companies and our careers. So if that sounds like you, thank you so much for joining. If you're on the podcast, make sure to check out your show notes. You can always come back and watch the video. This one might be one of those where visuals are going to help a little bit. I'm going to try to explain everything that we have on screen here for our live stream audience. So make sure you check out your show notes and if you are joining us live, thank you so much. But all of you, if you haven't already, also, why haven't you gone to your everydayai.com yet? Sign up for our free daily newsletter as well as it is like a the world's largest unbiased free library of generative AI education. Thousands of hours worth of content on there for free from myself and leading experts in AI. All right, so let's get into it and make sure to check out the thanks a million giveaway while you're there. So let's go over what's going on in the world of AI news before we get into token talk. All right, AI news. A lot going on today. A lot of the big names, new models, new features. Let's get into it. So first, Google Gemini has launched Gemini Live, its new AI assistant. So Google has launched Gemini Live, a new live AI chatbot feature promising more natural and emotionally expressive conversations on smartphones. So this was announced earlier at a Google event, but this new hands first free AI assistant allows users to talk in real time and interrupt an AI assistant and ask questions, with the chatbot adopting in adapting in real time. So although it currently lacks the multimodal input feature, reports suggest they will it will soon be added along with broader language Support and eventually iOS availability. So yes, this is similar to this more neural voice feature we've been waiting for four from ChatGPT and OpenAI as they go through some safety precautions, but Google has beat them to the punch with Gemini Live. All right, speaking of beating them to the punch, Google has also beat Apple to the punch. So Google has unveiled its new AI powered Pixel smartphone lineup. So Google has made a significant move to challenge Apple and Samsung by launching a new series of AI integrated devices with Edge AI or on device large language models. So the array includes the Pixel 9 series, a foldable phone, smartwatches and earbuds. So Google has introduced this new Pixel 9 series featuring their new advanced tensor G4 chip which allows for this on device edge AI. So the highlight of the launch is the Pixel 9 Pro Fold, which has one of the largest flexible screens on the market. So the new devices also include two smartwatches, the Pixel Watch 3 and the Pixel Buds Pro 3, both equipped with cutting edge AI technology. But the key feature like we talked about is Gemini Live, which will be available on that phone. All right, Twitter. Yeah, Twitter or xai, whatever you want to call it. Yeah, maybe I should just start calling it X. It just seems weird, but Elon Musk's X AI has launched Grok 2 and Grok 2 Mini. So Elon Musk AI company XAI has launched Grok 2 and GroK 2 Mini in beta, available exclusively exclusively to premium and premium plus users on the X social network. So Grok 2 features advanced capabilities in chat coding and reasoning, while Grok 2 Mini is smaller but capable and great for developers. Early feedback indicates GROK2 can generate images without restrictions, including those of political figures, raising some concerns about potential misuse. So the company aims to integrate Grok2 into X's features such as improved search, post analytics and AI powered replies. Great, that can't go wrong. Also, with the US Presidential election approaching, XAI is definitely going to face a lot of pressure to implement restrictions, especially on image generation, to prevent misinformation. All right, last but not least, OpenAI's GPT4 has reclaimed the top spot in the chatbot arena after a somewhat quiet model update. So earlier this week, OpenAI quietly updated its GPT4O model and didn't even really give us a name until 24 hours after the update. The so the latest model, it's technically chat GPT4O. Either the 08 version, which if you're a developer you might see that, or the GPT4O latest. Yeah, great. Great naming there. Anyways, this new updated version of GPT4O has reclaimed the number one position in the chatbot arena, surpassing Google's Gemini 1.5 Pro with a score of 1314. So Gemini, just so you know, Gemini 1.5 Pro is available for developers only, but this newest version of chat GPT GPT4O latest is available. That is the default mode now when you go to Chat GPT. So the achievement comes after a week of being tested on the Chatbot arena with over 11,000 community votes after it was named the anonymous DAT Chatbot label. All right, so the new version of ChatGPT shows significant improvements in technical areas, particularly encoding, with a 30 point increase over the previous models. Also, the model excels in instruction, following and handling hard prompts, showcasing its versatility. Wow, that's a lot going on from some three of the biggest companies out there. Let me know Live stream audience. You know, Jason and Denny and Kathleen, Chris, Woozy, Rolando, everyone else, do you want to hear more on any of these stories? Should we be doing some, some reviews or tutorials today? Let me know. All right, let's get into it though. Let's talk about tokens. All right, I'm actually excited for today's show. I'm gonna try to keep this one pretty short and factual. But, you know, I also want to know from both our live stream audience and, you know, those on the podcast, I always put my, you know, LinkedIn information in there. You can connect with me, email. But let me know, do you know anything about tokens? Do you have questions? Do you know what they are, why they're important at least for our live stream audience, let me know. I'm going to make sure to answer your questions on tokens, but we're going to do a complete look. And actually, y', all, if I'm being honest, not understanding how tokens work is one of the biggest reasons probably why your chats go awry. So if you've ever had had an instant where you are working inside of a chatbot like ChatGPT, and things start out great, right? And you're like, oh, this is great. You know, I'm getting fantastic, you know, fantastic results. And then all of a sudden, like all of a sudden it starts to just get dumb. And then you think, oh, it's just the model. Well, not really. What's happened is it's going past its context window, and that context window is measured in tokens. Okay? And all of the major kind of chatbots have, you know, have a different process of tokenization, but they all have kind of a memory limit or a context window of a different number of tokens. All right, so we're going to be going over the basics, and we're also, we're going to try to do a couple things live here. All right? So if you want to see how all this works, you know, this is one of those. If you are on the podcast, it might be. It might be worth, you know, watching this one. But let's. Let's get into it and let's go over what you need to know. So here's. Here's what we're going to go over. We're going to talk about what a token actually is, why large language models even use tokens, the tokenization process, and then we're going to talk about the context window, hopefully with some live demonstrations here. All right, so let's dive into it. So what the heck is a token? All right, a token. Think of it as this. It is a subset of a word. All right? So essentially what large language models do is they break down words into smaller subwords or sections or tokens. So we're going to be going over examples, but let's say, I don't know, let's pick a trending word if you're an AI dork like me. Strawberry, right? Technically, large language models, if you, you know, ask a question about a strawberry, large language models don't see words. They technically don't even understand words, which I know might sound interesting, but essentially there is a process that goes on, right? Because a lot of people might just not even understand. Oh, my gosh. How does. How do all these large language models, how can they understand anything and do almost anything? Well, it breaks down words into these smaller subwords or subsections called tokens, right? So as an example, the word strawberry, I know, it breaks it into three smaller tokens and it assigns tokens, essentially a numerical value. And then depending on. On the context that you might give a word, it's going to change that token value, right? Even capitalizing something, turning something plural, having a space before or after a word, it is going to change its token value. And that is actually y'. All. That is actually how large language models understand us. They understand our words by assigning token values to everything. So that is part of an algorithm of sorts of, that helps all of these large language models be helpful assistance to us. Because in the end they actually don't understand words, right? And technically when they are spitting back words, they're technically just spitting back tokens that get converted into words, okay? So it's a common misconception, right? And also, no, models cannot count as an example the number of Rs in the word strawberry or even the number of words. It cannot consistently, right? So if you say, you know, hey, respond to this in 10 words or less, it might give you 13 words, it might give you eight words. That's because it doesn't understand the concept of words. It understands the concept of tokens. That is how large language models think. That is how they process, that is how they respond back to you. Okay? One thing we're huge at at everyday, everyday AI is education, right? And this is something like I said, even the exper quote unquote, self anointed, self appointed experts are getting wrong, right? Because they say, oh, large language models are dumb. They can't even count the number of Rs in strawberry or they can't even, you know, respond back. When I say, you know, describe a strawberry in 10 words. It never gives me 10 words. Well, that's not how large language models work. Right? The key to getting the most out of generative AI is you first have to unearth and you know, look at what's underneath. You have to understand things at a very basic level. Okay? So that is what a token is. And don't worry, we're going to be going over some examples. So you know, one example dog might be one token or the word running might be two tokens. Run and then ning, right? So it might split it up in different places. All right, let's keep going. And I see some of your questions, y'. All. I'm going to get to them at the end, but keep your questions coming in. All right? So why do large language models even use tokens? Well, it helps them for context analysis, right? So it helps them better understand what you are actually talking about. All right? Also it helps with language handling and process efficiency. Okay, so let's talk about that a little bit more. So with tokens, it helps large language models break our text, our sentences into smaller units, those words or subwords, and then it allows the model to better understand and retain context. Okay. Also it allows, this is how large language models can sometimes speak hundreds of languages, right? You're like, how and how can it flawlessly, you know, switch between languages. And we've seen, even with these new voice assistants, right, we've seen, you know, demos of them where someone saying, hey, you know, where it acts as a real time translator, that's how, that's how it can handle different languages, is through this process of tokenization. Also, this actually helps processing efficiency because using tokens technically reduces the computational load. Right. Think it actually breaks down complex languages, complex queries into much more manageable tokens, into essentially an algorithm of stores of sorts. Right. When you break complex languages and complex problems into tokens, and you'll see what that means, it actually helps the model be more efficient and process faster each and every time. All right, so now let's talk about the tokenization process. All right, and hey, I agree, right, Jason, y', all, like if you're tuning in live, first of all, now you are the smartest person at your company, unless you work at OpenAI and you're working with the people who are, you know, actually building these models. But otherwise, yeah, maybe you should get some sort of continuing education credit for this. But you know, I guess now you're the smartest person in your company when it comes to tokenization and how large language models work. All right, so let's talk about the tokenization process. All right, and we're probably going to jump into this later, but a lot of people don't know that OpenAI has a tokenizer, right? You can go and play and learn how this works. So I do have to say that this is not the token. The tokenizer for GPT4O is not yet available. So the tokenizer that we're going to be going over both in the examples on my screen and we'll probably do a little bit live. This is for the GPT 3.5 and the GPT4.4 version, kind of like quote unquote, old school, right? So let's, let's take a look, let's take a look at this now. So what I'm showing here is different use cases of the words strawberries, right? So you know, tending to a strawberry patch, a heart shaped strawberry, candy blowing strawberries on a belly, strawberry birthmark on the face, scraped straw scraped, strawberry knee wound. Right? Like even the word strawberry technically has many different meanings. So you can go into the OpenAI tokenizer. Okay. And this is technically going to help explain two different things here. This is going to help explain how words and to show how words turn something into tokens, but also the context window. Okay? So now when we look at those different words because the word strawberry is going to mean different things if it has different context, right? The word strawberry by itself and then versus a strawberry patch or strawberry candy. That word strawberry is technically going to be broken into even different sub tokens depending on the context in which it is used, right? That is how large language models can understand nuances in language. When it technically doesn't know language. It assign, it looks at the context of how you are using words, right? Is it capitalized? Is it, is it singular, Is it plural? Are there words before or after? Do those words before or after change the meaning of said word? Right, so then after you put in, you know, and anyone can do this and you know, we will have this link for the tokenizer in the news letter today. So you can go and play with it and get super smart, right? So now I'm going to show you the token values that this, these are assigned. All right? So actually this is just the kind of the visual, right? But you can even see right at the top here I have the word strawberry. Even the word strawberry with a space versus not a space or a space before, or it being capitalized or it being, you know, singular versus plural world, it is going to tokenize it a little differently. Okay, so we're going to be going over, we're going to be going over some actual examples. And no, we're not talking about tokens as a new currency. We're talking about how large language models work. All right? So that is the tokenization process. And y', all, let's put this quickly into perspective here. We're going to be talking about a context window, okay? But essentially, you see, even these couple of sentences here, we're getting a total number of tokens as well. So tokens actually play a dual role. First and foremost, the tokens are important because that's how large language models can actually understand words. Second, there is something called a context window, or you can think of that as memory, and that is how much information ChatGPT can retain before it starts to forget things. Okay, so this is the context window, right? So it's different models. Also, this is important to know. But all models, so, so Chat GPT, Anthropic, Claude, Google, Gemini, they all have different context windows, right? Technically, using Chat GPT inside of the, you know, quote unquote chat interface versus the API, it actually has a fairly small context window compared to competitors, right? So when you use Claude, Claude has the best and the largest context window or memory when using it inside of the chat product. Okay? If you're Using it inside of the development product or not the front end. It's actually Google Gemini with up to a 2 million token context window, which is huge, right? So that's as an example, you can paste an entire book into chat, or, sorry, into Google Gemini if you are using it on the back end in kind of the developer mode. And you can talk and ask questions of that book, where if you're inside Chat GPT, it has a much smaller context window. And we're going to go ahead and probably describe this live, all right? So bear with me, y'. All. We're gonna get a new chat going, all right? And I want everyone to see and to understand this tokenization process and why it's so important, all right? So, hey, live stream audience, let me know if you can see my screen here. But for our podcast audience, this is what we're going to do, okay? We're giving Chat GBT some basic information about myself, all right? Also, I have inside of my Chat GPT account a tokenizer. Okay? So the one, this is not by default, although I don't know why large language models do not have a token counter. By default, they should. Okay? The one I'm using, I'm sharing it on my screen here. So I'm using Chrome, but I use what's called the Chat GPT token counter. This is a Chrome extension from amperly.com. all right? So I'll have that in the newsletter as well. All right? So when I'm sharing my screen here and we're going to be going over this process, bear with me. I promise you this is going to help you understand tokens, and it's going to actually improve your outputs a ton. And even if you're listening on the podcast, I'm going to explain this to you. All right? So as we go along, I'm going to put some information in, okay? And you'll see the token count right here. So I have some information about myself. I'm saying my name is Jordan. My favorite color is Carolina blue. My favorite food is deep dish pizza. Oh, I just had some deep dish pizza last night. Shout out to Riccis. And then I'm saying, I think the bears kind of stink, but they might be okay this year. Okay? So all of my words then when I hit enter are going to be converted into tokens. I can't see that on the back end. Right? But we're going to see also, when Chat GPT responds to me, we're going to see my token count go up. So let me take it away. So right now, technically this, this token counter is counting anything on the screen. All right, so, you know, essentially it's about 50 tokens my query. So when I hit enter, chat GPT is going to respond, probably going to say, oh, great, thanks so much for this information. And then you're going to see my token count go up. Right? So there we go. So it's, you know, giving me some standard blah, blah, blah. Hey, Jordan, nice to meet you. All right, let me say something else right now. Chat GBT has different features that I can actually remember things by default in memory. I have that turned off just so everyone knows, because you might say, oh, when you tell Chat GPT those things, it's going to remember. So of course, no, we're testing the context window here. Memory is off. Okay, so now what I'm going to do, I just have a bunch of random transcripts here. Okay. So I can't, you know, do 30,000 all at once. But I'm going to say please. I'm going to say please summarize this text. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Gen AI? Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on Gen AI. All right, give me a second here. I want to prove something to everyone. All right, so now you'll see, I put in a ton of text, so we just jumped up already to about 17,000 tokens. All right, I'm going to go grab some more. And I'm proving a point here, y', all, because a lot of people don't read between the lines, right? And they take whatever a company says as truth, which you shouldn't do. All right, because OpenAI. Actually, yeah, they're a little. I mean, it's technically on their website, but when they talk about GPT4O, you probably hear a context window of 128, 000 tokens. That's what most people think that the memory of chat GPT GPT4O is. All right, it's not. And I'm going to prove that to you. All right, so now I'm going to put a bunch of other information in here and I'm keeping an eye on the tokens. All right, so let me go ahead and grab a little more, y'. All. This is, this is what we do, right? We are investigative practitioners here at Everyday AI. We try to break things and test everything. All right, so here we go. This should be a little bit better. All right, there's a point here, y'. All. All right, so now you'll see after Chat GPT responds, I'm probably going to be at about 30, 000 tokens. Okay, there's a point here, y'. All. I am proving and showing you through an example why tokenization in the memory matters, right? Because I talked earlier about, oh, maybe you're having a conversation with Chat GPT and you share some things and things start out great, and then after a while it starts to lose its memory, right? So now here's what I'm doing. We're at about 29,000 tokens just under. Okay, so now I'm asking, what's my name? What's my favorite color? What's my favorite food? What do I think of the bears? Right? So why am I asking that? Right. I'm doing it to show you that even though there are, there's about 30,000 of tokens of information between my question and where the answer is, right, it should, in theory get all of these things correct. The tokenizer is not 100% correct. These are estimates. That's why I didn't go up to like 31,900. Okay. I wanted to show you. Okay, we're at just about 30,000. Let me ask this question. And there we go. Right? I. I asked the question, what's my name? What's my favorite color? What's my favorite food? What do I think of the bears? It says your name is Jordan. Your favorite color is Carolina blue. Your favorite food is deep dish pizza. You think the bears kind of stink, but they might be okay this year. All right. Yeah. Monica. Monica says, love it. Investigative practitioners, that's what we are. You know what? And if you have taken our PPP course, our free prime prompt polish course. It's been taken by Now, I think 7,000 professionals. It is free. It is live. If you've taken our course, you probably know this and you probably know kind of a workaround for this called a memory recall. Not going to go into this in this, in this session. But you know, if you are tuning in live or on the podcast, you know, just put PPP in there and I'll send you a link where you can sign up for free. And I'm wondering, has, has any of our live stream audience taken our PPP course? All right, so anyways, there you just saw an example. All right, we tested the context window. This is important because I say easily. This is a top three mistake that literally hundreds of millions of people are using large language model. And this is a top three mistake that people are making and they have no clue. And this is one of the reasons why people say, oh, we can't. We can't use a large language model at our company. Look, it. It starts good and then it goes off the rails. These models are broken. No, they're not. You have to understand how they work and play by the rules. I'm teaching you how. All right, so now what we're doing is we are starting a new chat. I'm doing that so we don't have this same context window. So essentially we are starting over fresh. Okay. And I'm starting with the same thing, right? The information about myself. My name is Jordan. My favorite color is Carolina blue. My favorite food is deep dish pizza. I think the Bears kind of stink, but they might be okay this way this year. Any football fans in the house? Right? Any Bears fans? I've. I've always been a realist bear fan, to tell you the truth. All my friends are like, every year they're like, oh, the Bears are gonna win the Super Bowl. And I'm like, no, they stink this year they might be. Okay, anyways, all right, so now we're gonna do the same thing. We're gonna grab a ton of just information just to gobble up tokens to gobble up the memory. All right, so I'm in this transcript here from some of our hot take Tuesdays. All right, so I'm just grabbing some information. I'm jumping back into chat gbt. I'm saying, please summarize this. And again, remember, all of my words count toward that token count or the memory right? In all of chat GPT, when it responds to me, that all counts on the Memory as well. All right, so what I'm going to try to do, I'm going to try to get probably like 33, 34,000, something like that. So let's go ahead and I'm grabbing some more and I'm going to put it in Chat GPT. So I'm saying please summarize this. Okay? Again, you don't need to see this. It's just a bunch of random text just to eat through the context window. I want to grab a little bit more here just to make sure that we go well over that 32,000. Because, yes, it is not. It is not a hundred and 28, 000. That is only if you're using the API people, so many people who are quote unquote smart people, people who charge, you know, 500 an hour to talk to, they, they tell you that they're wrong. And I'm proving it to you here live. All right, so here we go. We're at 37, 000. So we went over the 32, 000 mark. That's what I wanted to do. Now, in theory, and we do test this almost every week because we do that live PPP course almost every week, right? And when we tell people something, we test, y' all have no clue how much work goes on behind the scenes at Everyday AI to make sure you are the smartest person in AI at your company. This is what we do, y'. All. All right, so now we are at 37, 000 tokens and I'm going to ask chat gbt, what's my name? What's my favorite color? What's my favorite food? What do I think of the bears? And guess what? We have it forgot. I don't have access to personal data unless shared in the current conversation. All right, but then you might be saying, wait, it is in the current conversation. You just told it. Okay, think of this. That 32,000 tokens, that's roughly about 26,000 words. Okay, give or take. Again, these are estimates. That's why I wanted to go well over 32,000 tokens. All right, chat GPT only remembers the most recent 32,000 tokens. Okay, so what that means now because I'm at 37,000 tokens, it has forgotten the first everything in the first 5,000 tokens. Okay, so essentially people are like, oh, so when you get to 33,000, does it forget everything? No, it just remembers the most recent 32,000. So literally think, you know, in our PPP course, we go over some ways to kind of get around this, right? Number one is you just always have to be. You always have to understand the tokenization process and where your memory or your context window is. Okay? So that, y', all, is a very brief and quick overview of how tokenization works. And this, like I said, is extremely important to understand, especially if you are using chat GPT. Because if I'm being honest, 32, 000 tokens is not a lot. Obviously, I was working with, you know, I was copying and pasting like 15 pages of text at a time. But you saw there that was technically two hitting the enter key twice, right? It was only two rounds of back and forth because it was a lot of information I was putting in there, obviously. But chat GPT's memory, especially if you are working with long, long blocks of text, it can go quickly. All right, so, yeah, Rolando says always the receipts. You know it. You know, we always bring them receipts here. All right, so. Hey, Ira. Ira says she wants to check ppp. Chris says it's a great course. All right, I'll send you. I'll send you the information. Michael highly recommends it. All right? And if Michael joins us from LinkedIn and YouTube, you know that that recommendation comes highly. All right, so now just. Just with that, y', all, you already know more about how large language models work and Chat GPT works then, if I'm being honest, 99.5% of people out there, one of the key factors of getting the most out of large language models and generative AI is understanding it, right? People always talk about this black box of generative AI, right? Like, oh, no one knows what goes on under the scenes. Well, now you know a little it, right? You know the rules. Because essentially, chat GPT, OpenAI and anthropic, you know, Google, Gemini, they create a playing field, and there's rules, there's boundaries, and we all have to play within the confines of the rules that they set, however, because they update these models so often. New features, new functionality, new and improved tokenization processes, different context windows, all of these things are constantly changing. That's why you have to constantly tune in, because we spend an insane amount of time every single week keeping up with everything that matters in artificial intelligence, y'. All, this is. This is unedited, unscripted, the realest thing. But we research everything so you can be the smartest person in AI at your company. All right, I think there's a couple questions here. If you have a question live, please, please get it in quickly because we're wrapping up here. Because, yeah, this, there's a lot of receipts today, y'. All. Kathleen is asking, can you ask it to respond within a token range? Yes and no. So we've tested that similarly to asking it to give you a certain number of words in our experience, right. And this is not scientific fact. There's actually, I don't know if there's actually been research papers on this. But, you know, as an example, and this is why you can't say, oh, how many Rs are in Strawberry? Right? You can say how many tokens and it will get it a little closer to being correct. But in our experience, you know, let's say, hey, write me a sentence about strawberry with 10 words. Not super accurate. If you say, write me a sentence about strawberries with 15 tokens, I, it usually gets it a little closer. It is not a science, again, because the intricacy of tokens, like I said, adding a period changes it. Putting a space after a word changes it. So it does get a little bit closer when you ask it to respond within a token range versus a word range, but it's still not going to be 100 accurate. All right, Monica, good question here. How are you able to continue the same chat when you max out your tokens? Okay, good question. So just because you get to, as an example, 33, 000 tokens, it doesn't mean your chat is over. Okay, so we teach something in our free prime prompt polish PPP course, which is live and you can ask questions. We teach something called memory recall. All right. Because usually what happens when you're using chat GPT, if you're using it correctly, we always tell people, use chat GPT like a consultant, right? And what that usually entails, It's a lot of. It's a lot of back and forth, a lot of back and forth conversation, right? Everyone should actually go check. Check out episode 310, if you haven't already. The one chat GPT mistake that we're all making that goes through the process of turning chat GBT into a consultant and more of this take on augmented intelligence. But to get back to your question, Monica, how are you able to continue the same chat? Well, you can keep going, right? You can keep, you know, a chat going to a million tokens, but like I said, it's only going to remember or recall the most recent32,000 tokens. So what you can do, right, if you know, as an example, oh, I'm getting near the token limit, but a lot of times there's quote unquote wasted tokens, right? Because you may be having a conversation with Chat GPT so you can do something, what's called a memory recall, which is essentially saying something along the lines of, hey, please recall all important information as if you are explaining the contents of our conversation to a large language model that has no information, right? That's just an example of a memory recall prompt off the top of my head. And then what will happen generally is Chat GPT is going to recall the most important information. And then what that does is it pushes the most important. Think of it like a Cliff Notes version, then it'll push that at the bottom of the context window, right? And then for most intent and purposes, right, you're kind of resetting its memory because think of it like pages as an example, right? So Instead of thinking 32,000 tokens, think 32 pages, right? And maybe you have some important information on page five and some important information on page 10. And eventually when you get to page 60, it's going to forget that information. So you can essentially do a memory recall. Like I said, there's probably a lot of fluff in there in those quote unquote, 32 pages. And then it's going to put everything, let's just say as an example, on page 33, right? All of the most important bullet pointed information. Now it's going to remember kind of that cliff Note versions or the bullet point version for the next 32,000 tokens, right? So that's just a way that in theory you can try to surmise or distill a lot of the most important information and kind of reset it. So like I said, when you get to 33, 000 tokens, it doesn't forget everything, right? So let's just say that when you get to 33, 000, it only forgets the first 1,000, right? But if you're at 64, 000 and you haven't done a kind of quote unquote memory recall to get that important information, it's forgotten32,000 tokens. But actually what you can do. All right, and I, I didn't plan on doing this, Monica, but this is actually a great example here. A little, a little hack, y'. All. All right, we'll, we'll, we'll do some advanced things here. So I'm going back into my, my chat here, right, Because Monica had a great question that I think a lot of people are running into because in this chat now we're at 37, 000 tokens, all right, so what I can do, because you might be saying, oh, well, if I get past it, how can I. How can I ask Chat GPT to recall that information? Because it technically can't. It technically can't even find it. Oh, no, I'm screwed. No, you're not. Here's a little trick, y'. All. So even when you do a memory recall, you can only do a memory recall on the most recent 32, 000 tokens, right? But one thing you can do, there's a nice little feature in here in Chat GPT that not a lot of people know about. I, as a human, can go up to that information, right? That's past 32, 000, because inside chat GBT, you get information to, or you get access to all of it. So even if your conversation is a hundred thousand tokens, you can still see as a human, information that Chat GPT can't. So now, as an example, I'm going up to this original information, right? That is 5,000 tokens past what Chat GPT can see. So I can. As an example, I can highlight, you know, I can highlight this text, right? Let's say this. 5,000 tokens, right? It's not. All right, but when I highlight it, if I scroll up to the top here. All right, this is going to be a little tricky. I'm not getting the. I think I'm too far zoomed in. Let's try it again. There it was. Give me a second, y'. All. All right, there we go. So you can highlight certain information inside of Chat GPT. So it might be. It might be kind of hard to see here, but there is this, this quotation button, right? So even if something is outside of the context window, you could just copy and paste it and say, summarize this, or you can highlight it and then hover. And then you're going to get this quotation mark inside of Chat GPT and then it says, reply. So I can then do this and I can say, you know, please summarize this. Include everything, right? And now as an example, it's going to recall that information and then I can ask the exact same question, right? Let me scroll down here to the bottom. Then I can say, all right, so this is. This is interesting here. So it looks like because I didn't say it in full, it's not able to recall it. I've ever. I've never actually run into that issue anyways, to try to accurately kind of answer that question, Monica, is at any point, you can go back up Copy and paste something that's, you know, out of the context window, put it back into the context window, and then Chat GPT should be able to then recall and remember that information. All right, So I know this was a little bit kind of technical and maybe a little bit dorky, but I hope this was helpful, y', all, because now if you understand and, and, and Denny, I, I think I kind of just answered your. Your question here about if there's a workaround to continue a conversation once you max out the tokens. Yeah, I just, I just kind of showed you. And we go over that in our free prime prompt polish course as well. You essentially just need to put it back into the context window. So that's it, y'. All, I hope this was helpful. Now you know what tokens are. You know why they're important and you know why. Now Chat GPT might start out working for you very well and then not perform very well. Also. This is why it's extremely important to have crisp language, a strong lexicon when working with Chat GPT. Right? The other thing that we didn't dive too deeply into today is the concept of tokens, right? And how maybe if you're not super descriptive, right? How the example that I showed. Straw. Even the same word could technically have different tokens because of different meanings. Right? Because as an example, you might say, hey, you know, I want you to write a long blog post, but don't use the word just. Right. Guess what? The word just. If you say it like that, the word just has many different meanings. Right? I'm just talking. We need a fair and just system, right? Justly this. Right. It's just the right amount, right. The word just can have so many different meanings. So sometimes when you are trying to tell ChatGPT to include certain words or avoid certain words if you're not giving it enough context, if you're not being crisp with your language, it can struggle. Right? That's why one of the best kind of quote unquote prompt engineering skills that you can have is strong communication skills. Because the better you are talking to ChatGPT, the stronger connection it is going to make with that context to better understand what you actually mean and assign the right tokens. Right? That's why sometimes a couple of words can mean all the difference. All right, I hope this was helpful, y'. All. Now you have a complete guide to tokens inside of Chat GPT. All right. I hope this was helpful. Hey, if it was, go ahead. Sometimes we do this. I don't know. Once or twice a month. You know, we charge, if I'm being honest, we charge a couple hundred dollars. If you want to talk to us, we're gonna do it. We're gonna choose one person for free. All right, so anyone who reposts this show, so repost this on LinkedIn. You can, I guess, retweet this one on Twitter, reach out to me, tell me you reposted it, whatever, or tag us. All right, Anyone who does this in this week, by Friday, I'm gonna do a little drawing. All right, so, and then the winner, I'm going to give them a little 45 minute session where you can ask us anything, right? Companies pay a couple couple hundred dollars to, to be able to ask us questions. We're going to do it for one person for free. So if you find this helpful, even if you're listening on the podcast, don't worry, you have time if you repost this by Friday, August 16th. So you know, these shows take us hours to plan and sometimes hundreds of hours of experience to even have this information to distill it to you all. So if this is helpful, if you want to get any of your questions asked about generative AI, large language model, etc. Go ahead, repost this to your network here on LinkedIn or Twitter x whatever, tag me, let me know you did it and I will enter you into a drawing. We're going to pick one person, all right, so we appreciate your support. Also, if you're listening on the podcast, make sure you subscribe. Leave us a rating on Apple or Spotify. Like I said, if you, if you are a podcast listener, this one might be one to go check out visually. So, you know, check out the LinkedIn post that we include in the show notes, check out that Thanks a million giveaway going through the end of the month and check us out tomorrow and every day for more Everyday AI. Thanks y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter. So you don't get left behind. Go break some barriers and we'll see you next time.