Intrusion Prediction Explained for Beginners | Cyber Security + AI

So hello friends, today I'm going to talk about the super important AI technology which is must to understand in 2025 as well, because this is making so much more in AI and this has contributed a lot in the AI market. So let's understand it on my screen. So when I talk about AI and the models and the multiple parameters, then attention plays a major role. Attention is what we are going to discuss today in this video. So before I jump onto the attention, let's first understand the popular architecture that we already have in market. So for example, you might have heard about RNN, CNN, and MLP models. Yes, these are the popular architecture that we have till now, but these models are having some cons. It means there are some disadvantages of using it. Now what are they? For example, if I ask you, or I ask the model to read the popular architecture, how will you do it? As a beginner, or as a human even, we read it word by word. We go popular, we read architecture. So first word, and this is the second word. When we jump on to the second word, we remember the first word that we read. Same thing happened in RNN, CNN, and these models. Now how this model works, this model will go word by word. It will read first word, second one, and also remembers this last word. This is what actually happens in the model, but this makes the process slow. Which process we are following is we are following step-by-step process. It means a step-by-step algorithm where we go word by word.. And when we go word by word, it becomes slow. And when it becomes slow, we can say that we cannot use it for long sentences. So for long sentences, they cannot be used. And even they, they can be used, or they will be used by someone, but they will eventually give the lagging, or even there are multiple different things that might happen while you try to train it. When you will try to train it, it will take so many days.. And even though they want to get trained perfectly, so RNN models are having so many discrepancies, we have to figure it out. So researchers found out a way which is called attention. Attention is a technique which is used in the AI market to give the better performance, better accuracy, and to do the things faster. So self-attention and attention is important. Just like in real world, as a human being, attention is important. Same way For models also, attention is important. So let's understand what is attention in terms of machine learning. Now, what is attention? Attention is basically a way, a mechanism which is used by the Transformer models to give us the best result in terms of translation. Attention is like a smart way of the model to decide how much focus to give each part to the input. Each part of the input is requiring how much focus is what attention decides. Attention gives the spotlight. Just like in concerts, you might have seen singers are getting spotlight. You might have seen in the dancing competition or something, the dancer gets the spotlight. Same way in attention, in machine learning model, attention gives the spotlight to the word which needs focus. So attention is a smart way which helps the model generate something better and understand something better. So it takes 3 things. Attention takes 3 things. Now attention takes 3 things. Attention takes query, attention takes keys, and attention needs values also to work in the better way. It works by taking these 3 things. What is query? Query is what you want to find out or focus on, like it can be a question. When I talk about key, it means it is the list of labels or the pointers for each part of the input. Values are basically the actual input that you are providing, or the actual data that you want to use. Now what will happen? The model compares the query to each key to see how well they match, and then they assign the weight. It means the importance of that weight of that word. For example, the attention in this question, like, what is attention? Is the spotlight should be on attention? Same way What attention mechanism will do? Attention mechanism will give a weightage to each, each words, and then according to the values that they get, we combine the value and the weighted by their scores to produce the output. So attention works in every sentence. Now I will give you the example of both, how popular models or popular architectures are working and how attention is working. So for example, let's say this is the word How RNN and CNN will take this sentences— let's suppose the sentence is, the cat is sleeping. So they will take each word at a time, right? So they will go word by word. So the cat is sleeping. Now when we jump onto from first word to second word, we have to remember the last word. So it becomes the cat. Then we jump on to the third word is, then it will become the cat is. So we combine every time we jump on to the other word. Though it becomes slow, we cannot do anything with the long sentences. So what we have in the market, as I told you, it is attention. Now, how attention works, or how attention type of a model works, is let's suppose this is a sentence that we have: the cat sitting on the mat chased the mouse. It means first is there will be a spotlight, there will be a weightage given who is sitting. Who is chasing? It means there is certain queries, there is certain question which are being raised. Where is cat sitting? So these are certain type of a question that comes from this particular statement. So what the entire attention mechanism will do, will keep a spotlight on any word. Let's say, let's say these two words, okay? No matter how many words are there in between, but it will take it up it will take those words, will try to create the relation between them using the queries that you have passed. That the mechanism, attention mechanism, is created. So if I take another example, let's say she loves chocolate. So how mechanism, uh, attention mechanism will work is she loves chocolate, let's say. So what is this she loves chocolate is the query could be the current word position where, uh, like we are trying to translate, and key is each word in the input and value is the meaning of those. So attention mechanism will do what? The attention mechanism checks which input keywords are important. For example, she might find like— let's say it might find like she and loves are highly relevant and focuses more on those while ignoring the less relevant words. This is what it do in attention. So attention is important, and there is a model which is called transformer model, which is using the mechanism of— so there is a model called Transformer model which uses this mechanism called attention. And this is how the Transformer model is designed. In this Transformer model, we have main two things which is called encoders and decoders. Encoders and decoders are making this possible. The Transformer model consists of these two parts, encoder and decoder. Encoder is something that gives the number to the decoder, and decoder is something that decodes it. Simple, but we'll understand it better. For example, let's say, how are you is a sentence which I am passing to the transformer model. Now what will happen? Now transformer model and even the attention mechanism, as I told you, there will be some weightage allotted to each word. So how are you will have some numbering, and that numbering will be passed to encoder. And when it will be passed to encoder, encoder will have some numbers and they will be in the multiple layers. For example, let's say in this particular uh, image which I have, which is having 6 models, okay, 6 layers in that particular transformer-based model. So understand that encoder is something that is made up of 6 layers stacked one after the another, where there are multiple heads, where there are multiple values with each encoder layer. So understand that encoder is a part of the model that takes the input sentence turned it into the set of numbers and will represent the meaning of each word in the context. Now what decoder will do is— now decoder is something that takes the number, that takes those numbers and generate the output sentences. For example, if you wanted to do English to French translation, then you have passed 'How are you?' in English. It will be passed through the encoders. Each and every word will be taken each and every word will be combined. And once every weightage is allotted, like the weightage is allotted to each word, then that weightage will be compared and the spotlight will be given on to the important words. And those words will be passed to the decoder. And once the decoder will take it, decoder will try to fetch something from the French similar to this and will give you the output. This is how entire transformer mechanism works. It uses the same encoder and decoder, but the main thing in this particular transformer model is self-attention layers. Self-attention layers which are stacked on each other and which is helping encoder and decoder to fully connect the neural network and process the information faster and even with the perfection. So transformer models are important. Why attention is important is because it is helping us to work on long sentences. Even if you have this sequence which is like a combination of multiple characters or something, You can use the attention mechanism. It also gives us the better understanding, better context understanding from the sentence. It focuses on relevant part only. It doesn't focus on the irrelevant one, or it doesn't even focus on the, uh, doesn't focus on the useless words. It enables the parallel processing rather than using the step-by-step, which is obviously the, uh, slow one. So there are multiple GPT models which are created. Recently we have the GPT-5. So those are again the modern AI models like a transformer one, which is helping us for the translation, text generation, and much more is coming up in AI. So attention is all which is making it possible. Now if I say about the proof, so yes, this particular transformer model has been trained, tested, and there are some results which are the inferences for you to understand. So we— the, the researcher tested it on the dataset like English to German and English to French, which is on WMT 2020. 2014 dataset, which used that WordPiece tokenization and BytePair tokenization. For English to German, we use BytePair, and for tokenization of English to French, we use WordPiece with a shared vocabulary. Now, when I talk about the hardware and the training duration, there were 4 different type of models that were generally prepared. So if I talk about 4 of them, then this— these were there: Transformer-based English to German, Transformer Big English to German, and Transformer Big English to French, and other sort of models. Now to understand the performance, we have evaluated the model as well, but we'll talk about it later. First, let's understand what we exactly did. If I talk about hardware and training duration, then we used 8 NVIDIA P100 GPUs where base model trained on the lakh steps, which took around 12 hours for training. And big models are trained on 3 lakh steps, which took around 3.5 days. Now when we talk about the optimizers and learning rate schedule, then we have added some optimizers as well to increase the learning rate to form up like linearly. There are for first, uh, 4,000 steps, then decay proportionally to the inverse square root of the step number. Now when we talk about the regularization technique which we applied to prevent overfitting and to improve the model generalization, we have applied the regularization technique as well. When I talk about the results, then these are the results. This is the BLEU matrix, which is an F1 score which we use to evaluate the model. So Big Transformer, which is English-French, is giving the best accuracy of like BLEU score. And this is the model variation. We have tested multiple models. N is the number of layers, D models we have. So we have seen that base model with the 6 layers is giving us the best BLEU score, which is 25.8 within the train step of 1 lakh. If I talk about the big one, which is with the again 6 layers with the training step of 3 lakhs, it gives us the BLEU score of 26.5. 4. So you can easily see that how attention is playing a role, how it is making everything possible for us. So attention is playing a major role in this industry. If I talk about the— as I told you, this is all about translation that we did. So there are multiple parsers which are used, there are multiple training which are used, like SAMMI supervised we do, we did multitask, we did generative, and then we have the F1 score for each and every model that we created. Now why I am saying in the entire video is like Transformer is breakthrough is because it uses the attention, it skips the slow step-by-step processing, it connects each word with other word instantly, which is best for the long sentences. Even if I talk about employing the attention heads, then it quickly understands the relationship and it also trains faster, it performs better than the other model. So it is all powerful things that we have in AI. If I talk about impact which attention have made in this industry is with the NLP. Natural language processing has been revolutionized because of this attention mechanism. And it also— there are multiple advanced AI models which are coming up in like GPT-5, as I mentioned. So there are many more impact in the world of AI. So I hope you got it and you are— you were able to understand it. So till the next video, stay tuned and happy learning.