Google Gemma Released | First look at the Technical Report

So last week Google released their second version of their proprietary models called Gemini 1.5. However this morning they really shocked the whole industry with the release of open weights model which are called Gemma. It's a family of lightweight state of the art open models. They get inspiration from the same research done for Gemini models. Currently they release two different variations, one is 2 billion and the second one is 7 billion parameter model. In terms of performance on different benchmarks they seems to be state of the art for their category. However, keep in mind that these are not completely open source model but rather open weights model which is very similar to what Meta and Mistral AI has done. In this video we will look at some of the technical details that are highlighted with the technical report that they release. Apart from the release of the models with the models with the model they also released a number of different toolkits and frameworks which are I think very important. One of them is this responsible generative AI toolkit. This is basically a framework for building generative AI applications. So they provide safeguards both for checking user input as well as model responses before showing the output to the users. As you can imagine, Google is big on model safety and has come under criticism for some of the outputs that their Gemini models are producing. Now before looking at the technical report, these Gemma models are available on Google Colab, Kaggle Notebooks, Hugging Face MX text. It also has supports for Nvidia tool including Nvidia Nemo as well as the TensorRT LLM. The Gemma models are integrated within Keras NLP so it makes it way easy to train and fine tune those models. If you're not familiar, Keras is a deep learning library from Google and it's completely open source. It's a potential alternative to something like Pytorch on Hugging face. Both the 7 billion as well as the 2 billion variants of Gemma are available. For both of these models there are the base version or the pre trained version as well as the instruct fine tune versions that are available. Hana In a follow up video I'll show you how to run these models locally using the transformer package as well as Ollama. But here's the code provided by Google if you want to use transformers to run the Gemma models. Now let's quickly look at the technical report because there are some really interesting things in this report. So based on the benchmarks results this model is able to outperform similar sized models on 11 out of 18 text based task which represents really great performance improvement. Here's a plot that they provided for comparing llama 2, 7 billion, 13 billion Mistral 7B and the Gemma model. Now for question answering and reasoning, it performs very similar to the other models. However, where it shines is this maths as well as coding task. It really outperforms everything else by quite a big margin. In terms of model architecture, it's very similar to llama2. So I think most of the improvements are actually coming from the data that is being used to train the model. And for that we need to look at the training data. There are two different variants. The 2 billion model is trained on 2 trillion tokens, whereas the 7 billion model is trained on 6 trillion tokens, which is far more than what was used for training the Llama 2 models. We don't know anything about the training data of the Mistral models. Now this is primarily focused on English data, so there is no multilingual support at the moment. And this is also not multimodal like the Gemini models. Another important thing to notice is the vocabulary size of 256,000 tokens. In comparison, llama2 has a vocabulary size of only 32,000 tokens. And this seems to be using a subset of the tokenizer data for Gemini. So the architecture, or at least the tokenizer, seems to be very similar to what Gemini is using. Next, let's look at the model architecture. So this is based on transformer decoder only. So it's not an encoder decoder model. And they have a context window of 8,000 tokens. Now this is double the context window of llama2, which is 4,000 tokens, but smaller than something like the mixral moe, which is 32,000 tokens. They are also adding some other innovation. So they're using multi query attention. So this is a multi head attention Mechanism for the 7 billion parameter model. For extending the context window, they're using rope embedding. The activation function that they're using is not the normal RELU activation function, but rather they're using this new GGLU activation. So usually in deep neural networks, whenever people do batch normalization, they simply normalize the inputs of different neurons. But in this case they are normalizing both the input as well as the output of each transformer sub layer. It will be interesting to see how much this new normalization technique is actually adding to the performance improvement compared to the other models on the list. In terms of the actual training, they used 2 trillion and 6 trillion tokens for pre training the 2 and 70 billion model respectively. After that they have the instruction tuning step. In which they use question answer data set to further supervise fine tune the model. Now they are also using reinforcement learning from human feedback for model alignment. This is very similar to what we have seen from Meta Mistral AI as well as from OpenAI. They use the RLHF technique for model alignment. There are automated ways such as dpo. Now when you put everything together, here's the performance that you get. It literally outperforms the Llama 2 family of similar size as well as the Mistral AI models of similar size on almost all the benchmarks which is extremely impressive. The major improvements that we see are on programming or coding and mathematics which I think are going to be the most important use cases for these smaller LLMs. It's a very well written report and I really appreciate Google sharing the technical details in a lot more details compared to some other companies. If you want to test Gemma you have quite a few options. Google is actually providing a number of different Google Colab notebooks which will get you started. So for example you can use this directly as a part of Keras NLP and here's a very detailed notebook that they have provided which will get you started. There is also a tutorial on how to fine tune this model within Keras using Lora which is pretty awesome. I will be making some more videos on fine tuning Gemma in terms of licensing. So the terms of use permits responsible commercial usage and distribution for all organizations regardless of size. Probably they are taking a shot at Meta because they put a limit on which organization can use their Llama 2 models. But in order to use the model you actually need to fill out this Gemma access request form which is pretty straightforward but you do need to submit that even if you're trying to access this on Hugging face you will need to apply for access. This process is very similar to what Meta was using for their Llama 2 models. So this was a very quick overview of what Google released today. Will be really exciting to see the community coming up with different fine tuned version of the Gemma model. So you we might see a Gemma Dolphin, Samantha Nose, Hermes and so on and so forth. I will be creating more content on how to fine tune this model, even how to use this as a part of local GPT. If you are interested in that content make sure to subscribe to the channel. Thanks for watching and as always see you in the next one.