Meta's Code Llama 70B is an open-weight model that beats GPT-4 on coding benchmarks. It comes in three variants, runs locally via Ollama with a 38 GB download, and the video shows quick demos in C and HTML before a full GPT-4 comparison.
Questions this video answers
77 (HumanEval test, as reported in the video).
67.0 (HumanEval test, as reported in the video).
The same license as Llama 2 (per the video).
1 trillion tokens (500B pretraining + additional 500B for 70B).
Approximately 38 gigabytes (download size reported).
Chapters
So Meta AI just released their biggest Code Llama model. This is the first open weight large language model that is able to beat GPT4 on coding related tasks. They updated their original blog from August 2024 that introduced the Code Llama series of models. The latest update is regarding code llama 70 billion model, which is the largest and best performing model in the Code Llama family. Now this is going to be available in three different versions. The first one is the code LLAMA 70 billion model that is foundational model which can be further fine tuned for any specific tasks. There's a specialized version for Python and then there is an Instruct version which is fine tuned for understanding natural language instructions. Later in the video I'll show you how to run this locally on your own machine. The Code Llama series of models are not a new architecture, but it's basically built on top of llama 2. According to Meta, in their own benchmark testing, Code Llama outperforms state of the art publicly available LLMs on code related tasks, which is very impressive when you compare it with GPT4. So the original GPT4 that was released by OpenAI had a score of 67.0 on Human Evolved data set. Human Evolve is a popular benchmark for programming. In comparison, this new model is able to achieve a Human Evolved test score of 77 point, which is substantially higher than any other open source large language model out there. And it's higher than the score reported by GPT4. But keep in mind, this score is for the original version of GPT4. We don't know the Human Evolve scores for the latest version of GPT4s. Now this was released under the same license as llama 2, which means this can be used for both research as well as commercial purposes. Before showing you how to run this locally on your own machine, let me walk you through how this was trained. So all the core Llama models are based on the llama 2 architecture, which are further trained for 500 billion tokens. When it comes to 7 billion, 13 and 34 billion parameter versions. For 70 billion version they use additional 500 billion tokens, which brings the total tokens to 1 trillion tokens. For the Python version they used a Data set of 100 billion tokens for supervised fine tuning. And then since this model is supposed to have a longer context window of 16,000 tokens, they used additional data of 20 billion tokens. For the instruct fine tune version they used 5 billion tokens and for the base version they just used 20 billion tokens. Additionally, in order to access the model from Meta, you will need to fill out this request access form. But if you don't want to do that, this is already available in the hugging face format. And by the time you see this video, the bloke probably is going to have a quantized version as well. Now let me show you how to run this locally using Ollama. So first and foremost you will need to download Ollama from their website. So here I'm running this on a Linux machine, so you just need to copy this command for Mac os. There is a downloadable file and support for Windows is coming soon. Here I'm using a VM for Mass compute and I have partnered with them on my local GPT project. If you're interested in that, check out link for more details. After running that command, Ollama is installed on our machine. Now, in order to run this 70 billion model, we're going to use this command run Olama, code llama and then 70 bill. You can also select specific quantized version. So if you look here for different models, there are different quantization versions or quantization levels. So here are the different quantization levels for the 70 billion model. So if you're interested in a specific version or specific quantization level, you can choose Downloads. Here it downloaded the model. Just keep in mind it will take around 38 gigabytes. And now we can start typing our message in here. If you don't have access to hardware, you can use the Perplexity lab version of Codelama 70 build instruct model. So here I asked the same question. Write me a function that outputs the Fibonacci sequence. And here's an implementation in C. Here's another quick test. So I asked it to write HTML code for web page that has a single button. Whenever we press the button, the background color is supposed to change to a random color and it's also supposed to show a random joke. So it wrote the code here and I copied the code. Okay, so we pasted the code here. Let's click Run. So it does show a button that says click me. And yeah, it changed the color as well as it should. A joke. What did the zero say to the eight? Nice belt. Yeah, like the joke is not that funny, but at least it's doing what we asked it to do. So this is pretty cool. I'm going to do a more comprehensive comparison between this model and GPT4 since it's supposed to be on the level of GPT4 for coding related tasks. Which is pretty impressive for an open weights model. Okay, here's the implementation from the ollama version of Codelama 70 build model for our prompt right me a function that outputs the Fibonacci sequence. Now in this case it is actually using Python which is pretty nice. Anyways, this was a quick video on code llama 70 billion model. Really exciting to see the progress that we are making. Let me know what type of videos you want me to make on on this specific model. Would love to do a comparison between this and GPT4 so stay tuned for that. Thanks for watching and as always, see you in the next one.
