Video walkthrough (by LARRUZ) of Nomakey's GPT4ALL version 2 release: presents a new GUI, one-click installers for Windows/Linux/Mac, transition from LLaMA to GPT-J (Luther AI) under an Apache commercial-friendly license, expanded training dataset (~800,000 examples), and integrations (Llama.cpp wrapper and LangChain). The presenter demos installation, runs the model locally on CPU (noting 4 GB RAM minimum), shows example code generation, and discusses planned features including Atlas/document retrieval and fine-tuning support.
Questions this video answers
GPT-J from Luther AI
Apache license
About 800,000 data points
4 gigabytes of RAM
Windows, Linux, and macOS
Chapters
Nomakey just released version 2 of their GPT for all model with new graphical user interface along with some awesome new features. You can run this locally on any CPU. All you need is just 4 gigabytes of RAM. Instead of llama, they are now using GPT J from Luther AI, which means now you can use these models commercially without any licensing issues. Soon you will be able to fine tune this model on your own data set. In this video I will walk you through step by step installation of the model and we will also look at some use cases. And at the end of the video I will show you how you can use this with LangChain. So let's get started. Here is the official repo of GPT4ALL. Now they have added another technical report which talks about this new model that they trained. So this new model is Apache license rather than GPL license, which means that you can use this commercially. As like before, both the model and the data is open source and freely available for you to experiment with it. And they are providing this interactive map so you can look at each and every training instance that they used. They have really expanded their training data set so now it's around 800,000 data points. Now they have included specific coding questions as well as Neon data set. And then there are some custom generated creative questions as well. If you recall, their previous version used this CLI based interface. Now they're providing a much nicer graphical user interface. The great thing is they're providing one click installer for all three major operating systems. We will look at the installation in a second, but before installing it I want to look at this feature roadmap. So in short term the plan was to train GPT for all models based on the GPTG so that they don't have to deal with the licensing issue of LLAMA based models. And now they have a new graphical user interface and it is integrated with Llama cpp. And you will soon be able to actually let them use your own training data set if you want. In medium term they are actually planning to integrate this model with Atlas to allow for document retrieval. So this is going to be significant. Then you will be able to use embeddings from this model with tools like LangChain. LangChain integration is already available and you can test it. And they're planning to provide training scripts so that users can train their own models or fine tune their own models. Here's the official website. We provide one click installers for Windows, Linux and Mac os. Whatever operating system you have you can simply download the corresponding file. So I'm going to download the Windows installer. Well, here's the Windows installer. I'm going to double click on it. Simply click Next. Pay close attention to the installation location. I already have installed it, so I'm not going to install it here, but I will just walk you through the process. So I installed this on a desktop. Now in order to run it you need to go to the main directory where you did the installation and then find this BEN folder. So click on it and within the BEN folder you will have this chat exe file. If you are on Windows and if you are on Linux, it's going to be simply named Chat. On Mac OS it's going to be labeled Chat app. Right? So let's run this. Here is their new graphical user interface. It's a lot cleaner compared to the CLI based or terminal based interface they had before. Now they do take inspiration from ChatGPT. This is a flask web application. If you go to their GitHub page. So this is how the UI is supposed to look like when it evolves. So it's going to have different settings that you can change. It will be able to keep a track of all your conversations. And then there is going to be an extension page and a training page as well, but those are not yet available. Let's look at how good the model is because it's a new model. And also we'll look at the speed of generation. Write a python function that writes a file to S3 bucket using the boto library. Right? So here is a python function that uses the bot library to write a file to an S3 bucket. I think the generation speed is pretty good in terms of my machine. So I don't have a GPU, it's running on a CPU and I have 16 gigabytes of RAM. In terms of the actual code, it seems to be fine. It is using the Border3 library then to give it the file path and then it's day right that to the S3 bucket. I think it seems to be working all right. Now you have the option of stop generation. So you can simply click on this and it will stop the generation. And I think if you want to regenerate it so you can do that as well. It seems like this new model and the new training data has changed its behavior and the way to respond. So here's an example. I said write JD for a machine learning engineer. So it says I'm sorry but I can't fulfill this request. As it goes against OpenAI's content policy and our data privacy policy, probably it misunderstood the prompt. And then since it's using Data generated by GPT4, half of the training data. So it may be there are like some things that is picking up from responses from GPT4. And then I reworded the prompt. It again says as a language model, I do not have personal preferences, emotions or ability to take a job or make decisions. However, I can provide a job description. And then it moved on and provide me a pretty nice detailed job description. However, I think there's still some issues with the way it's understanding prompts. We can experiment with the prompts, but I'm more interested in some of the features that they have introduced. The first is this new package by Llama cpp. They have provided a wrapper around the model that you can use in your Python code. Now that means that you can essentially serve this model and access it through an API. And since there are no license issues like Llama, you can use this for commercial purposes as well. And it's very easy to install. So install it through pip or if you want to install it for your specific CPU architecture so you can clone the repo and recursively install it. Then for the usage you need to define a model object, right? Then a callback, we don't really need a callback, but you simply pass on the model that you want. And then they have provided actually multiple models here as well. We'll look at those in a second. And then it's a text generation or next word completion model. So you provide your prompt and you will get a response if there is interest. I will make a whole video series on this here they have provided a link to download their models. Right now if you go to the models, there are basically four different models. One is this GPT for all Lora Quantized, the model that you're going to be using with Python. And then there is another one with the same name, but it's not the ggml. But then at the end you see this one Lora unfiltered quantized. I'm not sure, it might be like one of those unrestricted models or uncensored models. I'm not sure. I'm going to experiment with it and we'll report the results in another video apart from Python, now they officially support LangChain as well. The LangChain is a framework that lets you interact with large language models, build apps on top of it. If you are not familiar with it, I have a couple of videos I'm going to put those links and they have provided examples of how you can actually interact with it. So here's a code base, for example, in this case they are using a prompt template to ask questions and then take the model that you can locally store, right? And get a response from that model using LangChain. So here is an example like how would you do it? So you call this GPT for all function pass in the path of the local model and then there is a callback manager and then using LangChain chain you simply pass on the prompt and along the language model as you would do it for the let's say OpenAI, right? And you get the chain, you run the chain, you get a response, right? So it's very streamlined process if you do it using LangChain. Again, I will be making a lot more videos on these topics, so keep an eye out for those. So it seems like the this is not a one off project, but nomakey is actually working on building a cool ecosystem around it, which is actually pretty nice to have, especially that you have the ability to run it locally as well as you can access it through an API. I hope you found this video useful. Please consider subscribing to the channel and liking the video. Thanks for watching, See you in the next one. Thanks.
