EP 249: The next AI trend: Small language models?

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

It seems like we just got used to large language models, but now we're hearing more and more about small language models. Like what the heck? What are small language models? What's the difference between them and the big large language models? And when should we be using small models? We're going to answer those questions today and more on Everyday AI. Welcome. What's going on, y'? All? My name is Jordan Wilson and I am the host of Everyday AI and this is for you. We are a daily live stream podcast and free daily newsletter helping everyday people like you and me not just learn what's going on with generative AI, but how we can all actually leverage all of this information to grow our companies and to grow our careers. And I think, you know, the more and more prevalent generative AI becomes in our day to day lives, you know, using in the workplace, we have to become more comfortable with all of these, you know, buzzwords and in all of these acronyms, right? So if you're not already using large language models or small language models in your day to day at work and you live and work here in the US you probably will be soon. So I think it's important to understand the difference between the two and what small models are good for. So we're going to be diving into that here in a second. But before we do, as a reminder, please go to your everydayai.com today's episode. There's going to be a lot of depth and detail. So if you're listening, you know, whether you're walking your dog right now or you know, you're on the treadmill or whatever it is, maybe you can't type down notes. So we always do that for you. I'm a human, I'm a former journalist, I write the newsletter, right when I'm done with this podcast in the live stream. Also as a reminder, y', all, this is live. A lot of people don't know. This is essentially live unedited. It's the realest thing in artificial intelligence right now. So also on the website, if you didn't know, we have more than two, I think 210 episodes or something like that now you can go listen to every single episode, Go watch every single episode, go read the newsletter that came along with every single episode. I, I, I, I argue we are probably the number one source for free generative AI information in the world. I don't know of a single other source, single source that has all this information. So let me know right now. I would love to know for our live stream audience or just let me know also if you're listening on the podcast. Do you use small language models? Do you know what they are? What, what questions do you have? I may not have every answer. Right. Some of them I'll be able to answer live here on the show. So if you are joining us, please get your question in about what small language models are or what questions you have about them. But hey, if I can get to them live, I'll make sure to answer them in the newsletter so we can all learn together. All right, so we're going to talk now about a. What is a small language model? And I'm going to give you 14 facts that you need. Receipts? No. All right. No, no clickbait here. Just straight facts. You know, we bring receipts. All right, so let's start here with small language models. So it's important to know that the definition is always changing, right? As actually large language models get bitter, bigger, what we consider a small language model is always changing. So keep that in mind. All right? The goalposts are always moving in terms of these definitions and there is no one overarching definition that everyone adheres to. So, you know, you could even say, oh, something that we might have considered a small language model, you know, two or, you know, a large language model two years ago. People might be calling it a small model now. So keep that in mind. So previously, a small language model was considered a model with fewer than hundreds of millions of parameters. But now, like I said, that definition is changing a bit. So as large language models like GPT4 get bigger and bigger, right? When we jumped up from, you know, GPT 3.5 to GPT 4, right. As the large language models get bigger, we're starting to call other models. Oh, yeah, these are small now, comparatively, Right. So it's important to know and it's all kind of judged by parameters. All right, we're going to get into here in a second what those parameters are and what they mean. So essentially, large language models have billions to trillions of parameters, allowing complex text tasks of all varieties. Right? So your, your GPT4, your Gemini Ultra, think of those as the ultra, ultra big boys, right, in large language models. And they can perform any task and they essentially know everything. All right? So that's not how small language models work. Okay. Small language models have fewer parameters, making them more efficient, and they are more for specific tasks or to be used locally on devices with more limited resources. Right, we're going to get into all the intricacies but you know, if we want to talk big picture, that's the best option, right? Or that's the best way to think of them. Large language models are the, the behemoths that can literally do anything and everything. And probably most of us actually use large language models like Chat GPT and, and Google, Gemini and Anthropic Claude. We probably use large models like that more than we do small models. All right, but small models are more for specific tasks. So it's not something that you can really get the best results sitting down and asking, you know, 100 different questions from 100 different, you know, walks of life. It's not really what small models are for. They are for, they're fine tuned for specific tasks. All right, let's keep diving in. Let's look at some examples. So again, even when we talk about parameters, because I'm going to define what a parameter here is in a second, but first I'm giving some examples of large language models in small language models and their parameters, right? So a lot of times companies don't even say how many parameters their models are. So even as I throw these numbers out there, some of them are unconfirmed but reported to be true. So keep that in mind. Let's just look at the difference here. As an example, large language models, probably the two most popular ones are GPT4 from OpenAI, which is reportedly about 1.8 trillion parameters. Gemini Ultra from Google, which is a reported 1.5, 1.6 trillion parameters. All right, then let's look at some, some popular now small language models that maybe would have been considered big, you know, three years ago, but they're not, they're small now. So Phi 2 from Microsoft is about 2.7 billion parameters. Then you have Llama as an example, very popular open source model From Meta, about 7 billion parameters. All right, you know, don't we can, we can pick hairs on the parameters all day. Again those are estimates, those are reports, it's, etc. But you know, those are some, some big names out there, right? So we're talking OpenAI, Google, Meta, Microsoft. So you know, there's these smaller, more flexible models like Phi 2 and Llama. Then you have your big, you know, your, your big ones, you know, open AIs, GPT4 and Gemini Ultra. All right, so now let's talk parameters, right? Because that is essentially the difference. That's the difference. There's a lot of intricacies and how they play out differently. But the kind of, the first thing that we look at to separate large models from small models is parameters. So I just gave you the examples, you know, 1.7 trillion versus a couple billion. All right, so here's what a parameter is in very simple terms. All right? So again, I'm sure if you're a machine learning expert with, you know, a decade of experience, you know, you might have qualms with my definition here, but we're talking to the everyday person, we're trying to simplify this. So a parameter, in very simple terms, it's. So in simple terms, a parameter in a large or in a language model refers to the variables that the model uses to make predictions. All right? Each parameter represents a concrete part of the model that can change or adapt based on the data it's trained on. All right, that's an important, that's an important piece, right? Because all of these models and all of the parameters that they contained are trained, right? So which is why obviously these large language models are much more expensive to create. They're much more expensive to upkeep, they're more difficult to train, right? Because they are so much more complex, right? It's like the way I like to think of it is if you know how to work with a large language model, you can essentially ask it anything, right, in the history of existence. And if you know how to work a large language model, you can probably get a pretty decent answer. Not necessarily the case with small models, right? Small models are generally trained in different, you know, categories of work or different, you know, different types of outcomes you may want, right? So as an example, a small language model may be trained or built specifically for a type of customer service, right? To handle just, you know, inquiries from, from customers, right? And it might be fine tuned for a specific use case like that, right? So if you have a small model that's maybe for customer service, right? Maybe it's an open source model that you can, you know, tweak yourself. But let's just say that, right? That small language model that's built specifically and tailored and fine tuned for customer service to respond to customer inquiries, you aren't going to be able to code on that, right? You're not going to be able to develop a website, right? You're not going to, it's, it's not going to be able to spit out images for you, right? That's what large language models do, right? It's, it's the multimodal the multimodality of large language models, right? Being able to input photos as prompts, being able to output, you know, different types of code and multiple language, all of these things. A lot of times small language models, not like that, right? They're built for one very specific purpose or they are just made for smaller purposes. Like okay, this small language model excels at creative writing, right? So can it create an outline of, you know, how the stock market has changed over the last 30 years? Maybe, but that's not really what it's for, right? So again, different use cases, different types of training, different parameters, it changes, right? Can complete difference in what a large language model in a small language model should be used for. All right, and hey, so great, great question here from Woozy. So have any specific products you like that are being run entirely with small models? I do have some, some examples, Woozy, but I'll try to share those in the newsletter. So I don't go off track here. All right, so now let's talk about 14 things that you should know about small language models. All right? And again, I think most of us, myself included, are more familiar with large language models. So as we go over some of these facts and things to know, I'm going to be comparing small models to large models, right? Because that's what most of us know, you know, so think the large models, the, the GPT4, the Gemini Ultra, the Claude anthropic etc or Anthropic Claude. So some, some of the most important differences. All right, so small language models require less computational power, making them more accessible for users with limited hardware resources. All right, that's the most, one of the most important things, right? These small models can live locally on devices, right? The new Samsung phones have Gemini Nano, right? Which is technically a small model, but it lives on the hardware, right? Which technically requires less compute. Because when you're using large language models, people don't like to talk about this, but they are extremely resource heavy, right? There's. We've talked about it on the show before, right? Every couple hundred prompts, you know, it kind of can tell you how many, you know, what is the environmental toll, right, on all of these prompts? Because large language models require a lot of compute power, you know, but small language models don't because they live locally, right? So they're not having to, you know, essentially send your query and compute it in the cloud, which can be very expensive and very resource heavy. Small models, not like that. So even with that, in the same vein, small language models are faster at training and inference due to their smaller size compared to large language models. Right. They're faster. Right. It's faster when there's way fewer parameters, especially when you're, you know, obviously using a small language model for what it's good at. It's faster, it's on, it's on device, it has fewer parameters to look through. Right. So, so think, think of it like this. You know, what's, what's technically faster? You know, if, if you had to read through a 500 page book to, to learn about the history of everything or a five page book. Right. That, that gives you the history of things that maybe you care about. It's kind of like that. Right. It's just faster. Small language models are faster, but obviously way less robust. All right, some more facts to know. Small language models are more energy efficient, which we just talked about. So they're reducing the carbon footprint essentially associated with the training and the running of AI models. That's another part. It's not just the running, you know, large language models that are expensive. It's, it's the ongoing training. There's so much compute. Right. That's why you have Sam Altman out trying to raise 7 trillion. Yes, trillion with a T. $7 trillion for more GPUs. Right. To build a new class of chips. Right. Because these GPU chips power all of these generative AI models, not just your large language models, but you know, all, all generative AI models are run off these very hard to get very expensive GPU chips. So right now, you know, we are, I wouldn't say it's a compute crisis but you know, all, all of this computing power is scarce, it's expensive, it's resource heavy in the long run, taking a toll on the environment. So small models, I think in that regard are important to keep an ion. Also, small language models can be deployed on mobile devices and embedded systems, unlike most large language models. Yeah. So again, that's your edge AI, Right. Or your edge computing. So that's bringing these language models locally to small devices. Right. So you're seeing it on actual phones now and just talked about that with the new Samsung S24 having Gemini Nano. Also reportedly Apple should be announcing their generative AI offering in June at the, the Worldwide Developer conference. So Tim Cook just announced that, I think it was earlier this month. So we are presumably going to be seeing a small language model in an upcoming iPhone. Right. Or maybe in upcoming, you know, MacBooks or iMacs. Right. So that's another important thing to keep an eye on is we are seeing that as well with Nvidia's chat with rtx. Right. They just announced that this or it was actually announced a couple of months ago, but it was just released, you know, in the last like 36 hours. Nvidia's chat with RTX. Right. So we don't know how many parameters it is, but it's a pretty, it looks like a pretty solid small, small, small language model that can run locally. Right. You have to have a certain Nvidia GPU running on your computer to, to run chat with rtx, but the same thing, you know. So this is a big shift that we're going to see because it's better privacy as well. Right. That's one of the biggest things that people are concerned about with large language models and it makes sense. Right. So not just data sharing, but training data. Right. How are these companies using any data that we upload into their systems to train their models? Right. So when you think of smaller models that run locally, they are not sending information back and forth. Right. So it is the concept of running large language model and generative AI locally on a device. So much more privacy, much more security. Right. Just like if you're opening, you know, Microsoft Office, let's just say, and you're not connected to the Internet, that's kind of, you know what it's going to look like in the future if you're working with a small language model on your, on your phone or on your PC or Mac once it comes out. Right. We're waiting on Apple. All right, some more facts. Here we go. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead of. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen. So small language models are suited for real time applications, such as on device language processing, where quick responses are crucial. All right, next fact, next fact. Small language models have a lower capacity for understanding complex language nuances. Nuances compared to large language models. Right? That's the other thing. Large language models, if you know prompt engineering 101, there's really not much in the world that you can't do with a large language model, right? You can translate languages, you can build advanced web applications by just asking, you know, a large language model to code something for you. And, and you can technically build generative AI with generative AI, right? It's, it's. So the, the large language models are extremely complex and they're only going to get more powerful and more robust as we see new models, right. When we see GPT5, you know, and Sam Altman at OpenAI has been saying, oh, it's going to be much better at reasoning, it's going to be able to, to rationale, you know, more, more multimodal capabilities. Right? So these large language models are going to get even more powerful and even more robust, right? And even right now, large, large language models across like MMLU benchmarks, we talk about that on the show a lot. But you know, the current version of GPT4 and presumably Gemini Ultra, you know, are about three to four times better on these benchmarks like MMLU than the average human, right? So right now, large language models for the most part are much smarter, much, much smarter than any one human. All right, so that's important to keep in mind. So it's, it's a big difference here. All right, let's keep it rolling. So another thing you need to know about small language models, they're often used in applications where speed and efficiency are more critical than deep language understanding. Again, tailored applications, small language models can also be fine tuned more quickly and cheaply for specific tasks versus large language models. Right? Y', all, like, even though GPT4 I think is still, I think it's still more powerful, at least right now than Gemini Ultra, you know, we might see that shift, you know, as Gemini Ultra from Google, you know, starts to get a little more stability and just a little bit more improved. But y', all, GPT4 is like almost two years old now, right? Which is, which is crazy to think about, right? But it's also important to know that and to see the difference, right. Presumably OpenAI has been working on GPT5 for years. So these large language models are extremely expensive to create, to train and to maintain. Right. It's like a titanic ship in the ocean versus a jet ski. Right? You can't use a jet ski for everything, but for a specific task, a lot of times a jet ski is much better than a huge, you know, you know, cruise ship maybe that 10,000 people can, can go on different, different applications. Different, different vessels for different applications. All right, couple more things you need to know. You need to know about small language models. And yes, if you do have questions, I'm going to try to get to them at the end. So keep them, keep them coming if you do have them. So small language models, they're much easier to maintain, obviously in updates due to their simpler architecture. Small language models can also be more easily integrated into software and web applications without needing extensive infrastructure. That's an important one. I think every, you know, all these different, you know, web applications and software early on, you know, just jumped on, you know, OpenAI's models because their API was, was good. You know, they've been making it cheaper and cheaper and faster to work with. But I think we're going to see a shift here in 2024 maybe to a lot of these, you know, pieces of software, these, these different web applications instead, you know, using small language models. Because again, as an example, let's just say if you're building, you know, if you're a large company and you want to, you know, have your own version of a model for customer support, do you need a model as big as, as big as, you know, Google, Google Gemini Ultra, or as big as OpenAI's, you know, GPT4. I don't know. You know, you might be better off as an example with a model like Mistral or a model like, like Llama, right? Something that is maybe, you know, more limited, more fine tuned. You know, another thing to keep in mind, which I think is important, important is large language models. People struggle with them, right? Because let me tell you this, if you're using, and I know I always, you know, old man Wilson getting on his porch and shaking his fist at the kids who don't know what a large language model is or how to use it, right? So so much of what you see on, on the Internet and social media is, you know, oh, use my prompts, use my prompts. You know, here's 15 prompts that'll make you rich tomorrow. Those prompts don't work. They literally don't work. That's not how large language Models work. They're too big. They're too big, right? If you tell, you know, GPT4 as an example, you know, you're, you're a copywriter with, with 20 years of experience. That means nothing. That means nothing. You know, for a large language model, because guess what, it has gobbled up all of the information on the open web and closed web, works of art, things that we don't even know about. It has essentially the history of humankind in its data set in its, you know, 1.8 or 1.5, depending on what model you're talking about. You know, those trillions of parameters. So guess what, it's also gobbled up all this information that's bad information, right? People that say, oh, I'm an Expert copywriter with 20 years of experience, guess what? There's a lot of people that say that on the Internet that are garbage and they're not good copywriters, right? So when you're working with a large language model with trillions of parameters and you think you can use these copy and paste prompts and get great outputs. No. Would you get better outputs if you were using a small language model that is specifically trained for copywriting or creative writing? Absolutely right. That's why I think so many individuals, so many businesses, especially early on, wrote off technologies. You know, these, these very powerful and robust technologies such as, you know, chat GPT, GPT4, even you know, Google Gemini Ultra, because they're like, oh, well, I can put one big prompt in here. And it's not fantastic. It's because it's a large language model with trillions of parameters. You can't just put one prompt in and expect something great out because its brain, its big neural network is too big. It's too big. It's not fine tuned for a very specific task. So this is just a small mini rant brought to you by, by old man Wilson. If you're working with large language models, you need to understand the basics of prompt engineering, right? You need to essentially train your chat that you're working with, right? You have to like what Tara is saying here. This is what we teach in our free prime prompt polish course that has been taken by thousands of people. Thousands of, you know, business leaders across the world. We teach them the basics. Most people are using large language models incorrectly. They're using it like it's a small language model, not how it works. Sorry, rant over. Let's keep going. Small language facts you got to know. So small language models offer a balance between performance and resource usage and it's ideal for many practical applications. All right, so good examples here. Small language models can power chatbots, they can power search engines, they can power voice assistants. Whereas large language models are advanced and used for every single task. All right, last couple facts you gotta know here. You gotta know small language models can be used in both cloud based services or by downloading them. Yeah. So that's the thing. You can't download a large language model. It's not how it works. I don't know if there's a single, you know, computer gpu, you know, on any one physical device that you can download these, the entirety of, you know, like a GPT4. There's been people that have, you know, forked it and they've created smaller versions of these large language models. But you know, for the most part, small language models, the big, big thing to keep in mind is yes, they can be downloaded, they can be cloud based as well. Right. So there's great resources out there. We'll mention them in the newsletter. You know, Hugging Face is probably one of the leading resources for, you know, working with and downloading small language models. And you can run them locally on your machine. Right. You don't have to be a tech expert to experiment and to download and to install large language models because that's what they're for. They're for, you know, on device use, for very specific use cases. All right, so I'm going to get to your questions here, but we're going to wrap up and I'm going to ponder with you and let me know if you're joining me live. What do you think the future is for small language models? I'm not 100% sure. Right. I talk about large language models every day. I read about small language models, use all kinds of models. So I'm very curious about what the future of small language models is. I think what we're seeing as an example with Samsung and you know, Google teaming up to bring Gemini nano to the S24 to a mobile device, that's huge. Right. So I think the future of small language models actually is going to kind of rely heavily on the successes or failures of these first couple large scale commercial rollouts. Right. So even if we could, we could count on our hand a handful of, you know, we could call them, you know, high, highly visible small language models. So you have your, you know, your models from Meta. Right. Your, your Llama models, very popular right now, very popular, you know, for people to run these models locally. You know, just talked about Gemini Gemini Nano. I think we also have to talk about Nvidia's chat with rtx, right? And you can use other models on chat with rtx, which is great. Same thing, you can upload your own documents. You know, we haven't even talked about, you know, the power, the power of, of rag, right? The power of rag, which is something that is kind of tied in with these small language models. So it's, you know, retrieval, augmented generation, combining with small language models. So essentially bringing in your own database of information and combining it with small language models, you know, then you can, I think bypass so many of these security and privacy concerns. Right. When you can work with a small model that works on a device, it's faster, it's more efficient, it's cheaper. And then if you can bring your own data in and work with it in a secure fashion, I think the future of small language models is extremely promising. Right. It's almost like, I think kind of, you know, the large language models are kind of like the Trojan horse, you know, it infiltrates all our daily lives and we see how powerful they are and we all start using them. Hundreds of millions of people are using large language models on a daily basis now. But then we are also then concerned, right? And we didn't know, I guess like some of the best marketing maybe for small language models is large language models, right? And people using them incorrectly because we see this, this robustness and how powerful models in general, general, you know, language models are right to turn unstructured data into something that we can use and we can create with. It's extremely powerful. But we are also now, over the last 18 months we've been exposed to the downside, right? And now we're becoming more cognizant of privacy and trust. So I think small language models, it's kind of been a wait and see. But I do see them gaining popularity, you know, with the, you know, Gemini Nano, with the new S24, with whatever Apple's going to be announcing with Meta's open source local models and also with Apple. Right. Presumably we're going to see some sort of small language model with Apple. You know, it could be a large language model as well, it could be a combination. But presumably we're going to have some, some edge AI, some on device large language model in a future Apple offering. So I do think that working more with small language models is going to be the future. All right, that's it, y'. All. I hope you enjoyed a somewhat, you know, random look. Right. We kind of went all over the place on this one, but I gave you 14 facts you need to know about small language models. We talked about the big differences. We talked about what they are from parameters and the future. So if you want more on this, we're going to break it all down in our daily newsletter. So go to your everydayai.com Sign up for that free daily newsletter. We're going to get to some of the questions we couldn't get to Live and more, as well as more AI news, more fresh finds from across the web. Our daily tutorial. Check it all out@your everydayai.com so make sure to join us tomorrow and every day for more Everyday AI. Thanks y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit you your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.