How to Build an AI System part 5

So welcome to our first live learning lab ever. This is super exciting. We, as part of the AI Leadership Institute, we wanted to make sure that everyone had an opportunity to put fingers on a keyboard and actually practice some of the things that we were talking about and learning about in the AI leadership classes that are available. So this is is we do one of these every month. This month, I thought it would be cool for many of the people who have joined us recently are either building a business or in a business. Almost every business has a website. Most of them, if they do have a chat experience, it's probably not powered by generative AI yet, and it can be complicated. So I thought what better thing we could do to have some fun is to each build our own chatbot for a website or business of our choice. So you can choose any business. Technically, you're going to go in and you're gonna put this chatbot on their website. It will be in a test mode only, but it is going to overlay their existing website. So you will wanna pick a business. It could be a local pizzeria. It could be a local daycare. It could be a dry cleaner. It could be any small business. Hopefully one you know and you own, because when— or not you own, but one you know. I am actually gonna use my own company because I'm a small business. So I'm gonna use my own company. You can use your own company as well. But it is nice nice if you end up building something and you are a customer of this business. You could go and be like, hey, I built you a chatbot. You never know. Last time I delivered this workshop, several people from that workshop decided to start their own businesses building these types of bots for their local community small businesses. And they ended up going to the Chamber of Commerce and delivering a talk about this similar to mine and starting an entire business. So you never know where this might take you. But the best thing about it is it will give you experience not just in building a model, but also in deploying that model to a real endpoint. Like, how is somebody going to consume the model that you've built? And we're gonna talk about it responsibly as well. Like, what do we need to do? But I'll go ahead and, and talk through all the pieces. This is a 5-session live learning lab. All of our live learning labs are 5 sessions long, which means that every day, or, you know, if you're watching this on the replay, you'll have 5 opportunities to go in and learn and progress your learning lab. So you won't have to do all of it at one time, especially if you're doing this live with us every single day during the live learning lessons. You will only get to do a certain piece, and then you have to rest and come back tomorrow for the rest of it. Some of it you might get really excited. I'm gonna give you basically beginner to intermediate steps. There are also advanced steps that you can do that you can go ahead and implement. There is no bounds to what you can build using these live learning exercises. But just know we're only gonna talk about kind of beginner and intermediate type steps. So we are today gonna talk about, of course, go through the welcoming part. So welcome. Glad you're here. This is awesome. Thanks for being here live. It makes it so much easier and better and more fun. And I'm also gonna then walk you through kind of the, the structure of the next 5 days. And then our 2 most important parts, the first middle 2 days, like tomorrow and Wednesday, we're gonna talk about how to create your model. And then Thursday and Friday, we're gonna deploy and test our model in, in production, really, but it won't be in production. We will talk about everything you need to get it to production and then talk about the importance of testing, monitoring, and optimizing your model. So super exciting. This slide deck will be available inside the classroom. So if you wanna look at it, but I will tell you I basically described to ChatGPT what I wanted to build. And I, I took that description and put it into Beautiful.ai with zero edits. It generated this deck for me. So just to give you a sense of how I kind of symbiotically integrate using these tools all the time. So if there's anything in here that I don't agree with, I did take a quick look at it, but I'll let you know. But for the most part, it's a great start. So first thing we're gonna do today is we're going to take a look at how you might build a model that's gonna support this small business. Now, you can— I will encourage you to look at building it inside of an existing cloud system like Azure OpenAI Service or Amazon Bedrock or Google's, you know, using Gemini. Any of those types of systems because they're built in the cloud and the cloud is secure. We also, though, have the opportunity to use GPT. ChatGPT is built on Azure AI. Azure AI is secured. It has a bunch of content filters and toxicity and different measurements for making sure that your model deployment is safe and responsible. So ChatGPT is also a good option, but you don't get as much control over the content. We're actually going to leverage a platform, a tool for helping us put all these pieces together. But I think it's really important that you get kind of the foundational components of an AI model and how you build it. So we're gonna talk about how to build a model, how to create an endpoint, how to use the API keys over the next 2 days. Now, everything that you do should be able to be done for free or with your— let's say you're not an Azure AI customer yet, and you go and sign up for a free account. They give you $200 in credits. That is plenty to do this live learning lab. So feel free to do that if you wanna do it inside Azure. I will be doing it both in Azure and in ChatGPT. So the first thing we're gonna do is build the model. The next thing we're going to do is, of course, look at ChatGPT as an option. So I mentioned this before, but you don't have to use Azure. Azure is a cost center. It's a cloud platform. There's lots of other shiny buttons to click on. It can be very confusing. We may just wanna simplify. If it's your first time ever working with the generative model, you may just wanna use ChatGPT because it's very simple. It is running on Azure infrastructure, so it's protected to a degree. You don't have as much control, but it might be a great first step. So if you have Azure, use it. If you don't, start with a generative model like ChatGPT. All of our examples will use ChatGPT and Azure OpenAI. Now, here is the process that we are going to use over the next 5 days. We're gonna start by building our model, I, and creating a deployment inside of Azure OpenAI, and I'll also show you how to do that in ChatGPT. Then I'm gonna talk to you about what you need to do to make sure you're using the right model. I'm gonna pre-select that model for you. Because I just want you to be successful. But many times, this is an opportunity for teams. Like, let's say you're gonna work inside of an organization that's building these. Many times, you might have teams test a hypothesis against multiple models to see how they perform. I use a tool. Many of you might have seen it. I referenced it in a recent, lesson inside of the AI Leadership Masterclass. I use a tool called Poe.com that allows me to test multiple models with the same prompt. So you can test out models before picking one for deployment. But for the purposes of this live learning lesson, we're just gonna pick one, and it'll either be ChatGPT or Azure depending on what account you have access to. Then, and if you don't have access to either of those, I'm gonna encourage you to get free access to ChatGPT. Last year, I gave people access to my own Azure account because ChatGPT didn't have all the features that it needed. Now, just recently, about a month ago, ChatGPT updated to make all of the paid features we all have been paying for for the last year free to everyone. So it really is starting to become an open AI system as opposed to just having OpenAI in the name. So, so definitely sign up for ChatGPT and I'll, I'll help you protect yourself against doing anything that might be harmful or hurtful to your implementation because you are using an open model. But for this demonstration, this demo that we're gonna build over the next few days, ChatGPT is great if you don't have another system. We're gonna then talk about configuring the model. And really, that's a— it, it sounds complicated maybe, or maybe a little intimidating. But to configure an AI model means asking it questions and getting answers, and then educating the model on what good answers look like and what bad answers look like. The machine learning term for this is fine-tuning. Fine-tuning, that we're gonna fine-tune the model on very specific prompt and answer pairs. But in order to do that, we first just have to test it out. And you'll see us doing this. I am using my own company, but I'm probably gonna also build one. I saw recently, so we're, we're an, a family that does pizza every Friday, whether we like it or not. It's Pizza Friday. I've established this pattern. My kids are like, it's Pizza Friday, mom, nothing else. Unless we're, like, in a different city or country. But, but every Friday, for the most part, it's Pizza Friday. So I may end up going to my local pizzeria. It's called Del Vecchio's and helping them build a chatbot so I can actually share it with them because that's pretty cool. And I also, like I said, my heart is for small businesses, for those companies that are trying to make it. And how do we level the playing field, right? Like, companies like Adobe don't need me going and building a better chatbot for them. But a small business that's trying to sell more pizza and, and, you know, make life for their family. Like, let's do that. Let's help those companies. So we'll configure the model, which means not only are we going to test out some of the questions we wanted to answer, but we're also gonna feed it the data that we have access to. Now, what that means for a small business website is I'm gonna go to that small business website and feed it the menu, feed it the frequently asked questions, feed it and any questions maybe I might ask and answers that I want the model to be able to provide so that we're focusing the model's attention on very specific data. And that data, of course, because we're building a website customer service chatbot, will be the information already presented on the website. Then we'll test it. I will encourage you, in the testing mode, we're gonna do this on Thursday, or on the 4th day of this, challenge that we're doing. You're gonna test the model. I, of course, think you should test it yourself, but it even becomes more fun when you bring someone else along. You can post the link to your chatbot into the community. I encourage all of us to go and test out those of us that are building it. Go ahead and test it out for them and let them know your feedback. But another thing I would do, and I do this myself, is I ask my kids. I have teenagers, so I ask my teenage kids to test it. Ask my spouse to test it. So just find people around you that are willing to just talk to a chatbot and ask it questions. The main reason for that is to give you experience knowing you cannot, should not build these systems alone. I always say we want, I mean, it's best if you have pizza eaters and you're building one for a pizza restaurant, find pizza customers like people who we are an entire household of pizza customers. So, so that's great. If I built a chatbot for the AI Leadership Institute, I might ask some of you who are members of the Leadership Institute to actually test it out so you could see, does it actually serve the needs of the people who need it the most? And then finally, there are about 10 or 15 things that happen to every bot over time. I wanna educate you on those common challenges that occur and also show you how you can troubleshoot and fix a bot in real time because that's gonna happen. The most important thing to educate yourself, and of course, if you do decide to go to a customer and say, hey, I can build you a chatbot, is to let them know that certainly in the beginning, the process is very iterative, right? As the model learns, it might make bad decisions. It might pull from the wrong information. It might get a piece of data out of context. Our job is to consistently and constantly educate the model on what's right and what's wrong. And that is an all-day job all the, all the time. So when you build this chatbot for someone, I always do educate them that once you deliver this, you do kind of have to have a person who's gonna care and feed for this model over the long haul. So these are the processes. I'm not gonna go into it in too much depth here. I wanna jump into the first day, but I wanna give you a, a roadmap for the rest of the week. The next 2 days, we're gonna work on these activities. So we're not gonna actually— I'm gonna show you where these parameters are. Some of these are more advanced steps that you can take, but the beginning level steps, are collecting customer data, pizza menus, FAQs on the website, and training the model. And then also optimizing the model. Now, there's one thing, one hyperparameter on a model I'm going to encourage you to play around with. And a hyperparameter are these variables, these elements that you can change in a model that changes the way it, it gets an answer, the way it derives or generates an answer. One of those is called temperature, and I'll talk more about this on Thursday, but you can use temperature to change the level of creativity in a model. In other words, how creative do you want its answer to be? If I'm asking, do you have a soup on the menu with beans in it? I'm not gonna want you to create an item on the menu. Like, I'm gonna want you to go to the menu with zero creativity and just say, hey, it looks like this item's on the menu, or No, there isn't. Like, it's yes or no. I don't want you to be like, yeah, I just handcrafted this, this soup. Here's what it's called and here's a fake price. That's called hallucinations or model errors. And we definitely don't want to propagate that. But training the model is kind of the most fun part because you're going to give it information and then test it on what it knows and then educate it on the good answers and the answers that didn't quite make the mark. So I think that'll be really fun, fun exercise for us to do. The last thing we'll do is we close out the live learning session is we're gonna go into embedding the chatbot on a website. Now, I'm gonna show you a brand new tool. It's called ChatGPTBuilder.io. I think they changed the name to ChatGPTBuilder.ai. They probably bought it or something, but both go to the same place. But this ChatGPT Builder allows you with no code. So I remember one of our members was like, Do I have to be a coder? Do I have to have any, you know, preexisting experience? You do not. If you do, awesome, but you do not have to have any coding experience whatsoever. You are not gonna write any code at all. You are going to instead, you're gonna be given code. You'll have to copy and paste that code. You will have to know where to put it, but I'll tell you where to put it. But for the context of this live learning lesson, I'm just gonna show you how do you embed it and then embed it in a test website. It's gonna look exactly like the customer's website or your website or whoever you choose. But you're gonna overlay that chatbot so you can test it and it'll look like a real experience. Actually putting it in a production website, we won't do, but I will walk you through the steps to do that. You can absolutely do that, if you choose to. The reason we don't do it is because by the end of 5 days, you have not built a production-ready chatbot. You have not done all of the work it would take to research, beta test, alpha test, beta test, customer test, inclusivity, design, racism, sexism, toxicity checks. Like, there is a good couple of weeks at least of work you'd have to apply to this model to get it ready for production. But you will definitely be ready to present it to a customer in order to get them to buy into potentially the opportunities you have this built onto— on their website. So testing and deployment best practices, that's how we end the week, getting you ready for, you know, the world. Now that testing piece does have a couple pieces in it. I would like to— again, this will be day 5. Day 5 is all about optimization strategies. Like, what are you gonna do now that you've built something and it kind of works and you're pretty impressed with it? How do you actually get it to production? How do you optimize it? How do you make sure it's ready for prime time. So I'll show you a couple monitoring tools and techniques that I use, how you can implement continuous integration. We have some agile, agilest amongst us in the community. Maybe they can chime in on this as well. But giving us the ability to actually create an iterative process of building, you know, training the model, deploying the model, getting people to use it, them giving us feedback, and we use that to actually update the training data. And go through that cycle again and again and again. And of course, I can't push you out of the nest as a chatbot builder until you know how to make sure that you are abiding by the ethical considerations that are in most legislative acts that are currently either being decided on now or have already passed, like the EU Act, but also those ethical considerations that are held by companies like Microsoft and Amazon and even Meta in what's called the Responsible AI Standard. So I will educate you on that just to make sure that you can, like, clearly stamp your chatbot with a responsible AI stamp, responsible AI standard stamp of approval before you go into the world. Now, here's the good news. When we finish, here is the test. Most important, in order for you to get your certified chat builder AI Leadership Institute badge, which you will get, and you can post on LinkedIn. It will credentialize the work that you've done. In order for that to happen, you will need to— and you do not have to be on video as I am with you now, but it would be good if you did get your voice to explain. You'll have to build a video, and it's like a— what do they call it in those university programs? It's like a capstone project. So when you're done, you're going to summarize what you learned throughout the course. It doesn't have to be like, on day 1 I learned this, day 2— like, just summarize what you've learned and then talk us through the application that you built. Once we review that, and we will review it, we have AI to assist us a little bit, but we will— I will personally look at every single entry, and then based on that, we'll provide active feedback and anything that maybe you've missed or anything we think that you could do to improve your bot experience. And again, the idea is not that you're actually gonna go to production with this, but we wanna give you a real, real experience in getting proper feedback on these types of systems as you build them. And then you'll get your badge, but you cannot get a badge unless you record a video that you're willing to share with the community. And again, your face does not have to be on it. Do not worry about that. You just have to talk through it. Some of you are gonna be worried about this, and you're like, oh my gosh, talking on camera. So I will also invite you to think you could offer to just come to me directly, and we can set up a call if it's easier for you. But I'm gonna encourage you, this is part of, you know, the, the first thing you need to do is know these technical skills. The second most important skill in the world of AI is knowing how to convey what you know to the world. That's why they're going to hire you for a job. That's why they're going to pick you as the company to work with is how you convey this information. So consider it a critical, important part and step of the process in order to, in order to share what you're learning with the world and start to get employed, whether as a business, you know, selling your services or as an employee inside a company. You're not only gonna need to know the tech, you're gonna need to know how to, to describe it. And it's what differentiates us at the Institute. We like to encourage you to create content, which is gonna be part of your brand building over time. Yes, it looks like it. Okay, cool. So I'm gonna start first, uh, with the hardest part. Um, there's a book, I might even have it in my bookshelf, it's called Eat That Frog, and it basically says do the hardest things first. And the reason why is because we sometimes— I'll just speak from experience— I procrastinate on hard stuff. I'll do all the easy stuff. Like when I'm building a business, I'm like, ooh, Let me design the logo and create the business cards. And like, as opposed to thinking, what service am I gonna provide and how am I gonna charge for it and how am I gonna sell it? Like, those are the hard questions, but I'd rather answer, like, building my website or stuff that's easier to do. So I'm gonna go and eat the frog on your behalf of doing the hardest thing first. And that is, talk you through the— oops, let me just type it in. Is it thinking? Going through the process of getting your model set up. So you'll notice I went to a different URL, so I'm going to copy this and paste it into the chat. Oh goodness, here I am over here. There you are. I'm going to copy it into the chat really quick. I'll also include these in the notes of the video so you'll have access to that. So if you have a ChatGPT subscription, or if you've become a free member, this is— you'll have access to this. And you'll have 2 choices, ChatGPT, which is just the engine. You can just talk to it in the browser, which you're welcome to try and play around with if you haven't done that. But we want the API itself. The API means that I'm gonna build a model that I can talk to, but I want to be able to point it to a website. I want it to be able to— I want a website you know, conversational box to be able to talk to this model that's gonna answer questions for me. So I just wanted to give you— you'll notice when you do this, it's gonna take you right into the playground. Almost all of these systems call their environment the playground. And one of the most challenging things for AI projects is to navigate from the playground to production. We're gonna get, like, 75% of the way there in this conversation, but the AI Leadership Institute, that's what we focus on. So you'll get lots of experience doing that. But the playground allows you now to build a model, and you'll see it has lots of different models to choose from. So by default, this is the model box, and I will try to zoom in on this. Maybe I can zoom in a little bit more. Yeah, there we go. That's a little bit better. So you could see here it's got ChatGPT-3.5 Turbo as the model, and I can now start creating some conversation with this to see what it knows about my pizzeria. However, this is a great performing model, meaning that it's super performant, low cost, low latency, but it doesn't have as much data at its disposal for answering questions. It's also not very current. It also doesn't have a lot of the responsible AI policies and practices built in yet. So if you drop this down, you can actually see more models, and there are lots of them to choose from that you can use. Now, I personally— again, this is if you only have ChatGPT, it doesn't matter. You can use 16K. This number at the end represents the context window. What, what that means is how much information can I pass to the model in order for it to make a decision? In our case, our questions, like, what's on the menu? Are there— what's the kids menu? Like, if, if I'm using those types of questions, the context window is pretty small, right? Like, I'm just using 3 or 4 sentences. If I wanted to pass in a PDF file instead, and that PDF file was 50 pages, I would need one of these larger context windows so that it could read everything in, in order to answer questions. But we're not going to do that. We're going to— the context window doesn't really matter. But you can keep that. Turbo is fine. So you can use this model. So this is one option that you have at your disposal. Then what I'm going to do is show you, and again, it's just a different model. It's kind of fancier model, of course. But if you go into Azure OpenAI, so I'm gonna go to the portal. Oh, it might force me to log in. Nope, I'm here. Okay, awesome. So inside Azure, I created, and you can go here, you'll see there's a resource here called OpenAI. This is a little bit more complicated. So don't worry if you— this is for, you know, the other side of our audience that's a little bit more technical. So this is the part where you can fast forward or play at 2x speed if you want to, if, if you're not using Azure. But in this case, I created this environment for hosting my service. I called it— it was last year, so I built a demo for it. And this is So I have a subscription. In this subscription, I've now enabled this service. So that's step number 1. Now, if you go— let's see here. If I wanted to— let's see, I'm gonna say develop. So over— oh, let me go back. So in the Chat Playground, what you'll notice over here is that it gives you the ability to build it all out. And then down here, you'll see the API keys. These are the— this is the piece of information you're gonna pass into the chatbot builder to say, use this model. I trained this model. In Azure OpenAI, same thing. It's gonna give you these keys. So in order for you to actually build a chatbot, there are two pieces, the user interface that's on the website and the model that has been trained on all the questions. You're gonna have to connect them together. And the way you connect them together is through an API. An API key. So this key literally is the key that the website is gonna use to unlock access to the generative model that you created. It does require— again, it is code. You don't have to write it, but it is a line of code that you will have to copy and paste into the right place, and we'll do that on Thursday. Alright. So first things first, you will have to get a subscription if you don't already have one. I'm gonna actually go back to what's called the Azure OpenAI Studio, and I just usually type in Azure AI Studio into Google or Bing, and it pops right up. Once you're in the studio, it'll look a little something like this as soon as it comes up. And hopefully not too long. There we go. And you'll notice here on the side, it has a couple Playgrounds. You get the idea? So we're gonna go into the chat playground, and it's going to ask you, what deployment you wanna use. So you can see it over here. There are a few deployments to choose from. GPT-4o, not zero, but o as in Omni, is an Omni model. And what that means is that the Omni model can support not just text, which is fine. You can go and copy and paste text from a website. But wouldn't it be better if you could actually get their menu? Like, let's say you're at the store and you take a picture of their menu. You could take a picture of the menu and build an instant chatbot that tells you what's on the menu, or gives you advice or dietary, you know, guidance or whatever. So GPT-4o versus GPT-3.5 or 4, those are not omni models. They're text-only models. If you want image generation or image usage, it would have to— you'd have to go to a separate model like DALL·E to get that. GPT-4o or Omni is available now for testing purposes. When you go in, you just wanna play around with ChatGPT. I'm just gonna go to the chat interface. Many of you have probably seen this before. The chat interface has GPT-4o. It's just not accessible as an API yet. So again, can you buy into that? Absolutely. But for the purposes of the lab, it like the, the learning lab that we're doing, 3.5 using ChatGPT is gonna be perfectly fine 'cause we're feeding it all the information. We're not asking it to generate content from the internet. We're not asking it to learn about anything. We're not. So every either model is fine. But here, like, I can use ChatGPT 4.0. I can, like, I can paste in. If I click on this, I can add in content from my Google Drive, a PDF file of the menu for a restaurant. Awesome. But I cannot generate an API to connect it to a website from here. So that's gonna be a limitation. So we don't wanna do that. Because the whole point of this is to build a model and then connect it to a real website and actually let people interact with it on that website. And that, I think, is gonna be really fun. Okay. So inside Azure OpenAI, if you choose to use that, you'll be in the chat playground. You will have to create a deployment. And so you could see, oh, I, I made it big, so I'm gonna make it a little smaller so I can see everything. So over here in the menu on the side, on the left-hand side, you'll see deployments. These deployments are the deployments that you can use. I have multiple subscriptions. You might as well. So if you build a deployment and can't find it, you might have to go and look at your directories, right? You might have different resources that you've used, I have this resource, but if I switch it, oh, it, it looks like I don't really have any other resources. Alright, good. That's interesting. So I built a different resource. I must have built it in a totally different subscription. So, so that's good. Alright, awesome. So I'm gonna go back to Azure OpenAI, and inside of the studio, let me go back to the studio. It will Hopefully take me there. There we go. Oops, I went a little bit too far. So now inside the playground. Now you'll notice it's almost exactly the same as ChatGPT, right? Oops, not that one. Almost the same as ChatGPT. So it'll have in the chat playground, right, as system prompts, message, Remember temperature I briefly talked to you about? Pat Meeney's here. What? You made it. Cool. Great to see you. So there's other variables. We call them hyperparameters that you can use to change the behavior of your model. But we're not going to mess around with those at first. We'll talk about them at the end of the week. And then we'll also do future sessions on the importance of these in different contexts. But for today, like I said, you can either use this environment for training your model, or you can use Azure OpenAI to train your model. I am— if I'm gonna do it, I would do it here. But I'm also a Microsoft MVP. They gave me an award. So I like Microsoft. They're nice to me. So I use this. But you, you can use either one at the end of the day. Just so you know, from the technical perspective, the model in ChatGPT is the exact same model when you go to the deployment as the one they're using inside Azure OpenAI. Like, they are the same. We're gonna customize them, but they're the same at the— at— it doesn't matter which one you choose. The only difference with the one in Azure is that it's inside my tenant, which means that my security policies, my data governance, my responsible AI practices, my content filters, everything that I've configured is unique to me in Azure. In ChatGPT, you don't have the configuration parameters to even change that stuff. It's open. So that means that you are counting on ChatGPT to protect you. As opposed to building your own protections yourself. So that's something to think about when you're again deploying a model for someone. Okay. So, so now we've got our model in place, right? We, we've built this model. The first, again, just to start you from scratch, you're gonna wanna go to, and I'll even maybe open up an incognito browser for those of you who've never done this before. Right? I'll just type in Azure. Oopsie. Azure— oopsie— Azure AI Studio. So you just Google Azure AI Studio down here. You click on Azure AI Studio. Oh, that's going to be the product. I actually wanted this one. We're not seeing it because you're in incognito window. Dang it. Thank you though. Thank you for that. Appreciate you. All righty, let's see. Hmm. Oh, actually, I was wondering what's going on. Yes. Yeah, yeah. Thank you. I'm just going to sign out actually. And then that way I could show it to you and just see if I can get it to just give me the intro screen. Azure AI Studio. Hopefully it won't automatically sign me back in. So I Google Azure AI Studio, and you— yeah, you guys could see this. Now I go to the Azure AI Studio, which is ai.azure.com, and it automatically signed me in. So that's nice. But it wouldn't say in this corner— I'm gonna make it big again, sorry. In this corner over here, it would say sign in. You would click sign in and it would give you an opportunity to become— and you could do it. It'll give you free credits. If you do not see the free credits option, just stop and send me a message and I'll send you the link that I have that will give you credits. But that's how they get you to be a customer, is they give you free credits. So just be aware that what you're gonna wanna do once when you create this, the first step you're going to want to do, you'll notice here I'm in JusticeGPT. For those of you who don't know or haven't heard, I am the chief AI officer for a new project called JusticeGPT where we're building a justice model for a company to help people who are imprisoned get out of prison. So, and, and I don't mean like wrongly imprisoned, I should say. Like if they're supposed to be there, we want to keep them there. But if they feel wrongly imprisoned or feel like they didn't know the language or felt barriers, and they're stuck in jail and do not know how to navigate the legal system. We want to offer them some help or services. So, oh yeah, there they are. Okay, cool. So I'm gonna go back to the Azure Studio. So in the first thing that you're gonna do when you go to the home screen, which is where it should present you, you will have nothing here. You're gonna— first thing you have to do is create a new project. Once you create a new project, it's gonna ask you, how do you wanna collect all your resources? The reason for that, and you'll see, see, it says create new hub. The reason it's asking you for that, and this is such a good thing, it's evolution because we never had this before. When you create a new project and you can name it whatever you want, I usually name it whatever I'm building, like Del Vecchio's Pizza chatbot or whatever. So all of the resources you build, databases, chatbot deployments, all of it's contained in a single hub. So when you're done, you can delete the whole hub and you won't accidentally get charged for it when you're not using it. So it's a way of organizing all the resources associated with the project so you can get rid of them when you don't need them, shut them all down very easily. So that's something that I would— yes, definitely can. Well, you don't have a choice anymore, but that is a best practice. So you'll name your project. I'll call it Pizza, Pizza Pal. What am I, the namer of things? So I'm gonna create Pizza Pal. It's— I'm gonna create a new hub for Pizza Pal, which sounds like a plan, and I'll hit next. So now it's gonna add in a new resource group, which I can use an existing one, which I have one for every project I've done. You'll notice here I can create a new resource group. And so again, what this is doing is helping you go and build this project and set up— this is the scaffolding that'll hold all the project assets. So you only do it once. So many of you, if you already have a project, you could just put it in an existing project. I personally create a new project and hub for every project that I build, just so I have isolation of assets as well as the ability to easily identify anything that I've started up and be able to shut it down. So I'm not charged more than necessary. And that's happened to me. I've definitely woken up to a crazy large cloud bill accidentally before. Okay. So, Noelle Russell Russell, yes, please. In order to do this as well, we have to— is this free or is this paid? Yeah. So this is Azure's free, like, you can sign up. It's technically not free, but you get $200 in credits when you sign up on Azure. If you wanna do the ChatGPT one, this one's totally free. So this one's completely free. And this, it does require you still to sign up and give your information to OpenAI. Like, you have to get, you know, register for an account. But these are free, free access. Alright. So let me close this so I'm not confused. So those are the 2 options you can build, and either one is fine. The model you choose, in the free OpenAI will be a lesser model than the one in Azure. But that's okay, because what we're doing is not complex. You're not, you're not trying to author new, you know, life sciences work. So you won't be burdened by that. But it is something to recognize that the free version doesn't give you the latest and greatest models in, in OpenAI. All right. So now once we do that, once we build— oops. Once we do that, I'm gonna go back, 'cause you just say next, and it would build all the project assets. Cancel. Once you have that project, it'll then allow you to go in and create the resources. So that's when you go to deployments, which is down here, and you'll create a new deployment. Now I'm going to— when you— in my case, I already have a deployment. When you go to create a new deployment, you will see a message that says your account has not been enabled to use Azure OpenAI. That's okay. It's an automated process, but this is good news. This means that Microsoft cares about every single user that leverages this technology, and they wanna understand what you're doing and how you're doing it. So they will ask, instead of this coming up, you're gonna get a message that says, Apply to get access and you'll just go apply that application process. I will record a quick Loom video so you can watch me do the application process. But the application, I mean, it's just going to ask you, who are you? They will ask you for your subscription ID. So remember, I don't think I have it here anymore. So I'm going to go to the portal. It's in the portal again, the, the, the questionnaire. Instructs you on how to do this. I'm just gonna show it to you so you have a video so you can go back to it if you need it. But in the Azure portal, when you get access, it's gonna ask you for your subscription ID. So it's gonna be here where you can click on it, and it'll show you your subscriptions. Obviously, I don't wanna click on it and show you all my subscription IDs. Not that you would do anything unscrupulous. But, and you shouldn't either. Don't give anyone your keys or your subscription IDs. But this is where you would go to find them. If you can't see it, just Search, type in subscriptions, and it'll pop right up. But again, it'll give you those instructions when you apply for access. It takes 24 hours for that access, which is why I'm telling you to set this up today and why we're not building the bot right now. If you already have it, you can start educating the bot on the use case that you've decided on. But today you just need to set up your infrastructure. So that is the lesson, the, the, the part you wanna do. I'm gonna actually just open up for questions and we can hang out for a little bit. Because you're going to— let me close that out. Okay. Because setting this up, it's a couple steps that you'll have to do. If you're on Azure, it'll take a little longer because you will have to go and request access. But yeah, I just literally created a new subscription to test this out before I gave it to you. I think I did it right. Renee and I, we, we made the idea on Friday. I think on Saturday I created the group, my subscription. On Sunday I submitted request and it's available today. So it's very quick once you do it, but you do have to do it. And I like it. Like I said, they're one of the only cloud providers that force you to register and tell what your use case is when you use the service. And now they're gonna monitor you. They won't care much if you're using it for 1 or 2 calls for a small business. They might start caring if you all of a sudden start seeing 50,000, 200,000, a million requests. They're gonna come asking you like, what, what's going on? How are you using that? Are you being safe? Which I think is great. Okay. So that is step 1 is to create your environment. I think I am going to— yes. So I am now gonna create a dedicated deployment for this pizza pal guy. So you could see, so this is my project. I'm not going to do it in this project, actually. But this deployment name you can customize. So I'm going to actually cancel out of this and make sure I'm in the right project first. Build 24. Oh, I guess I should— no, I should build it. Let's do it. OK, any questions? I'm going to build this project while we're talking. But let me know if there's any questions as we go. Is this the same process that would be in the ChatGPT as well? Yeah. The only pro— the only benefit, I guess, with ChatGPT is you don't have to build a project. You don't need a hub. It's not gonna create a bunch of resources for you. It's— okay. So once you're in here, it's done. Yeah. You're, you're, you're done. Okay. So it's much easier. The, the trade-off, of course, is that you don't get the fanciest, newest model, but we don't need it for what we're doing. So it's fine. Yeah. So when you go into this, yeah, you could. So one thing you could do if you wanted to is create a project. I didn't because this is the— I don't use this very often, but you could create a project, especially if you plan on creating a bunch of them. So that'll— but again, it's, it's not going to require you to create resources or anything like that because there's only one thing in here, which is the ChatGPT deployment. Okay. Thank you. Yeah, you're welcome. I'm going to do that for those of you who are listening. I'm gonna call mine Pizza Pal. I'm gonna create a new project. So I just went to default project, dropped it down, said create. And now I have a new project. So now all of this is gonna be in the context of Pizza Pal. So when I train it, it's in this project. And yeah, I think so that was much easier, I think, than what you have to do on Azure. So let me go do any other questions. And you're welcome to just chime on in, otherwise I will keep building this thing. Yeah, I'm gonna go to all projects and then I'm gonna create a new project. I'm gonna call it pizza pal. And it does have to be a unique name inside of your environment, but doesn't have to be unique across all application projects ever. Which is the case in some, some of these resources. So it's generating my project. Yay! Project created. Awesome. So now what I want to do is create a deployment. And so now you can see in this breadcrumb trail at the top, it'll tell you where you are. So in order for me to actually use it, if I go to ChatGPT right now, it's going to use one of the deployments that I've trained for a different purpose.. So I could use these, but this is like, this one's for a demo project, that I built. This one is for another demo project that I built. So I want to create a model that's trained on just the world that I'm like, the pizza pal thing that I want. So I don't wanna use those. So I wanna instead go and create a new deployment. So I'll create a GPT, a new deployment, a GPT-4o deployment. And again, if you wanna match your friends in, ChatGPT, we could grab the 3.5 deployment if we want to. I'm always like, newer, better, faster, whatever the latest one is. Okay, so I'm gonna pick the GPT-4.0, and then I'm going to confirm, and then it's gonna let me name it. So I'm gonna call it the Pizza Pal deployment. I might even keep the model name in there just so I know which model it is. Model version deployment type standard. I do have a— this is the resource, if you remember at the very beginning, that's the resource that hosts my subscription. So that'll be the same no matter what. It tells you 140 tokens per minute is your quota. Again, we're not gonna be doing anything like that, so you don't have to worry about it. You can increase your tokens, but this will also increase your cost. So I just leave everything as the default. And choose deploy. And then it's going to take a, a while because it's actually creating software, like it's building an AI model that is directly contained inside my tenant that is a copy of that model for me. So you can now see all this nice information. I won't show you the key, but it generated a key. So now when I build my PizzaPal UI, my web interface, this is the key that I'm going to use. It's gonna basically tie my website to this model. So now I can open it in the playground, or I could just go back to the playground by hitting chat. And I would normally just click that button, but I just wanna show you now it should be available. Now you can see my pizza one. Makes sense? So now when I talk to this model, it will be different than any other model 'cause I will train this model specifically. So let's not get out of ourselves. I wanna go in and talk to you a little bit about training the model. Maybe I'll do one thing and, but This is it. This is your homework for today, is just get your environment set up and ready. Get your account set up. In the last— maybe I'll spend 1 minute talking about the first thing that you can start doing, and that is the system message. So this is part of Bot Configuration 101. How do you tell or instruct the bot to do what you want? And so one part about— one part of that— oh, I think I might— oh, I won't be able to share with you. I was like, oh, I have a slide for this. But one, one part of this, I'm actually gonna stop my screen. No, I'm not. I'm gonna keep it here. But one part of building a system message in, there's a concept known as an AI safety system. Every system you build with AI should be in a safe system. And that means that there's 4 levels of safety. One of them is the system message, which basically is how are you gonna prompt the model to do the right thing to make the right choice? So in this case, when I go into the system message, this is the only thing we're saying to it. What I might want to do is give it a little bit more guidance on who it's trying to serve, how we want it to serve, and what we want it to do when it doesn't know the answer. So I'm not going to do that for you because I'll do that for you tomorrow. But you can play around with this, right? So you could just go in and type in, you know, all of my system messages. I usually start with, don't tell lies. It's a good general rule. I tell my children this. Um, uh, you are a, uh, pizza, uh, customer service, but you have deep knowledge of the menu. Nutritional new— oh gosh, information and hours and location of the pizzeria. All right. So that's a good start. Now, over time, as we test it, we're gonna uncover the bot does weird things, and we're gonna use the system prompt to provide generalized rules to avoid bad things from happening and to also avoid it telling the user something that isn't true. So don't tell lies will protect it from, like, making up something. There are a lot more graceful and, like, nicer ways to say this, like, make sure that anything you provide is sourced in the menu provided, like, something like that. I like don't tell lies because it was the first thing I ever told a bot, and it's very nostalgic for me. It was the first system message I ever wrote. And so I, I always, I always use it. But you definitely want something that says care, you know, how are you going to care about the truth? How are you gonna make sure that this model is accurate? And then we'll get into more of these tomorrow. So that's something that you can test out. And then you could just start chatting now. Right now, I haven't trained it on anything. It has no data. So that's also part of tomorrow. Is how do we make this bot knowledgeable? But maybe I could say, are you familiar with— Noelle, don't you have to save the system message? I do. Apply changes. Even tells me with this big box. Thank you. Yes. Thank you for that. Um, are you familiar with— oh, you know what, actually, let's also say, uh, do not answer questions that are not about pizza or the menu or, or the Pizza-ria. And now when I apply those changes and I go and ask it a question, can you help me with my taxes? Oopsie, my bad. I didn't spell it right anyway. Oh, it's not ready yet. My deployment's still happening. Control-E. But like, now I'm going to provide a prompt that directly tests the system message, right? Like, so let me ask it a question that should— I should now, because I told it not to, you know, lie. And I told it not to answer questions that aren't about pizza. If I ask it, help me with my taxes, it should give me some answer that says, I can't help you with that. And I'm still waiting for my, my model to deploy. Let's see here. Do, do, do, do, do. Almost there. Oh gosh, sorry. Um, let's see, let me try it again. It does not— if you— oh yeah, see, so it says if you did in the last 5 minutes, keep waiting. So I have to keep waiting. I'm gonna try it again. Oh well, I'm impatient. Welcome to my bot testing strategy. Like, I'm just like, net Click, try again. Yes, I could also go and actually monitor the deployments if I wanted to. Let's see, where are we? Pizza. It looks good to me. Alright, let's try this guy. Maybe I did. Did I pick the right one? Yeah, it looks good. It looks good. Okay, and the last thing as I let you go, we're in the last minute here, and I didn't hear any questions, but we are going to do Q&A for 15 minutes at the end of every Live Lab session. You can ask the day before, the next day, any, you know, you, you can ask questions across the entire lab exercise. But the last thing I wanted to mention to you as a precursor to tomorrow is we do have the ability to add our data. So this add your data allows us to train the model very quickly by pointing to a source that has a bunch of PDFs in it. We're actually gonna use a slightly different technique, but this is what you could use if you have a bunch of PDFs of menus that the company has provided you. If you're like a consultant and they ask you to build this bot, be like, great, give me everything you've got. And they send you a bunch of PDFs, you put it in a file, a folder somewhere, and you point this to it. And now that becomes its source of data. This is also the first step to a a RAG architecture, a retrieval augmented generation type search, because we're going to basically control the retrieval by giving it a very specific dataset to go against. So that's super fun, but that's not till tomorrow. I know I get ahead of myself. I can't help myself. It's super exciting. Any final questions in our last few minutes here? Noelle Russell Russell, why didn't it save the system message? Oh, because I generated a new Playground. So if I go back to— oh, wait a second, there should be something in here. Pick up where I left off. Hmm, it's not here. Maybe I didn't save it at all. My deployments. Here we go. Pizza. Open in Playground. Chat. Yeah, I don't know why. I must not have saved it. I don't know what happened. I feel like I did. Yeah, I thought you did. But anyway, yeah, I don't know why it didn't save. Maybe because the resource wasn't created yet. So nothing is like, it's not, it's got nothing to save to. Because yeah, that's, I'm sure that's why. Because the system message is technically instructions that are being saved to the model itself. And it doesn't look like my model's been deployed yet. But by tomorrow, it definitely will. So that's a great question. Thank you for that. But yeah, and thanks for helping me save it. Alright, awesome. Let's see, stop screen sharing. Yeah. So we're on our way. So open up your accounts, whether it's ChatGPT or Azure OpenAI. Get to the point where you are in the playground. That is your goal for day 1, is get into a playground. You could start playing around with it. But tomorrow, we're gonna then focus now our newly minted playground to a very specific set of data. And start testing out our model. And then I already told you the rest of the days, but the next day, we'll then start tuning that model. Then on Thursday, we're gonna deploy it to the website. And on Friday, we're gonna optimize and test. So should be fun. I got a question. I'm still trying to figure out what business to build this chatbot for. Yeah. Good question. Do you have any, like, ideas on who I can think of or what companies I can think of? Yeah. So I would— one of the things you could do is just go to like Google and you could like Google Maps and look for a local business, like any local business, a dry cleaner, a pizzeria, you know, other than like not like a fast food place, but like someplace that, you know, is owned by humans. Another thing you could do is any nonprofit. What was that, Pimini? I was thinking of donating it for a dentist. Oh yeah, a dentist, chiropractors, doctors' offices. All of these are really cool options. Yeah. You could also do it for a nonprofit. The thing is, is they won't know that you're doing this because you're grabbing their public data off their website to create this bot. So it doesn't matter. But yeah, just pick a business you care about, a company like that you— but I, yeah, I would just look in like, it's like going to the Yellow Pages and close your eyes and pick a business. But yeah, your own dentist, your own doctor, um, any of those types of business, service business, a, a, a lawyer, um, a lawn maintenance, pool maintenance, like any of these, everybody has customers. Um, and all of them do not a great job of helping us, you know, figure out what we need before they have to talk to us on the phone. So most of these— will that— does that help, Eva? Yes, that was helpful. Thank you. And, um, you're welcome. Where can I find the replay? Because I know I'm still— I have ChatGPT, but I want to try the Azure. Yes. Yes. So the replay will be in the classroom. I don't think I hit save on the classroom box, but you'll see it. It'll say Live Learning Labs in the classroom. It'll be like on the— it's like the 3rd class in. So you'll see it in there. Yeah. And I'll post this. It takes about 15 minutes for it to process. And then I'll post it. Awesome. Will you teach us how to scrape the website also? Like, okay. Yes, yes, that's tomorrow. I can't wait. So once we got the model, then we— and we will— I'm gonna encourage you all to do it live. Like, like, be— have your model up and ready so you can go and put in the URL and watch it do the scraping so we can all do it live together. It'll be really fun. Looks like I'll have to come with my laptop tomorrow. I know. Well, yeah, you and me both, but it's in the morning, so I think it's— so I alternated because half the people could come in the afternoons and half the people could come in the morning. So I alternated so that only— I mean, that means some of us can't come every day, but at least some people will get to make it live. So tomorrow morning, I think at 10 a.m. Eastern, 9 a.m. Central. So I'm going to do this from my hotel room right before I head to the event, and then we'll meet there. All right, awesome. Thank you everybody, have a great day. I'll see you next time. See you tomorrow. Awesome, you too. Thanks. Bye. Bye.