Step-by-Step Natural Language Processing Workshop: From Data to Deployment

Sa.

Hi everyone, my name is Alice. I'm a Director of Marketing at Deep Learning AI. Welcome to our Step by step NLP workshop today. How's everyone doing? It's so great to see everyone from all over the world joining us today. I see people from Canada, from Nigeria, from India, from Germany, Argentina. That's fantastic. Why don't we just keep sending all the wonderful messages coming in. Start by saying hi to each other, just sharing a little bit about yourselves, where you're from and what you do. I'm personally very excited about today's event, especially because I know this has been one of the top requests from you all for us to host a workshop. We heard you. If you are here today to learn how to build and deploy a virtual chatbot using NLP and you are at the right place, if not, will weather just stay. I guarantee you will learn something new today. Before we start the workshop, a special shout out to our co host, 4th Brain. If you are looking to uplevel your AI career and not yet familiar with how 4th Brain will be able to help you, I highly recommend you check out their live online instructor led programs in machine learning. And if you are already thinking about enrolling in their programs, I have some good news for you. So for those who have signed up for the event today, 4th Brain is offering up to $9,000 in scholarship towards the tuition. So up to three people will be selected to receive $3,000 each of the tuition. If you'd like to learn more and apply, check out the link below in the description box. Heads up that the scholarship application deadline is coming up on November 30th next week. So fourth brain has an all star team of machine learning instructors today. We're so honored to have one of them, Greg Locknay, joining us. Greg is the lead instructor of the ML OPS program at 4th Brain where he teaches students how to deploy, scale and monitor ML models and in development and production environments. Without further ado, Drumroll. Let's welcome Greg.

Thanks so much Alice. I'm really excited to be here today. I'm going to go ahead and start sharing my screen now and I'm really, really pumped to see, as I saw in the comments too, what we can do in an hour. And I'm really excited to share with you, how to actually build and deploy something like a virtual chat bot, like an AI assistant. And to do this really in about an hour, I think it can be done. And so today we're going to walk through step by step, how to actually scope the problem, how to actually get the data in the model, how to build the web app and how to really deploy things into the cloud. But, but first what I want to do is I want to talk a little bit about, you know, why I'm here giving you this presentation and the background and the lens through which I'm coming into this problem and taking a look at it from. And then I want to talk a little bit about chatbots in general and sort of the ubiquitous nature of these things up and coming in the world. Then we're going to talk just a little bit about some of the things that we should be thinking about as we go to create NLP MVPs in either sort of zero to one startup environments or even if you're working for a larger company and you're thinking about deploying a tool like this. Before we get started, I'm just curious. One of the things that we want to do today is we want to make this interactive. And one of the things I wanted to kick it off with is just this idea of figuring out how many people have actually used a chatbot. We're going to use this aha Slides thing today and we're going to make sure that we are interacting with each other and learning from each other in real time. If you want to interact, if you want to be part of this, go to ahaslides.com/dlabot dlaibot and then, you know, actually just start, you know, smashing those yes, no, or really can't be sure buttons. Now we're gonna have a number of questions coming up and really this is going to be an opportunity for you to ask things that you want to know and an opportunity to kind of learn to see what, what other people think and what other people have done and what their experience is. So I'm seeing some, you know, some great responses coming in. We've got, I'm going to go ahead and start sharing here. It looks like, you know, it looks like basically, you know, the vast majority of people have used chatbot. I think, you know, it's hard to think about exactly all the places that we see these virtual assistants today. And a lot of times, you know, my inclination is that can't be sure is going to rise in the years to come here. It's really going to rise. So I think basically what we're seeing is that everybody uses chatbots even today. And I think that's why, as Alice said, this is one of the most highly sought after things. Today we had computer vision, now we have nlp okay. So yeah, thank you all for that very much. We'll go back to this sort of DLAI bot thing for the rest of the presentation. All right, thanks a little bit. About Me I basically grew up on the east coast, lived in the Midwest for about 15 years, moved to San Francisco recently. I came up as a mechanical engineer with then a material science background that got me really into computation and really into modeling, simulation and data driven design of first research and then development. And ultimately that led me into wanting to get more involved with digital product. And in sort of after grad school when I had this degree in sort of optimization, everybody came to me in about 2015, 16 and they said, hey Greg, where's the machine learning? Where's the deep learning? Where is all of this stuff that I'm hearing about? You keep talking about optimization, but I'm not hearing machine learning from you. And it was kind of this really interesting space to be in because I had actually studied gradient descent like very, very highly in depth solving convex optimization problems throughout grad school. But what then happened is as you all know, that was useful knowledge early on in 2016, 17, 18, even in the past couple years that's less important to know this kind of thing anymore. And so what we're seeing is, we're seeing this, this explosion of ease of use of these ML and AI tools. And that's what really I think is most exciting about the space today. And that really is the lens that I'm coming from today is that sort of entrepreneurship, product focused lens. So we actually are interested today in deploying something that's really end to end. And the purpose of this is to really, if we could frame the context here. You know, this was kind of a prescient thing to have said in 2019 Harvard Business Review article talking about like in the next decade AI assistance will actually become the primary channel, like the channel through which we get not only all information but essentially all of our goods and services. You know, you may have heard of the, you know, we ship to shop or we shop to ship, that's the Amazon business model today. But you know, when the business model actually makes more sense to ship then shop, that's going to happen. And so like, you know, the, what's, what's really a side effect of this is that marketing itself becomes this, this battleground for our attention. And that's sort of where, you know, this, this exciting space of like, of bots and you know, people basically talking to us through program chatbots. I mean this is, this is going to be, this is just going to be crazy. It's going to be absolutely nuts. And then when the dust settles, really, we'll have a handful of general purpose AI platforms left and then most of us will only probably use one and our assistant will be incorporated into our house, into our car, into our smartphone. Going to be nuts. And you know, this is what we heard two weeks ago from, from Louis, from hugging face, right? I really loved this quote from Louis. Like, we use text in business for everything. And this is so true. It's like this is the thing about chatbots and virtual assistants is that if we can kind of automate something that right now a human does talking or texting or messaging someone, then we are going to. And there was another great question asked in this event as well, and it was sort of like framed. How can we build services like Grammarly, how can we build services that are like these kind of NLP powered services? And this is kind of the focus of today a little bit, is to say, hey, well, you kind of have to know about nlp, but more and more you can be abstracted from nlp. And it's actually more important to be able to say, well, what's the front end and what's the back end? And then how does NLP fit in the middle? And how can we actually then take this simple app, simple web app, simple smartphone app and go start testing it with some customers? This would work in a big company, in a small company. This would be something that you could do really anywhere. And you can do it very, very cheaply. And that's sort of the point of today. Today is meant to give some insight into those digital product fundamental pieces of how to take NLP and how to use it. Now, of course, there's this other side and there's this other side that's this burgeoning industry of, you know, the big players in this game. We have aws, we have gcp, we have Azure. And really what we're seeing is they, they gave us machine learning platforms, they gave us AutoML platforms, and now they're all giving us specifically chatbot and virtual assistant focused platforms. So Amazon, Lex and dialogflow, these things, the Azure bot service is the kind of equivalent there. Even recently Microsoft has released what they're calling Azure Power Virtual agents. These virtual agents, these are saying, what they're saying is, you know, this moves beyond hr, beyond sales and quote, to any channel or domain imaginable today. This is wild stuff. This is. Today you could do this. Okay. And so even players like HubSpot, like SAP. These people are getting in the chatbot game through acquisition. And so there's a lot of smaller companies out there. A shout out to chatbot.com, like, that was a sweet pickup at the right time. I mean, these companies are going to, are getting acquired like mad, and there's really like a lot of opportunity space for a lot more players to start coming into this mix and into this fold. But there's this other side and there's this sort of MVP side. How do we get this minimum viable prototype? How do we get this minimum viable product? How do we actually just do this without dumping a bunch of money into an ecosystem like AWS or GCP or Azure? And that's what we're going to focus on today. All right, so let's do it. Let's do it. But first, I want to know how many of you have ever actually built one of these things before? So go to dlaibot and sort of like, let's look at Saw. How many have used one, let's see how many have kind of built one. And for all of you that have built one, if you could just smash the comments with like a little bit more information about what you built, I think that would really help out kind of all the deep learners out there. And this could be a really, a really interesting discussion. So let's check this out. I'm actually, you know, this is pretty surprising, actually, to see how many people have, have really built their own chat bot. And I think this number, again, this lower number is going to go up. We're seeing. You know, this is great. You know, I really, I really love this, this engagement here. This is awesome. Yeah. So, so we're seeing that kind of a lot of people have built chatbots. Actually, you know, what is it, 1, 1 5th, 1/4 of people have actually built their own chat bot. I think this number, even though it's pretty high already, 20% or so, is going to go up. Yeah. So thank you for that. Okay, so. So let's do it. Let's build it. Let's build it. All right, but wait, hold on, hold on. Are we. I feel like we're forgetting something. I don't know about you, but I've. I've kind of jumped into building in my life a lot. Maybe too fast sometimes and, you know, so it's important to not necessarily start with what. Right. But to start with why and to start with the problem that we're solving. Right. To start with the framing of the context of why we're doing this chatbot thing. So I want to do this by telling a story. Okay. And once upon a time there was a product manager named Paige, all right? And Page worked for Dorical. Dorical was a company trying to revolutionize the fortune telling industry. Okay. And they've actually developed a solid fortune telling platform and business so far. It's a services business at this point and they've got a web app and they've got software infrastructure and customers and fortune tellers are actively connecting with one another today. But really, you know, they've only gotten some seed funding. They're still a small team. Page is the first product hire after the CEO. And until they can show investors that the service model is really scalable, they're not going to be able to raise Series A funding. So in short, what they're trying to do is they're trying to move from this services business to this software as a service business. A so called product led company is what they hope to become. So that, that will allow them to show investors, hey, we understand the game and we know what we need to do to become masters of scale. All right, so what that, what they need to do first and foremost is they need to convince investors that they're going to be the company that revolutionizes this industry. There's a lot of players in this game. There's, you know, one of the big things they can do is they can show that they know how to produce real value with the hottest thing out there. And that is ML and AI first products. The first order of business then is to basically help create value with those immediately. And that means looking at the repetitive tasks that the sales team does today. And that is the information gathering on both sides of the platform. Hey, fortune seekers, what exactly are you looking for? Hey, fortune tellers, what exactly are you looking for? Let's make a connection. This piece, this information gathering piece has just been outfitted as the first team. This is the team that Paige is leading, the info gathering team. And this really aligns with more of a rule based classical AI algorithm approach that is going to allow the sales reps to basically do better at what are the highest leverage activities that they can do. So rather than be focused on info gathering, acquisition phase stuff, they're going to go further downstream and become a true customer success team. And that's kind of the real value creation piece. However, they've also got a big moonshot project and that moonshot project Paige is also overseeing. And the plan while for information gathering is to like run it in shadow mode, go through AI assistance, partial automation and then fully automated version to like release salespeople. The AI psychic is our top secret. Build out the future of fortune telling. And if we can position our product to leverage the latest and greatest models. We hear that the super intelligent AI might be coming soon and if so, we, we will be able to leverage transfer learning to build it directly in to what we hope is going to be something that actually accurately predicts people's futures. All right, so today we need an MVP though. And the problem that we're trying to solve is that based on our proprietary market analysis, there are not enough fortune tellers to fill global consumer demand of people who want their fortune told. Okay, so this is the problem now if this is the problem we're trying to solve, we have to look somewhere for the data and the models to get started. If this is the problem we're trying to solve. Where would you look for data and models? I'm interested here. We're going to open this up, you might say anything. You can enter up to three different things. Where would you go look today to try to solve a problem like this? I'm very interested to hear from all of you where you would go look for data and models. There are places that I would go and there are places that we did go to solve this problem. And I love this. Right? Hugging face, killing it. Stack, overflow, YouTube, GitHub. So there's, yeah, there's a lot of places and I think Kaggle is crushing it here. Kaggle is crushing it. I think that's primarily what we're hearing from, from a lot of folks is hugging face. Google GitHub. Yep. Okay. Very cool. Kaggle is king still today, 2021. Kaggle is king. So cool. The stuff we get to learn together today. Yeah, Very, very cool. Awesome. Yeah, keep those, keep those, keep that feedback coming. We'll share this feedback. Yeah. Kaggle is king, basically. Kaggle is king. Okay, so what did our team do? Well, I'm going to let you know what the team did, but I want to let you know too. All the resources that I show today for how to build this chatbot are available in the YouTube live description right now. So if you want to get to work on building this chatbot before you even see the rest of this story. Although I encourage you to stick around and check out the story, then go ahead and smash those links in the YouTube description below. So what happens next? Well, Paige goes to the first data science hire and his name's Jay. Jay is a recent data science grad. He this is his first data science job. He's really not sort of a software engineer. He doesn't love the data engineering side. He doesn't love the sort of ops side. He really loves playing with the data that's already structured and the models. So he actually starts browsing around on Kaggle first. But ultimately he did hear about this new really cool up and coming company. And thinking about it, he decides he wants to check that out. He wants to check out this hugging face company. And so with that in mind, he kind of says, okay, ideally if this goes well and we can get this on the roadmap, there's a lot of cool work to do. Maybe we could even hire another MLE or an mlops person, so I could keep doing my data science. As the team grows, as he's thinking, Paige points him to some information about the customers. Our customers really are the psychics and the psychics are the people who kind of create the culture of the platform in large measure. And so it's interesting that Paige told Jay that the community of psychics that we've had the most success with on our platform is directly from the psychic subreddit, which is the largest psychic community forum on Reddit. And we did actually pull a number of people from other traction channels within Reddit that are leveraging our platform today. But this is really kind of the flavor of the psychic that we want to create. So Jay takes this information and he goes and he looks on hugging face and he, he actually stumbles across a really awesome Reddit focused pre trained chatbot model and it's called DialogPT Medium. And this is trained on 147 million different Reddit discussions. And essentially the human evaluation results on the Turing Test here are killer. I mean this is a great model and look how easy it is. This copy paste of this code looks like it's going to be a piece of cake in jupyter notebook. Now Jay knows that the whole to create the 27 gig structured data set, he's going to have to leverage almost three quarters of a terabyte of space. And he's going to download it over a couple days and pre training and retraining and all of this other stuff to customize the embeddings. This is a lot of work. He knows that what we want here is we want to go 0 to 1. First he starts to test this thing out, because you can test it out right on hugging face. It was super great. And it seems like, hey, okay, yeah, sure. Seems good enough, right? Seems good enough. And so he decides to move forward with this, with this data and with this model. And so he decides to show us exactly how he does this so he can start the conversation with other members of the team. And you can see this video. This video is live on YouTube for you. But we're going to see exactly what Jay does to try to solve this problem. So he uses Windows subsystem for Linux and he uses the Ubuntu distribution. But first he goes to the Hugging Face website. And the Hugging Face website is really easy to navigate. He's just like, okay, I need the medium one. I go to models, I go to dialogpt Medium and let me look here. Okay, Yeah, I guess all I need is that code. So I'm going to go ahead and open up my Windows terminal and my Ubuntu terminal. The first thing that I'm going to do is I'm going to update everything because I'm doing this from scratch. Put in my password. I'm going to go ahead and install Python and I'm going to install pip. I'm going to make sure that everything's good to go and I can install whatever I need to later. I'm going to set up my virtual environment. I'm going to create a new directory that I can do that in. And I'm just going to call it Chatbot Event. Right, we're going to call it Chatbot Event. And I'm going to go ahead and click into that. And, you know, so, so Jay is on his way. The, the next step is actually create this virtual environment and this virtual machine. We're going to call it Chatty vm, the Chatty Virtual Machine. Once we create this, we want to now activate this virtual machine, this virtual machine. Now that we're in, we're in the virtual machine. We went from Windows to Windows subsystem for Linux to Ubuntu. Now we're in the virtual machine. We want to install Jupyter, because we don't even have Jupyter yet. Now that we've installed Jupyter, we can launch it from our virtual machine. We can click the link and we can set up a new Jupyter notebook. All right, this Python 3 Jupyter notebook is going to be everything that we need for this simple code from Hugging Face again. Thank you, Hugging Face. Making our lives easier as data scientists. But we notice. Okay, wait, Transformers and Torch. We didn't install Transformers or Torch yet, so we want to go ahead and open up another terminal. We want to go ahead and install Torch, that's Pytorch that this particular model leverages. We're going to speed this up a little bit. Then we want to install transformers. Once we get everything set, we've got our virtual machine set up. We go back to our Jupyter notebook and the classic restart and run all. And we are approximately now on our way. Okay, so we're getting our sort of fortune told here. If we ask the right question, what are we going to ask? I don't know. Jay's kind of getting warmed up here. Will the sun come out tomorrow? We hope so. We hope so. Okay, so we've got our data and model set up and now we've got our Jupyter notebook set up. Now all we need to do is we need to take it and go beyond now. We need to create this web app. We need to create this ability to look at something beyond the ML model. If we had to do this, if you had to do this today, I'm interested. What would you choose to use? Would you choose for this web app to use fastapi? Would you choose Flask? Would you choose Django? Would you choose something else? Or do you just absolutely have no clue what you would choose? And, you know, if you can't read stuff in here, I apologize for the, you know, the size of the text. But we do have the videos live and I hope that those will be sufficient to kind of zoom in on and to help you out to build these on your own. And I encourage feedback like that continue to keep coming. So this is pretty interesting. You know, we're seeing a lot of our deep learners out there. You know, Flask and I have no idea. Very, very competitive here. I'm not sure which one's going to win. This is, this is pretty interesting, right? I mean, I think, I think Flask is the one that many of us hear about often and it's also the one that we tend to go to as data scientists first. At least today in my experience with the data scientists that I've worked with, I have no idea. Crushing. I have no idea. Crushing. Very, very cool. So, yeah, so this is, this is, I think, speaks to the importance of doing this. So I'm glad that we sort of align this with what everyone kind of expects. Jay takes his, his Jupyter notebook to the. To Brent. And Brent is the software engineer. Brent is the software engineer that does it all. He does back end, he does front end, he does everything in between. He is a true DevOps savant. And he also does front end. He doesn't really like believe in ML and AI. He really feels like it's kind of like all hype, right? He's really only interested in picking up tools, like if they're going to solve the problem and if they're going to solve the problem on the job that he's working on today. He's not a fan of buzzwords, not a huge fan of this MLOps thing, and he really aims at becoming the indispensable member of any team that he joins. He's a long term guy. He's the Brent, right? And he likes to make things happen. So when Jake comes to him and says he thinks flask like flask like, Brent's like, okay, that's fine. I haven't used flask before, but I'll figure it out real quick. Brent realizes though that he has to figure out a little bit of back end, a little bit of front end. And he's like, he hasn't even told me that he's done anything with the front end yet. Let me see if I can go pick something off of GitHub that looks like I could use it. Pretty quickly. Brent finds this Gotham Chatbot that has one star on it. So smash Gotham Chatbot that was created at the right time on Halloween in 2021 for him to leverage. Kudos to you, Gotham Chatbot creator. Really appreciate letting Dorical leverage this code. That's the beauty of open source. Brent sits down at Jay's computer and he says, hey, move over, let me show you how to do this. He says, okay, first what we need to do is we need to get Git installed. Then once we have Git installed, we can actually clone our git repository, the Gotham Chatbot repository. Just check this out. Super easy. Now that it's in there. Look, it's in there with your untitled notebook. J. Great job with your untitled notebook. And we're just going to go into this and we're just going to run it. And Jay's mind is blown because Brent already has this thing up and running. But he's asking him, he's like, hey, I'm Brent, but this bot I am. What am I jealous of? You're not too bright, are you, bot? What is AI? This bot is the Chatterbot bot and it's really not going to work for our fortune telling. So we're going to go ahead and we're going to copy paste this file folder called Gotham Chatbot. We're going to call it Dialoppt. Chatbot. What we're going to do is we're just going to look at both the requirements file real quick and we're going to look at the application file. Brent comes in and he's like, hey, we're not going to need this Chatterbot thing. As I look through, I don't really care about any of this other stuff, but I know that we'll probably need to. I think I saw in the Transformers documentation, we'll need to upgrade this YAML version and then I'm going to add Torch and Transformers. Okay, good to go. Requirements, good to go. Now here's my API. Where was that jupyter notebook that you had before? Oh, yeah, hold on, let me grab this and I'm going to just straight paste this into my little flask app. This is in Just use Notepad. That's fine. I've got VM code on my machine I would use instead. But whatever, do what you want, Jay. I'm going to go ahead and just replace the library installs. I'm going to replace the tokenizer and the model. This is just where we get the bot response. Here's where we just go ahead and get our bot response. Looks like user text is where we should put this input to leverage this ui. It looks like we don't need to spit it out generically. Rather we can spit it out through the UI in this form of a string. Then, yeah, I guess we don't need that. And we just need to tab over. We should be good to go. Let's check out this index because I think it said Gotham Chatbot. We're going to find that in templates and we can open that just in Notepad as well. Let's make it something that looks more reasonable for what we're trying to do. I can predict your future. Ask me anything. I tell fortunes. Great, now we're set up. Brent says, okay, that's it. We're just going to shut that down and we're just going to go ahead and get into the next folder that we just created, dialogpt. And all we have to do is we just have to run it just like we did with the Gotham Chatbot. Again, don't do this for production. J. But it's fine to test with page and with customers. So let's see. Hi, dialogpt. I'm Brent. Hi, Brent. You're brighter than Batsy. I'm not. Well played, well played. Dialogpt. What do you think of Jay? I think he's a good Guy, will I become rich one day? You will become rich one day. Brent's like, yes. Okay, so great. And with that, I want to kind of see. We can take a couple minutes here before we go into the next phase of this thing to see if anybody has any questions that, that might need answered. So we're going to kind of take, like a bit of a. Bit of a break here. If you saw anything that kind of confused you or that you wanted to dive a little bit deeper into, we'll give it just a minute here or so to see if we have any questions that we can answer. And sorry for all of you that cannot see the code. We will. Okay, so go ahead and you can start uploading these things. Let's see. All right, we'll wait a couple more. Okay, let's see what we've got here. So how do I find. So, okay, how do I find data sets specific to the business that I want to implement the chatbot for? So this is really where the rubber hits the road for value creation. I mean, this is where we really have to work hand in hand with the business leaders and with the people who know our customers the best. Generally, in a startup environment, that's going to be your product manager, it might be your user researcher, it might be part of the design team, but you want to understand who the users that you're building for are. This is the most important thing. And so to find data sets specific, this is what takes the creativity, Honestly, this is what takes the human creativity. And this is the real piece that we're never going to be able to automate, quite frankly. I mean, this is the piece. And so this is a tough problem. And when you can solve this problem, that's when you're really creating massive value for your company, for your. The team, and then hopefully for your family, for your community, and for the world more generally. So, yes, a very great question. It's. It's a tough problem. I think. You know how to do this in foreign languages. I think we heard a little bit about this two weeks ago in the ama. This is very tough to do in foreign. Foreign languages. Now, there's a lot of. There's. There's a lot of work going on in that space, but right now, I mean, English is by and large the absolute. The absolute leader in this space. And so I'm not sure if it would be best to sort of train it in English and then try to convert it to something else later. But you can imagine stacking these translation tools on top of these prediction tools in various ways as they both continue to mature. So that's also a great question, you know, can we fine tune the model for a specific domain? I mean, this is exactly where we want to make sure the answer is yes. And we want to make sure that this is going to be a useful thing to do when we send Jay to spend, you know, a month working on this, developing custom data from our customers, flavoring the model with dorical specific information. So we want to make sure that we've got an ability to say we're going to spend this much money on it, it's probably going to be worth it in the end. And so, yes, we can and yes, we should, but we want to make sure that the business case is there first before we start putting a ton of J's time into it. Okay, Very, very cool. Thank you all for your great questions. We're going to have another opportunity for questions at the end of the session here, but for now we're going to continue the story. I see. Do you have a step by step written up for that being such a high question? I am going to write that up and make that a little bit more clear in the GitHub repo rather than just the videos. Jay and Brent continue the story. Jay says, hey, nice. Did you say we're going to use Docker to deploy the web app? Can you show me that, Brent? That'd be so cool. And he says, yeah, we'll use container images like download Docker desktop. We'll keep it easy, we'll keep it GUI based, and I'll show you how to do it on your computer. They continue the conversation and Brent says, hey, remember how you had to set up a chatty VM from the ground up with Python, with pip, with Git, with your NLP libraries and all of that? Jay's like, hey, for sure. And we did that all on Ubuntu, on Windows subsystem for Linux, the second edition. And Brent's like, exactly. We'll effectively use ChattyVM through Ubuntu as our operating system to deploy containers locally. That's what we're going to do first. Jay's like, sounds simple enough. You're saying we'll actually put our containers inside of our virtual machine, inside of ChattyVM. Brent's like, right, just like any other computer here, I'll show you. Brent goes, and. And he's like, hey, just move over. Thanks for downloading Docker Desktop. Let me show you how to do this. I'm going to go ahead and create a new folder. I'm going to call it dialogpt, Chatbot, Dockerize. I'm just going to create a new folder here called Source and I'm going to put everything except the requirements in there. I'm going to show you how to create this Docker file now too. The dockerfile is the key file for Docker. No kidding. What we have to do is we have to create this Docker file first. We're going to take Python from Docker Hub. We're going to set our working directory. This is within our container. Within our container. We're going to copy the requirements document to that working directory in our container. We're going to install the requirements from the document by running our classic command to install requirements docs. Then we're going to copy the entire Flask app, back end, front end stuff. It's all there in Source and we're going to call the application. Then we've got this port that we're going to expose 8080. You've probably seen this before. This is best practices for simple Flask apps. We want host to be 000 so we can have external IPs accessing this thing. What we're going to do is we're going to basically go into now this new folder and we have to call now one new command and it's Docker build. And you see, we're kind of speeding this up here. Once we build it, there it is, we can see it and we can see that it's almost four gigs large. And we can just straight up use the Docker Desktop App Port 8080 to run this thing. We can look and see how it would look in the terminal as it's moving, as it's operating. And then we can, boom, hit the open it up. And at this point, Jay and Brent are starting to believe we're starting to get a little more complicated. Will my fortune be perfect? Thanks. Thanks. Fortune bot. Fortune bot. Okay. Always savagely messing with me. The Reddit trained fortune telling bot. So at this point, let's pick up the conversation where we left off with Brent and Jay, where Brent said, yeah, yeah, well, just like any other computer. But then we're going to ditch chatty, right? We're going to ditch it and we're going to create a new VM in the cloud. We're going to download and install everything that we need right there in the cloud. And then Jay's like, yeah, then we'll have a public link Right. To test with our customers and we can give it to Paige and, you know, so Jay's starting to get it. Jay's starting to get it. All right, so you're Jay, what are you going to do? What are you going to use to solve this problem? You need a cloud platform and we all have our preferred cloud platforms. And I think it's going to be really interesting to see where your minds go as deep learners out there in terms of cloud platforms that you feel like we should be leveraging today in 2021 for models and for products like this. All right, let's check out the race here. Okay, We've got a head to head race full on Sprint between Google Cloud platform and aws. AWS has had an early lead here and GCP seems to be, you know, competing, bringing it Azure, trailing something else. Very much trailing aws. It's hard to compete. It's hard to compete with aws, you know, very, very cool. Again, if you're looking to contribute to this discussion, go to ahaslides.com dlaibot and we'd love to get your feedback on which cloud platform you would use. So it's kind of a toss up. It's kind of a toss up. And in this case there's really no decision to be made for Dorical. Well, why is that? Well, because as Brent says, hey, we basically use GCP for everything. So we're using gcp. Jay. Most of our infrastructure is on gcp. And so there it is. I already have an account, it's already set up for billing. I'll just show you how to do this real quick. And so basically that cloud infrastructure is what we're going to leverage for this. So Jay says, okay, let me get this straight. We're going to create a virtual machine in the cloud which has its own operating system. Brent's like, yes, correct. Then Jay's like, in that host os, we're going to leverage Docker to create a containerized application by building a remote image container in the cloud. And Brent says, you know, you're really starting to sound like a developer, Jay. Like, I like it. Keep going, my man, keep going. So Brent says, okay, stick with me. I feel like you're a fast learner, so I'm gonna go a little bit faster this time. Gcp, let me show you how it's done. So gcp, we're just going to go to our console again. You have to have this set up for billing. I was messing around with some stuff earlier. We're going to open the shell editor and we're going to make this full screen. You can see we're starting our virtual machine, we're starting our cloud instance and we're starting our editor. We first want to just upload exactly that Dockerize folder that we used locally. And once we upload all of the items in that folder again, this folder is available for everybody out there on GitHub. We want to click into that folder and then simply. We just have a couple commands to run. The first one is build the container image in the cloud. GCloud builds submit. Simple as that. We want to authorize this API call to gcp. And this is sped up here and it says, hey, don't run on root. It's like, hey, we're not running on root. It's just root of the VM in the cloud. Don't worry about the red text. Brent says to Jay, but it takes a little while here to create the image. Now we've got one more. Remember, we've got four gigs of container that we need to actually deploy to the cloud. It's a 256 megabyte default here. When we run the deploy command, we want to make sure that we further specify we're getting a pretty long winded folder structure title here. Dialoppt, chatbot, docker eyes, GCP. But let's go ahead and specify that we need those four gigs. We need all of them. And now we don't really need a service name, so just hit enter. Yeah, we're on the west coast, so let's go ahead and do it. 25. That sounds great. Allow unauthent authenticated invocations. Yeah, sure. This is a, you know, this is a zero to one. And look, we're spinning, we're spinning, we're spinning and we're going to speed up the spinning up and oh man, like, we're almost there. We're almost there. Jay's getting pumped. Jay's getting amped. Jay is. He's like, oh my gosh, are we there? Are we there? And Brent's like, yeah, let me show you man here. And if you don't believe me, let's go ahead and just open up an incognito tab. Just like make sure this isn't like some local thing. Now go ahead and send it to Paige. Send it to your friends. Let's play with this. Now we're getting. Let's ask it some classic fortune teller questions here. So what will my finances look like in the future? Right? You'll be Able to afford a house and a car. Savage Reddit. Nice. Will I get the promotion at work? I'm sure you will. Okay, excellent. This fortune teller is going to be killer. What are some things you can tell me about my partner? That's a tough one. I'm not sure what you're asking. Oh, okay. Well played, well played. Let's see, what can we ask it? Maybe some classic, like when will I. When will I what? I don't know. Some classic ones. What are they? Have a child. When you have a child? Yeah. Nice, nice. Okay, when will I. When I get married. When you're ready. Oh, excellent. Then Brent says, hey, look, here's what you can see. Let's call this monitoring J. This is level one monitoring. You see this green hump? Yeah. This is what we just did. Boom. Now you know how to monitor too. Of course. You could create custom stuff as you go from one to N. You could do all sorts of different things as you go from 1 to n. But no, that's not really what we're doing today. So product data, science and software success. This is. Jay and Brent are like, hey Paige, check it out. Built and shipped. She says, hey, this is so awesome guys. Give me a few days to start some customer development activities and feedback coming soon, right? Feedback coming soon. So as Jay and Brent leave work, Jay thinks, I hope users don't ask questions that are too hard. Right? And Brent's like, yeah, I hope that like only a few users ping this thing at a time. This thing is pretty fragile. It's like a little fragile infant product and I'm not sure how well it's going to stand up to abuse. So the next day Jay comes in and Paige is like, hey, that's it for the AI psychic for now. Follow up with the dev lead on the info gathering team for next steps. Like, I'll bring you back in once the AI psychic has some more movement like over there. They're not building something brand new though, so no ground up prototypes needed. Jay, they're working on something that exists already, so. So go see how you can help them. Look at the data that exists, investigate technical debt and try to figure out how to make iterative improvements to top level KPIs that flow up to business objectives. And you know, Jay's like, yeah, yeah, sure, Paige. He thinks like, maybe I should have taken more time on stuff. I actually like totally could have spent more time researching the fortune telling domain and like training, custom embeddings and all of that. Well, hopefully she'll be back. And I'm happy to report that one month later, Dorical raises their Series A. They impress the investors with their new ML first and AI first products and their adventures from data to deployment and beyond and beyond. The adventures continue. Okay, so if you want to compete with Dorical or you want to build your own chatbot again, we've got the links in the description below. We've got YouTube videos for now, but I'm hearing a lot of feedback on the comments about how can we can sort of make this more step by step. I'm going to go ahead and make those updates over the next day or so to the GitHub, so look out for that. But I want to go ahead and just thank everyone. Feel free to reach out anytime. Greg at 4th Brain, hit me up on LinkedIn or on Twitter. And you know, last but not least, shameless plug for the MLOps cohort that starts next, early next year if you like learning this stuff. Basically we went from zero to one today and we have a cohort program that's active right now with many of these beautiful people that you see right here, all these ML practitioners here that I get the privilege to facilitate their learning. We are actually going to not just deploy and test 0 to 1 in here, but we are going to figure out how to scale and manage horizontally on the, on the, you know, usage side as well. We'll figure out how to set up custom monitoring dashboards and how to think about bigger infrastructure plumbing tools that we might consider using in the future. So we've actually got a special scholarship that we want to, you know, let you know about. Alice mentioned it in the beginning, you know, half off basically for three people that are here today. So check out fourthbrain AI dlscholarship. And so with that I'd like to kind of give a final round of any questions. So we'll open the questions back up and we'll bring this back for just a few minutes here before we close out the session. So can we fine tune the models? We've answered that step by step instructions written up. I will definitely do that. I will make sure that that is there for everyone over the next 24 hours. Why was the model not trained on Reddit data? So we actually took a pre trained model that was trained on Reddit data. So this data is actually sort of implicitly in the model already. And so this pre trained model embeddings that we used was those were learned from Reddit data directly. We just didn't have to do the training ourselves because that would have been a massive, massive undertaking and certainly beyond the scope of one hour. So feel free to pop some more questions in here if you have other questions. Hugging Face. Hugging Face has it all. So I would encourage you to check out Hugging Face for all of their models. They've got tons of BERT models, tons of pre trained models. They even got moved into computer vision and all sorts of other things. They're a fantastic resource and, you know, plus one for Hugging Face. Great stuff they're doing. And this is the kind of thing that it enables all of us to sort of leverage to create value out there in our, in our jobs for the companies that we're working for and that we're trying to build. So yeah, they've got a lot GPT and BERT and in between. All right, let's see. How can we actually create our own chatbot? So, yeah, I'm not sure exactly what, what's meant by that, but yeah, I think basically it's kind of related to this other question of what would the best approach be to fine tune the dialogue GPT model to another domain like psychology? Well, I think the first thing you want to do is you want to basically figure out how to curate the right data for that fine tuning. You have to know, again speaking of psychology, you have to know sort of the psychology of your user, of your customer, of the people that you want to leverage this sort of bot. So that's why when we talked earlier about Microsoft Azure, said we have this bot service, but then we have this power virtual agent service. And the difference between the two is bot is for customer service and virtual agent is for beyond hr, for sales. And fundamentally what you need is you need dialogues between people, you need dialogue data to really help you with chatbot stuff. And so if you can find that dialogue data in your domain, fantastic. If you can't, then it's the kind of thing that you want to start figuring out how you can go and try to either collect, try to buy some data, try to curate that data, try to get people to create that data for you, like the Amazon mechanical Turkish sort of methodology. And you know, really this is, this is the challenge is how do you create something specific from sort of a generic, you know, previously we thought object detector, but this is now sort of like a generic kind of, you know, sentiment analyzer document, you know, reader or, you know, chat. Bottom. Now, today is a lot of these things that we're thinking about. So look very carefully at where your dialog data comes from and Think about which dialog data would be best and then if you don't have it, start collecting it today. Especially if you work in a business that has lots of users and you can set up something that might give you the data you want to get in the future. So, okay, so maybe, yeah, so maybe using a pre built model comes with bias. Yeah. So is there a debiasing phase? Well, so I think, you know, I think bias is an important thing to consider, you know, in all cases. And the fact that you're even targeting a particular domain is a bias towards that domain. You know, I like to think about this idea of bias is you either have sort of like, you know, you're going to have one or you're going to have many or you're going to have something in between. Right. When you go to the supermarket, right, you've got, you can have like one choice for ketchup or you can have like 100 choices for ketchup. It's like how many choices for ketchup do you want? That's kind of, you know, you want to balance that bias, variance, trade off. And so when you're thinking specifically about your problem and your business case and exactly what it is that you're trying to solve, that's where you're going to be able to put that bias lens appropriately on the data that you're using from a pre trained model perspective or from the data that you're using from a, from the ground up model perspective. And then when you combine those two then it starts to get even potentially more complicated. So this is a big problem that there's a big opportunity space for lots of you to help out companies do this type of work in the future. So yeah, thank you all so very much for your great questions. I'm going to go ahead and stop screen sharing now. I think I'm going to go ahead and invite Alice back up on the stage to close out for us.

Hey Greg, thank you so much. Can I just say that was fantastic. That was such a great presentation.

Thank you Alice, great meeting you.

Paige J and Brent, love the Personas, love the user story. Thank you so much and thank you everyone who joined us today. I know we only had an hour and sorry that we're not able to dive into too much details here but feel free to just check out all the playlists in the description box. And again, highly recommend you check out the online programs on 4th brain again if you missed any part of today's workshop, we will be sending out a follow up email including a copy of today's slides and a recording. And again, if you're interested in signing up for the scholarship program, the deadline is coming up November 30th, next week. So, again, thank you so much, everyone. Thank you, Greg. We look forward to seeing you again very soon. And as always, keep learning. Bye.