How the World's Biggest Companies Design Machine Learning Applications | ML Monthly April 2021

Friends, welcome back. Or if you're new here, welcome. It's that time of month again. It's very, very exciting. We're up to Machine Learning Monthly for April 2021. Now, if you've never seen this before, Machine Learning Monthly is a newsletter that I write once a month that collects the latest and greatest from machine learning, but not always the latest because we want things that aren't just about hype. We want things that are battle-tested through time. And this is actually a very special edition of Machine Learning Monthly. It's the design edition. So not going to do any more talking. If you've been here before, you know the drill. Let's jump into it. So we have here Machine Learning Monthly, April 2021. If you want the link to this article, you don't want to listen to me talk, the link will be in the description. You can see it there. It's on the zerotomastery.io blog written by yours truly. Let's jump into it. At the top— oh, by the way, if you want to get this in your email inbox, you can subscribe here. It'll send you this article at the start of every month. Very, very exciting. We start off with my work. The Zero to Mastery TensorFlow for Deep Learning course has launched. Amazing. Let's go to the GitHub. If you've seen any of my other videos, you probably well and truly know that this is launched by now. But if you haven't, here's where you can find all of the course materials on the GitHub. Of course, the details to what you need are on this page. But if you want to learn TensorFlow, if you want to learn the fundamentals of deep learning, if you have some experience— because there are some prerequisites for this course— if you have some experience writing Python code, a little bit of machine learning, jump into this and try it out. If you want to give it a go before you want to sign up and pay for it, well, the first 14 hours are available on 2 YouTube videos on my channel. But that's enough about that. You can see all the details here. Let's go into Machine Learning Monthly from the community. Now, this is very important. If you would like your own work in a future episode of Machine Learning, something that you've worked on, anything in machine learning, can be beginner, can be intermediate, can be advanced, doesn't matter what it is. I'm here to promote your stuff. There is nothing more than I love than seeing people create things that they're interested in. Today we have a submission from Louis François. I might be saying this wrong, Louis France, Francis, but Louis has a YouTube channel called What's AI. Let's jump into this. And he has some amazing videos. So Infinite Nature. Oh, this was a paper that came out recently, which is about using GANs to generate massive video scenes from a 2D image. That's very exciting. There was also another video that I watched the other day, which was how to use Transformers for computer vision. As you may have seen in a previous issue of Machine Learning Monthly, we covered the Vision Transformer architecture, or the paper, and it's using the transformer neural network architecture for computer vision where CNNs have usually reigned supreme. So I found that video very interesting. But go check out Louis's channel, What's AI. Big shout out to Louis. Thank you for the— or Louis, I may be saying this completely wrong, sorry. Big shout out for the submission. But now let's go into the internet. So the best resources, the latest and greatest from the internet, but not always the latest. As I said, this is a special design edition of Machine Learning Monthly. One of the most often questions I'm asked aside from, "I want to learn machine learning, how do I start?" The answer to that one, by the way, is you just Google it and you go down the rabbit hole for 6 to 9 months and then come up and see if you can breathe there. Because there's a lot out there. But the next question after that is, what kind of machine learning project should I work on? And I usually answer with that is follow your curiosity, because truly there are an unlimited amount of answers I could give for that question. But I'm not going to read through the post exactly, you can read through that. And long story short, I say follow your curiosity because that's ultimately going to drive you to where you are best suited to whatever you're going to work on, machine learning or not. And now, the more and more I learn about building machine learning full-scale projects, not just models in a, in a Jupyter or Colab notebook, is that machine learning is an infrastructure problem, which means the ML code is actually quite a small part of any big large-scale machine learning project. So if you go to my blog, mrdiberg.com, I've got a beautiful little graphic that emphasizes this. 'How the Biggest Companies in the World Design Machine Learning-Powered Applications.' This is a repost of this article. But this graphic here is just exemplifying the pieces of the puzzle. Of course, these are not all of the pieces of the puzzle around your ML code. These are just an example. But that's the theme of this series, or this post of, or this edition of Machine Learning Monthly, is how do you design all of these things around your machine learning code. And what I've done is, as we'll see in a second, is collected resources from the biggest companies in the world that are building machine learning powered systems and put them in one place so you can see how the best do it and then apply that to your own projects. So let's keep diving in. And what I mean by infrastructure is all of the pieces around your machine learning code. But big caveat here, none of these— these are guidelines throughout here— none of these will replace the knowledge you gain from experimenting yourself. So read these as a guideline to give you a little picture in the clouds of what you might like to build, but none of them will do it for you. Let's jump in. The first one is Apple's Human Interface Guidelines for Machine Learning. We'll just dive into the resource. I'll let you read the post if you want. But so why, why is this important? Let's discuss. And I'm sweating already because first of all, I'm very excited to share these resources with you. Second of all, Australia, it's supposed to be nearly winter, but it is like the sun's beaming through my window and it is ridiculously hot. Anyway, no complaints. Let's jump into this. So why is this important from Apple? Well, I haven't got my phone on me, it's in another room. Very important, if you're doing something important, keep your phone away from you so it's not distracting. This is running on a MacBook, but they have, I think it's somewhere over a billion devices around the world. And every single one of them runs machine learning, whether it be on the system OS itself, so the system's operating system, or on an application. And so this developer guidelines is if you wanted to build an Apple application with machine learning, in it somewhere. What should you think about when you're building that application? How could you get that on the App Store, get that into the hands of others? Because ultimately that's what we want to do, or that's what I want to do, is get machine learning powered applications into the hands of others. So if you want to do the same, these are some great things to think about. So how could you design the user interface and user experience, often referred to as UX, of a machine learning app? One of the most important questions here that they go through is defining the role of machine learning in your app. And I found this very, very thought-provoking. So is it critical or complementary? That means, does your app— is machine learning a critical requirement for the app, or does it enhance your app? So if you imagine, autocorrect isn't critical for a keyboard, but it may enhance how the keyboard works. However, a service like Face ID, requires machine learning. So it uses your face to unlock your iPhone, requires machine learning, uh, to work at all. So therefore it is critical. They also ask whether it's private or public, like where does the data come from? Is it proactive or reactive? So does a proactive feature do it, does it work on itself or does it only work when a user or a person puts in input? And that's another thing I really liked about Apple's Guidelines is you'll see a lot of software refer to people as users. I don't really like that terminology, so I'm trying to fix that in my own wording because everyone's a person, right? If you— it's very easy to get lost with numbers on the screen and call them number of users that we have rather than that's an actual person who's interacting with your service or application or whatever you've built. So that's what I'm trying to personally do. So catch me out if I'm using the wrong language. Visible or invisible? Does it happen in front of a person or behind the scenes? Dynamic or static? So I'll let you read through that. As I said, I'm not going to go through them in detail. And then it also breaks it down to inputs and outputs. And I really like this terminology because what does a machine learning model do? It takes some kind of input, finds patterns in that input, and then produces an output that you've defined. Now, Apple, it says if you want to build machine learning into your application, use that, that ideology to design it into the application as well. So what inputs does your app take? What outputs does it give a user? And you're going to notice a trend across all of these guidelines. And the inputs and outputs is a big one. But the one I'm really a big fan of is mistakes. So if we jump into this, It's inevitable that your app will make mistakes. Why is that? Well, because machine learning algorithms are probabilistic. That means that they are— so probabilistic and deterministic. Probabilistic means that they output a probability of something happening. Deterministic means that it's, it's certain, the output is certain. So if a machine learning model is outputting probabilistic outputs, it's inevitable that it will eventually get something wrong. So how does your app handle when your machine learning algorithm makes a wrong prediction? So this is very important to think about. And then it goes through, uh, understand the significance of a mistake's consequences, make it easy for people to correct frequent or predictable mistakes. I kind of hinted on this in my, um, deploying a machine learning model, uh, video that I did a couple of weeks ago, is if your application makes a wrong prediction How can you entice a person to help you correct that prediction so your model improves over time, in turn, eventually creating a data flywheel? So really, really recommend reading through Apple's Human Interface Guidelines for Machine Learning. I got a lot of value out of reading through this, and it's certainly something I'm going to be using going forward for creating my own projects. And then I made a little colorful graphic which breaks down the role of machine learning in your app, which just literally takes the roles here and rather than just being straight up text, makes it colorful and pretty. You know, you've got to add a little bit more color to life. All right. The next one is Google's People and AI Research, which is the acronym is PAIR. But if we jump into this, this is another similar guidebook to Apple's Human Interface Guidelines, whereas Apple's is more designed for creating a machine learning application for an Apple device. The People in AI Guidebook is more of a, I would say, a wider scale thing to think about from Google. So if you imagine Google, one of the biggest machine learning companies in the world, you use their search engine. I mean, I'm using Chrome right now, which uses machine learning several different ways. But if we wanted to get started, they are, they break it down into— no, thank you. There's cookie— man, cookies are popping up everywhere. I just want to read a website. They break it down into 6 chapters. And what I really like about these chapters is they come with worksheets at the end, which means after you've read through a chapter— so let's jump into a big thing. You'll also see another trend is defining what success is for your application. So this is the User needs. Again, I'm not a fan of that user term. I prefer like, what does a person need? What does the person on the other end of what you're building, what do they need and define success? So let's jump into this. So even the best AI will fail if it doesn't provide unique values to users. So again, I would like to change that to people. What's new when working with AI? Because the users, what you can just get lost, you forget that you're dealing with another human being. And what's new when working with AI? So I like this one as well. Find the intersection of user strengths and AI strengths. So if you think about what problems is machine learning good for, that's what this question challenges you to do. Because machine learning is not always the right solution to, to put into your application. That's it. We'll see that later on. That's one of Spotify's principles. And then if you go through this, again, I'm not going to spend time going through this. But this is— I found this was a great read through, reading through all of these chapters. The big overarching narrative that I'd like you to take home from this video is the overlaps. So as I said, you'll find there's a lot of overlaps between Apple's Human Interface Guidelines as well as Google's PAIR, People and AI Research. See, they use people there. Why, why not just keep that trend anyway? But going back here is the key concept. So you can answer the worksheets, really help you to start thinking about machine learning applications and products, not just from a modeling standpoint, but from the whole ecosystem, putting together everything in this paradigm. So let's jump back. I kind of got distracted when I was telling you the overarching narrative of this is the— oh, there we go. What do you do? Errors and graceful failure. What do you do when your model is wrong? What's the difference between an error and a failure? A self-driving car stopping at a green light could be an error, but running a red light could be a failure. You don't want your self-driving car to run a red light, but if it started to stop at a green light, that might be considered okay. Of course, you don't want it to do that either, but what's the significance between an error and a failure? And you want to look for the overlaps in each of these because when there's overlaps between smart thinkers, that's where the, the truth is. Another great way to sort of learn something is when two smart thinkers disagree with each other. So as much as the overlaps are valuable, look for the points where these guidelines may disagree with each other. So the next one is Microsoft's Design Guidelines for Human-AI Interaction. Let's jump into this. So Microsoft break it down into 4 main areas, which is— let's get this graphic up. This looks a bit better. Can we see that? Beautiful. So the guidelines for human-AI interaction. So what do you do at the start? Make clear what your system can do. What expectations is the person using your system? What are they going to get out of it? Remember, machine learning is— it can seem like magic to someone who's never used it before. But if you're building the system, you probably know where its flaws are. If you're doing your experiments right. Make clear how well the system can do what it does. So again, we're setting expectations. We're building this entire narrative around what a person is looking for, what our application, what our project can offer them. When I was working at Apple, one of the big things we would talk about is what are the benefits of someone purchasing an Apple product rather than the features? Because a lot of people don't necessarily care how state of the art your model is. They care what it does for them. How does it improve their life? That's what you want to think about. The during the interaction. So what that means is while a person is using your service, what happens? Uh, are they— do you have time services based on context? Like for example, um, Google Maps might show you food places around where you live or something like that. Show contextually relevant information. Again, this is all while someone is using your service. When wrong, another overlap. This is what we're saying, like all of these, again, machine learning is probabilistic, not deterministic. So it's inevitable that your model will eventually make a wrong prediction. So how do you make that the best experience for someone using it? And I like these three here, these three cards. Support efficient invocation. So make it easy to invoke or request the AI system services when needed. So if you were talking to a voice assistant, can you talk to them quickly? Because speed is the name of the game in technology. Support efficient dismissal. Again, if you don't want to use the services anymore, you want to deactivate the voice assistant, you need to be able to tell them to stop quickly and it'll just stop. So for example, I don't want to set it off, but I have a smart speaker out there. If you say the word A-L-E-X-A, if you pronounce that word, it will trigger. But if I wanted it to stop, I just say stop and it stops immediately. And then again, if it becomes, if something's wrong, if your application is wrong, how does someone correct it to make it improve over time? That's the final step. So what does your system do over time? Learn from, again, I'd like to change this, learn from behavior. Update and adapt cautiously, encourage granular feedback, provide global controls, etc., etc. Some great guidelines here, and they also make it into a colorful graphic. So I am a fan of that. All right. To the next one. Oh, here's a little trend that I drew with arrows. You start with some sort of problem. What problem? Does your, does your problem, does machine learning's strengths overlap with your problem? If not, you might need no ML. So problem, create a solution, set the expectations for what a person's expecting, allow feedback for people to say, hey, this is good and this is crap, have a mechanism for when it's wrong, and then improve over time. And then you loop back to the problem. I couldn't really draw that arrow with text, but you kind of, you get the picture. Next is Facebook's Field Guide to Machine Learning. Now this is an amazing series of videos to think about out when you're building. Now, this is more focused on machine learning modeling. So if we go back to this little graphic here, machine learning modeling rather than all of the steps around it. So if we jump into this graphic, again, it's defining a goal. How do you start? What data do you need to service that goal? How do you process that data? What model should you create? See, I love here that they've got multiple models, because if you've gone through the TensorFlow course, you'll know that a machine learning practitioner's motto is Experiment, experiment, experiment. So try lots of things. This is the scientific method. It's combining engineering and science together. And then of course, evaluate your model. If we jump in, we've got 6 steps here. I actually used this video series as the foundation. You can go through these videos. I'm not going to go through them again. As the foundation of my beginner-friendly machine learning and data science course. So if you've gone through that, you might recognize this graphic. Well, here was the inspiration from Facebook's Field Guide to Machine Learning. Again, this Facebook's Field Guide to Machine Learning— keep in mind that although the previous 3 resources focus on many of these steps here, the Field Guide to Machine Learning really only focuses on once you have a dataset and modeling that data rather than data collection and deployment. So yeah, if you want expanded versions of that, you can go through the Zero to Mastery Data Science and Machine Learning course. Or you can also read— I have an expanded version on my blog. So there's me with a mohawk/mullet. And I put everything on a whiteboard to make it official, and then explained it with text, what I learned from the videos. Let's go back in here. This is one of my favorites. Again, Spotify's 3 Principles for Designing ML-Powered Products. Now there's a lot of information to take in here, and again, I said at the start, none of them will replace working and running your own experiments, but we're looking for overlaps here to keep in mind when we are working on our own projects. So let's jump into Spotify's 3 principles, uh, for designing ML-powered products. Now Spotify have over 250 million customers around the world that listen to streaming music. Of course, you're going to need machine learning somewhere. But how do they think about doing it? Let's jump in. They have 3 principles: identify friction and automate it away. So where is a person in your service, where are they running into sticking points? So if you imagine for Spotify, you're looking for music, that might be a sticking point. So can machine learning help recommend different playlists for someone to, to listen or discover new music? Maybe it can, and I think you'll find it will if you read through the full article. Ask the right questions again. What are we trying to do? You can— a person on the other end of your application does not care how state-of-the-art your model is. They care what your service can do for them. That's what you have to think about. Benefits, not features. And then finally, go manual before you go magic is, do you need to start off with machine learning? If you can use a simple rule-based system Start with that, implement machine learning if it improves that system. So you can read through those. I'm a big fan of, I use the Spotify application. I was actually, while I was writing this article, I was using, where is it? What was I listening to? And in my case, it looks like they must adhere to their two principles because whilst building, whilst listening to or writing this article, these tracks I'm listening to were absolute bangers. So I was listening, they use machine learning to create create their Discover Weekly playlist, what I was currently listening to while writing this article. I was listening to that this morning while driving to breakfast. It works. So I'll let you read through that. Principle 2, ask the right questions. If you don't know what you're building, you could end up designing a product in the wrong direction. Go manual before you go magical. Lastly, we have From Big Data to Good Data by Andrew Ng. Now this is another trend that I'm noticing in the field of machine learning. Um, RoboFlow did a great summary of the main points. So big shout out to RoboFlow. I'm a big fan of their product too. They build computer vision tools, so go check them out if you're into computer vision. But Andrew Ong did a talk recently at Scales' recent conference. You can see it here. Uh, you do have to sign up and put your email in. It's free. I've already done this, but you can watch the whole talk there. Some of my favorite points from the talk were getting deployment is a starting point. So I remember when I was at my last machine learning engineering role, we would build proof of concepts, we do a lot of the modeling points, we get to deployment, and it'll kind of fizzle out after that point. But that is what Andrew was asking, is a starting point, because we all know the real test of machine learning model is not just on the test set, it is once it's deployed to production. Getting actual people to interact with it. The next, my favorite point, was from Big Data to Good Data. So let's be real, we're not all building giant companies like Facebook's and Google's and Spotify's and millions of users, so we don't have the big data resources that they have access to. We need to be smarter about that. MLOps' most important task is ensuring high quality data in all phases of the ML project lifestyle. So we're going to be working with small datasets. How can we automate the ML parts and build the data parts around it so that we're building this eventual data flywheel? If you want more of that concept, look up a talk by Josh Tobin, Data Flywheel. We'll go to youtube.com. You get a little insight into my recommendations here. I'm into RuneScape, I'm into health. Um, Josh Tobin, Data Flywheel. Highly recommend. Oh, far out. I'm the first video on this, but all credit to Josh Turban for the idea of the data flywheel. Go check that out. But essentially what you're trying to do is just build a system that completely keeps collecting data. The model improves by itself and you're, you're spending most of your time programming the data. And when Andrew goes to be a consultant on machine learning projects, He tells them to freeze their codebase, freeze your modeling codebase, get a, get a good working model, such as if you're on computer vision, maybe EfficientNet V4 is enough for your image classification problem. Freeze that model and then program your data. So that's another thing I'd like to see come out from Kaggle is Kaggle's like, or maybe Kaggle or another data science competition is Kaggle, you usually, you have a dataset and you iterate on the model. I'd like to see a Kaggle competition flipped where you have a set model and you have to iterate on the dataset. That would be really cool. But go check out that talk. I'm a big fan of that. This point really speaks to— if we come back up to the top— the trend that I'm seeing in machine learning in general is that machine learning is an infrastructure problem. So once frameworks and pre-made models get better and better, where really just going to— I think ML, machine learning, in the next few years is going to be— it's just going to be like a— this is going to be an API call. We're seeing this with Hugging Face's models, services, and all that sort of stuff. The model is going to be taken care of. Your biggest task will be collecting the data and feeding it to that model, creating that data flywheel effect. Let's finish up here. Now, of course, I've thrown a lot of information at you. Of how to think about building your own machine learning projects. And you thought I was going to leave you there high and dry with all these principles, right? Because principles, guidelines, schmiedlines, it's all right. I said nothing is going to benefit you more than actually working on these things. So that's why we have a bonuses section. The above are all guidelines, how to think about building your machine learning powered systems, but they don't necessarily show you or teach you how to go about doing so. And so I've got some resources here, which will go through the technical bits around this little graphic. So a great resource here, I'm not going to dive through each of these. But we have Software Engineering for ML, which is— this is a beautiful resource, which really puts together, combines a lot of— so when I started learning machine learning, I missed out on a lot of Software Engineering Skills. I'm not going to dive into what those are because this covers a lot of them. So you could spend 3 months reading through all of these engineering or software engineering best practices for machine learning. That's a great resource here. The next one is the Machine Learning Engineering book by Andrey Burkov. I have a copy of this usually within arm's reach. Boom. I have a copy for this, as you can see. Hands-On Machine Learning: Deep Learning for Coders with Fast.ai and PyTorch. Amazing book available for free. Read it online beforehand. CS329S Machine Learning System Design. So if you wanted all of the steps above in course form from Stanford taught by the amazing Chip Hewitt, go and check out this. Yours truly teaches the machine learning deployment section and deployment— deployment tutorials. So you can go see all those, but this is an outstanding course. There's also Full Stack Deep Learning, which I'm personally going through right now. The 2021 version is out. All of the videos are on YouTube. You can see all of the notes here. This is absolutely outstanding to learn again, learn all of the steps around designing and building a full-scale machine learning project. Finally, the Made with MLOps curriculum by Goku Mohandas, which is absolutely outstanding as well. So this is more of a smaller, individual lesson-based style. I would say the Full Stack Deep Learning is longer style. So they have 12 lectures or 13, versus ML Ops here has a lot more— they're broken into smaller pieces. So check all of these out. You don't have to necessarily go through all of them. I'd recommend at least one of these. To go along with reading through these. So again, these will teach you how to think about what goes into a machine learning system. These courses then will get you hands-on, and then your next step after that— and my next step, in fact— is to build your own full-scale machine learning project. But I'm not going to talk about that for now. That will be coming soon. Far out, we have covered a lot, um, If you want to see the issue of this in your inbox next month, make sure you sign up. The link will be in the description. But otherwise, that has been Machine Learning Monthly for April 2021. Once again, if you have any submissions that you'd like to see in a future newsletter or a future video like this, you could see yourself on TV. How cool would that be? On YouTube. I mean, TV is dying, isn't it? But send an email to daniel@mrdbourke.com with the subject Machine Learning Monthly submission, and you may see it in a future episode. Again, could be anything that you've worked on in the machine learning world. But this has been the design issue of Machine Learning Monthly. I hope you enjoyed it as much as I did researching it. I've got a lot, a lot, a lot to think about for my machine learning projects going forward. But as always, keep machine learning, keep creating, keep moving. I will see you next time. Peace.