Deep Learning’s Most Important Ideas | Machine Learning Monthly November 2020

Can you tell? Surely you can tell. If you can't, that's all right. I'll spoil it for you. I'm very, very, very excited for this video. Reason one, oh, I'm excited in general. Reason one is because it's summer, so we're probably going to work up a sweat in this video. Reason two is I spent most of the morning Dancing around, grooving, moving. You know, it feels good when you move your body. Reason number 3, reason number 3, and the reason you're probably here is because it is time to go through another Machine Learning Monthly. That rhymed. Reason number 3, we have Machine Learning Monthly for November 2020. Now, if you are new to this style of video, every month I write an article called Machine Learning Monthly. Which contains all of the beautiful things that are happening in the world of machine learning. And I say all, but that's, that's not correct. It's really just a curated list of some of the most fascinating and important things that are happening in the field of machine learning, both recent and things that have stood the test of time. That's going to come back up later in the video. So if you want to read this article, you want to skip this video, I'll leave this link here. You can read it on the zerotomastery.io blog. I'll leave this link. It'll be the top link in the description. You can read through it. You don't have to listen to me blabber on about it. But without any further ado, if you do want to listen to me blabber on about it, check the timestamps below. There will be little segments of different things we go through, but let's jump right in. So what you missed in November as a machine learning engineer. Whoo! I can feel a few droplets forming on my brow already. That's the Australian summer for you. Let's get into it. So, my work in progress. The structure of this, by the way, is usually at the top I'll do a little bit of the things that I'm working on. Then we have some from the community. If you want to submit anything to a future issue of Machine Learning Monthly, please send me an email, daniel@mrdbourke.com. And then we have the best from the internet. So my work in progress is a new deep learning with TensorFlow course. Now, if you've done a beginner-friendly machine learning course, possibly like this one, I'm not sure who created it, but I'm pretty sure it's, it's, uh, oops, spoiler alert, that's the person who created it. But I'm biased in saying that this is a pretty good beginner-friendly machine learning course. If you've done some sort of beginner course, you might want to start getting into deep learning. And if you're going to get into deep learning, you're probably going to be using TensorFlow or PyTorch or C++. But this one, what I'm working on is specifically a new deep learning course with TensorFlow. You can check out the GitHub repo. Here we go. It's still a work in progress as of now, but I'm going to be updating this basically daily with what I'm working on. You can see the projects preparing for launch. I want to make this a work of art, so I've got a lot of things to do, but that's all right. Like, what else are we going to do, you know? And so if you see here, yeah, let's zoom right in at this. The repo may look rough at the moment, but give her a month or so, maybe a little longer because next month is Christmas, you know, or this month actually. How fast did this year go? And she'll be a work of art. So stay tuned for that. That'll come soon. If you want to stay up to date on the progress of that, uh, the GitHub repo is the best place or this newsletter or this YouTube channel or students of the Zero to Mastery Academy, which is the academy here, this little link, will get first access to the videos once they're released. All codes and other resources like slides and whatever will be public on GitHub, but the videos will go to the zerotomastery.io website first. Anyway, let's get into the best from the community. So Alvaro submitted a great guide to serving PyTorch models with TorchServe. Now, if you're not sure what PyTorch is, it's a deep learning framework. I mean, if you're watching this video, you've probably heard of PyTorch, but let's just have a look here. So PyTorch, an open source machine learning framework that accelerates the path from research prototyping to production deployment. Whoa. Research and prototyping is mean you're trying things out. Production deployment is as in you want to get the things that you've built into the hands of others. And so Alvaro, in terms of getting those things into the hands of others, made a guide here on how you can use TorchServe. So TorchServe is the ML model serving framework developed by PyTorch. And I just want to point this out. I actually really like the way Alvaro shared this project because it's all in GitHub. So it's like a, it's like a blog post, feed, code, at the same time sharing a project. So if you're like, people say like GitHub is the new resume or something like that, or if you're looking to share your work, I'm a big advocate for first of all having your own blog. So like at your own website name where you can just post whatever you want. Like I've got mine, mrdeburgh.com. I put basically anything and everything here. Um, but if you want to go code specific, I'd suggest GitHub and then do something like what Alvaro's done. So you've got requirements, dataset, modeling, deployment, Docker, usage, credits. My goodness. So here we go. If you want to get started, you can install the requirements. The dataset he talks about is a subset of the Food 101 dataset. Be careful looking at these images. I've got a jiu-jitsu competition tomorrow and I've got to lose about a kilo. So I'm not looking too much at food today, but that's all right. It's all part of the fun. Modeling. So Alvaro used a ResNet model, which is built into Torchvision. And then once the model was created, this is deployment. This is actually going to be a big theme of my work probably in the next year or going forward is, is not just building machine learning models, is deploying them. As I learn more, I will share as much as I can. And then deploy TorchServe. So this is how you deploy TorchServe. Docker usage, if you want to use it. There we go. So if you send it an image, it's going to return back some sort of prediction probabilities array in JSON form, I believe. So there we go. Must have sent the model is most confident on— looks like it's a picture of a hamburger. Wonderful. And there we go. So that's how you would, you would send it a custom image. GitHub actually formats Python code really beautifully. Awesome. So thank you so much, Alvaro. Have I starred this? Yes, I have. Beautiful. If you want to check out that guide, if you want to— if you're building models in PyTorch, you want to figure out how to serve them, go check out Alvaro's guide to using torch serve. Anyway, back to the best from the internet. And I am— this is— this issue of Machine Learning Monthly, I mean, they're all bangers, but this one especially is an absolute banger. I had a lot of fun writing this one and I'm going to have a lot more fun going through it. So let's get into it. So the first one is TensorFlow for Mac and TensorFlow Recommenders. And what I've decided from going forward, the best from the internet, I think I'm going to limit Machine Learning Monthly to like 7 beautiful resources from the internet. So it's not just an overwhelming abundance of things because at the end of the day, like there's, there's just too much information on the internet. And if you try to keep up with it all, you're going to get overwhelmed. Overwhelmed. So I want to keep it as curated as possible. Just a few simple things. Well, not— maybe not simple, but a few amazing things that may help, or the things that I'm finding most interesting that you might also be interested in. Here we go. So number 1, accelerating TensorFlow performance on Mac. Why am I so excited for this? Well, let's have a look here. If you're not quite sure what happened, Apple recently released the M1 chip and I'm not going to go through all of this, but basically, where is it? Where's the architecture? Uh, neural engine. So it's got a CPU, a GPU, a neural engine. So where's the machine learning part of it? Have we got machine learning? The next generation of machine learning. That is what I'm really excited about. Here we go, a 16-core neural engine. So what does that mean? So traditionally, TensorFlow, you could run fast TensorFlow code if you had an NVIDIA GPU. But where we're slowly getting towards, and what I believe this is— I haven't quite tried this out yet, I plan on trying it out this week— is TensorFlow on the Mac. Is— I've got a Mac, I'm recording this video on a Mac. Have a look at this, about this Mac. Where we go? Graphics. AMD, can you see that? I'll just read it out. AMD Radeon Pro 8GB. So I have a fairly beefy graphics card on this Mac. However, I can't run TensorFlow without doing a bunch of, of jumping through a bunch of hoops. So what's happening in Apple's Mac-optimized version of TensorFlow 2.0 is that you'll be able to run TensorFlow natively on your Mac. Which is super exciting. So if we come here, Apple have created— this is coming in TensorFlow 2.4, I believe. Yeah, there we go, TensorFlow 2.4. Uh, you will need macOS Big Sur and above, but essentially, um, what does it do? This pre-release delivers hardware-accelerated TensorFlow and TensorFlow add-ons for macOS 11 Plus native hardware acceleration is supported on Macs with M1 and Intel-based Macs through Apple's ML compute framework. Woo! That is exciting. So if we come back to the blog post, where is it from TensorFlow? Look at this, we get a fairly large speed up on the MacBook Pro. So if we go TensorFlow 2.3 with Intel, this is a gray bar. And then if we have the accelerated TensorFlow 2.4, we get some fairly dramatic— like, I mean, look at the M1. I mean, if you, if you have an Intel chip, you get a little bit of speed up. But if you have an M1 chip, I mean, what's that, 20% of what it was before? So about a 5x increase across the board. That is exciting. So instead of what this means in practice is I'll be able to hopefully run prototype models on my Mac and then scale them up to larger amounts of compute on, say, Google Cloud or AWS or something when required instead of having to consistently be connected to the cloud and running on a, uh, NVIDIA GPU. Very exciting. Anyway, we go back here, and by the way, the M1 chip is just phenomenal. So because this is a laptop-based chip, um, this is kind of an aside from ML Monthly, uh, if you've seen anything about this chip, it's just, just, just just the performance reviews are just out of this world. I'm really excited to see what Apple can do with a chip that is dedicated to like a larger machine. So not just in a laptop, like imagine the chip that's going to be in the next iMac. Whew. Anyway, where were we? We had TensorFlow Recommenders. Boom. So what is this? Another thing for TensorFlow. This is actually really exciting as well. So if you wanted to build a recommendation engine, so say when I go to YouTube.com, this is going to be probably pretty revealing of what I'm into, right? What do we have here? George Hotz, 1 hour of powerful Viking music. Protein folding explained by DeepMind. Actually, that was a phenomenal paper. That'll be in next monthly, I reckon. Some RuneScape, really into RuneScape. Lex, Lex Friedman's podcast. Boom. Training deep neural networks on a GPU. Have a go at that. That's awesome. Bodybuilding channel and Mark Riblet just making some groovy tracks. Anyway, so if you wanted to build a recommendation engine— oh, what did I do there? If you wanted to build a recommendation engine like that, traditionally you'd have to build a pretty intricate system. And now with TensorFlow Recommenders, you now have a TensorFlow framework to build a recommendation engine. So just like the things that power YouTube's homepage or Amazon's front page, or if you read articles on Medium, things that deliver you recommended articles. Look at this. Boom. Run in a Colab notebook. I remember when I was working at my last company for, as a machine learning engineer, and we had to build a recommendation engine for one of Australia's largest e-commerce websites. And it took me basically 2, 3, 4 weeks of figuring out how a recommendation engine worked, and then another couple of weeks to format their data in the same way. I mean, data preprocessing is always a challenge with whatever problem you're working on. And so basically I went through this little recommendation, TensorFlow Recommenders library. Basically what it's done is it's sort of packaged up everything I created in that 4-week block into like a few lines of code. So if I had this, maybe the prototype, uh, the proof of concept I was building could have been delivered sooner. Anyway, beautiful resources from the TensorFlow team there. So if we go here, next one is Yann LeCun's New York University deep learning course. So have you ever run a convolutional neural network? Well, if so, Yann LeCun invented them. And so if we come here, the full— his full deep learning course, and I mean credit to Yann, in the first lecture he actually says that this course is mostly not me, it's the people that have helped me do it. And is my internet busted up here? Oh no, just taking a little while to connect. Boom. So it is coming up. Well, it is summer here for me and I'm thinking about what my summer of code will look like. And so I'm collecting resources like Yann LeCun's deep learning course and figuring out, hey, where does my skill set need improving and what resources can I use to, to improve that skill set? So this is a full-blown deep learning course. If you would, if you were to go through all of this. So these are the beautiful people that have put in some hard work to create these resources. If you were to go through this, I would say you would have a fairly solid foundation of deep learning. Not fairly solid, a solid foundation of deep learning. So look, we've got lecture and there's the key slides. So we've got slides, boom. We got YouTube video and Jupyter Notebook, the triple threat. This is, this is what I like to see. This is the type of— this is how I learn. I like to read things, I like to watch things, I like to listen to things, and then I like to absorb that all into my brain, remix it, and then spurt it out into my keyboard and see if I understand it. So deep learning, boom, there's the slides, here's the videos, and boom, PyTorch deep learning. So this one's taught in PyTorch. So if you want to learn deep learning with PyTorch, You want to learn deep learning in general, I would highly recommend checking out this curriculum if you're looking to tee up a Summer of Code. If it's not summer where you live, maybe you want to tee up a Winter of Code. I would, yeah, go check out this one. Yann LeCun's, what's it called? Center for Data Science. I believe that's what it's called. CDS at New York University. So this is the beautiful thing about the internet. New York University, maybe you don't have access to go to New York University, but you can go through the materials. Go through the materials, add them to your GitHub just like Alvaro has done. Boom. Maybe you could replicate the same thing here, but with the models you build throughout this course. That would be cool. Anyway, let's go in here. Machine learning created art. Now this is absolutely beautiful and it just goes to show the broadness of what machine learning can be used for. Mlart.co, a showcase with creative machine learning experiments. And now I'll warn you, if you are— if you get queasy easy, maybe some of this stuff is not— is not for you because it gets pretty wild. What do we have here? Vadim Epstein. So painting by Viktor Vasnetsov, animated with Animating Landscape and multi-domain multimodality translation, L2L translation. Oh my goodness. PoseNet, using PoseNet to create art using your body parts like eyes, hands, chest, and knees. So maybe if you're like me and your art teacher in year 8 told you that your drawing of a ninja turtle wasn't up to scratch, it didn't fit the criteria of what a good drawing should look like. And that really broke my heart. I thought for a long time that I, I couldn't create art. How wrong I was. Anyway, if you want to— so maybe your drawing skills aren't up to par, but you can code machine learning code, potentially mlart.co has some inspiration for what kind of artworks you could create with your skills. So what do we have here? A GPT-3, a GPT model trained on 30K font bios to generate descriptions of Speculative fonts. Oh my goodness. A robot that uses a camera and translates the image feature to RNN features and writes poems on the beach. That is cool. A poem writing robot features on the beach. Look at that. No. Poet on the shore. This is wild. That is amazing. Poet on the Shore is a part of my MFA dissertation that investigates designing machines with autonomy in the age of artificial intelligence and challenges the fundamental anthropocentric assumption about machines and other non-human objects. Whoa. I'm going to save that for read later. Pocket. Great extension. Going to throw in the ML tag. Boom. That's amazing. Check that out. MLArt.co. I believe it is by Émile, who is a resident in the Google Arts and Culture Lab. So big shout out to Émile. Thank you for the work there. Go give him a follow on Twitter. Émile shares out some amazing stuff. And then we go here. We are whole. Now everything in this article is phenomenal, but this one may be— I think this might be the title— The Most Important Ideas in Deep Learning. You ready? You ready for this one? This is a doozy. I got excited when I was reading through this. Deep Learning's Most Important Ideas: A Brief Historical Review. So this is by Daniel Bourke, who has an amazingly designed blog, by the way. I would like to create something like this and possibly one of my favorite. Actually, let me ask you a question. How does a good idea— don't look at the screen, the spoilers on there. Look at my face. How do you test whether something is a good idea? Is it popularity? Is it if the idea works or not? Or is it— now you can look at the screen— if the idea has stood the test of time? And I mean, the idea kind of has to work for it to stand the test of time. Another droplet of sweat coming down. That's how intense we're getting with this video. I'm sweating. No, this is just, this is just Australia at 9 AM. So what are the things in deep learning that have stood the test of time? Now, put another way, I will try to present a minimal set of ideas that most of the basic knowledge necessary to understand modern deep learning research. So if you want to get up to speed quickly, now, one of the greatest recommendations I've seen here is This could be a great Summer of Code project. I highly recommend trying to reproduce some of these paper results from scratch in raw PyTorch. You could use PyTorch, or you could use TensorFlow. You could even use NumPy if you want to get adventurous without using existing code bases or high-level libraries. That sounds like a fun project. I may, I may even do this myself and create videos on that. So let me, let me know if that's something you would like to see. But anyway, if we go in here, I'm not going to go through all of them. I'm going to let you do that. 2012, tackling ImageNet with AlexNet. So potentially what caused the explosion in deep learning that we've seen over the last decade or so is when AlexNet, which is a neural network architecture, broke the record on ImageNet using deep learning methods. So prior to that, I don't think deep learning sort of— it wasn't as effective as what it was when AlexNet first came out. And I think that's what also introduced dropout, which is— I love the beauty. So this is what I've kind of noticed, actually. A little tangent here. In machine learning, if ever you add like simplicity, like as in make more of the problem learnable by an algorithm, you often see an improvement. So dropout is like such a simple thing that kind of makes the learning challenge difficult. So— and you see generally better regularization with dropout. Or if you increase the amount of randomness in the problem. So again, I guess that comes back to making more of the problem learnable. Anyway, we're getting another tangent. But then in 2013, we saw playing Atari with deep learning. This is from DeepMind. So this is kind of what caused the DeepMind revolution. I believe this is kind of what maybe got them— maybe, maybe sparked Google's interest in acquiring them as a research company. And then 2014, we had the encoder-decoder networks with attention. Now this is one of my— I need to reread this— Attention Is All You Need, a beautiful paper. So this is what introduced the transformer architecture. And basically I think what we're seeing at the moment with transformers and natural language processing is basically with any— saying basically a lot, far out. You gotta catch yourself on your errors, right? Improve, uh, what's my optimization function? My loss function is to reduce the amount of times I say conjunctions. Anyway, attention is all you need. What I'm seeing is that if you add attention to almost any type of model, you see some sort of improvement. I need to reread this paper and possibly re-replicate, uh, attention, an attention mechanism from scratch because just reading a paper doesn't mean you necessarily understand something. That's what I get trapped in all the time. Until you can replicate it sort of with code from scratch, well, you don't really understand something, do you? Let's go back. In 2014, we had the Adam optimizer, the ever reliable Adam. Adam is safe. Remember Andrej Karpathy neural network recipe, one of my favorite blog posts. This is in a previous episode of Machine Learning Monthly. And we got Adam is safe. I love that point. Just try Adam. Simplify your workflow, right? And then we had 2014-15 GANs, generative adversarial networks, residual networks. That's what gave birth to ResNet. Transformers. What's taking— there we go. Attention is all you need. And 2018 BERT and fine-tuned NLP models. So this kind of 2018 was a massive year for NLP, and also 2019, in terms of introducing, um, transfer learning into natural language processing. Transfer learning, which had revolutionized computer vision, is now making its way into— oh well, it's well and truly into natural language processing. Um, and Hugging Face are kind of just owning the game there in terms of large-scale or state-of-the-art NLP models, or making state-of-the-art NLP models accessible. So we see here 2020 and beyond. Now I think big language models, self-supervised learning, this is a big one, self-supervised learning coming up, but I think we're going to see DeepMind's, let's just go to DeepMind blog. There we go. I think it's this one. This is a spoiler for next month's monthly. AlphaFold. So this is probably going to be— I would add this one. So copy and paste. Now, Danny, if you're watching this, I'd put this one maybe in here depending on— I mean, this has only been out a few days. We've got to see if it stood the test of time. So this one might slot itself in here, you know. Anyway, phenomenal article by Danny Britz. If you want a research project, something to upskill yourself in the field of deep learning, I would say spend 3 months creating a GitHub where you replicate deep learning's most important ideas. That would be a phenomenal thing to add to a portfolio. You wanted to go to some sort of ML researching role or ML in production. If you wanted to go into production, you could just make an API for each of the most important ideas. That would be pretty cool. And if you do, share it with me so I can share it as a community resource in a future issue of Machine Learning Monthly. Anyway, let's get on to the next two, which is Continuous Delivery for Machine Learning. This is a hearty read. Now, I'm not going to go through this because this is about 35 minutes worth of content, but I'll just show you the top level of it because, as again, I keep saying this over and over again, we're starting to see, um, major developments in the field of MLOps, which is once you've built your model, it's all the little pieces around your model so that someone else, a user of your product or service, can access that model and potentially benefit from it. So continuous delivery. What do we got here? Let's read this out. Machine learning applications are becoming popular in our industry. Oh yeah. However, the process for developing, deploying, and continuously improving them is more complex compared to more traditional software. Such as a web service or mobile application. So in those things, you might just have code. You have to maintain, or you might have a database. It's all in the database, right? And you think about most applications, it's once you've got the database down pat, your front-facing code is transforming things from that database. But anyway, in machine learning, all of your parts are basically moving. Your code is going to continually update itself. Your model is going to need to be retrained as you get more data and it breaks. And your data, so as you collect more data, How do you store that? How do you transform it to go into a model anyway? So their behavior is often complex and hard to predict, and they are harder to test, harder to explain, and harder to improve. Lots of hard there, but that's the fun part, right? Because everyone likes a challenge. Continuous delivery for machine learning, CD4ML, is the discipline of bringing continuous delivery principles and practices to machine learning. Now, what is continuous delivery. This is, this is what I do if I don't know something. I just go down the rabbit hole and just Google terms. Continuous delivery. Boom. What is continuous delivery? Thank you, continuousdelivery.com. Is the ability to get changes of all types, including new features, configurations, changes, bug fixes, and experiments, into production or into the hands of users safely and quickly in a sustainable way. So Another resource that I found for this was, was just the Wikipedia article. You know, it's funny like that, right? Is the Wikipedia article here? There. CD is a software engineering approach which teams produce software in short cycles, ensuring that the software can be reliable, reliably released at any time, and when releasing the software, doing so manually. So the beautiful thing about CD is you're working on basically like small projects and then releasing them iteratively rather than going one big project. And that's kind of what I'm trying to do here with the TensorFlow Deep Learning course. So all of this stuff is, is not 100% ready. It's like the code is, is like close to 90% ready, but all the materials around the course aren't completely ready. So I'm trying to practice doing continuous delivery cycles with this course by just continually releasing new things to the GitHub repo. Um, so small update, release, small update, release. Anyway, now this is another great article by tecton.ai, um, why we need DevOps for ML data. So again, both of these resources I'm going to try in future issues of Machine Learning Monthly is to group like similar categories together. Like at the start we had TensorFlow stuff, and now here this is, this is more sort of MLOps stuff. So DevOps for ML data. Now again, what is DevOps? DevOps, Atlassian. I believe that's an Australian company, so shout out to Atlassian. Boom. So DevOps is a set of practices that works to automate and integrate the processes between software development and IT teams so they can build, test, and release software faster and more reliably. So it's basically what do you need to do, the operations around what you need to do to develop software and then get that into the hands of your users. You've got a little figure-eight circle here. Anyway, we come back to this great article, Why We Need DevOps for ML Data. Is a lot of the time, where's the, where's the key points? Well, developing, deploying ML is still slow and painful. Look at all these steps. Basically, in any process, when you have too many steps involved, it's, it's— you get exponential delays. So say this step up here was delayed. That means— or say this step was, was delayed by 10%. This step might be delayed by 1,000% because it just increases with each consecutive step. So if we come down here, so what's the hard part? So discover and access raw data, develop features and train models, productionize data pipelines, deploy and integrate the model, set up monitoring. Look at all of these, these different steps that are involved. And so what I believe Tecton.ai, uh, what their product does is— where is it? Come down here. This is a great little figure. So you're writing a machine learning model code. It's usually only going to be a small part of your overall machine learning pipeline. You've got data collection, verification, feature engineering, testing and debugging, resource management, model analysis, all these little steps. So this is, this is like, this is an MLOps diagram. All the steps around your machine learning model. We come down here. Challenge: accessing the right data source, building features from raw data. Oh yeah, this is all, all good fun. Combining features into training data, calculating and serving features in production. Anyway, I want to get to the solution. I mean, there's a lot of problems here, right? Introducing Tekton. So this is a product, I believe, for a data platform for machine learning. We're building a data platform for machine learning to help with some of the most common and most challenging workflows in data science. At a high level, the Tecton platform consists of feature pipelines, a feature store— that's really cool— a feature server, an SDK, a web UI, and a monitoring engine. So what Tecton does— where's the feature store? Anyway, it's— what is it? It's DevOps for ML data, right? So say you had some data and you, you somehow want to serve— you train a machine learning model and you somehow want to serve the outputs of that prediction. However, that data, chances are when you ingest it into wherever you, you want to feed it into your model, you're going to do a lot of different transformations. So how do you, uh, store those different transformations that you do to those data, that data that's coming in, in a reliable way so that you could share that, those exact transformations with someone else. So they get your basically your whole pipeline, not just the model, right? Because you can save the model and export that pretty easily. But we all know that the model is fairly useless without input data. So what I believe Tecton are providing is a solution. You can read the full article. It's a great article. It talks about problems. Not only— it's not just a selling point. Talks about the problems and the solution of creating a feature store of basically what those transformations are. So you can transport that with your model as well, so that if someone else wanted to use your model, they can ingest raw data, transform, transform it, put it through the model, serve it. Boom. And if that's wrong, you can correct me. Anyway, finally, let's finish up. We have some beautiful papers here. That I'm really— I really enjoyed reading. And they come around the theme of how hard is your data to model. So speaking of modeling data, the first one is Data Valuation Using Reinforcement Learning. And the second one is Estimating Example Difficulty Using Variance of Gradients, aka VOG. And the first one is DVRL. So Data Valuation Using Reinforcement Learning. This is the paper, but it was also a blog post. Where is it? Data Evaluation Using RL. I believe it was on, it was on Google's— there we go. Google's blog. So what I like to do is I read the blog post first and then read the paper. The blog post is like a high-level overview, then the paper's like a technical overview. So I see the blog post as a warm-up. So when you go on to do some sprints, you do a little walk beforehand. You don't just dive into sprints. Similar process. Anyway, they used reinforcement learning. Where is it? Here. I like this part. Quantifying the value of data. Not all data are equal for a given ML model. Some have greater relevance for the task at hand, are more rich in informative content than others. So how does one evaluate the value of a single datum? Hmm, that's a good point. So if we come down here, the results. So model performance after removing high/low value samples. So what they did, they have this data evaluation for RL, so using reinforcement learning to, to value how much a single piece of data or each piece of data contributes to a model's performance. And then on the Fashion MNIST dataset or Flower dataset, if you fraction of removed samples. So here we go. The blue line is DVRL least. So if you removed about 40% or let's go 35% of the least, the samples which offer the least most value, no, the least amount of value to the model, you see an improvement. However, on the contrary, if you remove about 50% of the most valuable piece of data, your model performance just falls through the floor. And a similar pattern happens on the flower dataset. So this is incredible. And if we go a similar process here was with estimating example difficulty with Variance of Gradients. So they're, they're both solving a similar problem or working towards solving a similar problem in figuring out how valuable a single piece of data is. But they're using different techniques. So Google's paper uses reinforcement learning, whereas this paper by Sarah Hooker and Chirag Agarwal uses a different technique comparing the gradients of samples as they move through a model. So if— put it this way— if your gradient descent algorithm is between each iteration, if the variances of its updates are wild, so large, chances are that that sample it's trying to update the weights on is a difficult sample to learn. But if the variance is small, as your gradient descent algorithm updates the weights of your neural network as it's, it's learning on different samples, if the variance is small, like, so this one, the weights were here, this one, the weights were here, this one, the weights were here. If that variance there was small, chances are the gradient descent algorithm is having an easy time to learn that sample. That's the intuition there. I'll let you read the paper for a more technical overview. But come down here, I really like this, this image here. So they created a score, variance of gradients. So these images had the lowest variance of gradients. So remember what we just discussed, the gradient descent algorithm didn't vary updates of the weights didn't vary much as the neural network was learning. So these are pretty clear images of magpies. Whereas this one, highest VOG, so lots of variance in the gradient descent or the gradient updates. Look at this. These are all in comparison to the lowest score. These are all like fairly hard images to— well, some of them are quite like that one's pretty clear. Clear. That one's pretty clear. But in comparison to these, like, these are all similar, if you know what I mean. So why is this important? Well, as we begin to sort of— like, now we're entering the stage with trillion parameter models and really large models, is that they're sort of— they're not limited by, um, compute power, because I mean, there's, there's compute power is just increasing, increasing, increasing. They're limited by the amount of data that they can ingest. So I mean, GPT-3 ingested basically an entire internet worth of data. So we actually don't have more than that. Or we probably could. We could figure out different ways. But what if we could use methods like variance of gradients and reinforcement learning, data evaluation using reinforcement learning to figure out— because there's probably a lot of noise in whatever data sample you're passing to your model. So of course you want your model to have some sort of generalization ability by having some noise in your data. But what if you could also just remove the, the samples that aren't offering much at all? So like they, they show in the paper— where is it? Where's that fun diagram? Like this. What if you could find the perfect balance, like the precision-recall trade-off? You could do the, the data-value trade-off by removing the samples that contribute the least to the model and focusing on the ones that contribute the most and still provide the same level of generalization. That's— I'm excited for that going forward. Anyway, the final one we're going to finish— I lied before. That's not the final one. This one, this is a funny one. And it's also a lesson into how fragile AI systems could be. So, and what do we got here? A video of an AI video camera mistaking a referee's head as a soccer ball. And the referee— there was a specific factor about why it mistook this referee's head as a soccer ball. Look at this. This camera is, I'm guessing, running a model to, to follow the ball, but it keeps zooming on the referee's head because the referee doesn't have any hair. And I mean, you can understand, okay, maybe— look at that, the soccer ball looks looks quite similar to the referee's head. So maybe whoever the developers who trained this AI model would benefit from reading papers like this to figure out what their dataset might be missing. Or they could just do the simple thing and just create a class of not soccer ball and add lots of photos of bald referees' heads in there. Oh, so that's, that's a great funny one to leave on. So if you are building an AI system to detect soccer balls, make sure, make sure if you want to avoid a mistake like that, make sure it contains a— your classification model contains a class, not soccer ball. And then in that soccer ball include, be sure to include pictures of bald referees' heads. But anyway, that has been Machine Learning Monthly for November 2020. As I said, if you want the link to it, it'll be in the description. If you want to send me your submission for anything, could be anything at all. Like, I mean, you could do your own work, you could do someone else's work, whatever. Send me an email, daniel@mrdanielbourke.com. Leave a comment below what your favorite was. But in the meantime, keep machine learning, keep creating, and I'll see you next month. Peace. I gotta go, uh, lose a kilo for tomorrow's jiu-jitsu comp. Stop, stop, stop!