Who said robots can't dance? | Machine Learning Monthly December 2020

Hello and welcome to Machine Learning Monthly for December 2020, a show where we go through the latest and greatest of machine learning over the past month. Well, that's actually a little bit of a lie because sometimes the things we go through are older than a month, but the reason we go through them is because they're worth going through. So of course, this originated as a newsletter and a blog post. Let me just show you it here. There we go. So if you'd like to read this without listening to me ramble on about it all, The link will be below. You can also sign up. It will be delivered beautifully to your inbox at the start of every month. And as you'll notice, the video comes a little bit after. But Happy New Year and let's get into it. I'm a little bit nervous about this one because it contains some things that I don't quite fully understand, so I'm going to have to try and explain them. I'll just tell you why I find them important. How about we do that? And if you want to find your own explanation, the resources will be there. Let's get into it. December, woo-hoo! Now, what you missed in December as a machine learning engineer. So my work, I am working on a new course with TensorFlow. So TensorFlow Code First Deep Learning. Let's have a look at the GitHub repo. You can go check this out. I'm going to be updating this fairly often as I start going through it. There was a little update a couple of days ago. Here's what's happening. The biggest update so far of the README has been all of the exercises and extracurriculum exciting to go along with the notebooks. So how it's structured, I make these notebooks, I have a whole bunch of code and annotations, and then I record videos talking through all the code and annotations, and then I set exercises and extracurriculum. But enough about that. You can find the link for that where you find links. And so check that out if you're interested in learning TensorFlow for deep learning. You will require a beginner machine learning course. Such as the one I also teach. There's a couple of plugs for me to begin this video. But let's go in here. Another thing that I did, how good is Apple's new M1 chip for machine learning? So in the last issue of Machine Learning Monthly, I said that, or we looked at the fact that TensorFlow had just released a version of TensorFlow macOS, which is TensorFlow to run on Apple's, on Apple Macs. Because if you've ever used TensorFlow on a Mac before in the past, it hasn't been the greatest. So we did a little bit of a speed test. Where we go? We'll go to the blog post. There's also a video to go along with it. But I just did a speed test comparing the new M1 MacBooks to my 16-inch MacBook. Let me zoom in a little bit for that. And let's just say this, without going through all of it, you can read the title, Apple's new M1 is a machine learning beast. It definitely is. Like, it beat my 16-inch and it also, it's just, it's just fast to run in general. So read that if you want to have a look at the full test and whatnot. Let's go here. Now let's get into the best from the community. I'm already sweating. See, this is Australian summer. You know, it's, it's 8:00 AM. It's barely just past 8:00 AM and it's, it's hot, hot. But I'm also sweating because I'm excited to go through all these amazing machine learning resources. The best from the community. So this one is from Yogesh. So thank you for the submission, Yogesh. Of course, if you want to submit something to Machine Learning Monthly, you can find my email. It's daniel@mrdbourke.com. Send me it. Put Machine Learning Monthly submission in the subject. And if it's good enough, it'll come into the Machine Learning Monthly issue. Now, You've seen pre-built machine learning models, such as with TensorFlow Hub or something like that, but how about pre-built machine learning pipelines? Now, I was pretty amazed at this. Hopefully it's still live. This is from JREX Train Generators GitHub. So here we go here. I'll give this one a little bit of a star. Amazing work. What's your name? What's your first name? Oh, there we go. Johannes. So amazing work, Johannes, and thank you to your guests for submitting this. So what Johannes's web app does, it's a web app to generate template code for machine learning because machine learning is getting pretty, hmm, how would you say, pretty mature. So in terms of a lot of the frameworks and things that you do, see, look at this little beautiful graphic. Aren't GIFs just amazing for sharing work? Let's have a look. Try it out. So what this is, is a code generator built with Streamlit. Jumpstart your machine learning code. If we specify the model in the sidebar, click on it, closed training code will be generated, download and do magic. Okay, let's see what we should do. Task image classification. PyTorch. Yes. Why not? Which model? I would love to see EfficientNet here. Let's go ResNet. Yep, ResNet 50. 50 is a good number. Input, yep, I'm working with NumPy arrays. Use GPU, cross-entropy loss, beautiful. Adam, trusty Adam, always trust Adam. And, oh wow. Download.py. Look at this. Pre-processing. Oh my goodness. Model ResNet-50 training. See, this is really cool. So if you wanted to learn what some PyTorch boilerplate looked like for, for machine learning, you could come in here, generate it, and then copy it out yourself, like write it out by hand rather than downloading the whole thing. That way you would understand what goes into a machine learning pipeline. So amazing work here from Johannes. A really cool project to share what, what you can do with machine learning. And of course Streamlit is just phenomenal. That was in a previous episode of Machine Learning Monthly. Streamlit.io. If you want to create some machine learning based apps or data based apps, check out Streamlit. So another shout out to Streamlit. Anyway, let's get into the best from the internet. So this is one I don't quite yet fully understand. I mean, I've read the blog post, and I've watched a video or two, but that doesn't mean you fully understand it. So I wonder if the paper's out yet. But anyway, it's what was making headlines all throughout the machine learning artificial intelligence world last month. AlphaFold 2. Now they kind of confuse me when they put AlphaFold here, but really AlphaFold 2 was the one that got the really good results in, in biology. Anyway, it's a marketing thing. So long story short, How do we describe this? There was a hard problem in biology. We have been stuck on this one problem, how do proteins fold up, for nearly 50 years. To see DeepMind produce a solution like this, having worked personally on this problem for so long, and after so many stops and starts, wondering if we'd ever get there, this is a very special moment. So this is Professor John Molt, who's a professor at the University of Maryland, very smart gentleman. Saying that basically— have they got a one-liner of what it does? No, they don't. Is the paper out yet? Published a paper. Ah, okay. So I don't think the paper's out yet. So until we published a paper on this work— so I haven't fully read the paper. If there is a paper, let me know please in the comments. But essentially what DeepMind did, and this is the amazing thing, if I recall back to my biology classes, if you imagine an amino acid sequence is made up of letters. I actually wrote this down like this. A, C, T, C, T, T, right? So that's a sequence. Now, if you put those sequences together, they code for some sort of protein. And if that protein folds in a certain shape, let's go back to the blog post, folds in a certain shape, let's say, Where is it? Really complex shape like this. Look how complex that is. So if we start from just simple letters such as ACGT, et cetera, et cetera, times that by a lot, and then they combine to fold up into some sort of different shape. The really interesting thing about these shapes is that they, they tie closely to what the protein actually does. So its role. So where AlphaFold is sort of what, what AlphaFold can do is predict the shape of what a protein is going to, to turn into, to fold into. Now, why is that important? Because then you can start to predict what, what role a certain protein is going to have. Now, again, what are the use cases for this? Well, this is, this is groundbreaking research. We're not quite sure what they are, but I imagine things. Now, this is where my explanation is probably still in thought space. It's not confirmed. I imagine if you could predict the, the sequence of a protein and the shape it goes into and subsequently its role in the body, you could potentially develop some sort of treatment for some sort of illness. Again, big caveat on this. I'm not entirely sure. I've only read the blog post. I don't fully understand what's going on here. But my favorite thing was this. They used to make this was an attention-based neural network system trained end-to-end. So why is that exciting? Well, when I first got into machine learning, and I sort of knew a little bit about biology, so ACGT, etc., etc., I was like, wow, and then I learned about, sorry, LSTMs and sequence-to-sequence models. I was like, why can't we apply sequence-to-sequence models to proteins? And of course, I was young and naive, but it turns out the attention-based neural network system, if you're not sure what an attention-based neural network system is, Look up Transformers. They're an architecture which are basically blazing the field of NLP, which is essentially finding patterns and sequences of information. And they're doing the same thing here. Of course, that's a very simplified version, but it makes me want to dive deeper into learning about attention-based neural network systems. So go check this out. This is groundbreaking work. We'll probably look at this in Machine Learning Monthly Episode 355 and go, remember AlphaFold 2? Well, now AlphaFold 7 is coding up full humans. I don't know. Don't quote me on that. I'm not a fortune teller. Phenomenal work from DeepMind there. I need to take a quick break and get a towel. I'll be back. Oh, we will soldier on. We've got more machine learning, beautiful things to go through. Another one from DeepMind. Breaking records with no rules. This is MuZero. Another thing I'm phenomenally excited about. It was a massive back end of the year. For DeepMind and OpenAI. So let's look into MuZero. Again, this is one of those things I don't fully quite understand yet. I've only read the blog post. I need to go through the paper myself. Um, but nonetheless, it is phenomenally exciting. So MuZero, mastering Go, chess, shogi, and Atari without rules. Now, why is this one exciting? So if you remember all the cool things, why don't I move my head? Move me over here. Yeah. If you remember all the cool things that AlphaGo did, can you still see that? Oh, I was probably better up there, wasn't I? There we go. Haha. Working things out on the fly. Now, if you remember all the cool things that AlphaGo did, it became the first program to master Go using neural networks and tree search, but it had some knowledge. It had human data, domain knowledge, known rules, and then AlphaGo Zero, was basically the same thing as AlphaGo, but this time it didn't have any human data. It didn't have any domain knowledge. It only knew the rules of the game. And so learns to play completely on its own without human knowledge. And then AlphaZero comes out and it's able to do Go, chess, and shogi. So 3 games and it only knew the rules of the games. And then come down to MuZero. Well, it can do Go, chess, shogi, Atari without anything. I actually, yeah, how did it do this? I'm not quite sure. I read it and it's like, well, I need to, I need to reread this about 3 or 4 times to understand. But MuZero learns the rules of the game, allowing it to also master environments with unknown dynamics. So if you imagine Go, chess, and shogi, I believe, are perfect rule-based systems. So they have like a set, like chess has a set number of rules that you can do. But of course there are many different combinations you can do those rules, but the rules are set. Whereas where MuZero kind of shines is that not only can it do Go, chess, and shogi, so these like fixed rule games, it can do Atari games and get superhuman results. And so Atari games, of course they have some rules, but the way you can apply them, the space of the rule sets is just so phenomenally large. It's Incredible. So where is— so MuZero, it models. So in reinforcement learning, it doesn't often learn a model, it learns a policy. So it learns— the saying goes, it's something like supervised models learn a model from the data, whereas a reinforcement learning model learns data from the model. Someone smart said that and it rung true in my head, but I haven't fully quite grasped why. Anyway, μZero models 3 elements of the model— of the environment, sorry— that are critical to planning. The value: how good is its current position, where it is in the game? The policy: which action should it take next? And the reward: how good was the last action? So these are all learned using a deep learning neural network and are all that is needed for μZero to understand what happens when it takes a certain action and plan accordingly. So that's phenomenal. Now, I believe where the combination is— well, here's the Muse0 performance. I'm not going to go into that. All you got to understand is that Muse0 just wipes the board with basically everything that came before it. It combines deep— where is it— deep neural networks and search. Yeah, there we go. So it combines a model, a deep learning neural network model with AlphaZero's powerful lookahead tree search. So they're the two combinations. Now there is a paper available and you know what? I actually did even read through the paper, but it's one of those things, and this is important to know, like with complex topics and new things, like this is groundbreaking research from multiple scientists. When you first read it, it's probably going to sound like, what's going on? But then as you keep going over it, it starts to make sense. And then probably in a few years' time, we'll look back at MuZero and go, yeah, of course we do things that way. So what is the applications for this? Where is it? The funnest thing. Yeah, here we go. So MuZero's ability to learn a model from its environment and use it to successfully plan demonstrates a significant advance in reinforcement learning. And the pursuit of general-purpose algorithms. So that's so exciting that it can, uh, apply itself to 3 different rule-based games and then sort of an open space game like Atari. The ideas behind MuZero's powerful learning and planning algorithms may pave the way towards tackling new challenges in robotics, industrial systems, and other messy real-world environments where the rules of the game are not known. So one of my sort of things that I'm really excited to see if it can be applied to is self-driving, because the rules of driving are not really known. You kind of, well, there are some basic rules to driving, but the way that how you apply them is such an open space. So really cool. Check out that from DeepMind. Now let's get into the next one. New year, new TensorFlow upgrade. So TensorFlow 2.4 is out. With a bunch of quality of life and performance improvements. My personal favorite, I'm not gonna go through each one, is, where is it? Updates in Keras. So, mixed precision. Let's have a read. In TensorFlow 2.4, the Keras Mixed Precision API has moved out of experimental and is now a stable API. So, most TensorFlow models use the float32 dtype, however, there are lower precision types, such as float16, that use less memory. So Float32, how I understand, is, is a number stored in 32 bits of memory on your computer. But say your computer only had, let's say, a really easy number, 100 bits of memory. It has probably a lot more, but if it stores it in 32 bits of memory, it can only really store 3 numbers. So what mixed precision does is why don't we just store those numbers as 16 bits of memory and take up half the space? So now you can store 6 numbers in that 100 bits of memory. So mixed precision is the use of 16- and 32-bit floating point types in the same model for faster training. This API— this is the most important point— this API can improve model performance by 3 times on GPUs. That's amazing! More than 2 times, 3 times. And 60% on TPUs. So check out the mixed precision tutorial if you want to check that out. I'm personally trying to figure out how I can best use that, going forward as well, to use up less space and utilize my hardware as best possible. So, massive updates from the TensorFlow team, thank you so much. Again, we missed out on a whole bunch more, we only covered my favorite one, so check those out. Now, if there's one thing I'm searching in life, it's for beautiful explanations. So if we come in here, this is the kind of blog post I'd like to write. That's when I know something's good. If I read something, I'm like, Wow, I wish I did that. That's how I know it's good. Same with videos, et cetera, et cetera. That's my one rule actually for making things is like, would I like this to exist in the world? So Backpropagation 101, the algorithm that powers many of the neural networks and machine learning models that we build. How to trick yourself into understanding backprop without even trying. This is from the team at think.ai. So I believe think powers spaCy. Which is, there we go, thinks redesign is brand new, but previous versions have been powering spaCy, which is an industrial-strength natural language processing framework. So if we go into here, this is a phenomenal blog post. If you want to understand backpropagation from a high-level overview, you could go through this blog post and you will be tricked into understanding backprop. Otherwise, this is like a phenomenal way of like explaining something. So it's only with code and words. So backprop, as you know, if you've been familiar with it, it has a lot of math behind it. So of course that's important to know the math behind backpropagation. Where is it? Yes, you should know backprop. This is a great post from Andrej Karpathy, head of AI at Tesla. So that's a bonus point for Machine Learning Monthly, this episode. But anyway, this is a phenomenal explanation from the Think team of what backpropagation does. So I'll let you read through that. It uses the example of 3 work colleagues all working together to try and improve some system and how they receive feedback, et cetera, et cetera. And it tricks you into understanding what's going on in backprop when you're writing it for neural networks. CS50+AI. Now CS50 have a new 2021. See, this is confusing too, right? Because I went on to CS50. This is artificial intelligence. Let's just dive in here. There we go. Some great diagrams. And it's published on New Year's Eve. However, if you look at the CS50 website, CS50— oh, it's even changed color. Now where is, hmm, announcements. They said something about 2021, spring 2021. Look, I was on some CF50 pages, a fair few of them actually. So that's why I got confused. But they have a new artificial intelligence module which goes through. So neural networks have kind of taken over the field of artificial intelligence, or at least they are the ones you hear the most about. There's actually a lot of different other components such as decision making, decision trees, which are very important, minimax, depth-limited minimax, depth-first search, breadth-first search, heuristics, greedy-based first search, et cetera, et cetera, genetic algorithms, finally neural networks. So if you want a very good overview of most of the working methods and popular methods in artificial intelligence, definitely check out CS50's artificial intelligence course. I believe they also have— it's free. It's on YouTube. If you can find it, please leave a link in the comments because the CS50 website, although it has phenomenal information, it's quite convoluted to try and find things. So this is like their 2021 announcement. And I know it says 2020, but I'm pretty sure this is the 2021 version. Correct me if I'm wrong. But anyway, CS50, phenomenal to learn the fundamentals of computer science. And now they're getting into AI. So exciting. Anyway, I say anyway too much. I need to fix my speech. This is the benefit of recording videos is you get to listen to yourself and see, hey, you say um, and you say hmm, and you look down and you, you don't pay attention to what you're saying. Pay attention to what you're saying, Daniel Bourke. Adaptive models, models which update themselves based on unlabeled test data. So this is a key point here. Many machine learning models are built off the assumption that training and test data are from the same distribution. So say you're building a food detection app and it detects 10 different classes of foods. Your assumption is, and you're evaluating your machine learning model on the training, or you're training it on the training dataset and you're evaluating it on the test dataset. The assumption is that if your evaluation metrics are really good on the test dataset, is that those pictures that it's being evaluated on are quite similar. This is what it means by from the same distribution to what the images in the training set are. However, if you ever deployed a machine learning model into the real world, you know that that assumption quickly breaks. So what adaptive models is, is again, this is brand new research. I'm not quite fully around it yet. So this is why I link the blog post and the papers for you to read. Adapting on the fly to test time distribution shift. So that means test time distribution shift is when your model goes out into the real world, that's test time, and suddenly the data that it's being used on in the field is not exactly the same as the data that it's been trained on. In turn, your model's performing poorly. So where we go here. Imagine that you're building a next-generation machine learning model for handwriting transcription. Based on previous iterations of your product, you have identified a key challenge for this rollout. After deployment, new end users often have different and unseen handwriting styles, leading to distribution shifts. Yeah, so that's a challenge. So if we come down here, what does it do? Where is it called? This is it. Adaptive risk minimization. So The work here from the Berkeley AI research team is, our work proposes adaptive risk minimization, or ARM, which is a problem setting and objective that makes use of both groups at training time and batches at test time. This synthesis provides a general and principled answer through the lens of meta-learning, the question of how to train for test-time adaptation. In particular, we meta-train the model using simulated distribution shifts which is enabled by training, by the training groups such that it exhibits strong post-adaption performance on each shift. So what they do, they meta-train a model using simulated distribution shifts. So while the— this is how I understand it. You have training data of some sort. So it's over here. And while it's training, you, you create some sort of simulation as if that training data was changed. So you simulate the fact that that the training data has changed as if it would be to, to be used on unseen data. And you train the model on both the, the training data and the simulated change at the same time so that hopefully your resulting model is more robust to test time distribution shift. Again, this is— take that with a grain of salt. This is something that I've just gotten into and just found, so I'm not completely around it. That's how I understand it. Where is the results? Here we go. We found that our methods empirically resulted in both better worst-case and average performance across groups compared to prior methods, indicating both better robustness and performance from the final trained model. So some phenomenal research there for making your machine learning models better in the real world. Let's go back. Software engineering cross AI, a systematic review. So I had a question the other day from my email saying, Daniel, how do I— oh, excuse me, had to burp. I had a question, Daniel, how do I get into software engineering? That was a great question because I'm personally trying to figure that out myself. And software engineering for artificial intelligence and machine learning software review is probably the biggest piece of literature out there on the mo— at the moment. Again, caveat to this one, I haven't read through all of this, but if you want a massive resource on how software engineering can partner up with AI, look, we're still scrolling. Let's just keep going. You can go through here. So I'd probably look at this like the table of contents. We're still going, by the way. Look at the table of contents of this and see what sticks out to you the most. We're into the references. So what is it actually? It's about 40 pages. 40 pages of how ML and software engineering and AI combine together from a whole bunch of different resources. So this is some phenomenal work to whoever put this together. Thank you so much to— oh my goodness, I got 4 names there who I'm going to do a terrible job of pronouncing, but Thank you to these people. I'm going to leave it at that. Let's come back here. So yeah, if you want to look into software engineering for AI and machine learning software, this is a great literature review. I've skimmed it. I haven't read it all. But yeah, that's probably what you want to do too. We go into here. Bored? Just replicate yourself with AI. And who said robots can't dance? So this is, this is a fun part. An AI-powered news anchor just made its or her, what do I say? I'm not sure here. It's 2021. I don't know. In South Korea, artificial intelligence powered news anchor makes its first onscreen appearance in South Korea. So if you're bored of reading the news, why don't you just replicate yourself with AI? You know what? I'm done. I'm done with Machine Learning Monthly. I'm going to insert my AI here and he's going to take over the next one. So Jeeves. No, I'm kidding. I don't keep Jeeves in the cupboard. That would be mean. Keep him under the stairs. Um, so apparently the news anchor can actually read 1,000 words per minute. So not only can your news anchor read your news, uh, can read a lot faster than you can as well. So this is a really— how do I even describe it? Like a thought-provoking use case for what AI can be used in the future. So there's a video. Can I play this on my channel without getting banned? Who knows? That is wild. I can't speak Korean, but it's just like the mannerisms of this. If that's AI generated, that's Wow. I'm not sure what he's saying. Maybe it's sunny in Korea. But that's just, who knows what's going to be created in the future if we're at this stage already. And then finally, of course, we've got to finish this off with a bit of dancing. We didn't start it off with a bit of dancing. I know. Thank you for pulling me up on that. This is a phenomenal video by Boston Dynamics. Of a robot dancing. I can't play this one now. If you need, if you need moves for your next party, watch this video and copy these robots. Oh, can't keep playing that. YouTube's going to ban me. Damn it, YouTube. Um, but I watched this video, it warmed my heart. Such a great— I don't even know how to explain it. I know these are robots, but it just, it gave me this warm fuzzy feeling inside and it made me want to dance. That's how I know something's good. If it makes me want to dance, check out that video, copy the dance, bring it out at your next party. And I have a question. If a robot does a particular dance move, is it always called the robot? Oh, but that is it. Machine Learning Monthly for December 2020. Happy New Year. It is December. I mean, it is January 2021. I may do the next episode with my AI-generated version of myself, but as always, leave a comment below if you'd like to, or what was your favorite? But if you'd like to submit something, also leave a comment below or send me an email to daniel@mrdbourke.com with the subject Machine Learning Monthly submission, and we might see it in a future one of these videos/issue. If you'd like to read the blog post or sign up and have it delivered directly to your inbox, the link will be below. But as always, keep learning, keep creating, and keep dancing. It's not the same without music. I'll just put some music over the top of this so it looks like I'm actually dancing. I speak Swedish. We'll see you next time.