AI makes you an artist + Tesla's self-driving car updates | Machine Learning Monthly August 2021

Friends, welcome back. It is that time of month again, your favorite time of month, Machine Learning Monthly. The video, the article, the newsletter where we cover the latest and greatest of machine learning, but not always the latest because some things have stood the test in time. It is August 2021. Let's get started. If you'd like to get this article, the link will be in the description below. That means you don't have to listen to me talk. You can just click the links and all that sort of stuff. We start off with my work. I'm super, super, super excited super excited about this. My first book, my first novel, a fictional story of Charlie Walks. Charlie is a machine learning engineer working at the largest technology company in the world, writing code at night, working on his secret project, XK1. Oh, sorry, during the day he writes code, at night writes letters to his nephew Paulie, because Charlie is a machine learning engineer but wants to be a writer. So you can check that out, charliewalks.com. That's what I've been working on the past couple of years, and it is now out. The digital versions are out, the print versions will come later in the year. And so we have submissions from the community. Now I love to see this, especially this one. The Artificialis— I hope I'm saying that correctly, 'cause I believe it is Latin for artificial— is a Medium publication and Discord server created by members of the Zero to Mastery Academy. So that means people who have done Zero to Mastery courses, whatever they may be, have gone off and created their own community to share their learning journey and works with the world. So there's articles on the Medium publication such as Your Path to Become a Machine Learning Engineer, by Alessandro Lamberti. Big shout out to Alessandro. You might have seen him in the Machine Learning Data Science Discord channel. He is everywhere. He's amazing. And then of course, Networks, Neural Networks 101 by Ashiq Shaifi. Again, you might have seen him in the Discord channels. Next, we have Writing Online. Now again, these are community submissions. If you'd like to see your work in a future edition of Machine Learning Monthly, please send me an email at daniel@mrdbourke.com with the subject title Machine Learning Monthly Submission. And you might see it here. But as it says here, I am a huge advocate for writing online. I would say that starting my own blog, mrdiburg.com, writing on Medium, various other places online is probably the best thing that I have done in the past 4 or 5 years of creating things online. But we have last month's ML Monthly, we covered Gradio. So if you look at Gradio, which is the easiest way to share your machine learning demos Highly, highly recommend checking that out. Pierre Pablo Ippolito, I hope I'm saying that name right, saw that and said, hey Daniel, I've got a Medium publication where he writes online. And I've also got an article on how to get started with Gradio. So check this out if you want to get started, start demoing your machine learning models, get them into hands, into the hands of others with Gradio. And this beautiful introduction to Gradio By Pierre. So thank you so much for the submission there. And then we also have Nifasimi's blog. Uh, if we go here, Nifasimi reached out to me saying that they're hoping to start a career as a technical writer. And so what are they doing? They're starting the job before they have it, which means writing about technical things before they even have a job as a technical writer. And that is my one piece of advice always, if you want to get a job in anything, whether it be starting your own business or a job at a tech company, is start the job before you have it. So that is enough from my work. That is also the community submissions. Let's now look at some resources from the internet. Are you a data scientist, engineer, analyst who does ML, or are you something else? So Eric Bernardsson— I think I'm saying that right again. All these names, you're probably gonna be like, wow, can Daniel say a single name right?— had a beautiful article in last month's Machine Learning Monthly July 2021, which is a story of a data journey at a startup mid-sized technology company. This time there's a beautiful tweet here. I think this specialization of data teams into 99 different roles— data scientist, data engineer, analytics engineer, ML engineer, etc.— is generally a bad thing driven by the fact that tools are bad and too hard to use. So yeah, when, whenever someone asks me, should I be a data scientist, data engineer, analytics engineer, ML engineer, my, my opinion on that is really I'm very biased here, so keep that in mind, is I'm more of a generalist. So when you start to do one of these, you'll do a little bit of all of them. However, I— as Eric says, specialization is a good thing. I'm not going to reread the whole article. You can read it here. But the main argument of the article that I picked out was, are you a tools-orientated person or a goals-orientated person? And so the debate, uh, a question I get a lot is, should you use TensorFlow or PyTorch? Should you use Python or R? And it's like, you could spend all of this time trying to find the perfect hammer, or you could pick one and start hammering nails. Can you get the job done with the tools that you currently have? So go and read that article for some more, rather than an argument for and against specialisation, just taking a step back and going, hey, is this the root problem we're trying to address? Next up, we have "Want to become a data engineer? Here is a roadmap." Now this is a beautiful modern data engineer Speaking of data engineers, by the way, roadmap, uh, 2021. Now this is by the team at Data Stack TV. So thank you so much for creating this. If we come in here, when you first get into the data industry or tech industry in general, there can be a lot of different terms that just sort of throw you off, like CS fundamentals. In fact, I would have even put computer science fundamentals, because if this is for a beginner, they might not even know what CS is. Computer Science Fundamentals. If you start off here, you want to know basic terminal usage. Now, what's a terminal? We go into here. How do you write commands in here? So if we go cd Desktop, what does that do? ls, what does that do? Right? Terminal. Data Structures and Algorithms. APIs, REST. Now, all of these things, if you never heard of them, are going to sound foreign to you. But as you get into the world of computer science, you will figure them out. Other questions like, how does a computer work? How does the internet work? Git, what do you use GitHub for? CLI, Vim, so on, programming languages, Python. It's got a little heart here. It's close to my heart too. I didn't make this. They just put a heart there. We come down, it goes all the way down to different types of data processing and databases and messaging and networking. A lot of things I don't even know what they are here, but I'm glad to have these steps here to sort of put a— it's more of a, not necessarily a straightforward map, like as in you should do all of these. It's more of a compass to sort of direct you in the right direction. Direction. Like, hey, here's what the, the community is using, so you might want to check it out. It might not be for you, it might not. So on and so on. There we go. TensorFlow, PyTorch, Keras, etc. Oh, now this is an exciting one. Number— what is this, number 3? Number 3: if you can't beat them, join them. The combination of CNNs plus Transformers. So if we look in Facebook AI, this is some new research that come— that's come out of Facebook AI's team. Computer vision, better computer vision models by combining Transformers and convolutional neural networks. Now we've covered Transformers a fair bit in previous Machine Learning Monthlies. We all know how good they are for NLP and they're starting to make their way into vision. However, there are some sort of caveats between Transformers and CNNs. Like one has, one's very strong. If we go down, let's find it. Yeah. A CNN has a hard inductive bias. What that means, if you imagine an image, It means that every pixel is treated as the same. It gets the same weighting along the image, and they assume that pixels close together are closely related, whereas a transformer model sort of pays attention— thanks to the attention mechanism— pays attention to what needs to be paid attention to. Now you can dig deeper into that. That was a very, very high-level overview. The benefit of a hard inductive bias from CNNs is that they perform well with smaller amounts of data. However, they have a hard ceiling of how well they can perform with lots of data, whereas Transformers kind of have really no ceiling of how good they can perform. The more data you feed them, the better they get. So if you combine them, you get the best of both worlds. So this is a beautiful, beautiful blog post describing what they use to sort of combine them, which the main component is the gated positional self-attention. That's a mouthful. GPSA. And I believe, yeah, ConvIT. We have here sample efficiency. The new ConvIT architecture outperforms the data-efficient transformer using much less data. However, as the amount of data increases, the performance gains decrease. Onto the next one, number 4. So you want to become an AI artist or an artist in general? I remember in year 8, my art teacher— I drew the most beautiful Ninja Turtle ever. My art teacher told me it was a bad drawing and I thought I could never draw. But then VQGAN and CLIP came out and it allowed me to create AI-powered artwork. So for this example, I tried to create AI art. And if you're wondering, VQGAN, let's have a look. VQGAN is a type of generative adversarial network. There we go. We got this great article here generating AI art. With VQGAN CLIP. There we go. Vector Quantized Generative Adversarial Network and Contrastive Language Image Pre-training. Whoa. Okay. I prefer the acronym VQGAN CLIP. CLIP is another algorithm from OpenAI which connects text and images. So go and read that blog post there. I'd highly recommend that. We've covered that in a previous Machine Learning Monthly. Amazing, amazing research. But what do you get when you combine these two? All right. Well, you get an algorithm that is capable of generating some wild artwork. So for this example, I told it to start here with this little tightrope walker, which is related to my book if you want to read it. And I wanted it to try and get to recreating the COVID And I fed it a prompt with some text. Remember, CLIP connects text and images. Man with a mohawk walking between nature and city landscape. Unreal Engine. Now, Unreal Engine is a graphic design program. It's not my wheelhouse, but I read in the blog post that's linked here, if you want to create real-looking art, you can try use some tricks like using Unreal Engine. And it ended up creating something like this after 1,000 iterations with a Colab notebook. So sort of looks real, sort of looks funky, a little bit cyberpunk-y. But I did also try it with a random— like, so no prompt, no starter image, no goal image. And it created this psychedelic landscape. I mean, Be careful if you're on mushrooms and looking at this. I don't know what I'm seeing here. Maybe some floating cities that could be like a giant shrimp dragon floating through space with a purple flowery background or purple tidal waves and whatnot. Maybe that's like a little samurai of some sort. I don't know. What's your interpretation? Leave a comment below. If you want to generate your own AI-powered artwork, Roboflow have a great blog post explaining their experience and there is a Google Colab notebook where you can use Google Colab to create Here we go. We'll get out of this. Use the VQGAN algorithm and create an image like I've created with your own starter image, target image, and computer text prompt. Now we have number 4. Self-driving cars are always 5 years away, but it looks like they're getting closer. So recently, over the last month, Tesla, Comma, Comma AI, and Waymo, 3 of the largest self-driving car companies, had sort of either conventions or a blog post announcing their, the latest and greatest of hardware and technology and techniques that they're using to move towards full self-driving. And they've, they've come a long way. The biggest one was probably Tesla's Autonomy Day keynote, which I highly, highly recommend checking out. There are some beautiful things in here. Uh, one of my favorite was— let's jump into the auto labeling now. I've got a little image here which is describing half a dozen or so of my favorite takeaways. There is a clip labeling, how a clip goes from straight video collected from the Tesla car and then they automatically label it. And then if you wanted to find better examples, so you say you had a really rare example, I think the example they used was two people running along a highway with a dog. If you wanted to find rare examples, you can summon the fleet or ask the fleet. It's like, hey fleet, show me more examples that look like this scenario. I think that's beautiful. As we all know, in the world of data, if you want better examples or if you want a better model, you need to train it on examples that are rare. And then automatic mapping. So some other car companies are using HD maps. So they're crafting these maps by hand with, with LiDAR and other techniques and whatnot. But Tesla, because they have so many cars out there, they can create automatic maps from vision only. Because say you had an intersection, a four-way intersection, and over time hundreds of cars or thousands of cars even went through it. Over time you'd have enough data to sort of go, okay, I'm pretty confident that there's a lane here, lane there, a lane there, and there's a turn here, etc., etc. And the beautiful thing about automatic mapping is that it updates over time. Then of course, there's scenario generation, or scen-gen for short. I like to call that scen-gen. Scen-gen, it just rolls off the tongue really nicely, you know, is that you can create with the data they have. They've created photorealistic scenarios or simulations of actual driving, and then they train their models on the simulated environments. Now, of course, the models that are deployed in the cars are only ever evaluated and tested on real-world data. The scenario generation is only ever used for training, not testing. And then Dojo, which is Tesla's custom AI chip, the fastest by, by their metrics. I'm not too up with the computer chip in terms of flops and whatnot, what those, those speeds mean. But according to the graph that Tesla showed, it's the fastest out of everything that's currently available. So for machine learning training, and the beautiful thing is if you want to use it On PyTorch, you can just type in torch.device and feed it the dojo name. And then finally was the Tesla bot. And I really liked what— I have no idea where this is going, but I really liked how on the slide Elon said that the bot is friendly and he said that you'll be able to run away from it. We're setting it such that it is, um, at a mechanical level, at a physical level, uh, you can run away from it. And you should be able to overpower it. It can only lift up to 45 pounds, which is about 20 kilos. So if you've got some— if you got— if you're rocking pythons while riding Python, you should be able to overpower the bot. But highly recommend checking out the Autonomy Day video in full. Uh, beautiful to see some of the latest and greatest in the self-driving car space. Then of course there was Waymo's blog post, which was how they're moving towards full self-driving with The world's most experienced urban driver. And not gonna lie, Waymo produced some pretty cool graphics in this one. I think the main trend that I noticed in Waymo's blog post compared to Tesla's was the emphasis they used on different sensors. So let's see. Sensors. Our advanced sensors. Most advanced sensors. Comprised of complementary sensors. Most powerful sensors. The powerful combination of these sensors by our sensors, sensors, sensors, sensors. So they just, whereas Tesla's going vision only, so cameras only, Waymo have LiDAR and radar and a whole bunch of sensors that I, I'm not even, I can't even talk to because I've never really even heard of them or, or read about them. But their videos were very impressive. So check out Waymo's most experienced urban driver. They call it the Waymo driver. And now it's good to have competition, you know. So finally we have Commacon, which was Comma AI's— I introduced Commacon at the end of last week's— last month's ML Monthly. They're going vision only as well, and they're a much smaller team than Tesla. They have 22 staff from what I last heard. They've just released the Comma 3, which is a new device that you can retrofit in many of the most popular new car brands out there now to add self-driving features to your own car. So really, really respect the Comet team. I love their work. So go and check that out. Self-driving, it's always 5 years away, but it, it looks like it's getting closer. Finally, we have— I think this is number 6, if not I will correct it— we have machine learning in the wild. I love this. This is a rapid-fire round, right? So we're coming out, we're coming out the 6th round, we're throwing some quick jabs here. Applied ML, let's have a look at this. It's a beautiful GitHub repo collected by Eugene Yan, who's also an amazing creator in the data science space. So check out Eugene's work, he has a great blog. Applied ML collects curated papers, articles, and blogs on data science and machine learning in production. So all the way from data quality— so how do you keep your data in a quality condition— to data engineering, to data discovery, feature stores— what is a feature store— classification, regression, forecasting, recommendation, Search and ranking, embeddings, natural language processing, sequence modeling, everything you could imagine. Best practices in, and most importantly, it's in production. So go and check it out. This is a repo. Look, I did not go through all of these. You can't. There is so much here. It would take you months and months and months. So best if you're working on something, you need something around of, let's relate it to one of these topics. There's 29 of them or something. So say you wanted to weekly supervise your, your data. You could check out how Snorkel Drybell do it, or Osprey, or Overton. Actually, we mentioned Overton in a previous Machine Learning Monthly. From Apple, the big dogs. Speaking of Apple, Apple put out a beautiful blog post on how they use machine learning for running on-device photo recognition. So you may have an iPhone, you might not. I believe Google Photos does this as well with Android. But this is recognizing people in photos through private on-device machine learning. So you could argue that Apple is either 1, 2, or 3 of the leading companies in terms of using machine learning on device. Anyway, Photos has a feature that can recognize different people in an image, such as Anna, Luke, Laura, Sophie, Jane, Tom, which runs on your device. So the photos are never uploaded to Apple servers. That means machine learning models go through your photos on your phone, right? Keep it private. Machine learning models like It's not peeking at you unless you have some nudes on there. It's probably gonna see, it's gonna see your nudes, but it's just turning your nudes into numbers. Don't worry. It's gonna detect your face, upper body, match those two, create a face embedding, pass that face embedding to a cluster, plus the body embedding. So that's the beautiful thing, 'cause the face can sometimes be excluded or shadow or different lighting. But if you tried to match the face with the upper body model, That's when you get better results. And then they put it into your photos gallery and it all runs when you put your phone on charge. So it saves power. And finally, we have Google's AI team substantially improves their polyp detection algorithm. So way back in Machine Learning Monthly, one year ago to this episode or to this issue, August 2020, we have— let's go polyp. We had number 2 was using machine learning to detect deficient coverage in colonoscopy screenings. So what happens? You go for a colonoscopy, a tube goes up your lower end and cameras take photos of your intestine to see if you have polyps, which are sort of small little nodules that may turn into colon cancer. And what Google's AI team do is they use computer vision to reconstruct your intestine into some gangly looking thing like this. And they use computer vision again to detect whether those polyps are worth looking at again or whether they're present or not, because the polyps could turn into colon cancer, which close to 1 million people per year die from. And so if you detect colon cancer earlier, what can you do? You can start to treat it earlier. Well, good news is that they've updated their research and they've improved from— we have 97% The system detected 97% of polyps. So 97% sensitivity. What that means is that out of 100, it detected 97 of them. So it would miss 3 out of 100. At 4.6 false alarms per procedure, which is a substantial improvement over previous published results. Last time they got around 94%. So again, it's, this is an incremental thing, but this is amazing, right? And of the ones that it missed, of the false alarms or the false positives, Follow-up review showed that some were in fact valid polyp detection. So this is a beautiful thing about machine learning models is they can often discover things that annotators or performing endoscopists, uh, have actually missed. So the combination of the human endoscopist and the machine learning model performs better than either one on their own. So check that out. If you're interested in machine learning and health, I would highly recommend reading up on Google AI's research. They do a lot of things in the health space. That is it for Machine Learning Monthly of August 2021. Of course, if you like this article, the link will be in the description below. You can sign up, put your email in there, and that'll be delivered to your inbox at the start of every month. In the meantime, keep machine learning, keep creating, most importantly, keep dancing.