Andrew Ng shows how to find and use open-source neural network code on GitHub to speed up your work. He walks through cloning a ResNet-101 implementation in Caffe, explains why pre-trained models matter for transfer learning, and urges developers to contribute improvements back to the community.
You've now learned about several highly effective neural network and confnet architectures. What I want to do in the next few videos is share with you some practical advice on how to use them. First, starting with using open source implementations. It turns out that a lot of these neural networks are difficult or finicky to replicate because a lot of details about tuning of the hyperparameters, such as learning red decay and other things that makes some difference to the performance. And so I found that it's sometimes difficult even for say, AI or deep learning PhD students, even at the top universities, to replicate someone else's published work just from reading the research paper. Fortunately, a lot of deep learning researchers routinely open source their work on the Internet, such as on GitHub. And as you do work yourself, I certainly encourage you to consider contributing back your code and to the open source community. But if you see a research paper whose results you would like to build on top of one thing you should consider doing. One thing I do quite often is just look online for an open source implementation, because if you can get the author's implementation, you can usually get going much faster than you were to try to reimplement it from scratch. Although certainly reimplementing from scratch could be a good exercise to do as well. If you're already familiar with how to use GitHub, this video might be less necessary or less important for you. But if you aren't used to downloading open source code from GitHub, let me quickly show you how easy it is. Let's say you've heard about. Let's say you're excited about residual networks and you want to use it. So let's search for ResNet on GitHub. And so you actually see a lot of different implementations of resnets on GitHub. I'm just going to go to the first URL here. And this is a GitHub repo that implements resnets. A lot of the GitHub web pages, if you scroll down, will have some text describing the work or the particular implementation. This particular repo, this particular GitHub repo repository was actually by the original authors of the repository Resnet paper. And this code is licensed under an MIT license. You can click through to take a look at the implications of this license. The MIT license is one of the more permissive or one of the more open open source licenses. So I'm going to go ahead and download the code and to do that click on this link. This gives you the URL that you can use to download the code. I'm going to click this button over here to copy the URL to my clipboard and then go over here. And all you have to do is type git clone and then control V for the URL and hit Enter. And so in a couple seconds it has downloaded, has cloned this repository to my local hard disk. So let's go into the directory, let's take a look. I'm more used to Mac than Windows, but I guess, let's see, let's go to Prototext, and I think this is where it has the files specifying the network. So let's take a look at this file because this is a very long file that specifies the detailed configurations of the Resnet with 101 layers, right? And it looks like, you know, from what I remember seeing from this web page, this particular implementation uses the CAFFE framework. Um, but if you want an implementation of this code using some other programming framework, you might be able to find it as well. So if you're developing a computer vision application, a very common workflow would be to pick an architecture that you like, maybe one of the ones you learn about in this course, or maybe one that you heard about from a friend or from some of the literature, and look for an open source implementation and download it from GitHub to start building from this. One of the advantages of doing so also is that sometimes these networks take a long time to train and someone else might have used multiple GPUs and a very large data set to have pre trained some of these networks. And that allows you to do transfer learning using these networks, which we'll discuss in the next video as well. Of course, if you're a computer vision researcher implementing these things from scratch, then your workflow would be different. And if you do that, then do contribute your work back to the open source community. But because so many vision researchers have done so much work implementing these architectures, I found that often starting with open source implementations is a better way or is certainly a faster way to get started on the new project.
