Train a Production-Level Image Classifier

Today me and Joffre are going to present how to train production level image classification image recognition system. Well, you probably already know about wildframe image and defined. Net. Basically you can obtain this network directly by the. Net model of Wolfram immediate version v1. It was trained back in 2017 using an inception inspired architecture. Ever since then there have been a lot of development in the research of optimized efficient architecture in the machine learning community. Some basically optimize for the, for the architecture to be both light and fast. Some optimize the training like the training time and accuracy of the model. And for the, for our project we've decided basically to yeah basically attack the speed so that the models will be basically lighter and faster than our baseline model which is 45v1 but still yield basically similar or higher classification results. Let's say on top one top five accuracies and another task is also optimize the quality. So it might have like it can have similar model size or maybe a little bit like heavier but it should achieve much higher classification results. Yeah, but first let's recap the polyframe and gender value one. So as I said you can basically call the. NET model and then pass an image and you will get an output or you can directly use the image identify function. The advantage of using image identify function is that you can do more advanced stuff like for example in this image you can ask to retrieve the top 10 results connected with the entity, word, character, car and if you look at the outputs you can see that total sum of the results is more than one. Of course, like as usual in the neural networks the last output layer is basically sumsed one because you have a softmax layer there and in here the sum is not one because image identifier retrieves not only the classes on which the model was trained on but it also propagates basically is using an ontology knowledge to get more high level result that for example although it's a convertible it's like which like 37 probability and all the other like automobile type vehicles but like it's definitely an automobile with 99 point probabilities. Okay, so let's talk a little bit about the training set Wildfire Image identify. So it consists of over 3 million training images and over 4,000 classes. Exactly 4,315. And the training data set is not publicly available but you can see that basically a little bit of the insight into the training data distribution per class. Basically it's not very well balanced for all the classes. Like we have several classes which Have a few training, few training data. And it comes partly from the fact that we cover so many classes and all of them are like it's a very diverse representation. So it's very hard to get like a big amount of data for every type of class. So for example, to compare the outputs of image identify v1 with sufficient net v1 which is trained only on ImageNet. Well, you can see that wildframe image identify covers not only cases from nature or from fauna and flora, but also fictional characters, bacteria types, images, scenes, even planets, the locations. So the classes are very diverse. And in here you can see also the distribution of the classes, like what kind of objects we are covering in genetify. Yeah, so it's quite diverse. You have animal floor artifacts, abstract entities which are basically can be fictional characters and so on. Okay, so let's. For the WiFi image identify v2 we should choose training network architecture. And basically to do that we're using the WiFram neural net repository. And the plus is that WiFram neural net repository covers a lot of networks which contain images. And this is from blog post of one of our users. You can see that the networks connected to images are very up to date. So since our goal basically is quality and quantity. So let's compare several models in the repository to see which ones are applicable for training. Basically you can see on this plot like parameters efficiency versus the image net accuracy. So like this one is the efficient net v1 and each dot represent a parametric model in our repository. So this is B0B1 and so on. So yeah, you can see that we have very light models like Shufflenet V2 or Mobilenet V3, but they come at the lower accuracy. So these models are basically very used in the experiments to train the model for the quality goal speed. And these models were trained in experiments for the quality goal quality. So sorry, a performance goal quality. And in here I have also some latency measures on my personal computer for the single image batch size of 8 and batch size 16 and these results are like very. So if you compare to voidforming widget identify v1, all the models except for F net v2 small one are lighter than Voidformation 5 v1 and basically yeah, all of them Shufflenet wins definitely in the latency in the speed and also compared to the like parameter efficiency. Yes, so basically in our experiments we used so again shufflenet v2 for the performance call speed and efficient v1 for the performance goal quality. Since it's a much heavier model, it can be more representative and so basically the idea of the training, since we've been performing a lot of experiments, was basically to have some kind of trackable process over the experiments. We've decided that we should. For like each training experiment should have a training config which you pass to the script and then the script should calculate basically that should store the trained net. The trained object should contain like should store also checkpoints, intermediate checkpoints in case some. In case of the failed training source, like in case of some failures during the trainings and also some walks like the training calls and yeah, stuff like that. The end result also should be tested. So this is basically the basic workflow. And yeah, so to run the script like it's very basic. You just using wifram script, then file and then train the training file the training script and then pass to the config. Basically it's very useful because since you're training in the remote and you're not using the notebook interface, you should use the scripting. Yeah, you should run basically scripts on the remote. So. And I also wanted to show some configuration file to give an idea or an inspiration for others to make the trainings. Basically what you need is the path to store the results. Then the training data set file which contains the labels. You can have the. NET model arguments like let's say for which to extract the backbone so that the script can extract the backbone automatically. You can also do some operations on the. Net encoders and have options how to prepend the image augmentation layer or not do some regularizations and also change the. NET train parameters as as much. Much as basically you want. Yeah, so now I will stop sharing and we will continue with the techniques to improve and regularize the training.

So I'm going to talk about different techniques that are useful for improving the training on such image classifiers. The first one for example is image augmentation. Here we can use image augmentation layer in combination of the padding layer to augment the training data. So the basic idea is that we will create several kind of random reflections on the horizontal axis as well as shifting the image using some random padding. As you can see using. So using NET evaluation mode train there is a random augmentations but with the NET evolution mode test it's frozen. And then to add this layer into your models you can use. NET prepend and that's it. I think I have some like size augmentation on my notebook and there's like. That's why it's like so big I guess. But yeah, basically the image of multitude layers here. Then there is the possibility to add some dropout layers. There are like some of the nets in the net, all from. Net repository do not contain dropout layers. So one way to add them is just using Net insert after the final prediction layer and using dropout layer. In this case it was like 20%. So the next technique it's called also it's called label smoothing. And this is instead of using hard labels for crisanthropy, it's using smooth labels. This is to avoid models to be overconfident in the results. And here there's a small example using MNIST data and you can see that the top one error can be decreased several points. Transfer learning it's when we have already an already trained model and we don't want to retrain from scratch that architecture. So we can freeze all the layers except the last one using learning rate multipliers. And there is a link here to a nice tutorial using Wolfram languages for performing transfer learning. There is also the option of learning rate schedule that will allow you to down the learning rate over the different epochs during the training. And finally there is the blind training which is once the model is fully trained, the idea is that you train again also using the validation data. So in the next session I'm going to talk about using notebook templating for test analysis reports. So here the ideas. This is a nice introduction video to the notebook templating by Anthony Zubnit. And I think here, since we are running multiple experiments and training different models, it was a good idea to use kind of a notebook template so one can create using create notebook function a template notebook. I think I have it hidden here. So once this will open you a new template notebook which you can edit and create your own template. In this case I wanted to have the experiment name and then some code to compute the top one, top three and top five probabilities which we can specify that this cell with the cell behavior that it should be run but afterwards should be deleted from the final report. And then I'm also generating a report table using grid specifying certain parameters that will be introduced using the slot. And in this case the cell behavior I want for this cell is to be evaluated and hidden so we only see the table through it as well the same for the result object from the training and the same for the for the log files where we can obtain the validation loss as well as the error rates. So we have to specify the arguments that we want to pass to the template and once we have them we can directly apply templates apply the input the template notebook with arguments and it will compute the notebook and will not do it live here since it takes a while to create to compute the top one accuracies for all the testing set. So that's the final result of one such report notebook. You can see here the top one to five accuracies, the model size and the model speed. And then the result object is nice to check for how long was in the number of epochs as well as the different methods that were used. And I think finally the losses against the the error rate, the round losses are a good way to compare to check how the model was doing. There was overfitting or some other problems. Finally I'm saying that we were doing many different so we ran several experiments and different trained models. And here a good way to compare the experiments is to plot all the different validation losses and check out the most promising models with the lowest validation curve. Another thing that one can do with the trained nets is to compare them on other applications. Here for example, one can perform one shot learning using classify using a feature extractor and taking them. So in this case we're like examples of two different spider species that are not that were not in the training for the model. So it was doing job for this small example. Other functions that can be also run with Internet are fine clusters. And of course a feature space plot can also be a good way to to check different behaviors of the TrainedNet in comparison to the original model or some other ImageNet data TriNet model. Finally, I want to point out other future directions that we want to go is to also to improve the net graph by fusing some bias norms and compilations and also like removing the dropouts on the final trainer net with a speed up model. We also want to update the ontology graph as Maria mentioned and there's some data creation that we could do. For example, in the training data and test data there's like several placeholder images, images that contain non useful information that were mistakenly added in the training set. And then the next step that we could do is to go and train these models remotely. For the future I think we will create some tutorials on that on using. For example, this is a nice guide on Remote Badge as well. We might want to train them on some external Amazon server. Then here I'm pointing to you some image data sets available. This is in the Wolfram data repository. This is also open but not so external with many millions of different images from animals and flora. And finally, I also mentioned that you always have several sections on image classification in the introduction to the Machine learning book from Bernard and I think with this.

To complete.