TensorFlow Tutorial (Sherry Moore, Google Brain)

So I'm going to take a picture. So I remember how many of you are here. Smile, Like Sami says. My name is Sherry Moore. I work in the Google Brain team. So today I'll be giving a tutorial on TensorFlow. First I'll talk a little bit about what TensorFlow is and how it works, how we use it at Google and. And then the important part is that I'm going to work with you together to build a couple models to solve the most classic machine learning problems, so called get your feet weak for those of you from New Zealand, anybody from New Zealand. So hopefully at the end you'll be going home with all the tools that you have to build all the wonderful things that you have watched today, like all the image recognition, the training of different colors, arts, making music. So that's the goal. So before I go any further, has everybody installed TensorFlow? Yay. Brilliant. Thank you. And I would like to acknowledge. So I know the link here says Sheeran, but if you have Wolf G TF tutorial, it's perfectly fine. Wolf is actually my colleague who spent all the time verifying installation on every single platform, so I would really like to thank him. Thanks Wolf, if you're watching. And also I have my wonderful product boss, product manager in the audience somewhere. So if you guys have any request for TensorFlow, make sure that you go find him and tell him why that TensorFlow must support this feature. Or is Zach somewhere? All right, so there he is. So with that we can move forward to talk about TensorFlow. So what exactly is TensorFlow? TensorFlow is a machine learning library that we developed at Google and we open sourced it last November. And ever since then we have become the most most popular machine learning library on GitHub. How do we know? Because we have over 32,000 stars. Those of you who track GitHub, you know how hard it is to get one of those acknowledgement. And we also have over 14,000 forks and we have over 8,000 contributions from 400 developers, 400 individual developers, and we designed this specifically for machine learning. However, as you'll see later, because of its really flexible data flow infrastructure, it makes it really suitable for pretty much any application that can fit into that model. Basically, if your model can be asynchronous and fire when data is ready, it can probably use TensorFlow. Originally we worked alongside with all the researchers. As a matter of fact, I was really fortunate. When I joined the team, I sat right next to Alex, the person who invented Alexnet. So that's how closely we worked Together, as we developed TensorFlow, they would tell us, no, this is not how we use it. Yes, when you do this, it makes our lives a lot easier. And this is why we believe that we have developed an infrastructure that will work really well for researchers. And also being Google, we also always have in mind that we would like to take from research to prototyping to production in no time. We don't want you to write all the code that's typically just throwing away. We want you to write code that can literally cut and paste and save in the file and productize it immediately. So TensorFlow is really designed with that in mind. So we are halfway into your deep learning school. So can anybody tell me, if you want to build a neural net, what must you have? What are the primitives? What are the. Yeah, primitive I think is the word I'm looking for. What must you have to build a neural net? Anybody? What is in the neural net? Very good answer. So in the neural net you have neurons. That's right. So, and you need. So all these neurons, what do they operate on? What do all these neurons do? They process data. Data. And they operate on data and they do something such as convolution, matrix multiplication, max pooling, average pooling, dropout, whatever that is. So in TensorFlow, all the data is held in something called a tensor. Tensor is nothing more than multidimensional array. For those of you who are familiar with numpy arrays, it's very similar to the ND array and the graph I think one of the gentlemen earlier this morning described, there's this concept of the graph, which is a composition of all these neurons that do different functions. And all these neurons are connected to each other through their inputs and outputs. So as data become available, they would fire. By fire, I mean they do what they are designed to do, such as doing matrix multiplication or convolution, and then they will produce output for the next computational that's connected to the output. So by doing this, I don't know how many of you can actually see this animation. Yeah, so this is to really visualize how TensorFlow works. All these nodes, the oval ones, are computation, those rectangle ones are staple nodes. So all these nodes, they would generate output or they take input, and as soon as all the inputs for a particular node are available, it would do its thing, produce output. And then the tensor, all the data, which are how intensives, will flow through your network. Therefore TensorFlow. Yeah. So everybody's like, wow, this sounds like magic. How does it work? So who said is It Sir Arthur Clarke, that says any sufficiently. What's the word? Any sufficiently advanced technology is indistinguishable from magic. So that's what this is. It's just really awesome. Excuse me for a second. I know, I want to get through this as quickly as possible so we can actually do the lab that you're all dying to do. So as any good infrastructure. So this is, I want to give you a little image of how we design this TensorFlow. Just like any well designed infrastructure has to be really modular because being modular allows you to innovate, to upgrade, to improve, to modify, to do whatever you want with any piece. As long as you keep the APIs consistent, everybody can work in parallel. It's really empowering. I think that's one of the wonderful things that's done at Google. Pretty much any infrastructure at Google is really modular. They talk really well to each other. All you need to maintain is the API or stability. So in this case we have a front end. I think you guys must have seen some examples of how you construct a graph. So we have the front end libraries written in your favorite language. And if C and Python is not your favorite language, feel free to contribute. We always welcome contribution. So you write, you can show your graph in your favorite language and this graph will be sent to. We call it the core TensorFlow execution system. That's your runtime and that's what you all will be running today on your laptop when you open your Python notebook or Jupyter notebook. So the execution runtime, depending on where you are going to run this application, it will send a kernel to the corresponding device. So it could be a cpu, could be a gpu, could be your phone, could be tpu. Anybody knows what TPU is. Brilliant. Very nice. I was at Strata. Anybody knows what TPU is. Everybody's like, translation. So this is good. So just to highlight our portability, today you'll be running TensorFlow on your laptop. We run it in our data center. Everybody can run it on your iOS, on your iPhone, your Android phone. I would love to see people putting on Raspberry PI because can you imagine you can just Write your own TensorFlow application. It could be your security system because, you know, somebody just stole my bike and my security camera. Capture all this grainy stuff that I cannot tell. Wouldn't it be nice if you do machine learning on this thing and they just start taking high resolution pictures, you know, when things are moving, rather than constantly capturing all these grainy images, which is totally useless. So I think the Application, literally. Applications are limitless. Your imagination is the limit. So we Talked about what TensorFlow is, how it works, how do we use it at Google? We use it everywhere. I think you have seen some of the examples. We use it to recognize pictures. This is actually done with Inception. They can recognize out of the box one of a thousand images. You have to retrain it if you wanted to recognize say all your relatives or your pets. But it's not difficult and I have links for you to actually, if you want to train on your own images, it's really easy. They should totally try it. Wouldn't it be fun if you go to your relative, your 40 year reunion, you just go, I know who you are, I know who you are. Just show off a little, it would be brilliant. And we also use it to do Google voice search. This is one that's super awesome. So how many of you use Smart Reply? Have you ever used Smart Reply? Yeah, yeah, this is awesome. Especially for those of you who are doing what you're not supposed to do, you know, texting while driving, you know, an email coming in and you can just say, oh yes, I'll be there, you know. So based on the statistics that we collected In February, over 10% of all the responses sent on mobile is actually done by our Smart Reply. I believe if we have, maybe Zach can collect some stats for me later. Maybe by now It'll be like 80%. It's actually really funny at the very beginning when we train it, the first answer is always I love you. We're like, that's probably not the right answer. We also play games. All of you, I'm sure have followed this. There are all kinds of games that are being developed. It's really fun to watch. If you watch it literally come up with scenarios for you to play as well. It not only learns to play the game but learns how to make a game for you. It's fascinating and of course art. I think many of you have done this deep dream. If we have time at the end of the lab we can try this. So if we are super fast, we can all try to make some art and all those what I just talked about. Of course Google being this wonderful, generous company wants to share our knowledge. So we have actually published all our models. So if you go to that link you'll find all these Inception and captioning language model on a billion words. The Latest resnet on CIFAR 10 sequence to sequence which I think KWOK will be talking about tomorrow. And we have many other high level libraries so today my lab, the lab that we will do, will be on the core TensorFlow APIs. But there are tons of new higher level APIs such as some of the mentioned cares. And we have slim, we have pretty tensor, we have TF Learn, we have many libraries that's developed on top of the core TensorFlow APIs that we encourage people to do. So if whatever is out there does not fit your needs perfectly, go for it, develop your own. And we welcome the contribution and we publish a lot of that here. I might have blurred some of the boundaries, but these are basically all the models and libraries that we have produced and we really love contribution. If you have developed a really cool model, please do send to us and we would, you know, we will showcase your, your work. So that's the introduction of TensorFlow. How does everybody feel? Are you all ready to get started? All right, so okay, before you bring up your Python notebook, I want to say what we are going to do first. Okay, so as I mentioned, there are two classic machine learning problems that everybody does. One is linear regression, the other is classification. So we are going to do two simple labs to cover those. I do have a lot of small exercises you can play with. I encourage you to play with it to be a lot more comfortable. So the first one is linear regression. So I'm sure it has been covered in today's lectures. So somebody must have covered linear regression. Can anybody give me a one line summary? What is a linear regression problem? Anybody? The professors. Well, if you don't know, go Google it. So I didn't know the audience when Sami asked me to do this, so I wanted to. I wrote this for one of the high schools. So I think it still kind of makes sense, right? Because all of us have played this game at one point of our lives. Like, you know, if you tell me five, I will tell you ten and you try to guess what the equation is. We must have all done this. I think my friends are still doing on Facebook saying, oh, only genius can solve this kind of equation. And then they will be like, yeah, I solved it. I was like, my God, if anybody, I will unfriend you guys if you click on another one of those. But basically this is what we are trying to do in the first lab. So we will have a mystery equation. It's really simple. It's just a linear, literally a line. And then I will tell you that this is the formula, but I'm not going to give you weight. W and B, all of you have learned by now, W stands for weight and bias. B stands for bias. So the idea is that if you're given enough samples, if you're given enough X and Y values, you should be able to make a pretty good guess what W and B is. So that's what we are going to do. So now you can bring up your Jupyter notebook if you don't have it up already. Yeah. Everybody have it up? Yes. Can I see? Show of hands, everybody, those of you? Yeah. Brilliant. All right, so for pretty much any models, these are going to come up over and over again. And just to make sure that you're all paying attention, I do have. I asked him if I was supposed to bring Shrek and he said no. But I do have a lot of TensorFlow stickers and I have all kinds of little toys. So later I'm going to ask this question. Whoever can answer will get some mystery present. So really pay attention. Okay, so pretty much whenever you build any model, there are, I would say, four things that you will need. You need input, you need data. So you're going to see in both labs we're going to be defining some data. You're going to be building an inference graph. I think in other lectures it's also called called a forward graph to the point that it produces logits, the logistic outputs. And then you're going to have a training operations, which is where you would define a loss, an optimizer. And I think that's pretty much it. Hang on. And there's the fourth thing. Yeah. And then you will basically run the graph. So the three important things, okay, you'll always have your data, your inference graph, you always have to define your loss and your optimizer. And the training is basically to minimize your loss. So I'm going to be asking that later. All right, so now we know what we're going to do. So you can go start, go to that lab. Yeah. Everybody have it? So shift, return, we'll run the first one. You say, I have no idea what's happening here. We turn again, still nothing. However, let's see what we are producing here. So you can also do the same on your laptop. You can uncomment that plot. You're going to say, so you know what kind of data you're generating. So in this case, when he return, what are we seeing? This is your input data. This is when you try to make a guess. When your friend tell me, oh, you know, give me your X. And yeah, so this is when Your X is 0.2, your Y is 0.3, 2. So this is basically your input data. Yeah. Everybody following, if at any point you're kind of lost, raise your hand and your buddy next to you will be able to help you. So now. Ok, I want to say one more thing. So today the labs are all on really core TensorFlow APIs. The reason I want to do that, I know there are a lot of people who use Keras use another thing that we heavily advertise, which is Contrib TF contribtf learn. So I feel like I'm giving you all the ingredients. So even though you could go to Whole Foods and buy the packaged meal, maybe one day, you don't like the way they cook it. So I'm giving you all your lobsters, your Kobe beef, so that you can actually assemble whatever you want to build yourself. So this next one is very key. It's a very key concept. Here you see variables. So variable in TensorFlow is how is corresponding to the square. Any of you remember this slide? Ok, I'm going to switch quickly. Don't freak out. So actually I wanted you all to commit this little graph to your memory because you'll be seeing this over and over again and it makes a lot more sense when you have this visual representation. So in TensorFlow, the way we hold all the data, the weights and the biases associated with your network is using something called variable. It's a statefold operation. I'm going to switch back. Okay, so this is what we are doing in section 1.3. We are building those square nodes in your network to hold these weights and variables and they are the ones when you train. That's where the gradients will be applied to so that they will eventually resemble the target network that you are trying to train for. So now you have built it. Wonderful. Okay, so you can shift return. Do you see anything? No. So exactly what have we built? Let's uncomment and take a look. So these are called the variable objects. So at the bottom of the slide for this lab I have a link which is our Google 3 docs, the API docs which is available in GitHub. I think you should always have that up. So whenever you want to do something, you would know what kind of operations are possible with this object. For example, I can say here, what's the name of this? Oh, it's called Variable 6. Why is it called Variable 6? Oh, it's because when I create this variable, I didn't give it a name. So I can say Sherry. Sherry Weight. I hope that's not but so see now my variable is called Sherry Weight. Same thing with my. So this would be a good practice because later. Oh, because I ran it so many times. Every single time you run, if you don't restart it, it's going to continue to grow your current path. So to avoid that confusion, let me restart it. Restart. Now I have to work, sorry. So now. So we have done build our input, build an inference graph. Now we can actually build our training graph. And as you have all learned, we need to define a loss function, we need to define an optimizer. I think it's also called something else, regularizer, maybe some other terms. And your ultimate goal is to minimize your loss. So I'm not going to do it here, but you can do it at your leisure. You can uncomment all these things that you have created and see what they are. And I can tell you these are different operations. So that's how you actually get to learn about the network that you have built really well in the next line. I'm also not going to uncomment, but you should at one point. This is how you can see what you have built. So actually why don't we do that? Because this is really critical. And as you debug this would become. So this is the network that you have built. They have names, different names, they have inputs and outputs, they have attributes. And this is how we connect all these nodes together. This is your neural net. So what you're seeing right now is your neural net that you have just built. Yeah. Everybody following? So now the next step, now you're done, right? You build your network, you build all your training. Now you can. Let's do some training. So in TensorFlow, do you remember in the architecture that I showed you have the front end C and Python front end. You use that to build your graphs and then you send a graph to your runtime. And this is exactly what we're doing here. Whenever you. This is how we talk to the runtime. We create something called a session. You get a handle to the session and then when you say run, you're basically sending this session your graph. So this is different from the other machine learning libraries, I forgot which one. Those are so called imperative. It happens as you type. TensorFlow is different. You have to construct your graph and then you create a session to talk to your runtime so that it knows how to run on your different devices. That's a very important concept because people constantly compare and it's just different. Okay, so now you can also comment that to see what the initial values are. But we are not going to do that. We're just going to run it and now we're going to train. The data is not. So what do you think of the data? Did we succeed in guessing? Is everybody following what we are trying to do? Yeah. Yes. No. So what was our objective before I started the lab? What did I say our objective was? Yes, to guess the mystery function. So have we succeeded? It's really hard to tell. All right, so now all of you can go to the end and uncomment this part. Let's see how successful we are. So the green line was what we have initialized our weight and bias to. Yeah. The blue dots were the initial value, the target values and the red dots is our trained value. Makes sense. So how successful are we? Great. Big success. Yeah, I would say so. So any questions? Any questions so far? So what other things? So everybody should play with this. You're not going to break it. This is a notebook. Python notebook. The worst that happens is they'll just say okay, clear all like what I just did and change it. So what can you play with? Since today you learn all these concepts about different loss functions, different optimizers, all this crazy, different inputs, different data, so now you can play with it. How about instead of, let's pick one. So instead of gradient descent, what are the other optimizers? How can you find out? I guess that's a better question. If I want to know what other optimizers are available in TensorFlow, how can I find out? Very good. Yes. The GitHub, Google 3, the G3 doc link with the APIs. I'm going to switch one more tab. Bear with me. So this is when you go there, this is what you can find. You can find all the. Let me make it bigger. So you can find all the different optimizers so you can play with them. So maybe gradient descent is not the best optimizer you can use. So you go there and say why are the other optimizers? And then you can literally come here and search optimizer or you can say, wow, I have other delta Adagrad Adam. I'm sure there are more momentum. So we also welcome contribution. If you don't like any of these, please do go contribute. Write a new optimizer, send a pull request. We would love to have it. So I would like to say this over and over again. We love contribution. It's an open source project, so keep that in mind. We would love to see your code or your models on GitHub. So back to this one. How is everybody feeling? This is too simple. Yeah. Should we go register? Yes. Can I say that one? Oh, is that right? Hit tab to see all the other optimizers you meant. Oh, brilliant. See, I didn't even know that. Learn something new every day. Let me go there. Tap here. Oh, yay. So this is even easier. Thank you. Clearly I don't program in notebook as often as I should have. So this is where you can all the wonderful things that you can do. Thank you. This is probably a little too low level. I think it has everything. But that's a very good tip. Thank you. So anything else you would like to see with linear regression? It's too simple. You guys all want to recognize some digits. All right, so that sounds like a consensus to me. So let's move. If you just go to the bottom, you can say click on this. So this is our mnist model. So before we start the lab. So once again, what are we trying to do? So we have all these handwritten digits. What does mnist stand for? Does anybody know what does mnist stand for? Very good. See somebody can Google Very good. So it has, I think makes National Institute of Standards and Technology something like that. So they have this giant collection of digits. So you know, if you go to the post office you already know that it's a trivial, it's a solved problem. But I don't know if they actually use machine learning. But our goal today is to build a Little network using TensorFlow that can recognize these digits. Once again, we will not have all the answers. Right. So all we know is that the network, the input will give us a one and then we'll say it's a nine and then we have the so called ground truth. And then they will look at it and say no, you're wrong. And then we'll have to say, okay fine, this is the difference. We are going to train the network that way. So that's our goal. Yeah. Does everybody see the network on the side? So now we can go to the lab. So can anybody tell me what are the three or four things that's really important whenever you build a network? What's the first one? Your data. Second one? Inference graph. Third one, your train graph. And with this lab, I'm going to teach you a little bit more. They are like the rock. You know, like when you go to a restaurant, I not only give you your lobster or your Kobe beef, I'm also going to give you a little rock so you can cook it your Data. Second one, inference graph. Third one, your train graph. And with this lab, I'm going to teach you a little bit more. They are like the rock. You know, like when you go to a restaurant, I not only give you your lobster or your Kobe beef, I'm also going to give you a little rock so you can cook it. Okay, so in this lab, I've also teached some absolutely critical additional infrastructure pieces, such as how to save a checkpoint, how to load from a checkpoint, and how do you evaluate your network? I think somebody at one point asked, how do you know the network is enough? You evaluate it to see if it's good enough. So those are the three. Okay. So in this lab, I've also teached some absolutely critical additional infrastructure pieces, such as how to save a checkpoint, how to load from a checkpoint, and how do you evaluate your network? I think somebody at one point asked, how do you know the network is enough? You evaluate it to see if it's good enough. So those are the three new pieces of information that I'll be teaching you. And also I'll teach you a really, really useful concept. It's called placeholder. That was requested by all the researchers. We didn't used to have it, but they all came to us and say, when I train, I want to be able to feed my network any data we want. So that's a really key concept. That's really new pieces of information that I'll be teaching you. And also I'll teach you a really, really useful concept is called placeholder. That was requested by all the researchers. We didn't used to have it, but they all came to us and say, when I train, I want to be able to feed my network any data we want. So that's a really key concept that's really useful for any practical training. Whenever you start writing real training code, I think that will come in handy. So those are the, I think four concepts now that I will introduce in this lab that's slightly different from the previous one. How to save checkpoint, how to load from checkpoint, how to run evaluation, and how to use placeholders. I think the placeholder is actually going to be the first one. So once again, we have our typical boilerplate stuff so that you hit return, you import a bunch of libraries. The second one, this is just for convenience. I define a set of constants. Some of them you can play with, such as the maximum number of steps where you're going to say, useful for any practical training. Whenever you start writing real training code, I think that will come in handy. So those are the I think four concepts now that I will introduce in this lab that's slightly different from the previous one. How to save checkpoint, how to save all your data, how big the batch sizes are, but some other things that you cannot change because of the data that I'm providing you. For example, the MNIST pictures. Any questions so far? So now we'll read some data. Is Everybody there in 2.3? I'm at 2.3 right now. So now I use if you don't have slash temp, it might be an issue, but hopefully you do. If you. If you don't have slash lamp, change the directory directory name. So the next one is where we build inference. So can anybody just glance and tell me what we are building? What kind of network? How many layers am I building? I have two hidden layers. You have all learned hidden layers? Yeah, today. And I also have a linear layer which will produce logits. That's correct. So that's what all the inference graphs will always do. They always construct your graph and they produce logistic outputs. So once again, here you can uncomment it and see what kind of graph you have built. Once you have done the whole tutorial by yourself, you can actually run tensorboard and you can actually load this graph that you have saved and you can visualize it like what I have shown in the slide. I didn't draw that slide by hand. It's actually produced by tensorboard. So you can see the connection of all your notes. So I feel that that visual representation is really important. Also, it's very easy for you to validate that you have indeed built a graph that you thought sometimes people call something repeatedly and they have generated this gigantic graph. They were like, oh, that wasn't what I meant. So. So being able to visualize is really important. Any questions so far? See here. I have good habits. I actually gave all my variables names. Once again, the hidden layer one, hidden layer two. They all have weights and biases, weights and biases, et cetera. So now we're going to build our train graph. So here is actually here. There's no new concept. Once again, you define the loss function. We once again pick gradient descent as our optimizer. We added a global step variable. That's what we will use later when we save our checkpoints. So you actually know at which point what checkpoint this corresponds to. Otherwise if you always save it to the same name, then later you say, wow, this result is so wonderful. But how long did they take? You have no idea. So that's a training concept that we introduce is called Global step, basically how long you have trained. And we usually save that with the checkpoint. So you know which checkpoint has the best information. Yeah, everybody is good at 2.5. So now the next one is the additional stuff that I just mentioned, that piece of rock that I'm giving you now to cook your stuff. So one is placeholder. So we are going to define two. One to hold your image and the other to hold your labels. We build it this way so that we only need to build a graph once and we will be able to use it for both training, inference and evaluation later. It's very handy. You don't have to do it this way. And one of the exercises I put in my slide is to try to do it differently. But this is a very handy way and get you very far with minimum work. So as I said in the slides, I know I don't have any highlighters beams, but you see there says after you created your placeholders, I said add to collection and remember this op. And later we'll see how we're going to recall this op and how we're going to use it. The next one we're going to call our inference, build our inference. Is everybody following this part? Okay. And once again we remember our logits and then we create our train op and our loss op. Just like with linear regression. Just like with the linear regression, we're going to initialize all our variables. And now at the bottom of this cell, that's the second new concept that I'm introducing, which is the saver. This is what you will use to do checkpoints to save the states of your network so that later you can evaluate it or if your training was interrupted, you can load from a previous checkpoint and continue training from there rather than always reinitialize all your variables and start from scratch. When you're training really big networks such as Inception is absolutely critical because when I, I think when I first trained inception, it took probably six days. And then later when we have 50 replicas, it took still like stay of the hours to two and a half days. You don't want to have to start from scratch every single time. So yeah, everybody got that, the placeholder and the saver. So now it's 2.7. We're going to go to 2.7. Lots of code. Can anybody tell me what it's trying to do? So this is an yes. So it's trying to minimize loss. We can actually see this. So we'll run it once okay, Where did I go? Okay, very fast. It's done. But what if I really want to see what it's doing? So python is wonderful. So, so I would like to actually see. Did somebody show how you know your training is going well? They show the loss going down. Going down. Oh, I think my training is going really well. So we're going to do something similar. Sorry. So I'm going to create a variable, what do you call losses, which is just an array. So here I'm actually going to remember it. Append. So what am I collecting? Matplot mapl. Anybody remember this plot? Let's try this. Oh, look at that. Now do you see your loss going down? So as you train, your loss actually goes down. So this is how, when you do large scale training, this is what we typically do. We have a gazillion of these jobs running in the morning. We would just glance at it and we know which one is doing really, really well. So of course that's just when you are prototyping. That's a really, really handy tool. But I'm going to show you something even better. That's part of the exercise. Man, I don't have it. So as one of the exercises, I also put the answers in the backup slides that you guys are welcome to cut and paste into a cell. Then you can actually run all the evaluation sets against your checkpoint so that you know how well you're performing so you don't have to rely on your eyes glancing. Oh, my loss is going down. Or relying on validating a single image. But see, this is how easy it is. This is how easy the prototype and you can learn it. Very often our researchers would cut and paste their colab code and put in a file and that's basically their algorithm and they will publish that with their paper. They would send it to our data scientists or production people. We would actually privatize some of their research. This is how easy. Literally from research to prototyping to production, really streamlined and you can do it in no time. So for those of you who have run this step, can you do an LS in your data path? Wherever you saved that, wherever you declare your trainer to be. What do you see in there? Checkpoints. That's right. That's the money. That's after all this work, all this training on all these gazillion machines, that's where all your weights, your biases are stored so that later you can load this network up and do your inception to recognize images, to reply to email, to do art et Cetera, et cetera. So that's really critical. But how do we use it? Have no fear. All right, let's move on to 2.8 if you are not already there. So can somebody tell me what we are trying to do first? That's right. First we load the checkpoint and you remember all the things that we told our program to remember. The logits and the image placeholder and the label placeholder. How are we going to use it now? We're going to feed some images from our evaluation and see what it thinks. So now if you hit return, what's the ground truth? 5. What's our prediction? 3. What's the actual image? Could be 3. Could be 5, you know, but so the machine is getting pretty close. I would say that's a three. Okay, let's try a different one so you can hit return again in the same cell. Oh, I need to somehow move this. So what's the ground truth this time? Yeah, I got it. Right. So you can keep hitting, you know, you can keep hitting return and see how well it's doing. But instead of validating, instead of hitting return 100 times and count how many times I has gotten it wrong, as I said in one of the exercises. And I also put the answer in the slide so you can cut and paste and actually do a complete validation on the whole validation set. But what do you think? So you can actually handwrite a different digit. But the trick is that a lot of people actually tried it and told me it doesn't seem to work. So remember on the slide I said, this is what the machine sees, this is what your eye sees, and this is what the machine sees. So in the MNIST data set, all the numbers are between 0 and 1. I believe I could be wrong, but I believe it's between 0 and 1. So if you just use a random tool like your phone, you write a number and you upload it. Number one, the the picture might be too big and you need to scale it down. Number two, it might have a different representation. Sometimes it's from 0 to 255 and you need to scale it to the range that mnist that how you have trained your network. If you train your network with those data and then it should be able to recognize the same set of data. Just like when we teach a baby, if you have never been exposed to something, you are not going to be able to recognize it. Just like with Oreo, one of our colleagues. Caption, what was that? That program a while ago, Anytime when it sees something that it doesn't recognize. Have anybody played with that captioning software? It's super fun. So you can take a picture and say, you know, two people eating pizza or, you know, dog surfing. But anytime it sees something that has never been trained on, it would say, men talking on a cell phone. So for a while we had a lot of fun with it. We would put a water medal on the post and they would say, men talking on the cell phone. You put a bunch of furniture in the room with nothing and they would say, men talking on the cell phone. So it was really fun. But just like with your numbers, if you have never trained it with that style, like if I write Chinese characters here, it's never going to recognize it. But this is pretty fun. So you can play with it. You can see how well See every time. See, so far it's 100% other than the first one, which I cannot tell either. So what are some of the exercises that we can do here? What do you want to do with this lab? It's too easy because I made this so easy because I didn't know that you guys are all experts by now. Otherwise I would have done a much harder lab. Let me see what things we can do so you can uncomment all the graphs. Oh, so here's one. Actually, you already see it. So try this. Can you guys try saving the Checkpoint, say every 100 steps and you're going to have a gazillion, but there's tiny, tiny checkpoints, so it's okay. And try to run evaluation with a different checkpoint and see what you get. Do you know how to do that? Yeah, everybody know how to do that. So the idea is that when you run evaluation, it's very similar. So we typically run training in evaluation in parallel or validation. So as it trains every so often, say every half an hour, depending on your problem. So with the inception, every 10 minutes, we would also run evaluation to see how well our model is doing. So if our model gets to, say, 78.6%, which I believe is the state of the art, it will be like, oh, my model's done training. So that's why you want to save checkpoints often and then validate them often. If you're done with that already, did you notice anything? If you try to load from a really early checkpoint, how good is it when it tries to identify the digits? Just take a wild guess. Yeah, very bad. Maybe every other one is wrong. But this MNIST is such a small data set. It's very easy to train and we have such a deep network. If you only have one layer, maybe it won't get it right. So another exercise I think all these you can do after the session is that really try to learn to run evaluation from scratch rather than actually another part, but run evaluation on the complete validation set. That's a really necessary skill to develop as you build bigger models and you need to run validation. So I think this is the end of my lab. I do have bonus labs, but I want to cover this first. The bottom line is that TensorFlow is really, is for machine learning. It's really from research to prototyping to production, it's really designed for that and I really hope everybody in the audience can give it a try. And if there are any features that you find it lacking that you would like to see implemented, either send us pull requests, we always welcome contribution, or talk to my wonderful product manager, Zach sitting over there. He's taking requests for features. So with that, yeah, thanks and have fun.

Thank you, Sherry. We have time for questions for those who actually tried it.

See, it's so well done. Everybody feel like they're experts. They're all ready to go make arts now, Right? Right. Go deep dream. Cool.

If there are no questions. Oh, there's one question I think someone who's trying desperately.

Hi, my name is Pichin Lo and first of all, thank you for introducing TensorFlow and for designing it. I have two questions. So the first question is I know that TensorFlow have C API, right? So let's say if I use Keras or any of the Python front end, I train the model. Does TensorFlow support that? Just I can pull out the C model of it and then just use that.

Yes, you can.

So even if I use for example Kira's custom layer that I code using Python, I still can get those things.

That's correct.

Oh, that is correct.

It's just a front end that's different how you construct the graph.

Ah, nice.

But we are not as complete on our C API design. For example, a lot of the training libraries are not complete yet, so. But for the simple models, yes, you can, you can. So.

Well, let's say not the training, but let's say if I just want the testing part because I don't need to do, I mean the training I can also do in Python.

We do have that already actually. If you go to our website, there's a label images cc, I think that's literally just loading from Checkpoint and run the inference. That's all written in C. So that's a Good example to follow.

Cool. A second one. So another thing that I noticed that you support almost everything except Windows.

Everything except What?

I mean, iOS, Android, have no fear.

Actually we're actively doing that. But when I first joined the team, I think there were 10 of us and we have to do everything. Like before open sourcing, all of us were in the conference room together. We're all writing dog, we're fixing everything. So now we have more people. That's like top of our list. We would love to support it.

So I'm just curious because, I mean, when I look at the roadmap, I didn't see a clear timeline for Windows. But the thing I know that just like the reason why you cannot support Windows is because of Bazel. Bazel doesn't support Windows. So let's say theoretically or I mean, what you think I know Bazel, that just like you will get window five at some point in November. That is what they say. So once Bazel can run in Windows, can I expect like just. I could immediately do TensorFlow audio, foresee some other problem?

Maybe Zach would like to take that question. Okay.

Okay, that's it.

So yeah, let's talk offline.

Yeah, sure.

Thank you very much.

Hi, great presentation and session. My name is Yuri Zefovich. I have a question about TPUs. Are they available right now for testing and playing for non Google employees?

Are we. I don't think so at the moment.

Do you know when it might be available in the Google cloud?

Would you like to take that one? I'm so glad we have a Prada boss here so that he can.

Okay, thank you.

Hi, nice tutorial. I have a question.

Are there any like plans to integrate.

TensorFlow with like open source framework, like misos and hdfs to make the distributed TensorFlow run easy? So there are definitely plans. We are also always actively working on new features, but we cannot provide a solid timeline right now. So we do have like, we do have plans, we do have projects in progress, but we cannot commit on a timeline. So I cannot give you a time saying yes, by November you have what you know. So thank you. But if you have this type of question, I think Zach is the best person to answer. Oh, hi. I was wondering, does TensorFlow have any examples to load your own data? Of what? So the current example has a MNIST data set. Are there examples out there to load your own data set? Yes, yes, definitely. I think we have two. One is called the TensorFlow Poet. I think that one, that example shows you how you can know your own Data set, I think. Is there another one? Zach, are you aware of another one that might be loading your own data set? I know we have retraining model. You know, if you go to TensorFlow, we have an example to do retraining. Those you can download from anywhere. So in our example we just downloaded a bunch of flowers. So you can definitely download whatever pictures that you want to retrain. Thank you.

Hello, thank you for your presentation. I have a question concerning the training. You can train using torotensorflow in any. Virtually in any system like Android. And what about the model? Do you provide anything to move the model to Android? Because I'm generally you program in Java there.

Yes. So that's a beautiful thing. You remember the architecture that I showed?

Yes.

You build a model and then just send it to the runtime. It's the same model running on any of the different platforms. It can be a laptop Android.

You have your own specific format for the model or it's just you build a. Because the model is just bunch of matrix. Matrix and values. Is there any special. Special format for your model? Because sometimes it is bigger.

Yes, compression. So I will not recommend training, say inception on your phone because all the convolution in the back prop would probably kill it 10 times over. So definitely. So there will be that type of limitation. Like I think you guys talked about the number of parameters. If it blows the memory footprint on your phone, it's just not going to work. And if the compute like especially for convolution, it uses a lot of compute.

Yes, that's for training, but for inference.

You can run it anywhere.

Okay, thank you.

Yeah, it's the same model you just restore. Actually our examples like label image. That's the C version. I think I also wrote one. It's called Classify Image. It's in Python. That's also. You can run it on your phone. So any of these, you can write your own and load the SharePoint and run it on your phone as well. So definitely I encourage you to do that.

Thank you.

Cool.

Hi, I have a question related to TensorFlow serving. So I went through the online documentation and currently I think it requires some coding in C and then combined with Python. Is there only going to be, you know, only Python solution that's going to be provided or is it always going to be, you know, I think you need to do some first step, you know, to create a module and then, you know, just import it into Python.

I am actually surprised to hear that because I'm pretty sure that you can write the model in just Python or just C. You don't have to write it in one way or the other. They might have a special exporter tool. At one point that was the case. They wrote their exporter in C. I think that's probably what you were talking about. But you don't have to build it in any specific way. The model is just. You can write in whatever language you like as long as it produces that graph and that's all it needs.

So TensorFlow, serving the tutorial. Actually, if you go on the site, it had. Yeah, those steps actually.

Okay, so I will look into that. So maybe you can come find me later and I'll see what the situation is. I do know that at one point they were writing the exporter in C only, but that should have changed by now because we're doing another version of serving TensorFlow serving.

And is there any plan to provide APIs for other languages like. Like MXNet has something called MXNet JS and you know.

You mean at the front end?

Front end, yes.

Yeah, yeah, we have go. I think we have go. We have some other languages. Maybe Zach can speak more to it. And once again, if those languages are not our favorite, please do contribute. And if you would like us to do it, talk to Zach and maybe he can put that, you know, maybe.

I don't know because as somebody asked for the Android you need Java for. So I think that's going to help out in integrating these models with.

Yeah, that's great, great feedback. We'll definitely take note.

Thank you.

Thank you.

Hi, I have a question. I'm having embedded GPU board TX1, which is a ARM processor and I really wanted to work with the TensorFlow, but I got to know that it can only run on x86 boards. Okay, so when can we expect the TensorFlow can support ARM processors?

We will have to get back to you after I have consulted with my product boss, see when we can add that support.

Thank you.

Sorry, one last question.

Thanks for the presentation. Sherry, I have a question regarding the. When you have the model and you want to run inference, is it possible to make an executable out of it so they can drop it into a container or run it separately from serving? Is that something that you guys are looking into?

Just run the inference? Yeah, just have it as a binary.

As a binary, yeah.

You can definitely do that right now. You can, yeah. You are always able to do that. You mean just save human. Just save the. You want to.

What I mean is that if you can package it into a single binary source that you can just pass around?

Yes, yes, we actually do that today. That's how the label image works. It's just its own individual binary.

Okay.

It actually converted all the checkpoints into constants, so it doesn't even need to do this low, et cetera. It just reads a bunch of constants and runs it. So it's super fast.

Let's thank Sherry again. We are going to take a short break of 10 minutes. Let me remind you for those who haven't noticed yet, but all the slides of all the talks will be available on the website, so do not worry, they will be available at some point as soon as we get them from the speakers.

Oh, I forgot to ask my bonus question. But in any case, I have a lot of TensorFlow stickers up here. If you would like one to put will proudly display on your laptop, come get it.