TensorFlow in the cloud

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@robar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. Digital Ocean's developer Cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.9 uptime SLA and 247365 world class support to back that up. Digital Ocean makes it easy to deploy, scale, store, secure and monitor your cloud environments. Head to do co Changelog to get started with a $100 credit again.co changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence pract, practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changehaw.com community and follow us on Twitter. We're @practicalaifn. Okay, here's Daniel and Chris.

Well, welcome to another episode of Practical AI. I'm Daniel Whitenack, a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a principal AI strategist at Lockheed Martin. How you doing Chris?

Doing great. How's it going today, Daniel?

It's going well. Coming off of vacation, it only took me just about till 2pm on Monday to catch up on email, so not too bad. Had to jump in early and catch up from things, but all is good. How about you?

Just slaving away while you're on vacation? My wife and daughter actually went to Disney World without me.

Oh no.

So they left me at home with the pups and doing work.

Gotcha. Didn't go to see the new Star wars deal? Star Wars World or. I don't. I don't know what it's called actually.

No, I think they saw Harry Potter but not Star wars this time. Yeah, Universal.

Cool.

Well, I'm excited to be back from vacation. I'm excited to have with us today an awesome guest. Today we have with us Craig Wiley, who is Director of Product Management for Google Cloud AI Platform. Welcome Craig.

Thank you. Very excited to be here.

Yeah. Excited to talk to you today about a lot of TensorFlow related things and a lot of Google Cloud related things. But before we get into that, could you just give us a little bit of your background and how you got into AI and eventually ended up at Google.

Sure. So you know, as you said, I run product management for our AI platform here at Google. Previous to that, I spent a couple of years at aws building Amazon SageMaker and then previous to that I spent a number of years in Amazon supply chain group doing kind of optimizations and starting with pivot tables and moving on to classic econometric regression and then moving on to kind of more and more unsupervised and more and more deeper and deeper learning as we tried to solve some of these kind of unbelievably complex problems. And that the ever present goal of trying to make all of this go faster and yield stronger results got me just super interested in the tooling space. Space. And so since SageMaker and now at Google, I've been really focused on how it is that we can unlock the power of data within the enterprise and give companies and enterprises the ability to gain full benefit from the data sets they're collecting.

So then is your background more on the did you do like software engineering before that or is your background more in science in terms of how you got into working on supply chain stuff at Amazon and all that?

Yeah. So I was a philosophy major who believes deeply that the world can be regressed. If you had all the information, then I think you could kind of predict anything. Right. And so it kind of followed that sense. And as I joined Amazon, I found myself at a data driven company, increasingly trying to answer the questions deeply with data. And that next thing I knew, I was leading a team of machine learning engineers trying to do this at a scale that I had never imagined.

So does your philosophy major, do you feel like that gives you a unique perspective on AI related things or do you feel like you're pretty much in the weeds of the technical stuff and you're not thinking about philosophy too much?

It's a little of both. There's times when I'm pretty deep in the details here and there's other times when I think that ability can lend some context around some of the work we're doing in responsible AI. And under just I mean fundamentally, if we can think about these problems as here's the input and from this input we can determine an output. When we break the problem down into something that's that simple, I really think that we can broaden the customer group who we reach out to with regards to building these types of models.

I know one of the things that I really appreciated for quite a while is, and you probably know more of the background on this, but I remember that someone in Google came out with this sort of, I think it was called Machine Learning Guidelines or Rules for Engineers or something like that. I remember seeing it as a PDF originally and I think now it's actually got a nice website and everything. But it was very practical set of guidelines for engineers getting into machine learning problems and all of that. And I found that really useful. Maybe we can link that in the show notes, but I definitely think that, you know, your background probably has helped you dig into some of those problems and I, I really appreciate a lot of what Google has done in that area.

As a leader in this space, with great power comes great responsibility. And so we find ourselves building things like our AI principles in an effort to try and ensure that we're proud of the work we're doing and that our customers are doing and that others in this industry are doing. So it's a deep area of passion for many involved here.

Yeah, I'm not surprised. I know that kind of going to. Following up on what Daniel was saying a moment ago, as Google has published various principles in terms of how to do machine learning, how to do ethics as, which is a hot topic now that we talk about quite a bit, all these things are certainly very much leading edge and the rest of us in the industry are constantly looking to see what you guys are putting out there, which kind of, you know, leads to transitions us to another topic, which is TensorFlow, which obviously being, you know, certainly one of the two dominant, maybe the dominant tool out there, TensorFlow, to be able to do machine learning in this space, that probably the vast majority of our listeners are quite aware of it by this time? I'm kind of curious though, what is the relationship? Could you kind of tell us what TensorFlow is and what is the relationship between TensorFlow and Google @ large? And, you know, how do those communities interact and kind of maybe give us a little bit about the origins of TensorFlow at Google, how it got kicked off, you know, how people opted into that and how the ecosystem started developing on.

That. Yeah. So, you know, TensorFlow is, for those who aren't as aware, is kind of the premier deep learning framework for building these neural networks that have become so transformational in the kind of journey of artificial intelligence and machine learning. And TensorFlow, it's an open source project kind of started and led by Google, but with contributors from all over the world and from many different companies, corporations and organizations. And at the end of the day, I think when I think about TensorFlow, I think about the ability for people to do increasingly complex tasks easily so that they can get to the point of value creation faster. Now, you asked about how we sit organizationally things. TensorFlow is a part of our research group and internally has a charter or mandate to ensure that Googlers have the best tools they need in order to build the kinds of assets, the kinds of machine learning tools and assets that they want. Having said that, in cloud we work hand in hand with this group, to be honest, probably of folks who are not in my direct kind of org, this is probably the org I am closest with, meeting with them kind of on a daily basis to figure out how we can provide more value to TensorFlow users and how TensorFlow can provide increasing value to Google and Google's cloud customers when need.

Be. Yeah, and I know, I mean, even though I guess a lot of the decisions around TensorFlow and maybe the roadmap are made by people within Google, I know that for example, TensorFlow 2.0, there was a lot of community involvement in that and community feedback that led to more usability and that sort of thing. TensorFlow 2.0 is a pretty big deal. I think maybe you could kind of remind us what happened with TensorFlow 2.0 and how that was driven by a lot of this user feedback and that sort of.

Thing. Yeah, I think you're right that TensorFlow continues to be an active open source program and is one that the TensorFlow, both the Google TensorFlow community and the open source community are kind of constantly seeking a wide variety of input and contributions. One of the things that I was most excited about within this same development time period, just as TensorFlow 2.0 was starting to really take shape and get to a point where it would be nearing release, is at that same moment that we started to come together and bring TensorFlow Enterprise to our customers, which is basically offering for enterprises to be able to have a rich TensorFlow experience with the requirements that they need kind of front and center in a cloud.

Environment. And what are those? Could you just kind of briefly describe what are some of those requirements that are unique to like a production enterprise setting that maybe are harder to deal with in an open source.

Setting? Yeah, I mean, so you know, TensorFlow Enterprise, you know, it's really designed to accelerate the software development experience and improve the reliability for AI applications at the enterprise. And you know, it starts really first with enterprise grade support. And when I say enterprise grade support, really what I mean, there is a lengthening of the support window for previous versions. We know that Folks have developed exciting models and models that create a lot of value for their organizations on 114 or TensorFlow115 or something like that. When TensorFlow as an open source project, they only support older releases for one year. And what we've done in cloud is we've come through and said we will extend that by an additional two years so that you get a full three years of support so that these models that you've built in TensorFlow 114, you're not feeling pressure to move them out of production and replace them with newer models. You can keep them in their environment producing value, knowing that security patches will still be deployed and kind of fundamental functionality patches or bugs will still be fixed if you should incur them even after that one year window has elapsed. We also, we focus deeply on cloud scale performance and ensuring that the TensorFlow Enterprise binaries are highly optimized for Google's cloud environment and as well as for running on kubernetes should folks want to run it in a hybrid or on prem installation. And then finally, we believe there is a ecosystem of apps that are going to be critical for customers to gain the most value out of TensorFlow. And so for instance, we've launched a public facing TensorBoard experience and we've been open about the fact that we're working on an enterprise version of that tensorboard experience as well. So it's really not kind of one thing, but it's kind of a series of pieces that together we find we can kind of piece together and build a comprehensive edition of TensorFlow that can really speak to the needs and requirements of these.

Enterprises. So you have customers that are engaging in TensorFlow in the enterprise model and recognizing that the core TensorFlow project is open source, led by your internal Google team. From a customer's perspective, if they're opting into the enterprise version, what is the relationship like for the customer between engaging you with the additional support that you just talked about in the enterprise model versus that open source community? How do those two separate sides of the TensorFlow world at large interact with each other for that customer who has access to both.

Sure. So I mean it really comes down to, you know, TensorFlow is an open source project. Anyone can go and download TensorFlow, run it on their laptop, run it their own data center, or run it on their cloud of choice. TensorFlow Enterprise, having been built and highly optimized for Google Cloud is designed so we have a series of products such as our deep learning virtual machines, which are basically VMs within Google Cloud that are pre configured to run TensorFlow Enterprise on them. So all you have to do is come in, pick an instance size, do you want a small CPU or do you want a giant GPU? And say which version of TensorFlow Enterprise you want installed and we will outfit you with a machine so you'll be up and running instantly with everything you need pre installed. And then we have containers for folks who want to use any of our managed services, our Kubernetes managed services, or if they're running their own clusters using kubeflow or other Kubernetes frameworks for managing AI. It gives them the opportunity to get closer to their app if that's what they need to do, or move into an on prem environment if that's a priority for them. So really the way they can engage in TensorFlow enterprises, it's kind of the default engagement when you work with Google.

Cloud. So I'm curious, we've talked about open source TensorFlow, we've talked about the various elements of TensorFlow Enterprise. I was wondering also if you could mention a bit about what TensorFlow Hub is, since a lot of people, I think part of their first interaction with TensorFlow and maybe the problem that they're working on can be solved by pre trained models. So I was wondering if you could kind of give us a glimpse into that world and what's available and how that.

Works. Yeah, So I mean TensorFlow Hub is a library for the publication and discovery and consumption of reusable parts of machine learning models. Right. So users can create modules which are kind of a self contained TensorFlow graph along with its weights and assets that can be reused across different tasks or different applications. And so really the idea here, and I think we're increasingly seeing this both within TF Hub as well as other locations, is this idea of kind of composable AI where folks can build small pieces that others can then take those pieces, leverage them and make them more broadly available. And so TensorFlow Hub is really a collection of those modules to help accelerate the machine learning process. I'll tell you, as someone who has worked for a number of years and continues to work in cloud machine learning, a huge percentage of my kind of thinking is spent on how can we improve the cycle time of building machine learning models and getting them into production. Because for so many companies it comes down to a hey, I have a team of data scientists. That team of data scientists today can get me 10 models a year. And if I can come through and with the tooling that we put together we can help them double, triple 10x their velocity, then the ROI of machine learning in that organization increases substantively. And now the problems that they're willing to attack with machine learning gets much broader. And so I think any, I'm super excited about all of these technologies that give customers the ability to save time and effort and thought, really kind of like time of thinking and thought energy on these problems that have been solved by someone else. And you don't need to reinvent the wheel, you just need to go get their wheel, drop it in and use it in your case. And I think that's what TF Hub has shown specifically, you know, such a strong ability to.

Do. If you like this show and you aren't listening to the Changelog, hey, let's fix that bug. The Changelog is our flagship show and we've been doing it for over a decade. Adam and I seek out and interview the people who are pushing the world forward with software. We dive deep into the hacks, the innovations, and the leadership required to do what these amazing people do. One recent example is our conversation with Anders Damsgaard, a climate scientist from Denmark, who gave us a peek inside his work and how he scratched a common itch he has when gathering academic research from around the web. Here's a dorky moment from that.

Episode. Are you trying to be right or are you trying to solve the world's.

Problems?

Exactly. Like if you're a scientist trying to be right, well, then you're right may not actually be the.

Right. Yeah.

Exactly. There's another.

Saying. All models are wrong, but some are.

Useful. I like that one. Then there's another saying. All models are wrong. Except for mine. Mine's correct. We had a lot of fun with Anders. He's a fascinating guy. Continue listening@changelog.com podcast or search for the changelog on your favorite podcast app and find the episode called Open Source Meets Climate.

Science. So, Craig, if you are a practitioner out there and you're, you know, you're using Google's cloud AI services in the enterprise, you know, what are some of the things that you're seeing pop up over and over again in terms of use cases, in terms of applications that are kind of clearly leaning toward what your services are able to provide, are there certain use cases that you're seeing come up over and over again, are fairly common to the platform where people are able to bring value Fairly quickly to AI, you know, using TensorFlow in this context of Cloud AI, or, you know, is it just everything is different? What kind of trends Are you.

Seeing. Yeah. You know, this is one of these questions that gets harder every month for me to answer. You know, like, you know, I mean, if you had asked me this question a couple of years ago, I would have kind of said, oh, you know, people build great recommendation systems and, you know, and people build great forecasts or great, you know, vision models or something like that, you know, and the kind of application of models was still relatively narrow, but today, if I look at company like Unity. So Unity uses Google Cloud and TensorFlow Enterprise to quickly test, build and scale out ML models at a massive scale, allowing them to serve up kind of the most relevant ads and drive revenue for game developers. And so we don't think about when we kind of turn on the game on our phone or light up the most recent game that we're excited about. We don't think about the fact that there is tremendous technology going on behind the scenes to target those ads to pay for that game so that it can continue to be free for us. This is something that is very important to Unity and its developers and customers. And they have found tremendous success using TensorFlow, using TensorFlow Enterprise, and using Google Cloud to kind of be able to scale this problem that requires them to kick off hundreds, thousands, or tens of thousands of models in an effort to highly target their advertising and what have you. I take that as one example, and then on the other end, I can call out a company like GM Cruise. So GM Cruise is a strong leader in autonomous vehicles and is a TensorFlow enterprise customer. And the deep collaborations between Google Cloud, TensorFlow and GM Cruise have given us the ability to reduce their training times. I think when they arrived, it was something along the order of four days. And now it's less than a day when you're able to achieve a 4x reduction in cycle time. Now all of a sudden, they can potentially move four times faster in trying to get to the solution to their problem. And as the father of a young child, I have to say the idea that he'll never drive a car is a very exciting idea to me. And so I'm more than willing to help all of these folks get as far down that path as they can as quickly as.

Possible. It's funny you say that, by the way. I was just gonna say, because I also have a young child and I try to tell her the same thing, and other people and they struggle to believe it. I just found it very curious that you jumped in on.

That. My son has even said, At 17, I'm not sure I'll get my driver's license because I'm not sure those will still exist and things like this. But those two problems are about as different from one another as could be. And certainly we see problems across industries, whether it's fintech, retail, manufacturing. I continue to be humbled by the creativity that folks have in finding ways to utilize this technology to create value for their.

Companies. So I'm kind of interested. Since you see so many use cases in your day to day and have seen so much variety, I'm kind of curious to get your perspective on actually how enterprises are leveraging AI. And what I mean by that is kind of in my mind how I see various companies approaching AI is you've got, on one end of the spectrum, you've got companies that are like really investing in intense research and development and maybe they're actually, you know, have their own AI team and they're developing their own model architectures and very unique models, very unique combinations of models and doing those sorts of things. You've got other groups that are what I would consider more kind of like doing AI cooking in the sense that they get like a recipe, they get an existing neural network architecture or something like that, they plug in their own data to create their own model and then go from there. And then there's other end of the spectrum. Definitely a lot of people that are just scaling up inference so they might be using a pre trained model or a combination of pre trained models or modules, or maybe they're just doing a bit of transfer learning or something like that based on what someone else has previously done. And so they're not even contributing a ton of data to the situation in terms of training. I was wondering from your perspective, where are you seeing the biggest investment from enterprises these days? Is there a shift one direction or the other on that.

Spectrum? It's a really good question and it's one that we spend a lot of time thinking about and working with companies on how they're thinking about this problem. I'll say the companies who I've been most impressed when I hear their strategy around these things, it really comes down to where they choose to buy versus build. And from that perspective, one of the things that I think deeply about and that I think others do as well is this idea that if you can buy a solution, whether it's a contact center AI, our intelligent virtual assistant or contact agent in your call center, our document understanding these technologies that are powered by AI, these make a ton of sense for companies to buy because for most folks your contact center isn't the primary point of differentiation for you. And now as you move kind of further down the stack, I think the question becomes, where are you comfortable using AI to accelerate your business and gain efficiencies in your business? And where does that do those efficiencies? Where are they tied to your competitive advantage as a company? And fundamentally, the idea of buying your AI makes a ton of sense if it's something that you're very comfortable with your competitor buying as well. But when it comes to what you should build, it really comes down to where are the unique areas that you think you could kind of express some differentiation in that industry. So if I take retail as an example, we'll see retailers who are happy to buy a recommendation system, but they want to optimize the back end and their supply chain deeply because that's how they believe they can stand out. Or maybe they'll want to highly optimize their promotional capabilities or something along, or their forecasting capabilities. Whereas others may say, listen, forecasting and supply chain, I'm happy to work with a partner and get the best in class that is easily available to me. But I really want to stand out on our usability or our recommendations and things of that nature. And that's how I plan on setting us apart from others. I think that that's a conversation that companies have been having for a long time with regards to the software they're building. And if I think over the last decade of the number of industries where the leader in that industry has almost become a hybrid software company, whether it's a retailer who's become more or less a software company or a healthcare provider who has a giant software investment, those same decisions that have powered that kind of investment over the last decade, I think now we're seeing that same set of decisions being applied to how they'll invest in intelligent computing and where they'll choose to build versus buy. So I'm not seeing kind of a wholesale like, hey, people are giving up notebooks and are only going to go buy from now on, or vice versa. It's really around how companies will choose where they are willing to build versus where they're willing to buy and how that aligns to their.

Strategy. So I love your perspective there kind of the business take on where you're going to choose to invest in adding value and how you create competitive, competitive advantage using these types of AI tools for your own organization. And so I found myself, as you were saying that, I found myself wondering, you know, what do you think some of the big challenges that you see people trying to create competitive advantage for themselves, where it's not the run of the mill stuff where they're just taking their data and doing a little bit transfer learning and creating their own version of the same model. But the things where people are saying this is where our organization wants to make a mark, are there any examples that have particularly surprised you or caught your attention in terms of big challenges that organizations are staking themselves.

On? You know, it's an interesting one. You know, my life is spent trying to make it easier for them. Right. And I'll say I continue to believe that one of the biggest challenges in this space is actually far to the left of machine learning. And you know, when I say left, I kind of mean in a way, workflow. If I start with kind of data acquisition at the left hand side and I end up with a model in production at the right hand side, that kind of data acquisition, that data cleansing, I'll tell you the number of organizations who are where I'll talk to data scientists and they'll say, yeah, the data is decently clean and I know where to get it, but I can't get it because there's organizational silos and this other organization owns the data and I can't do machine learning on it or something like.

That. A really common problem.

Too. Yeah. And these are the things that just break my heart. And you know, one of the exciting things for me about working with Google Cloud is that the tight integration between our AI team and our analytics team results in some opportunities for customers to much more easily do these things. So a great example would be BigQuery. So BigQuery is Google's hyperscale data warehouse product that I'll say if you haven't played with it, you should because it is an order of magnitude kind of faster and more scalable than anything I've gotten an opportunity to play with. I came into machine learning by running data warehouses in many ways, so it's close to home for me. But today you can build machine learning models in BigQuery using SQL as your programming language. And so you can train an XGBOOST model or train a TensorFlow model, all from the SQL UI that data analysts would be accustomed to. It's integrations like that that I hope can help us break down some of these barriers that are, I'll say often they're blamed on tech, but often the problems are as much policy as they are tech. And, and I think the key is can we simplify the governance for folks, not remove it by any means, but simplify it so that it becomes much easier and much less scary, so that they can get to the point of extracting value from their data even more.

Quickly. I know one of the things that I've hit occasionally is because you're working in the cloud and people in large enterprises have traditionally had their data in their local data centers. And maybe there's restrictions like governance, things like you were talking about around moving that data around. Do you still see that popping up in terms of people being hesitant to utilize cloud resources for AI just because they have to move the data around? Or do you, do you see any progress on that front in terms of people being more willing to invest in moving their training data set to the cloud and managing it.

There? Absolutely. You know, customers are getting more comfortable. I think this problem is being solved in two directions. One is the clouds in general and Google in specific. With Google's Anthos, which is our ability to run a Kubernetes cluster across a data center and cloud as if it were kind of one kubernetes cluster, sort of like a hybrid deal, technologies like that start to diminish the damage of saying, hey, I can only have my data here and I can't move it elsewhere kind of thing. Right. And so in one direction there are strategies like that, and then in the other direction, as cloud continues to grow, and I will routinely meet companies who will say, well, my industry, and you can fill in with whatever industry doesn't allow me to put data in the cloud. And we'll talk to them about how many references we have in their industry who are putting data in the cloud. And it very quickly they realized that it's not their industry, it's their company or even their department that has built these rules. Probably years ago, before it was understood the benefits of things like cloud security and reliability and resiliency. And so I think those kinds of blockers are kind of falling by the wayside, either because companies are realizing that the cloud is not as scary as maybe they imagined, and then secondly, because the cloud is coming to the customers in a big.

Way. So one of the things, as we kind of talk About Google's Cloud AI and using TensorFlow in the cloud, I know for me, using COLAB has been a real game changer. I will just go ahead and say it. It is my favorite cloud environment to work in. And I know a lot of other people who feel the same way. And so I've kind of stuck my own bias out there. What kind of insights have you had into the future of how we're going to use these tools in the the cloud, where Colab might be going specifically, and what kinds of integrations will we continue to see and accelerators will we continue to see in these types of.

Environments? Yeah, whether it's Colab or whether it's notebooks on Kaggle or whether it's Google's own Google Clouds Jupyter as a service or Jupyter Notebook service, I continue to be excited and I'll be honest, slightly disappointed, and I'll get into that in a moment. I continue to be excited by the evolution of tooling for machine learning. I say slightly disappointed. I wish it were moving faster. I think you're starting to see that with more and more kind of opinionated workflows around, whether it's automl type of workflows or kind of rich templates to your point of the cooking recipes that can get people 90% of the way there and then let them adjust the part that matters to them. I continue to believe that the way we're doing machine learning today probably isn't the way we'll be doing machine learning a decade from now. And I often think about software development and if we were to rewind the clock on software development by 20 years, concepts like regression testing and unit testing and these kinds of things weren't a part of the everyday software development life cycle. Today, if you went to go deploy something and you hadn't passed it through a regression testing suite, we would kind of think you're irresponsible. And we don't have those same concepts yet for machine learning. And to me, one of the reasons why it's so exciting to work in this space is that I feel like we are getting to develop the next generation of standards around how people will develop machine learning. And, and whether it's things like pipeline technology to improve CI CD experiences, whether it's specific TensorFlow modules built into TF Hub or notebook examples on Colab or on Kaggle. This is an area that you can feel is very likely going to evolve very quickly over the next few.

Years. Yeah, I was just going to mention before we get off of Colab and some of the things there's there's certainly, as you mentioned, there's standards that need to be developed and rigorous things that need to be developed. And I know guests like Joel Grus and others on the podcast have talked about that. But on the other side of things, like I've been working with this Masakane collaboration recently which is trying to involve research groups and people on the African continent into developing machine translation technology for the languages that they care about. And essentially central piece of that is the fact that they're able to very quickly onboard participants into their working group because they have a set of Google Colab notebooks that they're able to spin up, they're able to run in their browser, they're able to have access to a GPU to get them started on training baselines and they don't have to have access to their own gpu. They don't have to even have a lot of experience in an and a local environment set up with a IDE that's all set up to do certain things and all of that. So that's really, I think, been crucial to that collaboration is the fact that they are able to onboard quickly into that. Now, obviously, as you productize things, as you were mentioning, as you work with enterprises that have certain concerns, there's definitely a matter, matters of integrity and that sort of thing and robustness that need to be dealt with. But I think some of those concerns are being addressed. But also on the other side, there's a lot of tooling that's being developed out there that's really helping make this technology more accessible to more groups of people, which is really exciting for.

Me. Absolutely. And I couldn't be more excited to be a part of the kind of the work that is going into this democratization process and widening the field. Machine learning developers and creators. And then fundamentally though, I'll say I always worry about whether or not those models are going to get into production, because one of the things we know is that the vast majority of machine learning that gets built never makes it into the area where the inferences can now start creating value. All too often it gets built on someone's laptop or maybe in the cloud, and then they're never able to actually integrate it into the application or integrate it into that point where it could create value. And this is an area where we invest deeply to try and ensure that. Like, hey, once I've built that model, whether it's in Colab or wherever I built it, that I can easily put it into a place where I can call it for inference and kind of gain the full value of the model. As data scientists, I think it's very easy to become attached to the training as the, the training and kind of the AUC curves or what have you as the point of success. But for me, really the success comes when those inferences are creating the value that we all set out or the goal we all set out to achieve when we first started trying to build the.

Model. That's a great point that you're making. We do tend to get caught up in our own training and thinking of the training is the thing versus the value that you're creating with the model after you deploy it. I would feel remiss. I'm kind of curious. I just wanted to go back a little bit to your personal bio for a minute and definitely not wanting to get into the politics of competition and stuff, but coming from aws, you came into Google, you had a hand obviously in creating SageMaker and obviously like you would with any job, moving to another. You learn and you get better and you gain knowledge as you do that. Is there anything that comes to mind that you learned from that previous experience? And definitely not putting down SageMaker, but just like, you know, you went through that process and you gained expertise and then you came to Google and you started working on Google AI. Is there anything that really helped you as you came across.

There? You know, I was unprepared for the benefit of sitting side by side with the Google's research groups. You know, Google is clearly a leader, if not the leader in AI and machine learning. And to get to sit shoulder to shoulder with Jeff Dean and others as they solve some of the most complex machine learning problems in the world and then taking the components with the kind of fundamentals or the primitives that they've had to use, extracting those out of some of these world class problems and putting them into an environment where enterprises of all levels can interact with them is something that I was just unprepared for. To give an example we're using, if you look at DeepMind and what DeepMind has done with AlphaGo and with StarCraft, and then we use that same technology for optimizing manufacturing facilities and optimizing the control systems in manufacturing facilities and that kind of research, the tight connection between kind of research and the practical applications and practical tools we're building is just something I don't think you could get anywhere in the world. Like you can get it at.

Google. I know one of the things that I feel a lot as someone who isn't at Google, but keeps a close eye on what's going on in AI, it often seems like there's just so much happening so quickly and obviously Google is doing a lot, other groups are doing a huge amount of research and it just seems like, oh, you know, like recently I remember Facebook released this like Multi language question answering data set. I was like, oh, this is really cool. This is like something I wanted to see for a while. And so I started playing with it and before I even got done like, you know, figuring out some of what they had done and how it worked and the format, there were already like, I think Google came out with another one and there was like another. There's been like three or four non English question answering data sets and models that have come out after that. As someone who is kind of helping shape the future of Google cloud AI, and in those discussions, how do you parse through all of the advances that are just happening so quick and what people are wanting to do? How do you keep from getting left behind? I guess that tight integration is part of it, but it still seems like a hard thing to keep up.

With. Yeah, herding ducks I think would probably be the right idea here. Right? I mean it is amazing because you'll sit there and think, my goodness, I couldn't be any deeper on this topic than I am today. And then you'll find a team will announce that they've done something that that's two orders of magnitude deeper than where we were. And it's kind of, you just sit there and are like, wow. And so really it's getting to know all those folks and keeping close with them so that we can look at not what they're doing today, but where are they going and where are they going to be six months from now, 18 months from now, and 36 months from now? And how do we ensure that we're building towards that fact? I mean, in kind of cloud, where I sit, I look at things like GANS or things like reinforcement learning and say, well, listen, most enterprises aren't running machine learning. Most enterprises, if they're running machine learning, they're running Scikit models at this point. What is the value of GANS to me right now outside of an academic environment? But then what we find is very quickly, far more quickly than I would have imagined. Companies start jumping on some of these technologies and we find new and exciting ways. And you know, so for example, we are running a Kaggle competition, a reinforcement learning Kaggle competition right now, first of its kind within Kaggle and you know, seeing some really interesting ways in which users and developers who have never done reinforcement learning before are starting to interact with this. And by watching the conversations they're having and the challenges they're having, I think it allows us to build better products, you know, as we come out of.

That. So I guess you Know, as we kind of work our way to the end here, people are listening to this, they've gotten a little bit of knowledge about Cloud AI if they haven't already. And TensorFlow, what would you recommend is the best way for people to get their hands started in TensorFlow and in TensorFlow in the context of Cloud AI as an enterprise customer? Obviously there's the crash course that Google has, which kind of gets you started there, but could you take us a little deeper into how people can steer into all these tools and get started with.

Your. Yeah, I mean, so, you know, I think if you're getting started fresh and, you know, haven't touched this technology before, then I would certainly recommend heading towards Kaggle and some of their educational resources. I think, you know, they have things like, you know, 10 weeks to, you know, coding in Python, coding in Python in 10 weeks and, you know, other things around, you know, learning a number of the basics and starting to get engaged with kind of machine learning. And so, you know, I think that's a great place to start. If you're newer, if this is an area where you have kind of skills and expertise, then absolutely continuing to utilize Kaggle and their competitions and their wide variety of code samples. Colab as well. But then going to cloud.google.com and heading to our AI tab and then honestly going to our deep learning environments and spinning up a VM with deep learning on it. But I remember years ago when I was working in supply chain, I had a really strong econometrician on the team and I said, hey, why don't you go install a deep learning package? I can't remember if at that point it was Theano or TensorFlow or earlier days. Install a package, get it going and let me know what you think. The next week I met with him and he said, I worked at it for, for a number of hours, couldn't get it running. And so I went back to the stuff I know because I knew I could create value there. Well, with our deep learning environments, our deep learning VMs and what have you, all you have to do is click on the thing and now you have a TensorFlow experience or open a notebook with a GPU as a back end and you have everything you need to run strong and run fast there. So I would just tell people, dive in, find some code samples, try and break them, try and mess them up and go from.

There. Awesome. Well, appreciate that perspective and we appreciate you taking time from all of your work to come and chat with us. It's been a great time. I know I've learned a lot and excited to continue exploring what's coming out of Google. And thank you so much for joining us. We'll mention all the things that you just did in our show notes and hope to stay in contact and see you at a company conference or something.

Soon.

Yep. Thanks a.

Lot. Absolutely. Thank.

You. Thank you for listening to Practical AI. If you're not following Practical AIFM on Twitter, you're missing out on clips and highlights from past episodes, links and repos from around the AI and data science community, and more. Follow Us Practical aifm. You won't regret it. Practical AI is hosted by Daniel Whiteneck and Chris Benson. It's produced by me, Jared Santo and our music is brought to you by the Beat Freak Breakmaster Cylinder. We have awesome sponsors support them. They support the show. Special thanks to Fastly, Linode and.

Rollbar for helping us do what we.

Do. If you aren't receiving Changelog weekly every Sunday, you are missing out. It's our take on this week in the world of software. What's interesting and why. Head to changelog.com weekly to subscribe. Get it for the price Price of a free cheeseburger. Thanks again for listening. We'll talk to you next.