PyTorch 1.0 vs TensorFlow 2.0

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. They now have CPU optimized droplets with dedicated hyper threads from Best in class Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application. This gives you immediate access to Python 3R, Jupyter, Notebook, TensorFlow, Scikit and Pytorch. Use our special link to get a 100 credit for DigitalOcean and try today for free. Head to dot co changelog once again dot co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community follow us on Twitter. We're @practical aifm. And now onto the show.

Welcome to Practical AI. I'm Chris Benson, one of your co hosts and with me is my partner Dan Daniel Whitenack. How's it going today Daniel?

It's going great. How about with you Chris?

I'm doing well. It's been a long time since we have put out a new show, so this one is long overdue. What have you been up to lately?

I've kind of been doing a little bit of traveling, went out to O'Reilly AI, recorded a couple episodes there which have been released that was, that was a lot of fun. Went on a little bit of vacation and also learned a little bit of data visualization stuff with package called Vega, which that was fun. But yeah, glad to be back on the on the show. I know you've had some health, broken bone issues, so I'm glad to see you're back in action and great to be recording with you again.

Yeah, it's good to be back after MIA for a little while for our listeners. I was actually going to be at O'Reilly AI in San Francisco with Daniel to do recording for those last couple of episodes that we had of those and I missed it because like two hours before I was supposed to be at the airport I broke my foot and so I ended up going to the emergency room and calling Daniel from the emergency room going I'm not going to make it. So thank you so much for handling all that alone. I know, you were even fighting a cold off with losing your voice and so you had your hands full.

Well, it was, you were missed. But the next conference we'll do that one together. And I should say too, for the, for the listeners, if first off, we'd love to have you join our slack community@changelog.com community and we'd love to hear about what events you're going to be at, whether that's O'Reilly AI or ODSC or Strata or Machine Learning Applied Machine Learning days or MLConf, whatever, whatever ones you're gonna be at. Let us know and let us know if, if you'd like us to be around at those, at those conferences, recording some content and, and interacting with the community. We'd love to, to meet some of you.

So yeah, and we, we get great feedback from you guys on the, in the Slack community and we also have a fairly new LinkedIn group called Practical AI. So if you're on LinkedIn, join that because we have some conversation going there as well. And yeah, I guess it's good to be back on the podcast now. I've done a couple of conference keynotes in recent days and I've been hobbling around on my cast, so I'm sure I looked quite comical as I walked up to the podium. So, anyway, on to the show today. Daniel, you want to start us off?

Yeah, sure. So it has been a while since we've had this sort of conversation. We're going to do another kind of news and update show for you guys. There's been a ton of news in the AI over the past few weeks, so we're going to dive into some of that and discuss it and let you know about some of the things that were on our radar over the last three or four weeks. And also we're going to share with you a couple of new learning resources. If you're trying to level up your AI game or maybe you're just getting into the community and want to start experimenting. We're going to point you to a couple of those resources later in the show, so stick around for that. But to start us out, I'd love to just start by some big news in the community over the last few weeks, which has been around PyTorch version 1.0. So in my understanding, at least at the time of this recording, PyTorch 1.0 is in kind of its preview or release candidate stage. Maybe by the time it releases, it'll actually have the full release cut. But first off, I mean, we Just want to congratulate the Pytorch team. It's a huge accomplishment getting to version 1.0 and we're really, yeah, I'm really excited about it. I just want to pass on, you know, our congrats to that team. And also, I mean this is just, it seems like the community is really latching onto this. Even, you know, Google Cloud is implementing a lot of implement Pytorch implementations in their, in their images and other things. And so yeah, it's, it's really great to see this. What do you think, Chris?

I think Pytorch is really rocking right now. It has come on so strong in the last year and you know, it is really just talking to people, so there's no scientific basis to this when I say it, but just observing, I mean, I really am hearing a lot about Pytorch and then obviously TensorFlow, which has been big for a while and talk about that in a moment. But Pytorch team, you're really rocking and so keep up the great work. It is a fantastic framework to work with.

Yeah. And I think maybe this is a good opportunity to just kind of give some perspective, at least from our very biased perspective as far as the Pytorch and the tensorflow community, the state of them, if they're kind of, I don't know, Chris, do you think that they're kind of reaching different segments of the, of the community and who do you see using one or the other?

So it's kind of funny, I see historically Pytorch among the people I'm running around with in data science and AI on a day to day basis has really been in the academic and research arena. And then to contrast that TensorFlow was kind of dominating the kind of corporate production teams. But Pytorch on their front page right now in big letters it says from research to production. And I think that captures exactly the feeling of where they've been going as they have moved to compete with TensorFlow squarely on that corporate roll things out to production front and compete with TensorFlow's existing tools that have been out there. Pytorch is now becoming just a powerhouse, not only in academia, but for corporate teams that once upon a time would only have said, well, we got to do TensorFlow for what they have in terms of getting this stuff out in the world. But so I'm just really happy to see Pytorch coming on the way they are. And from research to production is the perfect attitude for that team from my perspective. As an outsider.

Yeah. I think you, you've hit the nail on the head with that. I really see in the blog posts that I've read About PyTorch version 1.0, it does seem like a lot of the emphasis is on production, quote unquote production or system integration, scaling sort of things. Some of the things I'm really excited about is they're really tight integration with the onnx neural network exchange format, which standardizes kind of model format, you know, across Pytorch and scikit Learn and mxnet and all of all these things. So you can train your Pytorch model and then export it in this way and then serve it with MXnet. Stuff like that is just really cool. Also, you know, integrations that they're working on with things like kubeflow, which is a set of standards for deploying machine learning technology on top of Kubernetes, which of course really kind of zeros in on making Pytorch really useful at scale in a company's infrastructure. And then of course we see things even from Google Cloud that where they're working on integrations of PyTorch with TPUs, which is Google's accelerator technology, kind of similar to GPUs but different, as we learned last week from, from our guests from Nvidia. But they're also integrating Pytorch to be able to be used with TPUs. And so there's a lot of emphasis on that front. And for one, I'm really happy to see that because I think from my experience that's a lot of times where people get blocked in terms of implementing and applying these methods.

Yeah, the things that they call out as kind of highlights on their feature page include hybrid front end, distributed training, Python first tools and libraries, native onnx support, C front end and then cloud partners and all of those, especially the cloud partners. You know where TensorFlow has been so dominant lately that it's great seeing. I love having choice, I love having the option to go where I want. And so big congratulations to the Pytorch team.

Yeah, and I think that it's really cool to see that interoperable stuff because I think the pytorch and the TensorFlow community are just both very, very vibrant. I mean there's TensorFlow Dev Summit which is. Has a lot of momentum. There's the Pytorch Dev conference, I forget what they call it, we're just seeing about that. And both just are really vibrant. And of course the online community, the open source community, I Think one of the things that, you know, I always appreciated, and I use Pytorch now probably a little bit more than TensorFlow, but one of the things I always appreciated about Pytorch was the kind of Pythonic way it allowed you to build up AI applications without having to worry about, like the static graph computations that were that we're in TensorFlow. But I know that that's actually changing as well, and you were telling me a little bit about that. So what's going there?

So TensorFlow2 has been discussed with the TensorFlow team. They made an announcement a couple of months ago and then they've updated the site. And ironically, I think that part of the motivations in TensorFlow 2 that we'll address here kind of come from that competition with the Pytorch team, because Pytorch has been considered to be because of kind of putting Python first mentality. It's been so easy to use and TensorFlow has been notoriously difficult because of the graph mode. And so one of the big highlights of TensorFlow 2 is that they're putting the eager execution, which has been out recently, as the primary mode now. So you'll start in eager execution, and then if for performance reasons or a variety of other reasons, you're ready to move into graph mode, then you can do so. But I think a lot of people are going to welcome that ease. There was a video that I saw recently where they were comparing the two frameworks and they were showing kind of TensorFlow1 versus TensorFlow2 syntax. And you could just see it, it was much more readable and it was a lot. It was just, you know, putting Python first again. And so that was nice to see. And they're increasing support for, for platforms on the TensorFlow side, and they're starting to remove deprecated APIs and things like that in 2.0. So I for one, am really looking forward not only to this PyTorch 1.0 release, but also to the TensorFlow 2.0 release release when it arrives.

Yeah, that's exciting. Is there a timeline for that release or. I think they just announced that they're working on it. Is that right?

I think so. I pulled up the TensorFlow site and I don't see a date jumping out at me, but I couldn't guarantee it right now.

Well, kind of along the same front as the practical production, ready system integration and applied AI stuff that we've been talking about with Pytorch. One thing that. Another thing that I saw over the past few weeks is this kind of set of articles and resources from Google AI that's called responsible AI Practices. So if you remember, I forget if which episode it was in, we can put it in the show links. But we talked about Google's previous release of their AI principles, which really had more to do with maybe on the ethics side of things, things that they would or wouldn't want to do with AI.

Yep, I remember.

Yep. How they viewed that. So we'll find that, that show link and put it in. But these responsible AI practices, they really are more at the practical level of the AI developer, which I can definitely appreciate, I know we on the show can definitely appreciate. And they've had these break in or they have these broken down into a few different sections. So general recommended practices, fairness, interpretability, privacy, security. And I just find these, you know, really practical, really useful, some of their general recommended practices. And they really break these down into bullet points that you can follow easily. But generally they include things like human centered design, identifying multiple metrics to assess, training and monitoring, examine your raw data when possible, understand the limitations of your data set in your model, test, test, test, which is of course hugely important. And then last, continue to monitor and update the system after deployment. And they even include some links to ways that they do that in the article. And I just think, you know, what I was thinking about when I was reading through this is this would make a great. Like if I was an AI team or a new project, I would almost take these and map them to a checklist of sorts where we could kind of just check off that we've at least considered each of these points and we've either implemented some of their suggestions or we have a good reason that we're not doing that. And I think that would be a really great way to kind of move forward responsibly on a, on a project. What do you think?

No, I think that's great. And my last employer, I had to go in and build out the team from scratch. And so having there weren't, there was not, there was a little bit out there. But it's been really in the last year that Google and other key players in the AI community have released these kinds of guidelines. And I would very much have liked to have had them available to me in those early days as I was trying to put together my own playbook and figure out how do you build a team? What are the different skill sets? How do you divide those up? What types of work can you do? There are just so Many questions and, and I guess that that kind of leads me into another one that we had this week was Nvidia had an article, it's in Forbes.com actually from an Nvidia person. It's not as comprehensive, but it was, it was five steps to Build a Business's Deep Learning Workflow. And in that article they kind of walk you through some highlights that is somewhat similar to the Google guidelines that you just walked us through. And I want to note before we leave that behind that you only covered what was under the general category when you were kind of highlighting the sub bullets. There's another or four pages of things that Google had released and that was having those and combining like Nvidia's. Here they talk about identify business problems, build a data strategy, build and train models, evaluate model accuracy and deploy trained models. And each one of those has a number of bullets under it. And so I know as practitioners being able to go and look at all of these different guidelines and how to put it together posts that these, you know, major organizations are releasing out there and starting to get a sense of what your playbook should look like as you're building out an AI capability in your organization. It's a fantastic place to start and I agree with you on that.

Yeah. Is there anything like from your perspective? Because I know, you know, one of the things that I've appreciated about talking to you is you have kind of gone through the process of building up a team around AI and that sort of thing. Are there things that, that were particularly important for you as you did that that were maybe highlighted in these articles or things that maybe you didn't expect as you were going through that process?

Yeah, there's a lot there actually. So I'm just gonna touch on the tip and at some point, I know in the future we're gonna talk about how to put together organizations and hiring considerations and I'll go into more depth from my perspective when we get to that. But kind of all the things that I just called out on Nvidia are kind of high level processes and the bullets are not enough. This article alone won't help the way there, but it kind of tells you the categories you should be thinking about in the Google document that we were just talking about. Kind of goes through a lot of the process stuff that you need to be thinking about now in your organization. You're likely going to have to customize all these around your own size, your own operations, your own team capabilities. And so everyone's a little bit different in that way because you're having to put together your own AI capability and it's going to be a little bit unlike everybody else's. But these are good places to start. Just as a teaser for future. I actually have what I think will be some controversial opinions that I developed when I was doing the team build out. I'm busy writing a blog post I'm in the middle of right now, and I will throw those out into the. For everyone to, to have a go at in a future episode. So I'll just leave that. I'll just leave that. Hang in there.

I'm definitely looking forward to that and giving, of course, my highly biased opinions as well. Yeah, I like what you say. I mean, I think what we're trying to do here and I think what many, many people and organizations are trying to do is represent some type of scaffolding or like I was kind of saying checklist, but really it's kind of like a scaffolding where like, you need to be considering this point that might look different in your organization than other organizations, but you need to consider this point and not, you know, not ignore it.

Right, yeah, totally agree with that. It's, it's a very creative process is all I can say at this point in time. You know, we're, we're still in such early days in building out AI capabilities and the maturity of the community in general. For me, having been around the block more than a few times, it feels like when the Internet was coming to being in about 1993 on that. And if you think how far software development and software engineering has come in the years since, that is the road ahead for us in the AI community right now. So seeing these things and discussing how to put them together, it's the right time. We already know how to do this in other areas of technology, but we're still learning in data science in general and specifically, certainly in AI.

Yeah, for sure. All right, Chris. Well, I'm going to transfer our discussion to a slightly different topic and that of transfer learning. I ran across this article. It was published September 17, and it's called Deep Learning Made Easier with Transfer Learning. It came out from Fast Forward Labs, which is now part of Cloudera and is associated with, you know, with, with some bigger names in the, in the space. But I've really appreciated content that they've put out in the past. Have you, have you read any of their, their blog posts or content in the past?

I have. And you know, Fast Forward is, is, has been a great Fast Forward Labs has been a great source of, of information in the past. Obviously they're now part of, of the larger organization there at Cloudera, but I love seeing their stuff and I love this article by the way, that you found. I'll have some comments, I'll let you share a little bit more about what it's about and then I have some commentary on it.

Yeah, definitely. And I would encourage people, they've actually put out Fast Forward labs, I mean has put out a number of reports or kind of white paper ish sort of things on various topics. I remember reading their one on machine learning, interpret interpretability, which really kind of gave me a sense of what people are doing on that topic and what considerations there are. And they have a bunch of other content that's, that's really great for learning. So in a sense like these are kind of learning resources in and of themselves. We'll give you some more later, but this article I really appreciated because I think, you know, transfer learning is so important in terms of how people are implementing their AI strategy in their company. But the article kind of goes through and it tells you, you know what, what transfer learning is the sense that you're taking, you're taking a model that was maybe trained on a certain task and kind of starting from that starting point and building or generalizing that to another task, building in additional, additional knowledge. And they kind of walk you through that concept with, with a bunch of different compelling figures and even some, some code and some, some Pytorch examples and robot pictures and cat pictures. And I just thought the, the article was really good. So I, I would definitely highly recommend.

And I will kind of foreshadow another teaser of that same thing I was talking about earlier and that is that that this is transfer learning gives you the option of standing on the shoulders of giants. And, and so most companies out there that are creating capabilities are going to be on the implementation side. They're not going to be doing research the way Google Brain is and the way the Facebook team is and stuff. So you know, they'll do enough to get what their use case is and that's going to be the majority of production work in industry. And so if that's the case and if you're able to use your framework of choice and find some work that somebody has already done on a model and you can do that adjusting your way into your use case, transfer learning is really almost the default way that a lot of data scientists and AI engineers are going to be accomplishing their own goals. And that's certainly on the teams that I've been on, that has been the approach that we've used. And I think that that is definitely the major use case. And so I think the more people understand how that process works, the more useful it's going to be. So I think this is a great article in explaining that, yeah, there is.

A kind of a general misconception, I think, that people, when they think about AI, they really look to a lot of content that out that's out there on the web, which is really good content. But maybe it's from like, you know, DeepMind or OpenAI or something like this. And really the incentives of those companies around research and the projects that they work on and the way that they work on them is very different from the incentives in a typical company where they're really focused on these deep research questions and new model architectures and all of that. For the most part, I think, you know, when you're in a company, you're going to be, like you said, standing on the shoulders of giants. You're going to be taking model definitions and architectures that were developed at somewhere, maybe like OpenAI or somewhere, and actually applying them to your own data. I was actually teaching a workshop a few weeks ago now and this question came up and the question was really around like, what does it mean to have a custom machine learning model or AI model for your use case in your company? And the misconception amongst the crowd there was that, well, that always means that you're going to kind of make up your own sort of model definition and equations and expressions that are really kind of tailored to your particular use case and specifically designed to model your data. And I think that by and large is not how things are done. I think in pretty much every case of applying AI and machine learning, what you're doing is you're taking, you know, a model definition that has been developed somewhere like DeepMind or OpenAI. Maybe that's a recurrent neural network or a convolutional neural network or whatever it is, and you're applying it to your particular use case. But really when we say you're customizing it for your use case, you're not changing up the layers of the network. Even in many cases. What you're doing is you're just training that model on your own data to get your own set of, of, you know, weights and biases, your own set of parameters that parameterize that model definition that someone else has developed. And I think by and large that's, that's what people do. And of course, transfer learning provides even a layer of additional help on top of that in that you're not even starting from scratch when you do that training, but you're taking knowledge that was already developed in another tasks and you're kind of starting from a good checkpoint.

So I agree with everything that you said and I think that, I think between us we've identified what I think most people who have been working in the space would agree is kind of the way real life works on that. So great article. Thanks.

Do we live in real life if we're doing AI?

Good, Good question. Although I'm about to transition us into a little bit of a scary real life here, leading in. You know, you and I are always talking about the theme of AI for good. It's something that you and I care about very much. And we talk about you versus the horror stories about what could go wrong. We like to talk about AI being used for purposes that helps the planet, helps mankind, helps everything about us. And we've had some, some great episodes with people who are doing just that. But I want to turn to a darker story for a moment, at least from, from, certainly from my perspective. I ran across one. It's on. It's ABC News, I believe, in Australia, I think. And it's on. It's called Leave no D Dark Corner. And it's about the social credit system that is coming into being in China right now. And it is essentially a system where all 1.4 billion Chinese citizens are going to be monitored 24, 7, 365 all the time through all sorts of different channels. And they are essentially expected to tow the party line, if you will. The Communist Party itself calls it the so called social credit system. It's supposed to be fully operational by 2020. And a quote from the party, from the Communist Party says that it will, quote, allow the trustworthy to roam freely under heaven while making it hard for the discredited to take a single step. And I just, I read that in horror.

It's like something out of a novel.

It is, I mean it's, it's a, it's a 1984 theme again, but, but it's no longer, you know, in past shows we've talked about some of the, the darker stories about, about, oh my God, we don't want to go down a path like that in the future. But this is happening now. They're talking about it being fully operational in 2020, but it's already in place partially now. And they interview several people so if you're, if you are that Chinese citizen who is completely in sync with the Communist Party, then you are good to go because you're going to, you're living a prescribed lifestyle that is approved. And, but if you are, for instance, an investigative journalist and, and you discover that maybe high ranking officials in the Communist Party are corrupt and you know, there's been the big corruption crackdown recently in the Communist Party in China and maybe you upset certain people, then they give an example of one man in particular who does exactly that and his social credit is very low and he can't even travel within the country. He can't get plane tickets, he can't get train tickets within the country and rail is very popular in China. And it's just when I look at that as a Westerner and as an American with the biases inherent in that, that is exactly the opposite of what I hope my life to be and my children's life to be going forward. So I just wanted to draw that out. And if you're not aware of the social credit system in China now you are. And if that's not what you want as a listener, maybe be thinking about what you do want and how to get there.

Yeah, I think this was actually like literally, I think this was a Black Mirror episode on, on Netflix. I remember like people walking around and doing certain things and then they're like, you know, they would get a ding and their social credit, you know, went up or down or whatever. I forget what they called it. Maybe some of our listeners could, can, can remind us in our Slack channel. But yeah, and one of the things that I think beyond the fact of just the social credit system itself, it's like, you know, we've already experienced AI and machine learning being used like in the social media context in terms of engineering people's political views and all of that. And really what we've seen is that those systems and especially the ones that are driving advertising are really, really pushing people to the extremes of their, of their views. Right. And it seems like this is, you know, the same thing, but on a, on a greater scale in the sense that the people that are, that are just, you know, discriminated against or their social credit is pushed down, they're only going to be more radicalized than the people that are, want to get their social credit up. They're just going to turn more to kind of the norm of what's expected. So yeah, there's definitely, I think, think tons of interesting and scary implications and I think that people should Be aware and kind of watching what's going on, not just in the US and even in Europe, but in China and also, you know, like in India with the Adhaar system, of course there's, there's now like billions of data points of people's biometric data can be used and in various ways and you know, hopefully some good ways. But there's also a lot of, you know, potential dangers, of course, in that and you're already seeing, seeing people bring up things and talk about that in this context. So we definitely need to be watching and kind of involved in the discussion around this, around the world.

So sorry to give everyone nightmares here today talking on this topic. Just as a final note to balance it in the near future, we're going to have an AI for good episode. So that's coming up and hopefully that'll give you some inspiration instead of, instead.

Of the terror here. Yeah, and there's definitely always going to be a balance. I think it's with any technology, and we've mentioned this before, is technology in and of itself. And this has always been true of whether it was smartphones or the Internet or whatever, of course can be used in good and bad ways. And really what we want to be doing is promote the positive as much as we can and promote the responsible practices as much as we can to hopefully help people to be asking the right questions.

Amen, brother.

Yep. So from that, let me recenter my mind and I did want to bring up a couple of kind of interesting data visualization things that I ran across and I don't know if both are entirely new, but anyway, they're new to me. The first is this how to visualize decision trees. This is an article and a package for scikit Learn that came out from Terence Parr and Prince Grover. Sorry if I I mispronounce any of those names. At the University of San Francisco's Master's in Data Science program. Basically this package just kind of. It gives you a really, really nice way of visualizing and interpreting how your decision tree models were trained and kind of understanding the decisions that they're making at the various levels of the tree, which if our users aren't familiar, decision tree model is. It's kind of like a bunch of of if then statements. So your features are split up into certain ranges and based on those ranges or their values, then you kind of navigate through these various layered if then statements. And these visualizations, I just find them really compelling. And I think that as people are using neural networks and other more Complicated models that are increasingly less interpretable, although there are many people working on that very topic. I think keeping in mind this sort of model and even the fact that you can utilize a package like this to really visually understand how your data is transformed from input to prediction, I just think that's really cool. I think things like this should be used as much as they can. And they give a bunch of examples, of course, on like the iris data and diabetes data. Digit data. Yeah. I just find it really compelling, you.

Know, and it's funny because being able to use tools like decision trees and the visualizations around them are really important. We tend to think of them, you know, just within the data science world, but there's the rest of the world that we have to communicate with and that we have to explain things to that we've been asked to explain and show what the possibilities are going forward. So being able to do this and to visualize them well, and I'm just looking through all the great examples they have in this article, is really, really important for people who aren't necessarily in the same field that you're in. Just wanted to call that out. It's a great set of communications tools that they have here.

Yeah. So the other one that I was going to mention was this Anatomy of an AI System, which I think is kind of attempting to be an infographic that represents all of the interconnected pieces that are at play in Amazon's Alexa system, or more generally, kind of that type of smart speaker system, all the way from, like, the materials that are used to make the various devices to the neural networks that are used that are being used to the AWS infrastructure, to the kind of control flow and geography. So it's probably not meant to be a fully technical spec of the whole system, but I think it is meant to be kind of to give us an idea of the impact of the systems that we're building, both functionality wise and kind of otherwise, in terms of people, in terms of places and materials and all of that stuff.

Yeah, I totally agree.

Yeah, it's interesting. So take a look at that. I think you can download it as a, as a PDF and take a look through it.

Yeah, it's super detail. I'm just looking through it as you're talking and zooming in on different aspects. So definitely interesting to look through. After the show, I'll probably keep doing this where I can just take 15 minutes and explore it.

Yeah, for sure.

So I want to turn us briefly over to spending. You know, we talked about the scary China thing a few minutes ago with social credit. And I want to point out out that, you know, that China has already committed, the government has committed to actively building a $150 billion AI industry by 2030. And, you know, they're really behind it, whether you like what they're doing or don't like what they're doing. And other major players like Russia are as well. Vladimir Putin announced last year that he was in front of a bunch of university students and he, he said artificial intelligence is the future, not only of Russia, but of all mankind. And the industry leader will rule the world. And there's a little bit of an ominous tone in my view. No biggie there. September 1, 2017 was when he made that speech. And I came across an article on CNN business actually that talks about that the Pentagon is investing $2 billion into artificial intelligence. That was actually at the 60th anniversary conference of DARPA. And DARPA was talking about this $2 billion investment into them. And I'm sure the Pentagon is spending lots of dollars in other people places and stuff. But I just kind of wanted to say that there are other governments I know outside of the US and other Western countries that are, they're very focused on AI. I would like to see a level playing field throughout the world. I think everybody's going to be doing it, and I would like there to be no one that just masters it, puts their point of view across to everybody. So to Western governments, you might be thinking about making a little bit more investment on this and make sure that you don't lose the status of being a leader in the. That was even though it was a $2 billion price tag, after reading some of the other announcements from other places in the world, I was kind of let down and wanted to share that view.

Yeah, I definitely agree. And to any of our listeners that bag some of that 2 billion, then call us up and we'd love to go to dinner.

Absolutely. Daniel and I are available for your $2 billion budget.

And speaking of the community around AI and also the development of AI, of course, a lot of that is open source now. Now I just wanted to highlight that, you know, it's Hacktoberfest. If you don't know what that is. It's kind of a unofficial, I don't know if we'd call it a holiday, a season put on by DigitalOcean. And so if you're making contributions to open source during the month of October and you sign up on their website, which we'll have in the show Links, then you can get a, you can get a free T shirt if you do a certain number of pull requests. And so we encourage you get involved, find a project, whether that's Pytorch or, or onnx or these visualization libraries or something else that's interesting to you and contribute to the larger community. And that'll be a great way to get involved.

Sure. And are there some other. You have some other conference announcements coming up?

Yeah. Good reminders. So I've got. Just wanted to remind people that we're kind of getting into spring conference season as far as submitting proposals. I wanted to highlight Applied Machine Learning days. It's a really great conference in Europe if you're able to make it there. And they have a call for talks and posters. Also there's O'Reilly AI New York, that CFP is open. And then if you're more on the research side, especially on the image and video sort of side, cpvr, their call is open. I think it goes into November. So get ready for those things and definitely get out in the community and get involved and meet some people and in real life.

Sounds great. I encourage everyone to get involved. There are two other things I was wanting to mention. One I'm actually going to know I like to do as do you. I know, like to do little personal projects and have fun. I have a six year old daughter, Athena, that I'm. That I'm always kind of pushing fun technology things for kids in front of her. And I ran across something that even appealed to my wife who, who stays out of the AI AI AI space. It's not her interest. She's like, yep, you can have that. But it was a, a little thing where it just showed how my wife has a bunch of hummingbird feeders on our back deck and we have hummingbirds flitting all over the place here. And somebody had taken a camera and had put a mount on it through some 3D printing and was using image classification through a deep learning framework. And I don't remember which one they used right off the top of my head to capture when the hummingbirds were at the feeders. And it's just a kind of a silly little weekend project, but it's a lot of fun. It allowed me to get my family involved and I do that with my daughter often. I also have a little tello that I'm, I'm playing around with with her. And so if you're not doing little personal fun things out there in the audience and this isn't all serious stuff sometimes we can have a little bit of fun. I would encourage you to do that. And then the last thing I had was I just wanted to mention that Google has their data set search beta out. If you're not aware of it, you can go to, yeah, no kidding. You can go to toolbox.google.com datasetsearch and, and it gives you the, the usual little Google search page. But you can start finding publicly available data sets out there. And since this was released, I've been using it more and more to try to locate data sets to start putting together for my own projects. And I think this is, I think this is a fantastic tool for us. It's, it's, you know, it's just a search bar, but uh, it's one that I have open in a tab all the time these days. So that's it for me. Prior to us hopping into some learning stuff. Yeah, some learning resources.

And keep in mind, you know, when you're searching for data, it's, it's not always just what your company produces or what, you know is available to you. There's a lot of, you know, public data and other data out there, you know, and you can find it with the tools like this, like Google dataset Search. So this is great. So, yeah, we'd like to turn now to learning resources again. You know, Chris and I were always trying to learn more and keep up with the latest things, but also just remind ourselves of some of the fundamentals of machine learning and AI. And so we wanted to share some of those resources with you. The one that I wanted to share is a set of machine learning cheat sheets. So you might have seen a bunch of different cheat sheets out there, but these are from. They're made for Stanford CS229 class, which is a machine learning course. And I just found these to be probably like higher quality and better produced than many of the cheat sheets that I see out there. And they're good, really good reference cards for. They have one for supervised and unsupervised learning, Deep learning tips and tricks, probability and statistics and more. So I think these are a great thing to include and they're even available in a bunch of languages, you know, Spanish and French and Arabic and other things. So yeah, I think these are, these are really great to kind of bookmark on your browser and pull up when you need them.

This is fantastic. I went from the GitHub link that you provided and they list off to another website and I dived in, into the deep learning topic on that And I'm just looking through this and it's amazing. I am going to be using this all the time. It covers so much material that you're the kind of things that you are always having to look up or recall or whatever. So fantastic find, Daniel.

Thanks.

And my learning resource, I was just at one of our Atlanta Deep Learning meetups a few days ago and we are always having people coming in there asking for starting off and you know what to do do and we're always throwing throughout the usual things and some of those I've already put out. But my buddy Reza and a couple of other guys were pointing out that one that that we had not covered was Udacity's Machine Learning by Georgia Tech, which they have online. It covers supervised unsupervised and reinforcement learning. It's free. And they said that for them I have not been through this course but they said part of the nano degree program and that it had really provided them with a great base upon which to continue learning. So having had several people say this was definitely a worthy place to start out with, I wanted to share that with the audience at large.

Awesome. Yeah. And like we've mentioned, if you have questions about AI or maybe you have a good resource that we don't know about, get on our Slack channel, get on our LinkedIn page and let us know about it. We'd love to hear from you, hear what questions you're having, hear what resources you're using and of course we'll try, try keep bringing you some good ones. But in the coming weeks we're going to have more guest interviews. We've got some really great stuff lining up as Chris mentioned, about really technical topics and more use case stuff and ethics and all sorts of things. So keep tuned in and I'll see you again next week.

Chris I'll see you again, Daniel. It was a great show today and looking forward to next week. So talk to you later on. Bye bye.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor, go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by fastly. Learn more@fastly.com and we catch our airs before our users do here Changelog because of rollbar. Check them out@robot.com changelog and we're hosted online linode cloud servers head to lino.com changelog check them out Support this show this episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith, the music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcast for developers in your inbox every single week.

Thanks for tuning in.

Tuning in. We'll see you next week.

I'm Nick Niesse. This is Kay Ball.

And I'm Rachel White.

We're panelists on JSparty, a community celebration of JavaScript and the web. Every Thursday at noon Central, a few of us get together and chat about JavaScript, Node and topics ranging from practical accessibility to weird web APIs. I like your rhymes with Mafia idea.

Like that's a.

That's a good way to get a across. I'm trying to think what I can do. K Ball Rhymes with Ball Join us live on Thursdays at noon Central. Listen and slack with us in real time or wait for the recording to hit. New episodes come out each Friday.

Find the show@changelog.com jsparty or wherever you listen to podcasts. I'm Tim Smith and my show Away from Keyboard explores the human side of the world creative work.

You'll hear stories, sometimes deeply personal, about the triumphs and struggles of doing what you love.

I ended up in hospital with burnout. I just kept ignoring the way that it was making me feel and just kept powering through it. And then eventually my body started to give me physical symptoms to say like, hey, you should stop and listen to me.

New episodes premiere every other Wednesday. Find the show@changelog.com afk or wherever you listen to podcasts.