AI in healthcare, synthesizing dance moves, hardware acceleration

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. They now have CPU optimized droplets with dedicated hyper threads from Best in class Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application. This gives you immediate access to Python 3R, Jupyter Notebook, TensorFlow, SciKit and PyTorch. Use our special link to get a 100 credit for DigitalOcean and try today for free at the.co changelog once again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practicalai FM. And now onto the show.

Hey Chris, how's it going this week?

Doing great. I am super excited about the next few weeks. Got some cool things coming coming up, which we'll talk about.

Yeah, definitely. So I'm here, joined by my co host, my co host Chris, and we're going to bring you some news and updates from the AI world today and then also some learning resources to help you level up your AI skills in a practical sort of way. But yeah, there's a lot coming up. Chris. I know I'm traveling over the next three or four weeks quite a few times and we're. I know you've got some travel too, so there's a lot of exciting stuff coming up.

There is and there's so much happening in the news right now. I am going to just leap on into the first one, if you don't mind.

Sounds good.

Okay. So I was, I came across something that I thought was really defined what I've been noticing in the, in the industry and that is it was an article from VentureBeat called How AI is decommoditizing the Chip Industry. And it was a cool read because it really was pointing, pointing out something. It seems like you know, for years as you had, you know, with before the, the era of AI and, and chips and, and the commoditization of different things coming out in computing in general and now AI has kind of reversed that trend to some degree and that processing devices, CPUs and the like, GPUs et cetera, are really becoming more and more specialized and that is creating all sorts of entrepreneurial opportunities for different companies. And so we're seeing lots of chip startups instead of just software. And some of these companies are like Nirvana, whom some people may know from intel at this point. Graphcore, Cerebras, Vasith, there's a whole slew of them. And they're really challenging the big incumbents, which certainly in the AI era has been Nvidia kind of leading the way. But Intel's come in hard, come back from the CPU world into the new AI oriented chip world. Microsoft has theirs, AMD, Qualcomm, Google, TPUs and IBM, all these big players are getting huge challenges. So Nvidia really came in with early dominance with the GPU as they moved from consumer gaming into AI. And you know, they had been kind of the poster child for the AI world. Even they at this point are having to watch some of the new risers coming in because of what's called asics, which is application specific integrated circuits, and that is chips that are designed specifically for a particular application. They're completely optimized for that. And that's just fascinating when you think about it, because even though Nvidia has their Volta architecture and intel has nirvana, Google's TPUs, at the end of the day, this article is suggesting that the future lies in ASICS rather than commodity hardware. What do you think about that? Do you think they're right?

Well, I first of all think that you did an amazing job saying decommoditizing correctly on the first trial, which I think is maybe one of the more impressive things that I've heard recently. But in all seriousness, I don't have as much exposure to the application specific type circuit world. I think it'll be interesting to see that develop. What I do have exposure to is definitely the kind of resurgence of specialized hardware in the context of things like another company that intel owns now, Movidius, has things like these neural compute sticks. And one of the things that I really like about them is they've kind of found a new niche that they're filling, but it's also enabling really interesting new types of applications. So they've got these little VPU visual processing unit like USB sticks that you can plug into a Raspberry PI or to a drone or other things and run your neural networks on this specialized architecture in a very kind of, you know, low power at the edge sort of scenario. And you know, I'm at GopherCon this week actually and just had a conversation before I walked over to record this with someone said, oh, it's so cool. Now I have, you know, I've been doing all this drone stuff and now I can just plug in these sticks into my drone and have them follow around, you know, specific people based on their, you know, facial recognition and all that, you know, stuff. So people are really excited about these things.

You know, you have a great point there. And I've, I've, I've used a Movidius and I know so many software developers who are not data scientists. They're not coming from the traditional backgrounds leading into AI. They're software developers that have started in other areas. They might have been web developers and such and they've moved into this and having something like a Movidia stick or similar devices has really opened up the space for them. So since I was halfway thinking about suggesting you and I go create a startup where we create an AI chip for, that's designed around AI for good, I say tongue in cheek.

Yeah, I would be happy to do that.

There you go.

You provide the funding. A couple things that I found this week actually had to do with more on the research side which was kind of, I don't know, kind of different for me because I'm usually more on the non research side. But I really found these interesting. The first is this new paper that came out on the archive from some people at Grenoble University in France. And there's also a pytorch implementation of this network. And what it is, is it's a kind of new type of sequence to sequence prediction. So if you're not familiar, that's where you know a very common type of neural network that's used in things like machine translation of text and you know, taking sequences of something to other sequences and normally what happens in those is there's kind of an encoder and decoder stage of those networks. And this paper showed that they could kind of combine those two things into a single two dimensional convolutional layer which I think is really, it seems after you see it it's like, oh, that's a really great idea. But it took someone to kind of come up with that natural step. So it's cool to see even in things that are utilized in production so often there can be this sort of innovation and refining happening.

You know, it's funny because when you think about sequences, you know, most people automatically turn to RNNs but we, I know over the last year or so I've seen so many CNN applications for, for sequential applications. And it's, it's interesting to see how versatile different architectures in the, in the larger convolutional neural network space have been going beyond, you know, just the visual thing that we tend to associate with them normally.

Yeah, definitely. And like I say, there's an implementation of this already on GitHub in PyTorch, which I love working with Pytorch. And so I would love to try out some other examples and you guys can as well.

Gotcha. Well, I am going to turn briefly toward the medical world where it intersects with AI. There is a couple of articles that I've run across. One is called John Hopkins Researchers use Deep learning to combat Pancreatic Cancer. And that one really struck a nerve with me because incidentally, I'm on my second marriage, but in my first marriage I lost my mother in law to pancreatic cancer and we watched as she went downhill very, very rapidly. And in doing that, it made me very aware of how bad pancreatic cancer is in terms of only 7% of patients that are diagnosed. Make it another five years. It has the lowest survival rate of any form of cancer. And so this really caught my eye and that they're basically saying that early detection could lead nearly a third of all diagnosis to be made four to 12 months earlier, which could save a lot of lives or extend a lot of lives out there. So in our ongoing theme of AI for good, I really am encouraged by that. They use deep learning in combination with a CAT scan to look for minute textural changes to the tissue. So that was pretty amazing. And then the other thing I saw, which was on the, on the pharma side, was a system that they call release, which stands for reinforcement learning for structural evolution, which uses these two neural networks, one that's kind of a, a teacher neural network and one which is kind of a student. And they say that they can, the teacher knows 1.7 million active molecules in great detail and the student's able to learn from that and then actually create new molecules and evaluate those new molecule molecules with properties that researchers specify. And so this is where you're seeing deep learning being applied to pharma to create designer drugs much more rapidly. And between the diagnosis of cancer and the life saving aspects of that and being able to get to new life saving drugs sooner, I'm just really impressed with how deep learning is revolutionizing medicine in general.

Yeah, definitely. I think that especially coming from a background where I was exposed to computational chemistry and those sorts of methods. I think people don't, when you say all of those molecules and those sorts of things, maybe people don't fully realize that for even a single molecule, a small molecule like oxygen, if you ignore the protons and neutrons, you still have a bunch of electrons. 6 I believe, if I haven't forgotten everything. And each of those are in a three dimensional space and there's a time element and there's potentially external fields and there's just a lot of variables that happen in these sort of computational chemistry scenarios. And if there's anything we know about deep learning, it's pretty useful in high dimensional spaces sometimes. So I think that's really interesting to see more of those methods come out.

What else have you seen this past week?

Well, I saw this super creepy video which I shared with, I shared with our users on, not our users, our listeners in our Slack channel. It's this new work from Berkeley. And essentially the video that I saw, and maybe there's multiple out there I'm not sure is like a guy dancing, you know, it's like a Bruno Mars song. And what they did is they kind of taped two individuals moving around in some space to kind of train on their movements and the, and their body structure. And then they generated video of these two individuals dancing in the same way as in the Bruno Mars and then a ballet video. And it's just amazing. And I think you mentioned when you first showed some people, they didn't even realize that it was generated videos of people they thought they were actually dancing, synchronized in that way.

Yeah, it was actually my 6 year old daughter Athena. After you posted it in Slack, I saw it and she heard the music on it and she comes running up to my laptop and we were looking at it and you know, I'm marveling because they showed in the video, which people can see in the show notes, they showed the video of the, of the source dancer. In one case it was kind of a, you know, a funk type dance and then there was a ballet dancer and then these two people that they were using to superimpose the motion on and they didn't always line up. And so you'd have these brief moments, these subtle moments where the body was doing things the body couldn't do. And it was enough for me to, you know, I kind of knew what I was looking at. But my 6 year old daughter never realized that it was generated. Is she. She's grown up in a world, you know, where, where this is Normal, you know, AI doesn't even phase her at her age because she's seen it from me and, and just like mobile technologies and everything else, it's, it's normal. But later on I said, do you realize those people weren't actually moving like that? The computer made them do that? And she goes, no, I had no idea. You know, it's. And I'm just thinking, just two or three years down the road, where is this going to go? You'll be unable to distinguish generated motion from real life.

Yeah, I think not to give away all of our startup ideas, and I guess you can scoop us if you like, but another startup we should create is the computer generation of music videos where we kind of make obsolete all of the music video dancers and just get their training data and then we can reuse the same dancers in any sort of video and any sort of background and make it seem realistic. That's my second startup proposition.

I'm all over it, but just so long as it doesn't involve the two of us out there dancing. So long as we're not the models used in any of the stuff.

Yeah, well, I'm sure you could do better than those dancers in real life.

I don't know about that. My wife would tell you. Not so much.

So the last thing that I wanted to share with everybody is that Chris and I are going to be at O'Reilly AI in San Francisco, the O'Reilly AI Conference, which is, I believe, going on when this airs. It will be going on this week. So if you are at O'Reilly AI in San Francisco, come find us. We'll be walking around doing some interviews. We'll have stickers, we'll have some nice swag and all sorts of stuff. So come and meet us. We'd love to hear from you. We'd love to hear about your ideas and discuss whatever topics you'd like. So come find us there. I'm really excited to, to attend and.

And I, I second that. We're, we are just there to meet everybody and to have conversations, to do a bit of recording. So don't be shy. We're looking forward to meeting lots of people in San Francisco.

Definitely. Well, let's go ahead and move on to some learning resources. This is the, the part of these type of shows where we just kind of share a couple things that we've run across that have been useful in terms of learning new things within the AI ecosystem or new frameworks or new techniques or whatever it is. The first one that I found this Week, which I don't think is totally new, but it was new to me, is the site that kind of creates a data visualization, a map of all of these different data science and machine learning books. So it's called Hands On Machine Learning and it's kind of like a little roadmap of all of these different books that have been released on different subjects like deep learning and Python in general, and beginner books and expert books. And it kind of guides you to the different sections that you might be interested in. So if you don't know where to start or what books to look at, that might be a good place to just kind of explore what's out there and maybe avoid an expert book. If you're looking to begin and start out in AI, or maybe you're looking for a book specifically about deep learning or something like that.

I think this is great. I had not seen this before. So I'm looking at the link at this point and after we're done recording, I'm going to go snoop through it and see what I need to go get.

Sounds good. What do you got for us this week, Chris?

So there is a Udemy course that I decided to put that's pretty good. It is a paid course and the price, typically this is one of those things where I've never seen it at full price. It's always at some kind of discount. Currently it's $10 to get it, but it's called Complete Guide to TensorFlow for Deep Learning with Python. And it is quite lengthy actually, and you can kind of pick and choose, but it has 14 hours of on demand video. And so if video is your thing in terms of learning, it kind of takes you through everything. From the beginning is kind of what is machine learning to An Introduction to neural networks, TensorFlow basics, CNNs, RNNs, various other topics that they're kind of ancillary autoencoders, reinforcement learning, and even generative adversarial network scans. And so that that was. It had such a breadth of topics that it was covering that for 10 bucks. I thought it was a pretty good, pretty good investment to get people into. And so if you like to use Udemy as a platform for learning, then I recommend this one. It has 4.5 stars out of six and a half thousand ratings as I read this on their website right now. So that's what I found. I thought it was a good thing for a beginner to get into.

Awesome. And yeah, I should mention too that if anyone out there is looking for books on specific subjects or looking for specific types of courses or maybe just GitHub repo that has some relevant examples. Go ahead and and jump over into our Slack channel. You can join the practical AI and changelog Slack team by going to changelog.comcommunity and there's a practical AI channel in there. And just send us your question and we'll do our best to point you to whatever resources we know about and maybe some other resources that our other listeners know about. So make sure and leverage that. And otherwise we'll reconvene next week. We're talking with Susan Ettlinger about AI ethics, which I'm excited about. I know you are, Chris, because our listeners have expressed to us a lot that they want to hear about this topic and I know I want to hear about it as well. And Susan's an expert, will also be talking at O'Reilly AI about the same subject. So make sure and and join us again next week.

Yep, she is super impressive. I can't wait for that conversation. And as this gets released, I will be seeing you within a couple of days at O'Reilly AI and we'll meet a whole bunch of our listeners and do some recording as we go. So I'm looking forward to this coming week.

All right, see you there.

See you there. Talk to you later.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do. Share the show with a friend. If you enjoyed it. And bandwidth for Changelog is provided by Fastly. Learn more@fastly.com and we catch our errors before our users do here at changelog because of rollbar. Check them out at robar.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show this episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this@changelog.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox single week. Thanks for tuning in. We'll see you next week.