AIs that look human and create portraits of humans

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. They now have CPU optimized droplets with dedicated hyper threads from Best in class Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application. This gives you immediate access to Python 3R, Jupyter Notebook, TensorFlow, SciKit and PyTorch. Use our special link to get a 100 credit for DigitalOcean and try today for free at the.co changelog once again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're PracticalAI FM and now onto the show.

Welcome to Practical AI. This is Chris Benson and with me is Daniel Whiteneck. How's it going today Daniel?

It's going great. How about with you Chris?

Doing pretty good. Looking forward to going through all these cool news stories we have today. What have you been up to lately?

Yeah, I've been been doing a lot of learning myself. I've been doing a lot of learning about graph structure data. I'll actually mention a couple of things related to that later in the episode, but also been working a little bit and learning a little bit about Vega Data Visualization Library. I think I mentioned it in a previous episode.

Yeah, you did.

I've had a desire for a long time to learn D3, but I have no experience at all with JavaScript and but Vega is this cool data visualization library that that actually I think it uses D3 under the hood, but essentially the interface for the data visualization creator is JSON, which I am perfectly fine with. So that was a lot more approachable for me. And yet you can still get these really cool, interactive, appealing data visualizations out of it. So I've been kind of tailoring some of my graph data into that world and that's been a lot of fun. What about you?

Well, technically I've had my TensorFlow Tutorials tab open a lot this week and have been doing some of the tutorials that I hadn't gotten around to that I've been meaning to for a while procrastinated on. But I'm also getting ready to go to Nashville this weekend. Vanderbilt University, they're the Women in Computing Group sponsors the Emerge Conference and where they talk about emerging technologies, they're talking about AI. And I am really looking forward to going up there and talking about AI and actually talking a little bit about the world that, that my daughter Athena is going to grow up in. And so that is. I'm really excited about that.

Yeah, that's awesome. That sounds like a, like a great opportunity. I would tell you to get some Martin's Barbecue, but as you and I are both don't eat meat, that's probably not going to do any good but.

Yep, yep. Yeah, yeah, barbecue doesn't work for at least for me for a vegan.

Yeah, yeah for us both. But I'm sure you'll find some, some other interesting things.

I will find suitable rabbit food to eat.

Let's go ahead and get into some, some news and updates here again for those new listeners. We do these kind of news and updates show to highlight some of the things that have come across our radar in the AI world. The first of those for me was this article which I'm sure is, is also information is published about it elsewhere but where I saw it was on NPR and they have this article entitled AI Produced Portrait will go up for auction at Christie's. And so essentially what this is is there's a art collective that call themselves Obvious. They say that they, they make paintings using artificial intelligence. They've, they've kind of been doing a series of these and they have a picture of the painting. It kind of looks like a kind of rough person in kind of a, maybe an older kind of old masters sort of style. Kind of looks like a clergyman or something like that. And then in the bottom right hand corner of the painting there's this math formula. So that's probably maybe one indication that it's not quite a normal picture. But anyway, there's this painting and it's going up for auction and like I say, they've already sold one of these but this one is estimated to sell for around seven to ten grand. So seven to ten thousand dollars. And yeah, I think this is, this is pretty interesting. What do you think, Chris? Would you, would you hang this in your house?

You know, I probably would. Now I will say that I know nothing about art. I never took art history and so I'm just coming at it as your everyday guy looking at. But if I like it, I'll certainly Hang it up. But, you know, it really. I find this really interesting, but in that it's similar to some of the other things that we've seen lately about art and different creative pursuits being attempted in the world of AI. And a recent thing that I had heard was. And I can't remember exactly where I heard it, but it was about AI produced music being compared, where they were kind of recreating classical music that the great masters had produced. And they would let people listen to it, and they would either kind of bias them as the AI was maybe created by the human master, or vice versa. And they were switching that up with different groups. And they found that people tended to follow their biases, oftentimes thinking that humans would always be better than the AI but if they kind of switched them behind the scenes on them, they would stick with their biases even if they were listening to the other one. So I hope that made sense.

Yeah, it's interesting.

It really made me start to believe that. I think that there will be types of creativity that AI does very well at. And you know what? If it can create beautiful art like this, I would certainly hang it in my house, especially if it is less expensive than buying at the art gallery.

Yeah. And I mean, I think this is something not totally new in the sense that we've had, you know, compute computer and machine generated music and other things in the past. And a lot of that, you know, is digital signal processing and other things that have been utilized to create new types of sounds and soundscapes. And people seem all right with that. I think that this is kind of new for people in the sense that, you know, this art collective. I don't know what you would call the people in this case. If they're artists. I guess they're artists. The artists that created this AI that created the painting, they really wanted to give the AI a lot of free rein in terms of what it generated and kind of take themselves out of the picture, I guess. And it seems like they do give a little bit of technical explanation. It's not a technical article. Maybe there is another technical one somewhere that our listeners could point us to. But they do say that they fed in to the system a data set of 15,000 portraits painted between the 14th century to the 20th century. The generator makes a new image based on the set. Then the discriminator tries to spot differences with the human made image and the one created by the generator. So there's definitely kind of a feedback thing going on here with these models. And so there does Seem to be a lot of interesting tech behind it. And obviously it is creating something of value, whether people want to say it's a value or not, because people are assigning it a value. Right. Seven to $10,000. So, yeah, it's super interesting.

Yeah. And just for listeners who may not have picked up on it, you were referring to generative adversarial networks, which are also known as gans, when you were mentioning generators and discriminators. And they seem to be. That architecture seems to be really leading the way in creative efforts here.

Yeah, definitely. And hopefully we can have a. Have a show that talks about that sometime. But that would be a great episode. Yeah, it would be. I would be interested to kind of hear the makeup of this team. Obviously, they have very technical people on the team because this is not something that is, you know, I imagine that they teach in art school. Right. So I would be really interested to see hear the makeup of their team if they have kind of a combination of artists and data scientists or AI people. I'm not sure. I would be interested to hear that and hear how that kind of maps to teams we see emerging in industry in general.

Yeah. I think this kind of AI created art is going to become very, very common in the days ahead. So it'll probably become perfectly normal for us in the not so distant future. Yeah.

And speaking of AI generated faces, or maybe not generated faces, but faces and AI, you ran across something we were just discussing before the show that was pretty interesting. Maybe more on the a little bit disturbing or alarming side, but tell us a little bit about that.

I'm actually going to lead with my last point and say this is another thing that I think is going to become extremely common in the days ahead, and that is there is a company called Magic Leap who has a personal assistant that is very much like Apple, Siri or Amazon, Alexa. It's kind of grounded in AI and augmented reality. And they have named theirs Micah. And apparently Micah looks and acts human and she can give you the. And when I say she, if you look at the pictures that they have here, it is a woman. And if you interact with her, you can either do voice only, or apparently they have a pair of augmented reality glasses called Magic Leap one. And if you put those on, then you can actually see her in front of you and you can interact with her. And apparently people. They note in the article that people that are interacting with her in this augmented reality will oftentimes, like, if she leans in, they'll kind of lean back from A personal space standpoint and truly act as though they are dealing with a human being beside them. So very interesting and a bit creepy maybe for us. I'm betting that my 6 year old daughter won't find it so creepy as she gets older because I think they'll be ubiquitous. I think she'll grow up not knowing a world without them all over the place.

Yeah, and I think it's kind of one of those things. And I'm not a UI UX type person, although I do value design, but I know that there's this kind of principle and I forget, I think it even has a name where if you try to make something look human and you kind of slightly are off, then it's kind of, it comes off super creepy and weird. Whereas if you just created something that was really like avatar like and obviously not human but kind of mimicking human, then that could actually come off, you know, a little bit better in the, in the user experience. So it's interesting to see people going both of those directions. I don't know which will kind of win out. I don't know. I don't know that I want, I want to be interacting with a lot of micas in the future, but like you say, maybe that's something that will just become commonplace.

So as we are approaching Halloween, as this episode comes out, I'm going to challenge our listeners to either get a picture of Micah or any other human like personal assistant out there and put in our Slack channel your version of the avatar for Halloween. In other words, make your change. Let's create a meme for the next week.

Sounds good. So moving on to kind of a set of things that I have been running across and really I think I've been exposed to some of these things because as I mentioned in my personal work with the nonprofit Sal International, I've been doing a lot of work with graph structured data, specifically in the language space. So language families and populations, how they're related, what countries they're in, what resources are available in those countries, who's writing those resources, where they're coming from, what countries they're coming from. And so this kind of graph, very dense information that's represented in a graph structure. And obviously as I've been going through that, various ways that we could apply AI and machine learning have popped up. And I've been interested to see over the past couple weeks a bunch of articles from people that I didn't know were really working in graph data and machine learning pop up. And one of those is this article which I actually saw on LinkedIn, but it's from Helena, Helena do, sorry, mispronouncing the name from Elsevier, which is the company that at least one company that has journal articles and a bunch of other tech in academia. And she wrote this kind of summary spawning out of the or stemming from the International Semantic Web Conference in Monterey, California. And her kind of view on the state of things is that people are really interested in graph structured data and people are using machine learning on graph structured data. And she provides a lot of great links to things that people are doing, including helping people find relevant healthcare information and health data in knowledge graphs. There's also ones that are using graph structured data to find effective drugs for incurable diseases. There's people using graphs to kind of analyze documents and find related things, of course, and, and find groupings within social networks and all of that stuff that maybe is more obvious to us. But then she also gives some references to people that are doing deep learning and machine learning on graph structures. And another thing that I saw as related to that was this semantic Scholar project from the Allen Institute for AI where they're really using graph structured data and AI to help people to guide them to relevant academic works and scientific works. Because it's really hard to find that as you're searching through all these different papers from all sorts of journals. So that's really cool to see. They provide a bunch of tooling around that and then even from DeepMind. So DeepMind came out and open sourced this Graphnets library. So it's on GitHub @DeepMind Graphnets and this is a library for using TensorFlow on graph structured data. So to be clear, this is not. I mean, there's still a computational graph within TensorFlow in many cases, but this is actually doing using TensorFlow models on graph structured data. So a graph is your input, a graph is your output, and they have open sources. But there's also some really cool things to play with. There's some collaboratory notebooks, notebooks where they can show you how to kind of figure out and learn the fastest or shortest path between things in a graph, like in a social network or a graph of health resources or whatever it is. And so I would recommend taking a look at that and in general take a look at some of the stuff going on with graph structured data. I think there's a lot of interesting work going on.

Yeah, it's interesting. You just pointed out that this is to apply own graphs versus we think of graph data a lot, but we tend to think of it as being part of the framework or the model, the architecture that we're in. So as you were talking, it made me realize that I actually have a lot to learn on that. So I'm definitely going to dig into these links after the episode and try to ramp up on it myself.

Awesome. Yeah.

So I ran across an article this past week from mit, and it was actually on News MIT Edu, and they have announced that they are now going to build the new Stephen A. Schwarzman College of Computing. And you may, on initial just hearing that, you may say, okay, well, that's what MIT does. They do computing. But this is a little bit of a different approach to setting up a new college. They received a $350 million foundational gift from Mr. Schwarzman, who happens to be the chairman and CEO of Blackstone. And then on top of that, they have what amounts to a $1 billion commitment. And what they're trying to do here with this new College of computing is to build an. An interdisciplinary school around AI and the various fields that it touches on to try to kind of drive things into where we're seeing the future go as we're living it right now. The AI world has really revolutionized education in that space. And some of the things that MIT notes is that they want to reorient. And I'm reading from the article when I say this. Reorient MIT to bring the power of computing and AI to all fields of study at mit. Allow the future of computing and AI to be shaped by the insights from all disciplines. They have 50 new faculty positions to support it. It's going to give. There's a shared structure with other schools that they're doing, and they're really looking at trying to produce students that can operate in this interdisciplinary approach. And I think that's fantastic, personally, because in my own experience, AI touches on other fields almost every time you use it. It's never a standalone thing by itself. It always intersects other areas that you're using it on. What you think, Daniel?

I mean, I think this is great. I think that coming from, you know, I've always. I'm glad that I came from an academic and a physics perspective, and I really enjoyed physics.

I'm.

I'm really glad that I spent my time in that world. But kind of what I tell people when I talk to them about academia and different disciplines and different departments within academia is that, you know, physics is kind of almost in a lull in the sense that there hasn't been really, like, crazy Paradigm shifting stuff going on for a little while. And so it's maybe not receiving quite as much funding or having as much enthusiasm in terms of grad students going into physics and all that stuff. But on the other hand, computer science, especially with AI and then also in some ways biology are really seeing this surge of enthusiasm. And I think this is one of the evidences of this. I mean, 50 new faculty members is crazy. And I mean, one of the things that I'm hoping they do talk about education here, I'm hoping that along with some of the stuff that MIT has already released, that they're able to release some of these resources, whether it be lectures or tools or documentation or other things to the community at large and so that we might be able to benefit from this work as well. Because I know I love schooling, I love learning. I kind of like to go back and maybe go back to MIT and get this degree. But I imagine that probably where I'll intersect with it is with whatever resources they release to the community.

Yeah, I really envy the students that are just going in to college or graduate school this moment in time because just in the last three years, education around computing and AI and related fields has changed so dramatically and really taken off that if you came out of school five years ago, the curriculum that you went through is already changed since then. And so the rest of us that are, you know, past school at this point are having to continue to learn and catch up and do that. So I almost wish I could transport myself back to the beginning of college right now and just experience this because it's hard for me to imagine a better way to spend the time.

Yeah, definitely. And speaking of spending our time and community stuff, in our last news and updates show, I mentioned a few conferences and CFPs that were open and I want to continue to do that. I really encourage our listeners and encourage myself to make the effort to get out into the community, to meet in real life and have discussions with your, your AI community, learn from, you know, some of the people that are working in the space, hear some great talks. And the conference that I wanted to mention this time around is CSV conference or CSV conf version four, which is going to be May 8th through 9th, 2019 in Portland, Oregon. And this is a conference that actually I was aware of last year, but I think I had a conflict with some other event so I wasn't able to to go. But it's one that I'm definitely interested in attending this following year. And they have their cfp. It Looks like it's open, you can submit a talk. And it looks like just a really great, fun conference. They even have a mascot, the comma llama, which seems, seems pretty exciting.

That is cool.

So if you're a fan of data or llamas, this is the conference for you. But yeah, it's not only about CSVs and maybe you're thinking of spreadsheets or something, but I think they're. It's a non profit community conference which is really great and there's a lot of diverse topics. They're talking about data sharing and data analysis from science, journalism, government and open source. And I think it would be a really great conference to go to, to get exposed to a lot of different ways that data analysis and AI is being used across industry, how data is being shared, all of the, all of the subtleties that go along with that. So, yeah, I'm excited. I'm going to try to submit a talk and hopefully I can make it there.

That sounds good. I think I'll do the same. And you brought up a good point a moment ago and that is just getting involved in your community to take a second and share an experience I had back in late 2016. I was deeply interested in this space and I happened to be in Atlanta and I thought, I looked around at different meetups and different groups and no one was really tackling what I was interested in directly in these meetups. And I thought, well, I'm going to start a deep learning meetup. I have no idea if anybody will ever show up, but you know, I'll go ahead and give it a shot. If it doesn't work out, it doesn't work out. It has been hugely successful and that's just one case. The AI world, deep learning, machine learning, data science world, there are so many people. You don't have to have a PhD in these fields to be able to enjoy it. And so I would encourage anyone do what I did. You might be surprised at how many people will come out. I was shocked that we would have 60 to 120 people show up in person at any event. It was almost overwhelming and I wasn't sure anybody would show up when I started. So wherever you happen to be, I would encourage you to go out and either start a meetup or some similar group and get to know the people in your area or your region that are interested in this and you can help each other get along. So thank you, Daniel, very much for bringing up that suggestion.

Definitely. Yeah, I totally agree with everything, everything you just mentioned.

So I will dive into the last article before we go into learning resources. Periodically, we will talk about the world of, of medicine being impacted by AI. It's come up in several episodes. Prior and Physics World had an article called Deep Learning Algorithm Identifies Dense Tissue in Mammograms. And so the Massachusetts Institute of Technology, MIT and Massachusetts General Hospital developed a deep learning algorithm working together that assesses breast density in mammograms. And it does so very, very reliably. And this provides a tremendous tool for mammographers, if I'm saying that right, you know, doctors in this field to be able to use, to help save lives. Apparently, and I wasn't aware of this specifically before I read the article, but dense breast tissue apparently mask cancers on mammograms, the common mammograms that people are getting every day these days. It makes the screening more difficult, and apparently that can be an independent risk factor for breast cancer, the presence of it. And so in this case, the researchers trained and tested the algorithm on a data set that was 58,000 digital screenings that were mammograms. And so they divided that up into 41,000 for training and 8,600 for testing. And then during the training, the algorithm was given random mammograms to analyze. And in doing so, it was able to predict the most likely density category, which enables these doctors to save lives. And I just. It inspired me. I come from a family full of women. I'm the only boy. I have four. Four sisters, and I have a daughter as well, and my wife and my mother. And I see the impact of these things in their daily life. And so I was truly inspired by what, what's happening in this field and the fact that these doctors are getting better and better tools every day.

Yeah, this is awesome. And one of the things I was just reading through part of this while you were, while you were talking is there's a quote in the, in the article. It says then when radiologists view a scan at the work of workstations, they can see the model's assigned rating, which they can accept or reject. And I think one of the big things that that is emphasized here, which I'm really glad to see, is that this is really an AI augmentation of something the radiologists are doing. It's helping them actually do their job better. And it seems like the radiologists are very accepting of that. They want to do their job faster, they want to make better predictions, because obviously they care about their patients. And there's a Lot of pressure on them as well. So this is really a welcome AI augmentation. It's not an article saying, you know, we're going to replace all radiologists with this sort of, this sort of modeling. Right. It's an augmentation that is actually very welcome and makes things faster and cheaper and easier and better. So I think that's a really important point to mention.

Yeah, it's AI for good. And I think it shows the fact that it doesn't have to be an either or proposition. It's not humans versus the AI like so many people are always putting out there, it's humans plus AI make a much greater capability. And so I love seeing these examples of AI for good that can truly have a massive impact through our society.

Awesome. Well, let's turn now to learning resources like we do in each of these news and updates shows we provide some learning resources. I was just talking to a student the other day and I think that there is a kind of stereotype that us working in AI where we've got all the knowledge built up in our brains and we're never having to consult the Internet for things. But I always have, you know, Stack Overflow open in a tab and my Slack channels open in a tab and forums and GitHub issues and all of those things. So, you know, we all need to constantly be learning from one another and we want to share some of those resources with you. So one of the ones that actually came up this week, I'm teaching a corporate workshop and one of the students in that workshop, we were going through learning rate regularization, rate regularization. And some of these may be concepts that can be hard also in terms of just the jargon that you have to build up. And one of the students, they found the neural network playground@playground.tensorflow.org, and was saying that it was really helpful for them as they were thinking about these different, the different components that go into defining your model and the training process. Number of epics, number of hidden layers, regularization, regularization rate. And I agree, I think that this neural network playground, it's been around for a while actually. I remember it, I used it.

Yeah, I remember.

Yeah, it's been around for quite a while. But I agree, I think it's, I mean, it is kind of interesting in the visualization. It's a really nice looking visualization. But I think even more so as you're learning a lot of this jargon, it can really help you firm up what is regularization rate and learning rate Are they, how are they different? Why is there these two rates? You know, what does one do, what does the other one do? Those sorts of questions, I think can be answered really nicely in this visual way. And you can modify things and update them. It's all interactive and so. So definitely a really, really good resource.

Yeah. I mean, speaking for myself, I am a visual learner and I remember when this came out. It's a fantastic tool. I'm playing with it right now as we're talking. It graphically shows you as you've changed those different things, what that means to your architecture and what that output is. And it was one of the things that helped me grok how things would come out if I chose TANH or relu, for instance, because it will do it instantly for, for you there. So it's just a great way of if you've read up on something and then you can go play with the idea and see it right there. So highly recommend it. It certainly helped me.

Yeah. And the other one I wanted to mention just here quickly is actually comes from Lindsey Zulaga, who was our guest in episode 17, so our last show. And of course she did an amazing job at explaining bias in AI and how to fight that. And she mentioned this, this toolkit called the. It's from IBM. It's called the AI Fairness 360 Open Source Toolkit.

I remember.

Yep. And I went ahead and took a look at that after the episode and I was kind of pleasantly surprised in the sense that this isn't just like an open source toolkit that you go to a repo and look at it. They have a whole page full of demos, full of videos, papers, there's tutorials and example notebooks to help you kind of understand where bias creeps into the models. And also, you know, fairness metrics and state of the art techniques and algorithms to help you mitigate bias in your algorithm so you can actually create more fair and better models. And so I would highly recommend taking a look at this resource, watching some of the videos and I think it's a great place as you enter into a new project, really a good thing to revisit and think about. Okay, what can I. What fairness metrics or what bias mitigation can I apply in this new project?

It looks really great here. I had not looked at it prior to you bringing it up. It was on my to do list. But there are dozens of different topics to explore on this page and so I'm looking forward to. I'm going to leave this tab open. Like I have dozens of tabs open. I'm going to leave this one open for after the show and go exploring.

Awesome. What learning resources do you have for this week, Chris?

Well, I wanted to start off with one and it wasn't one that I was typically thinking of as a learning resource, but I found myself it's a medium site, as in medium.com, the publishing platform that is called Towards Data Science. And it has different topics like data science, machine learning, programming and visualization. And I find myself reading different topical posts on this site all the time and have for quite a while. Yeah, and I actually had a few tabs open this week from some of the articles I was looking through and it occurred to me that this was a learning resource for me personally. It's often where there'll be a particular topic that will get covered in a post and I may not have a lot of experience or exposure to that topic prior to reading. It's a starting point, it's a launching point for me to say this is something I want to go learn more about and then I'll go find other resources on it. But it's just an easy read. You can do it anywhere. You can pull up your Medium app in your car. Well, maybe not in the car, I was thinking when you're stopped, but hopefully, yeah, don't, don't go reading Medium while you're driving, folks. I'm sorry I said that. So, but yeah, it's a great place and it's, it's fun to read. And so I recommend Towards Data Science, I believe it's dot com at the end of that. And then the other thing is, I often get asked, you know, running the Deep Learning meetup and things like that, how reinforcement learning fits in and what exactly is it and how is it related to deep learning. And there was a Forbes article that went around all the different feeds this week. I have a whole bunch of feeds in the AI space that I read from and it came up in several of them and it's called Artificial Intelligence. What's the difference between Deep Learning and Reinforcement Learning? So this is a, it's in Forbes. It's a non technical explanation that's very accessible where the writer kind of, he introduces the topic and kind of he starts off with what is deep learning? And he offers a few paragraphs on what that is and then he goes into what is reinforcement learning? And kind of gives that. And then he covers the difference between deep learning and reinforcement learning and finishes up by talking about how you may use deep learning as a component in your reinforcement learning, but it's a quick read. You can, you can probably do it in two minutes, I'm guessing, but it might give you a start. And if you're one of those people who are trying to figure out how these different, you know, these different things fit together, it's another good starting point, particularly for the non technical folks in the crowd. So I recommend it.

Awesome. Yeah, I think that just the article's title in and of itself gives kind of tells that there is a misconception between, you know, that reinforcement learning and deep learning are necessarily different things or mutually exclusive or how do they fit into one another? And I think that that's really great. To clarify, I would also recommend we had a great discussion with Woj Zaremba, one of the co founders of OpenAI, and he let us know. He gave us a great introduction to reinforcement learning and robots and how they're using it in robotics. And that was episode 14, you know, included in the show, notes and everything. But that's another great resource for reinforcement learning.

He is a brilliant person and I personally learned a lot from that episode. I was very impressed that you got him onto the show. He was a fantastic guest and one of those that I keep going back and listening to over and over again for sure.

Thanks, Chris. I really enjoyed digging into these things. I also have a bunch of tabs open that I'm going to do some, some reading afterwards and look up a few of these things that you mentioned. And until next time, we'll try to keep my learning up and then we'll, we'll talk to you next week.

Sounds good. If anyone happens to be in Nashville this Saturday, then I'm looking forward to seeing you at the Emerge conference and I will talk to you later on. Daniel, take care.

Bye bye.

Bye bye.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor, go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you got to do. Share the show with a friend if you enjoyed it and bandwidth for Changelog is provided by fastly. Learn more@fastly.com and we catch our airs before our users do here at Changelog because of Robar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show this episode is hosted by Daniel White Knack and Chris Benson. Editing is done by Tim Smith, the music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcast for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.