Noelle Russell demonstrates Azure's Computer Vision object detection, from pre-built image tagging to custom models. She builds a Hidden Mickey detector and a Chip-vs-Dale classifier by uploading tagged images and training the model, then shows how this automates real tasks like produce inspection and defect detection.
Chapters
Hello and welcome to another episode of This Week in Cognitive. I am your host Noelle Russell, and it is my pleasure to share with you some of the latest and greatest tidbits of knowledge about Cognitive Services and Applied AI. This week, as I do with every week, I like to share with you all of the different pre-built and Applied AI models that we have available ready at your fingertips. If you are a developer and you know Java, Node, Python, Go, right? If you're a mobile developer, you have the ability to leverage SDKs to be able to use some of these models in your solutions today. I want to focus on a pretty interesting new feature that we've got available. Today, it's available for preview, so you can go and test it out right away. It's actually in the vision side of the house. So, we're going to go ahead and take a look at computer vision. Now, I always like to start with our website. You should go out to our doc site and take a look. You can actually demo these products right inside of the browser. So, if you look here, I'm actually inside of the azure.com website. I'm in Computer Vision and I have the ability to test out what Computer Vision does for you. So, if you're new to the vision side of pre-built AI, it allows you to pass in an image and then be able to get data or metadata back from that image. So in this case, I'm looking at an image that looks like somebody in a train station. You can see here there's tags that come out of that: train, platform, luggage, man. There's also the ability on this site to upload your own image. Or I could just flip around to other images and see how those tags change depending on what I've selected. So using computer vision gives you the ability to analyze and detect data from images, which which is not something as a developer that came very easy to us without some algorithm to help us out. So, what I wanted to do though is take this one step further. One of the things that we provide the ability for you to do is to take some of our pre-built models that use Microsoft data and Microsoft's algorithm and take that but now add your own data to accommodate some more customized use cases. I had a fun use case recently. I was in Orlando for a week and I thought, Oh my goodness, there's actually a couple of things in Orlando that would make a perfect example. So, I went over to Custom Vision which is our portal, customvision.ai, which is our portal for allowing you to build on top of these existing models. I signed in and actually created a couple models. One is for hidden Mickeys. I don't know if you know this, but there's a bunch of hidden Mickey Mouse symbols all over the Magic Kingdom and all of the other properties at Disney. I thought, wouldn't it be cool if I passed in one image, if it you could then show me where the Hidden Mickey was on that page. Another example which I'll show you, I'll dive into right now and show you exactly how I built it, was to be able to create and detect the difference between two cartoon characters, Chip and Dale. And not just cartoons, I actually wanted to detect it between all these pictures I have of my kids, which one's which. So, I went in and I loaded a bunch of images. All right. So, if I select trained images, you'll see all of these images of Chip and Dale that I grabbed off the internet. Now, my goal was to go into these images and tag the nose. Turns out, you can tell who Chip is because Chip has a chocolate chip black nose, and Dale, well, he's the other guy, or has a red brown nose. So, I actually did a couple, right? I went in and tagged those that were black and chocolate chip sized, Tag them Chip and then tagged the red one Dale. Now, I had to train the model. I'm training it on top of the computer vision model that already knows that these are big furry stuffed animals, and now I'm adding this new data of who's Chip and who's Dale so that when I pass in a new image, I have the ability to do that. So, I just go in, you can see Chip, Dale. I did this for about, you can see over here, let me close this out. You could see here I did this for about 22 images, maybe 21 images. Now, I only needed to do it for 15 in order to be able to train and test the model. But as I trained and tested it, and you'll see this in the performance, once I trained it, it generates some performance data for me. You could see that I've done this quite a few times. I'm now on iteration 11. Again, this is part of what is artificial intelligence, building these research experiments and tweaking the data in order to generate a model that gives us the percentages and the confidence that we need. So you can see, you know, I've got a bunch. This was— these were numbers that I was really happy with. So I went ahead and I decided to do a quick test. So a quick test means now I'm gonna go out and grab an image in order to do analysis and detect the difference. So I'm gonna browse some local files. Let's see, I think I have them in pictures. There is a collection of data here, right, of images that I downloaded, and I wanted to make sure I selected an image that was not already in my dataset. So, I'm going to go ahead and pick one. Let's say, I think this one. You'll see it was able to tag and identify Chip and Dale, Dale with 79% accuracy and Chip with 63% accuracy. Both confidence numbers I'm pretty good with. Now, imagine this is your project, right? What could you do? Now you can think about, like let's say you are trying to analyze produce, good produce versus bad produce. You've taken pictures of them, you know which one's which, and now you can flag produce that's not great. Imagine it also, we've got a great partner company that is building hardware, and they used to use engineers to look at that hardware to determine whether it was good or bad. Not the best use of an engineer's time, right? Visual analysis of hardware. So they use computer vision in order to identify, create an object detection model that said, this is what's good, this is what's bad, alert us when something bad shows up, right? Now we can use machine learning to do that work of identifying, and then we can use the engineer to do the work of analysis, right? To do the discernment work on what do we do with that information once it's created. So, this is just an example. I created it very quickly, but you can see just upload some pictures, tag it with what you want to identify, and then begin to tweak the model to get the performance that you want. So, it's just an example of computer vision and object detection. I hope you will find an opportunity to play soon. We actually have created a GitHub link with the images that I've selected and a short tutorial. So, if you want to take a look, you can click the link and go play with it today. So thanks for joining me on another episode. I'll see you next time.
