In this episode of This Week in Cognitive, host Noelle Russell introduces the Intelligent Kiosk – a Windows Store UWP application that aggregates many Microsoft Cognitive Services for rapid testing and prototyping. She walks through the app’s key capabilities, including Vision Explorer (Computer Vision, Custom Vision, Face API, Emotion API), real‑time driver monitoring, crowd insights, News Analytics, and OCR. The demo shows how Bing Search with auto‑suggest can retrieve images, how the Computer Vision service returns tags, descriptions, celebrity recognition, and dominant colors that can drive hyper‑personalized UI experiences. OCR extracts text from images, enabling downstream searches (e.g., for “Confucius”). Russell explains that developers can access all services via SDKs or REST calls, and she points viewers to the GitHub sample (aka.ms/intelligentkiosksample) with a README for building and deploying the UWP app. She highlights the ability to run the kiosk on edge devices such as a Raspberry Pi, and encourages community involvement through GitHub issues and contributions. The episode’s purpose is to showcase the breadth of Cognitive Services, provide a hands‑on learning tool, and motivate developers to build their own intelligent applications.
Chapters
Hello and welcome to This Week in Cognitive. I am your host Noelle Russell, and it is my pleasure to share with you some of the cool things we've got going on in Cognitive Services and Applied AI. Today, I want to show you one of the tools that we use internally and that you actually have access to in order to test out some of the latest and greatest services inside of Cognitive Services and the Applied AI organization. So I've shown over the course of some of our episodes, we talked through a lot of the different services, and it's always really great to be able to have a way of testing them, right, or building projects related to them. One way that we do that internally is through a tool called the Intelligent Kiosk. So it gives you the ability to test out many of these services, but we also give you the ability to build that kiosk yourself. So I'm going to walk you through just a couple of those things. So first, the Intelligent Kiosk is an app that you can download from the Windows Store. In your case, I'm also going to show you in just a minute how you can build that app and deploy your own. But let me just talk you through a couple of the things that are in here. Anytime you go to a conference or come to our Executive Briefing Center here in Redmond, Seattle area, you would have the ability to test this out. A very, very unique way of seeing a bunch of different functionality in a single application. So you can see here we've got things like Vision Explorer, right? Being able to test out computer vision, being able to do real-time driver monitoring to detect distraction. We also have things like News Analytics and Emotion API, Real-Time Crowd Insights. This is always a crowd pleaser at conferences, right? Where we put it up on kind of a big screen and add a couple cameras and as people walk by, is able to instantly identify using the Face API as well as computer vision things like a gender, age range, sentiment, are they happy, sad, bored? It's useful information typically at a conference, but it's super fun. You always see people walking by and they try and change their face to get a different age to show up, and sometimes it doesn't work, but sometimes it does. I do find that happy people look younger, so you should smile more often. But let's take a look at a couple of these. One, is the ability for us to use vision. Custom Vision, we're actually going to combine a couple of different services here. The first service is Bing Search. So, I need to go find an image to analyze. So, I go in, I use Bing Search as a service in the app. You'll see this in the GitHub sample code. Maybe I would type in, some of you are getting to know me, I like motivational quotes. So, let's pick some of those. You'll notice what happened when I started typing in a couple of letters, Auto-Suggest automatically suggested some options for me. So, that is also another service that you can take advantage of. So, a bunch of things come up here. I'm going to pick one. It's a nice motivational quote. Here are some interesting things that happen as a result of this analysis. Now remember, these Cognitive Services, the reason they're so interesting is that you as a developer, get to access them either through an SDK or a RESTful web service call. So, very, very simple access. You pass in this request, send in the image in your RESTful call, and you get this data on the way back for you to do what you will within your application. So, imagine here if you're looking at this, it gives you these tags. It also gives you a description. It gives you celebrities if there happen to be any using face recognition. But here's an interesting part, colors. So it allows you to take a look at an image. Maybe you have multiple calls to action on your web page. And based on what that user picks, maybe they pick one that has blues in it, as opposed to the one that has oranges. Now you know that they like this blue for whatever reason. You can actually hyper-personalize, take their experience and now shift everything to shades of blue. Because you know that that's what's driving their action on your site. These are really great opportunities for us to create better experiences for our customers and users of our application. Now, you'll also notice there's this feature of Computer Vision called OCR. OCR allows us to go in, this is an image, it's from gooddayquote.com. OCR or being able to click and drag this image or try and grab the text off this image is typically pretty difficult to do. But in this case, you can see it's able to pull all that off. Now, why is this important? So, imagine again, if a user passes you an image, you can now grab text from that image which was otherwise difficult to do, and now you can execute a search on any of the content that you pulled off. Notice here, it gives me back Confucius. So, now I could go in and do a search on Confucius and pull back things like YouTube or SoundCloud, audio or voiceovers or anything that's related to that content. Again, enriching the content that I can return back to the user as a result of their request. So, really interesting stuff, and it's great to be able to build an application that just demonstrates this capability. It gives you a chance to review the metadata that you get back from these services, and so as a learning tool, I think it's really useful. So, let's jump to how will you learn this. So, I will take you over to the GitHub project. You can actually type in aka.ms/intelligentkiosksample. We'll put that link in the show notes. You could see in this sample, it provides you the ability to rebuild this. As always, we provide a nice README file that'll walk you through what you need in order to build it. It is a UWP app, so be aware of that. Then running the sample, you'll need to have a development environment, you'll need to be ready to deploy it as a UWP app, Here's all the different projects that are included. It is not a direct match to what I just showed you, but we're constantly adding more and more projects into this area and updating it all the time. Most importantly is if you test some of this and you want some help, you can submit GitHub issues and our team will review those issues and even you could contribute to some of these projects if you want to. What a great opportunity not only to learn by doing, but then also to even maybe make a difference in another person who's trying to learn this by either fixing something, making something more clear, or even adding an entire sample. To the project. So, one last thing I want to mention is the ability to build this at the edge using a device, Raspberry Pi in this case. So, not only can you of course build this as an application that will run on a machine or in the cloud through a web application or through an application native on a computer, but you also now can start playing around with building devices and integrating these models into devices, really cool stuff. So, in this really quick week episode, we decided to just give you a chance to take a look at some of the projects that are available for you today that you could build that demonstrate some of the common capabilities provided with Cognitive Services. So, if you go in, again, take a look at the show notes, we provide the link to go and build this yourself, and you can also download the app and play with it there too. So, hopefully, you found this interesting and I encourage you to start building something really cool today. And I'll see you guys in the next episode.
