What's new with Speech Services

Hello, my name is Noelle Russell, and welcome to This Week in Cognitive, where I have the pleasure of sharing with you some of the new and awesome things going on in Cognitive Services and Applied AI. Always feel free to connect with me on LinkedIn or connect with me on Twitter. Be happy to be able to answer any questions you may have or just share with you some of the latest and greatest that's going on in this space. So for those of you who are new, thank you so much for joining us. This Week in Cognitive is all about getting just snackable content every week and being able to see some of the latest and greatest samples that we've developed just for you. So the first thing though I always like to do is make sure that you are familiar with kind of the architecture that we are talking about, right? So we have built an entire platform that makes it easy for you to consume machine learning models and in this specific episode, we are actually going to be talking about the pre-built models that are available for you to use. So, you can see here we have pre-built models that we call Cognitive Services. There's actually a collection of these we call Applied AI, and what it means is that you can apply these models. You didn't have to build them, you didn't even have to have your own data, it's Microsoft data, Microsoft models, but available for you to use as a web service. But sometimes that's not enough, right? Sometimes it doesn't completely answer your question. So, the very next thing we offer is the ability for you to customize those models. So, I'll show you today exactly how you can do that with one of our brand new generally available services. Even sometimes at that point, that might not solve your problem completely. AI, right? This is a research activity, you have to do lots of experiments. So, you may get to the point where you want to experiment on your own. Code first, build your algorithm from scratch. Obviously, if you already have an algorithm, you don't want to do that. But if you can use one that's pre-built, You want to do that, but sometimes you need to build your own. So in that case, we provide an entire stack of tools and frameworks and most importantly compute that will allow you to power your own models. So here you can see, we have a huge selection of computing power available to you no matter what your side of the architecture you're on, whether you're using our pre-built models or whether you're using models that you're building from scratch. Of course, I don't want to go without mentioning all the tools that are available for you. We want to meet developers right where they are, meet you where you're building these models, whether you're building them in a language through our SDKs like C#, Java, Node, or Python, or whether you're using some of the tools like TensorFlow, or Caffe, or Cognitive Toolkit. So, we want to make sure that you have all the tools you need to build what you want. Now, a lot of people consider Cognitive Services to kind of be a superpower like a magic wand. So, they're business ready, they run on Azure, they are part of your subscriptions that you have available to use, and I want to just give you a little taste of what that looks like. First, the languages. All of these languages are supported. You have the ability, we consider Java, Node, Python, C#, these are our primary languages. First-class citizens. So, if you are a developer in this space, we want to make sure that you know you have access to these models through these SDKs. But there are so many things that you can do. Today, I'm going to just focus on one, but look at all the different models you have available. You could do things in computer vision. I recently created an object detection model that could tell the difference between two cartoon characters. Just really interesting technology that can help you solve some fun things, but also can solve your business problems. So, everything from speech to being able to analyze text, to be able to create a bot very, very quickly that can answer all of the top-of-mind questions for your customers or your employees. These things are already at your fingertips. But today, what I want to do is show you very quickly how you can leverage our newest member of the family in Cognitive Services, our newly generally available Speech service. So, the first thing I'm going to do is actually show you the website. You can go out to Microsoft.com to our Docs site, and you can actually see that speech has changed. So, for those of you who have used speech in the past, you used to have to cobble together a bunch of different APIs in order to leverage the speech service. But today, instead, what you will find is Now, they're unified. It's a single subscription. So now, rather than having to piece all these things together, and they normally go together anyway, we've made it easier for you to consume. So, I'm going to actually show you a very special part of our speech service, which are some of our preview technologies. But I just think they're really cool and interesting, and I wanted to share with you what we've done with them. So, I'm going to show you Custom Speech, and this is the ability to do what I mentioned earlier, take an existing model, that we've provided and then adapt it for your use. Now, many of you may be familiar with the JFK files. You can go out to GitHub today, rebuild this project all by yourself, and if you want to play around with it, the benefit of this project is it allows you to take structured and unstructured data in a file directory, a blob storage, and be able to make sense of it and create analytics and relationships across that data to make it easy for business users to consume. Which is a problem every business has. Now, I'm not here to talk to you about that. I will in a future episode, but not in this one. In this one, I actually wanted to show you a bot that I built on top of this. Now, this is JFK, so you guys might remember. It's like a controversy that happened way back when. Well, I thought it would be important, this was based on a president getting assassinated. What if that new president wanted to ask someone questions? So, I went back, looked up the head of the FBI at the time, his name was J. Edgar Hoover, and I said, "Wouldn't it be cool if we built a Hoover bot?" So, I created this Hoover bot in order to allow me to leverage that cognitive search capability in JFK files, but interface with a chatbot. Not only one with text, but also with the ability to leverage speech. So, what I'm going to do here is just start a conversation with this chatbot, Mr. Hoover, and I'm simply going to Say, "Welcome." Nice to be here.

Thanks for having me. How can I help you?

Now, the voice you just heard was synthesized speech. It's a new service that you have the ability to use. So what I did is we took a bunch of audio from those JFK files, right? We also took some audio that we were able to find across the internet and we used about an hour of audio in order to create this synthesized voice. Now, I was in luck because this is like 1960s audio, so it's not awesome to begin with. But it does give you the ability to start thinking about what is your business going to sound like. This, I created kind of a fictitious version of Mr. J. Edgar Hoover for the purpose of this bot. But this is something that you're going to want to think about as you're building bots for your experience. Not only what does it sound like in the form of text, but also what is it literally sound like. Now, another thing that's really interesting is I want to provide the ability to do things that most default language models just can't do. So, here's an example. What's the most important cryptonym associated with GP Floor?

K-pop. It is the table indicator for the highest level of document sensitivity.

"GP Floor is above Royal Bath, which is above Eyes Only." So, the interesting thing about that is that there's two things that happened here. One is I'm using words that are not in the default language model, meaning if I spoke this to just a default chatbot, it would say, "What is GP Floor? I have no idea what that is." Most importantly, it would not be able to execute a query on the doc set and return this response. So, I had to train the language model on this custom word, So, I trained it not only on GP Floor, which definitely is custom, but even Kryptonium wasn't in the library either. So, I took 268 of these words, I trained it on the pronunciation data so that the model knew what it was when I said it, and then it was able to translate that to a query and actually pull that information from this blob storage full of documents of structured and unstructured data. This is incredibly difficult to do if you had to do this from scratch, if you had to build your own model to support this. Now, it's available to you in a web service. So, just to give you a quick wrap-up of what we did here and what you can do at home, you can actually go out to GitHub. There are three projects. One is Web Chat, which is this basic chat box you'll see here. It's almost exactly the same minus I slapped on a picture. Then, we grabbed Speech-to-Text and Text-to-Speech, and then we added the endpoints from these services in to create this experience. It literally took a few hours, Most of that was me just fine-tuning the way I wanted it to work. I hope you enjoyed today's episode. It's really great to have an opportunity to play with the latest technologies, and I really look forward to seeing you guys in the next one.