In this recorded demo from Voice Summit, the presenter walks through a project called the "JFK files" that uses Azure Blob storage, Azure Cognitive Search, and Azure AI services to index and explore historical archival material. The demo showcases a custom speech synthesis pipeline that creates a synthetic voice resembling J. Edgar Hoover by training on audio from the JFK files, and a custom language model that understands domain‑specific cryptonyms (e.g., GP Floor, K‑pop). The presenter builds a conversational bot – the "Hoover Bot" – that combines the custom voice, speech‑to‑text, and the search index to answer queries in a historically‑flavored dialogue. The demo highlights how businesses can brand their audio output, how to extend a base language model with specialized terminology without building a model from scratch, and provides links to three GitHub repositories and a chris.ai demo for viewers to try the sample themselves.
Chapters
Hey guys, so excited to see you. I actually just did a demo and session at Voice Summit. It was phenomenal. Thanks so much, Voice Summit, for inviting me. I had such a good time and I met a bunch of new friends. But for my new friends, I tried to do a demo and my HDMI or my audio cable— anyway, you guys couldn't hear the custom speech model I was demoing, and it's such a cool thing. So I wanted to share it with you. So I'm gonna really quickly go through that demo and post it up. You're probably watching this on YouTube just so you could see it, 'cause I love it. I just love it. I think it's really cool. But all of this, the JFK files is actually a project on GitHub. You can see that right here. So you can go check it out. If you want to know more about the speech model, the custom speech model and the custom language model, let me know. I'd be happy to share that with you. We are in the process right now of building that into a sample and a project. So really excited about that. Would love some help if you guys, uh, have an inclination. Um, so thanks so much again for watching, but, uh, let's dive right in. So I'm gonna go ahead and launch, uh, the JFK files. So as I mentioned, uh, in the session, JFK files is, uh, you know, this project was built a long time ago, a couple years ago. It was built based on a bunch of data, uh, kind of being dumped into a blob storage and, uh, no real easy way to interpret it. So they leveraged Azure Search and Cognitive Search, a feature of Azure Search, in order to create a way to interpret and understand the relationships within this data. So in this case, I did a search for Oswald. It popped up, you know, a bunch of different entities and tags that were intuitively discovered from that content. You could see it has like, you know, form data, cursive writing, notes, pictures, right? So it's able to do a bunch of that. If you click on one of the pictures, you'll see it uses computer vision in order to identify what's in the, you know, object detection, if you will, to identify what's going on or description of what's happening. Maybe some sentiment analysis because it says, you know, he's posing for a camera, posing for the camera, which isn't exactly what he's doing, but anyway, we get the idea, right? So we've got this really great way now to interact with this. Data, we also have this really interesting way of visualizing the data, visualizing the entities, visualizing how it works. So this is all great, but what I wanted to do was create something a little bit different. I wanted to be able to go in and say, well, in addition to be able to type in text and see, you know, textual rendering, I would love to be able to build a speech interface. I love speech. So, so what I decided you know, I thought through the scenario, right? We're thinking about like Lyndon B. Johnson. He's just taken over a new job, right? The guy who had the job before him was murdered, um, probably has a lot of questions. Who would he want to talk to? So we came up with this idea that he'd probably want to talk to the head of the FBI at the time, and that was Mr. J. Edgar Hoover. And J. Edgar Hoover, um, you know, being the head of the FBI, we thought he'd make a great bot. So we created the Hoover Bot. So if you go over here, and again, these, you could go hit this URL if you want to, but if you go over to this website, you would see the ability to go in and type. Again, I built this language model specifically for this demo, so it's not gonna support a ton of things, but the first thing it will do is support me kind of introducing myself, right? So let me try that. Welcome.
Nice to be here. Thanks for having me. How can I help you?
Now you'll notice the sound that you heard was actually a synthesized voice that we built by taking a bunch of audio off, you know, from the JFK files, feeding it and adapting the speech synthesizer to create this new sound. And if you think about it, this is something every business will do in the future, right? Every business will have like, you'll hear, it's kind of like the NBC chimes, or, um, like the bugles at the Olympics, right? You hear these sounds and you know what it is. And so every brand has an opportunity to build the sound of their business, and this is an example of how that's done. So that's pretty interesting, and it didn't take a whole lot. I— we did use some SSML to kind of fiddle around with it because it didn't sound super great the first, you know, out of the gate. Um, we used SSML to kind of control it, but But ultimately, I mean, after a few hours of work, we had a pretty decent synthesized sound that sounded a whole lot like J. Edgar Hoover. But that wasn't enough. You know, if I'm Lyndon B. Johnson and I'm curious, you know, I'm the new president, I'm curious about a bunch of things, I'm probably going to want to ask questions of J. Edgar Hoover. And if I do that, chances are I'm going to use some names or code names or what the CIA calls cryptonyms to represent things. So if I tried to use cryptonyms in a natural language model, a default English model for example, it won't work. It'll actually just render it as like, um, you know, whatever it thinks it might be in an English language or whatever language model you happen to be using. But it's like this default functionality. So what I thought would be cool is what if we add to that, right? Just like we'd want to add, you know, names of products or businesses or services or or, um, you know, let's say you're a game developer and you're building an entire alien species race, uh, you'd want all of that. You'd want all of those things to be able to be spoken, and you'd want your language model to understand it. In the past, you'd have to get an entire language team to like build you a custom language model. Now you can simply adapt— um, now you can simply adapt that information, uh, basically adapt the language model with the changes. So in other words, you're going to take the existing US model and simply add to it, which is brilliant. I don't need to build a model from scratch. I just want to add to it. So there's several cryptonyms. So let me, um, I'll say a sentence, uh, that will ask it using one of these cryptonyms, these code words, and then we'll see what happens. What's the most important cryptonym associated with GP Floor?
"K-pop. It is the cable indicator for the highest level of documents sensitivity.
K-pop is above Ryugan, which is above eyes only." So what you can see happened here is that, right, how interesting, um, when I said the words GP floor it didn't take it in as a default language model would take it in, right? If that was the case, it would turn it into like lowercase g, lowercase p, and like floor, like the floor you walk on. Instead, I needed it to identify that it was a kryptonym and that it was all caps and that it had a specific pronunciation. And so I used the custom speech and custom language model in order to teach it what these kryptonyms are. So I said GP Floor, it understood that it was a kryptonym, and here's the cool part, it took that kryptonym executed a query and pulled back information that had cryptonyms in it, and using speech-to-text, it translated that into a voice. So K-pop was actually said out loud. Ryebat was said out loud. Who knows what they actually sounded like in the '60s or how they pronounced those things, but we were able to create a model and train it on what we wanted it to do. The idea is full customization, being able to create a custom synthesized voice like Jade Gurhuver, and a custom language model that supports the words that I want. I can define words, I can define, you know, things that are not easily understood by a language model. I can make it a custom model and deploy it. This is like a new day, people. So I hope you're as excited as I am. There are 3 GitHub repos available for you to use to test this stuff out. You can also go to chris.ai and take a look at where I built this and tested it out. It's currently in preview. But really cool to try. So hope you like it. Thanks, you guys, for attending my session. Sorry audio didn't work, but hopefully you got a chance to hear it here. Thanks a lot. Bye.
