🔴 Let's Build An Audio-To-Text Generator in JAVASCRIPT for Beginners (Live Coding)

Alright, so I did post YouTube subtitle generator last night, but it got taken down because we're downloading YouTube content through an NPM package. So we should change the project to something like audio to text generator. The description will be using audio recording Navigator media devices get user media, capture audio and send to server. Server will then convert to mp3 and upload it. Then we will send to Deepgram API to get text using Deepgram listen prerecorded transcribe file and then we will return to front end the uploaded file link and text to be displayed. So what we will need are some NPM packages, Deepgram SDK express and some dev dependencies so we can do NPM install EEP install eepram express. So those are the first things we'll do. But before we do that I think we need to init the NPM NPM init Y. Now you'll see a package JSON and we'll go up add deep gram SDK. Now we'll let that install. Now we will also need some dev dependencies, NPM install m d and NPM install nodemon minus D. So now if we look at the package JSON we have dependencies Deepgram SDK and express. And then in the dev dependencies we have env which is going to help us use the environment file and nodemon to help us with when we save to restart the server. So what we need to do first is make an index JS file and in the package JSON if we wanted to use import and export statements we could put type module. But for me I like to use require so in the server. So I'll just stick with this one and we'll add a dev script and that's going to say nodemon on index js. I'll show you how that works in a second. So now if we just put maybe console log hello world and we put NPM run dev it should say hello world and nodemon is alive. So if we put like this it'll return, it'll restart the server. So let's turn it off and let's set up a simple express server. Now we'll bring up some documentation express so we can do like this, we'll just copy it. Const express equals require require express and to use the environment variables we can do const or no, no, require env config like so and const app equals express. And if we wanted to, I guess we can say const port equals process m port and if it's not there 88888 because that's what I set in the environment file. And from here I will want to const path equals require path and we'll say I just want to set this up express static files. So I will copy paste this guy and from path it's going to give me something called name. So we're going to say app use this on public folder. So we'll make a public folder and this is where our front end will go for now. And first thing we'll do is app listen on port and that takes a callback and we can just log that our server is running. So let's choose this server running on port port. Alright, now if we run npm run dev server running on port 888-888-8888 so now app crap app get. We'll make a ping slash ping function request res res sen. You could add like the status here if you want, but it's okay. So now if we go to localhost 8.8.8.8 whoops on ping it should say pong and we just change here enter it says pong pong. So that is our first step. And now we need to serve an HTML file. So we'll just make an index HTML here enter like this and we'll just make H1 audio to text and that should be enough. We're just going to keep everything in one file. Put script here. Just make sure just keep everything very simple. Hello world. All right, now let's just do like this. We open the dev tools index JS now we'll say app get on just the bare route we can say function request res and then we use res send file on their name + on/index HTML and that should send our HTML so audio to text. So from here what we need to do is we need to record the audio with Navigator media devices get user media. So that's the first step. So I will make two buttons. So let's just put a p tag here. ID equals recording status. We just put ready and we'll make two buttons. Button start button button stop and let's just make this a class equals container for now. Start record and this is ID equals stop record and in our script here we'll use const start but record equals button equals document getelementbyid start record and we have another one stop record button equals document getelementbyid stop report. Alright, so those are the two buttons we have. We can add some event listeners really quick and I think we're going to Need a couple other media global variables. So media recorder. I know we'll need that and I know we'll need chunks. There we are. So now we can say const our start record button dot add event listener on click and we have a callback here. And I will make it an async function. So we can make it async and use await and then we'll make a stop record button.add event listener, click async function like so. So from here we'll just put a couple logs. Start record and then here, console log, stop, stop record. All right, let's refresh. All right, now that says start recording and this will say stop record. Awesome. So when we start this it should say recording and then when we stop it should say stopped and then it should go back to ready. So let's do that real quick. So we do have the recording status. So let's just put a comma here. Recording status text equals document.get, get element by ID on recording status. So the first thing we need to do, I'll wrap these guys in, try, catch just to see the errors. If they happen, do the same for the stop button. So on try we're going to try to. What are we gonna do? We're gonna take the whoops. Recording status text. Inner text. We're going to equal that to recording. And then here we're going to say recording status text equals stopped recording. And then set. And then we're going to set a timeout and then after about one second we'll say recording status text. Inner text equals ready. All right. And inner text here as well. So let's see if it works. Start recording, stop, stop recording one second and then back to ready. Amazing. So now let's figure out how to record someone's audio in the client. Let's go. I know that first we need to use the navigator media media devices. Get user media and we need to check if that's available. If it's not available, then we need to alert. We can throw. Let's just, let's just alert. Alert. Device not available. Not available on device. Alright, so here, log ready. So if you press this now it says ready. But then if, if let's say if it was not there, then it would say start and then not available on device. So at this point we can grab the stream. So let's look up, get user media, record audio. So I like to look at the MDN web docs. I was looking at it earlier, get rid of that. So first thing we can do is consistent equals on audio true. And we can now say const stream equals a weight because we're using the async function navigator.media devices.get user media. And then we can put the constraints inside of here to just get audio. Now we have the stream. So this is what's happening. We have the stream. Now we can capture it, capture the media stream through this method called Media Recorder. So we're going to say media Recorder as the global variable equals New Media Recorder Recorder, New Media Recorder. And we're gonna stick the stream in and ready on click and we have the chunks variable here and we're gonna say media Recorder. And before you say add event listener and like this and then data available. And I think you can still do that, but this is much cleaner. In the documentation, you can say on data available equals on the event and we can say chunks push e data. So that's the event event data. So on this event, after the Media Recorder starts, we can say, hey, when the data is available, push the chunks to this array. And after we create it we can say mediarecorder start. So once it's starting and once the data is available, we're gonna push the data to the chunks global array. And from here we're good. And on stop, we can do like this. Oops. After stop recording, we can say media Recorder stop and then grabbing and using the blob. So since you have here media recorder.on stop equals e and we can say console log E. You can say console log Console log on the chunks and see how that looks. And it should have a blob inside. All right, so let's go. So we start. Media record is not available. Hold on. Oh, there we are. All right, typo. Start, start. So now it's recording, it's recording. And if we stop, we see a chunk here and it stopped and we have audio and the size of this. So we can check if it's working here. I think we can do something here. So let's make a container div ID audio clips container and let's just do like this. Audio clips container equals document getelementbyid audio clips contain clip. All right, so what's it saying here? All right, so after it's on stop, so we stop and then it triggers this on stop event. And we can say we can make const audio clip equals document dot create create element audio. And we can say on the audio dot set attribute dot audio clip dot set attribute and it's saying controls and I think that gives it the play button and stuff like that. And then we can just say audio clips, container dot append child to audio clip. And from here we can say const blob. Blob equals new blob chunks. So that's the chunks from earlier and type audio. We get MP3. Let's check Codex equals. And then we'll empty out the chunks now. And then const audio URL equals window URL create object URL from the blob and audio src will equal on audio URL. So that's. Let's see if that's working. Refresh. Start. Recording. Recording. Stop. Cannot access chunks before initialization. All right, let's try one more time. Hello. World audio is not defined. There we go. All right, Start. Hello, Hello, Hello. Stop. Hello, hello, hello. All right, great. Now we can. We have the blob and we're, we have. So let's check this, inspect this. So it's blob and it's giving me a link. So now we can send this over to the server as the blob and upload it as an MP3 and then display it and let's see what happens. So next we're going to need to upload the file into the server. So we'll start with this server app post and we'll just make an endpoint called upload async.function take the request and res and we'll res send Ready. All right, so from here what we'll need to do is we'll need to. So this is where we left off on the stop record and we had it working with the blob inside of the audio clip. But actually I would like to make this audio URL audio URL a link instead of the blob. Let's see how we can do that. I've been looking up things on the Internet. So there is this cool NPM package called multr which handles the multi part data and form data and all that kind of stuff. So let's look at multter here, you can see here. So let's start the server NPM install multiple and inside of our index js we can say const multer equals require on multir. That'll be good. Now I guess we said, we say const storage and we need to define where it's going to be saved. So multir disk storage and the destination. Whoops. Destiny destination request file and then it takes callback and then we'll take the callback null and then we'll say audio and that's where it'll be saved and then on there we'll say file name so this is creating the file name. So request file callback and callback null and let's just yoink this guy just do something like date now date now path extension name file original name. That's good. All right. Alrighty. And then there's some check file type so we'll make something called const Upload equals multir and we'll pass in storage and I think that's all we need to do for a simple just stick the destination all they did so maybe that's. That should be good and we can just pass it as a middleware. So we say upload single look at the stack overflow multir file name upload multir storage upload single audio and then we'll just say res JSON send success True. See if that works. All right, so let's go here and I think what they did was they made a form data so start audio true get user media save. So from here what we need to do is you have the blob so const data equals new form data and data append on audio and that's going to be our blob and now we can just send blob to the server so we can say Axios So I'm going to look up the Axios CDN here this is the first one if you guys wonder about this in my Google it's called it's from Grepper it's a. It's a Google Chrome extension makes it easier to find answers that's my script I'll put it in the head. There we are. Now we can just say axios post We'll a weight on this async await const res equals await post and then we just making a request to himself so there shouldn't be any port issue 8/ what I call it upload upload and let's look back at the stack order flow Just put data here and then let's just console log res really quick. Alrighty now let's refresh. Are we gonna run the server NPM run dev and I'm hoping that we're going to have an audio we're gonna have the file up in here so let's. So this is going to look for the audio in the form data and then put it in there and run this storage. Alright so start recording recording recording recording stop and then we have a nice audio file here. Great. And data success true start. All right, let's Just check if this is an MP3 file. MP3 file player online Electrofile Audio Audio I don't know if that's gonna start. Recording, recording, recording Record start. Amazing. What? Easier than I expected. So now it's here. So inside of the callback for the file name we need to put dot MP3 so we can try it one more time. Start. This is me recording 1, 2, 3, 4, 5 Stop. Start. This is me recording. And then I think this, this one select my file. Recording, recording, recording Record start. Oh, that's my last one. So let's see. This is me recording 1, 2, 3. All right, so that's uploading now. So great. So now we need to do the final part and send the MP3 file to Deepgram and get the text back. So let's start but first instead of using the blob here I'd like to return the MP3 file. So let's check on how to do that. How to send that file name from uploaded file from multip return file name of mult alright, so we have the file name and request file file file name. So we can just do link and then request file file name. Hopefully that'll work. And then maybe we can add the localhost 8888 first. So it's HTTP local host 8888 audio like so I'm not sure if we can yet. So we might have to do something. So let's check local host 888-88- slash audio cannot get so we need to be able to make it used. So we need to go back into the server. Can we do something like this audio? I thought you static. Why don't we just instead let's just put them in the public audio that might be a little bit easier. And we just move this audio folder to the public move. Now if we do start this is me now it should work. I don't know why this why this static earlier wouldn't work but now it's all good. So we'll return the link and then in the client after we make the request there should be something called res data. We can say audio URL equals const audio URL equals res data link and I think that should work. Start start start start start stop start start start start start okay, amazing. And then let's just inspect it and make sure it's coming from here. Local 88888 audio and then the file name. So that's pretty cool. So now let's make the. Let's Hook up the Deepgram SDK. So just going to do it here. So it's going to say const deep create client equals require add DPRAM SDK and we're going to look at the documentation Deepgram node JS and earlier in the readme I said we're going to use Deepgram listen prerecorded transcribed file. Let's look for that JavaScript SDK here and prerecorded audio transcription. So now in here we can say something like so this is all Malta stuff. So let's just put a comment alter. Usually we split this up a little bit, but let's just do it like this. All right, now that's good. Now const d RAM equals create client on and I, I have the Deepgram API key. RAM API key and now we do the pre so we can do something like this so it uploads the file and then when we're done I would like to do const result error equals await and maybe we could do it from the URL, but let's do it from there. Whoops. So just copy paste this and read file sync. We have the request file name so we'll just put this guy in. But from here we can just say dot slash public slash audio slash public like so. And let's just put the result and result here. And then if we hit an error we can say res status 500 send something went wrong. All right, all right, start, start, start, start, start, stop. So something FS is not defined. So something crashed. So let's refresh and we actually need the FS module. So const FS equals requirements FS and server should restart, I think. All right, 1, 2, 3, 4, 5, 6, 7, 8 stop and let's see the response and results channels alternatives 1, 2, 3, 4,. 5, 6, 7, 8. So let's go get that result here. So I think it's result dot results dot channels on an array dot alternatives on an array and then dot transcript and that's what I'd like back. Refresh start. Hi, my name is Phil, Please keep watching. And from the data start. Hi, my name is Phil. Please keep watching start. Hi, my name is Phil. Please keep watching. There we go. Now we just need to add that, add the text under the audio clips container. So let's just append two childs at one time. I think that should work. So we have audio clips and then we have the res. So we can say something like const audio text equals res.data. result. I think I put look at the server code result. So dot result. And then we can say const p tag result text equals document create element on p. And then we can say P.inertext equals audio text. And then on that audio audio clips container, copy paste here audio clips container, you can just add the result text and result text instead of here. And that should do it. Now we're done, and hopefully you guys learned something. Now we're done, and hopefully you guys learned something. Great. Thank you for watching. Now we're done.