Add Real Time Voice to ANY n8n Agent Using ElevenLabs

Today I'm going to be showing you how you can transcribe individual audio files and have two way responses using ElevenLabs for automatic responses using agents. And I am then going to show you how you can customize it to have conversational calls with back and forward research questions, whatever the business or client may require in order to generate the output. So in 25 minutes time you're going to be able to build both of these flows out using this, which is 11 labs, which is I'm on the. The API key page. Four 11 labs, telegram, messenger and Perplexity. That's all you're going to need. Let's jump straight into the build and we're going to start it off with a telegram trigger. Now the telegram trigger is going to be if you just either hit the node or hit tab, you hit. You want to search for telegram, we go telegram, we go on message. That means that when a message is received, you're going to be in a position where you, where this fires, this trigger fires. Now I have these accounts connected already. If you don't you literally want to head over to your 11 labs, search for Botfather, hit create new bot. What we're going to call it Test Test test. Spelled horrendously but Test test test. And now we want to choose a name for the bot. Test Test Test bot. And then we're going to hit enter and once we do that it's going to pop. It's now going to provide us with it's already taken. Test 1-23484/lot. Can be tricky to find the name of one which isn't taken. But there you go. There's a, there's, there's a bot that. That name wasn't taken obviously because it was very random. And there's two key things in here. One is that we have the link to the actual chat itself which we need to start a chat with in order to continue through the next steps. You will see. Let me just turn on my presentify. You will see up here in the top left hand corner the new, the new chat that we created. The new, the. The new bot that we created. And going back to bot father, you're going to see this API token. Now if we copy that API token and we go back into, go back into nan it create a credential. And we're going to call this Test Test test. However it was spelled. And I'm going to hit save. No thank you. Once it's saved, we now have a connection directly with Telegram and this trigger is now watching for that message. So what we're going to do is we're going to go into that individual chat and we are going to send a voice note. So I'll quickly do this now. Hey, Michael Elliot here. Nice to meet you. We fire that over. But that ain't gonna work because I didn't click execute workflow. So let's see if it, if it takes its time to pick it up or not. In the meantime, I'm going to send another one. Test, test, test, test, test. There you go. Did fire. Fired twice. Got both, which is good. Okay, so it picked it up because of the time frames anyway. I just want to make sure. But now we have a telegram trigger that is listening for voice notes that come through into that chat. The next step of this flow is for us to add another telegram node. But this telegram node is going to get a file because what it's receiving is a binary file that is voice. So watching for a message or getting a chat is not going to be enough. We have to get a file. And you will see that on the. Let me just delete this at the moment. This connection on this, on this file id we have down here the type, which is audio ogg, which means that it's an audio file. And then we have the file id. So if we just stick that into there and execute the step. I'm using the wrong credential. There we go with that lesson there. We now have the output that is the download of the, of the audio that I've just played. So let me download it and play it for you. Hey, Michael Elliot here. Nice to meet you. That was obviously the first one that I did. So great. We now know that we're receiving the files and that we're downloading them properly. Okay, so we hit save. And then from here we are going to use 11 Labs. Now 11 Labs is a fantastic tool. This is a community node. So if you are not on one of the latest updates of N8N, you need to just update. If you can't find it, that will be the reason why. But we hit this and what we want to do is we want to transcribe audio or video, but because obviously it's audio that we're using and we have this data. So I'm going to come out of here and I'm going to pin this and I'm going to pin this and I'm going to execute this step. Let me just hit save. This happened to me a minute ago. Let me unlink this. And look, look at this expression. JSON.message.voice.File ID. I have this somewhere. Let me have a look. There we go. JsonMessage voice. Of course, it's the same. So we're learning on the job, guys, which is, which is amazing. Yeah. So the download is the JSON message file. I don't know why that didn't work, but let me execute this step and unpin it. It still pulls through the data. It's got the file size. Let's now connect and see if this will transcribe it now and execute the step. Yes, it will. So there you go. The download needs to be, needs to be an expression. So, okay, we can see that up here. We have text. So it has text available for us. It's obviously transcribed the audio that I've got successfully. And that means that we're now in a position where we can, we're now in a position where we can look at opportunities for the AI agent to be used. Now, the AI agent here isn't really that important. We hit the plus icon and we hit AI agents. And the key and core element of this is the fact that we don't want the source for the prompt to be the trigger of this, like the chat trigger node. We don't want that to be the case. We don't, you know, it's not what we're using here. What we actually want the prompt user to be is we want it to use the JSON text that it receives from the output that it transcribes. So there we go. It's now got. We've. I hope you can understand that. From the schema in the, in the input to this node, we've pulled the text which has been transcribed by 11Labs and then we add option system message and the system message is going to be you. Like this can be anything. You're a helpful assistant, but you can be quite grumpy. Let's put that grumpy and funny. Let's just, let's just put that in just something I'm going to cover with you quickly. I'm going to give this a brain so chat model. You can use anything you want here. I'm going to use open AI. You can use open router. If you haven't connected your OpenAI account, you'll need to do the same thing with the, Create the credentials like we just did with the, with the bot. Which is not. Which is really why I didn't show you elevenlabs. But I'll just Very quickly, in case you had not, you've not seen this, you literally log in. You go down to the, the workspace at the bottom, API keys, and you hit create an API key. And what's important is this gives it permissions to what it can actually do. So the permissions might be text to speech. Well, we, we need that. Speech to speech. We need that. I just turn them all on. You got to be careful with your API keys. And when you, when you do create the API key to copy over, make sure you save it somewhere because you can't get it again. And you'd have to change your, your integration but back to where we were in the, in the individual step. Now I've pinned this data, pin this data, and let's just pin this, and let's just execute the workflow. Now this is going to show us what the output was. Oh, great, Another Michael Elliott. Just what the world needed. Well, nice to meet you too, Michael. What can I do for you today? So it's taken the grumpy approach, which is let's change this to or so that we just get varied messages back. That's just me being me. But that's, that's amazing. So you can see that it actually provided a response, but importantly the response. The output was in text format. Now, as we have previously sort of identified when we had the issue earlier with the, with the, with the speech node, we now need to do basically the complete opposite of what we did earlier. And what we did earlier is transcribe audio to audio or video to text. But now we need to transcribe text to speech. So we're going to go back to 11 labs and we're going to convert text to speech. Now, inside of this node, the speech, we want speech and we want text to speech. Now the voice that you choose. Okay. Is, it depends on your, on your output. So let's use Rachel. Okay. And the text is going to be the output variable from here. And if I execute this step, it's going to give me a file. Now let's download this and let's listen to this. Well, well, Michael Elliott. Nice to meet you too. What can I do for you today? Hopefully it's not just small talk. I'm here for the juicy stuff. Thanks, Rachel. Yeah, that's normally how women react to me. So we have Rachel, but let's just try something here because it's. Something just popped up and I cloned my voice. I'm not sure which one of this it is, but we'll have a Little bit of fun on this video. Let's see if it actually sounds anything like me. I only did half an hour of uploads from videos. Let's see. Well, well, Michael Elliot, nice to meet you too. What can I do for you today? Hopefully it's not just small talk. I'm here for the juicy stuff. Very sad, very southern, and I'm certainly not southern. Let's just see if it was this one. I'll execute the step again. We'll just see if it hits it. I'll do another video completely on doing that. I'm not trying to waste your time here, but if it got it right. Well, well, Michael Elliot, nice to meet you too. What can I do for you today? Hopefully it's not just small talk. I'm here for the juicy stuff. Stuff was a bit amplified, but that was me. Well, well, Michael Elliott. Nice to meet you too. What can I do for you today? Hopefully it's not just small talk. I'm here for the juicy stuff. Interesting. So I'm going to flick this off from, from Rachel. Okay. I'm gonna, I'm gonna go back to Rachel. That was. I'll show you that at the end. Now that it's actually worked. It's funny you saw the message, I believe, or it popped up during the, the, the sort of approach. So I'm going to execute it just one more time with Rachel just to make sure we get the Rachel data. Okay. And now what we want to do in terms of the next step of this is literally just send an audio file to Telegram. So we go to Telegram, we send. Send a, an audio file. There we go. We send an audio file. Now when we send this audio file over, we want the chat ID to be the telegram trigger. Okay. So if we go into schema and scroll all the way back and we find the telegram trigger. Telegram trigger. Here you will see that we want the. The chat id. Where's the chat id? The chat id. Telegram trigger item. Chat id. So we can pull the correct chat at the right time in order to send the audio. It wants to be able to send the right audio, obviously to the right, to the right person. This is not the correct one for me either. So I'll just change that back now, which is important. Now the audio is going to be. Obviously it's a binary file. And as soon as you tick that, it's going to give us data and it's going to. It's obviously called data. We know that from looking at the, the previous, the previous node where Are we transcribe audio? Which one was it? Anyway, we know that this is, we know that this is data. Maybe it was JSON, but the, the, the, the output is data in terms of the file. So, so we're pretty much set as far as I can see. Let's execute the step now that completed, if we go into Telegram. Well, well, Michael Elliott, nice to meet you too. What can I do for you today? Hopefully it's not just small talk. I'm here for the juicy stuff. So here for the juicy stuff, we've just built the first flow which allows us to be able to send and receive voice notes. If you had a conversation you wanted people to respond to people like that, you could do it, which is amazing. Now I have to say that inside of 11, inside of 11 labs, there are a huge amount of individual, individual voices and things that you can use. Like as you come up inside of here, you can see if you literally just hit Vilno is a Duvant Chikov, the soft female Russian voice if you wanted one. But if you wanted to come down here like Rugged Men collection, you can see that you can. And they have different categories. Well, the point of coming in here, meditation and life are not separate is just to show you that if you actually copy the voice id, you could go and change that inside of the individual, inside of the individual audio. You could transfer, you could do that inside of the individual audio. And instead of selecting from list like Rachel here, you could change it and select from ID and put the ID in and it would use that voice. So that's just important for you if you want to mix it up. And especially if you want to create a voice like your own, which is the one that you've just heard me do. So that's the first part out of the way. The second part is, is incredible because it can have two way conversations and do research and analysis. Let's jump straight into that. So now we finish with this. What we're going to be doing is building a two way conversational agent using 11 labs, which is where the true power really comes in. And for simplicity, I'm going to hit tab and I'm going to add a webhook. Now I'm going to search for webhook in full actually, because we are going to use both of these types of webhooks inside of this flow. Forget this one for now, but I just wanted to let you know you can create a webhook and you can also respond to it, which is really important in certain flows where you're trying to have a conclusion to the, to the, to the flow without passing it off and just ending it somewhere. The responding to the webhook allows us to do that and you'll see an important setting a little bit later on. But for now, let's hit webhook. Sorry, let me move that way. Let's hit webhook. We have this here and I'm just going to click this copy of the webhook URL. Now we have a test URL and a production URL. Now test URLs what we just want to use for this because it's a test. If it was in production, you'd want to use the production. But let's copy, let's copy this, this, this address and let's head over to 11 labs where the magic starts. So I showed you a moment ago how you can edit these voices and stuff. And there are so many. You can look at trending voices, all sorts. Like you can try different things. What we want to do though is we want to head over to this, this button top left that says, well, it's a drop down. It's got a creative platform and conversational AI and we want to go inside of here and we want to create a new agent. So if, if you go here and see agents on the left, we just literally hit that and we want to hit a new agent. But what we want to do when we have a new agent is we want a blank agent. We want to do, we want to build it ourselves. Okay, so generalized agent test. If you have a use case for this, call them appropriately, of course, but generalized agent test. Okay, now this is where the magic happens. In here we can do a lot of individual things. Just keeping your mind. We've just copied the web hook over. So we don't, we've got that in our clipboard. But inside of here we, we can change things. We could have, you know, different languages we can have. There's so many approaches to this. We could change the voice. I'm going to change the voice and I'm going to select my voice clone. Let's just do that. Let's see how that, how that works. I think that was the right one that we chose, but you can choose any. Rachel was one that I, that I used. Inside of there, you can search for her and she should appear. Rachel Legacy. There you are. Narration. You choose anybody. Doesn't really matter, to be honest with you. Nothing else really matters inside of here. You can change the speed, you can slow them down. There's lots of things you can do but I'm going to hit save using that clone and I'm going to go inside of here. From an agent perspective, English is what we want it to, is in this case what we want it to speak. Now the first message we want to have. Hello, how can I help you? You can add variables to this so that you can, you know, you can play around with different things depending on like you could for example that you might want to use this like the caller id. So if, if you have a system which matches a caller ID to a name, you could, there's so many things you could do with the variables but we're not going to, we're not going to do that now and we're also not going to mess with the system prompts as of yet. But it has a very cool feature from an AI perspective. The LLM that it uses, you can choose, you can choose whichever one you want. I'm just going to leave it as Gemini 2.0 flash and temperature is a parameter that controls creativity or random and randomness. Now this is basically how creative you want your, your agent to be. Your, your. Your your conversational agent to be. If you wanted it to be extremely blunt you would have it at zero. If you wanted it to be somewhere in the middle you would have it at 5. And if you wanted it to be in fairy tale land you would put it at one where it would just start talk. Its responses would just be. It would, would be. The 11 labs calls it more creative. It would just be like lackadaisical fairy sort of approach can be useful in, in certain use cases. I think sticking it at 0.5 year where it's creative is, is good but we can maybe. Let's drop it a little bit so it hits somewhere in the middle of the deterministic or creative doesn't. Honestly it doesn't really matter. Leave it where it was if you want. Now what we want is we want an end call option. These are the tools that allow the agent to perform specific actions so it can transfer to a number. If you had this as a filtering number, a number rang, this agent answered the phone, asked a few preliminary questions. You could then transfer the agent to a number so it calls a human phone for them to take the inquiry. You can see why these would be, why these would be useful. But we don't need to do anything here. We're not going to do anything else. The only thing that we are going to do here is we're going to Add a tool and you can see I added one previously which was Test Nanagent. But I'm going to add a webhook tool and I'm going to call it, we're going to call it Test NAN Agent delete later. And the thing is about this is that because we need to max the response time because we don't know how long it's going to take it to get the answer when depending on what the user's question is, we want to max it. But the method wants to be post because it's going to be posting information, it's going to be sending information back and this is critical. So the webhook URL we copied earlier, we are going to paste in here just in case you may have lost it. You literally just go into your webhook node, click the, click that it will copy it for you and it will bring it back in here and then you paste it. So we now have the, we now have this set up in a way which allows us to be able to, allows us to be able to effectively, effectively manage the, the, the, the, the webhook and the posting of the web hook backwards and forwards. Completely new. I've created this from scratch to, to help you. We don't need anything else. We literally just need to add the tool. So we add the. Well, in terms of the, the, the. The description, I mean I don't know what I put in the other one. You're helpful assistant, that is. Yeah, this is correct actually. Good job that decline helpful assistant that uses NAN to go out and research the user's query, providing specific answers based on facts. Let's just put that. Okay, if I could spell, I would be dangerous at all. We do want, we do want this. So this is really important. I've just nearly made a couple of mistakes because I was trying to be quick and efficient for you. But this is the data type that it's going to be sending and the data type is going to be a string and it's going to be a search query. Right. And is it required? Yes. And it's prompt prompted by, by the LL but by the, by the LLM. So I'm going to literally show live to you. I'm not too proud. Let me go and find what I put in the, in the, in the previous agent. Okay. So I created this one this morning when I was looking through this. Let's have a look. So this, this will come later. But inside of the, inside of the webhook what did we have? So extract the search Query. The user is looking for more information on what the user is searching for online. So this is what goes in that individual query, but it is a required string. Make sure that you, you know, that you have that as a required string. And the description was to, you know, get a. Research the user's search query. Extract the user search query. Yeah, bit of a mess, but what can you do? Extract the user's search query what the description wants. Yeah, extract that. What the user is searching for. So, yeah, there we go. Let's have the tool. Now we've got it. So test any and delete later. Webhook is now created and we have that. Now what we need to do in here is we need to create a system prompt. Now if you were going to use this in a business, you'd probably go away and professionally do it. What we want to do though is we just want to say you're going to receive information via a. You're going to receive information and your job is to extract the user's search query using NAN and provide specific answers. Keep this friendly and reasonable in length, three sentences max. Let's just see what this does. Now, if we hit this wonderful magic button, it's going to generate as a prompt in full on its own, which is amazing. So you're a friendly, efficient assistant specializing in extracting search queries and providing answers. You're integrated with an NAN workflow that provides. Provides user input. It's critical that you mention NAN or any other tools when you're building them later and expanding inside of this if you are going to AI generate them, because if it doesn't include it, then you're in trouble. So you have access to nan, Receiving user input and delivering your response. Formulate a clear and helpful response based on the query. Deliver response in a friendly and concise manner. Guardrails. You might want to tweak them. Looks good to me. Now let's. So the agent has been saved. There we go. The agent has been saved. Now that we've done that and make sure you do that. And now what we can do is actually we can go back into the flow and we can execute this workflow. So it's waiting for it and we can. Whichever labs was it, we can test the AI agent. Hello, how can I help you today? Hey, I'm just looking for the weather in Tampa. I am sorry, I am unable to get the weather in Tampa at this time. Would you like me to try again or is there anything else I can help you with? No, that's it. Thank you. Cheers. Happy to assist. Have a great day. Cheers, mate. Bye. There we go. So now if we go back into our individual flow, this will trigger the. The event as long as I put the correct. The correct URL in and once it does, once it executes the workflow. Have a look. Once it executes the, the workflow, it's going to give us. It's going to give us. It wasn't that long. So let's go into this. Listen it, right, we've just learned another lesson and I hope this is why my content comes across as raw. It's exactly what I want you to do. You want, I want you to learn with me. The response was post. Well, this needs to be post as well. You can't have separate unmatching webhook calls and analysis from each. Each side. They need to speak to one another under the same thing. That's exactly why that did not work. Now I hit save. So if I hit command S or you just hit the save button at the top. And I now execute the flow and I now start the conversation. Hello, how can I help you today? What's happening, mate? What's the weather like in London today? I've kicked off a search for the weather in London for you. I'll let you know as soon as I have the details. Amazing. Okay, I'm just waiting for the results to come in. Is there anything else I can help you with in the meantime? No, it's absolutely fine. In fact, I've got a go, so if you can end the call, that'd be great. There you go. He ended the call. He didn't say that he was going to end it or buy. But the point is, is that we've got the data now inside of this webhook, so I'm going to pin it and if you look at this, you will see that it gave us the output of what it was, it gave us the search query and it also gave us the webhook URL, which is, which is absolutely critical. So that search query is what we're going to use inside of Perplexity. So Perplexity is going to be the next, the next tool that we use. I'm not going to go through setting up your Perplexity account because it's. They're all very, very similar. Literally go to the platform, head to the settings, you know, and pick that. I'm going to change the model to Sonar though, and the model might be, might be required differently depending on what you are, what you are building inside of There, but very, very similar to earlier. If you hit the schema section and you find the search query, all we've got to do is drag and drop that in there and then you can do it. Just in case you want to know how. If you are, if you are fixed right, you can, if you press equals it will change it to an expression and you can actually write it yourself. So you could, you know, JSON.body.search.search query. You can do that. But dragging it over is by far and away the simplest way for us to be able to do it. So drag it over. We want to simplify the output and we want to execute the step, see what happens. Today in London, the weather is becoming cloudy with a brief shower or two expected this afternoon. If we look at the JSON, it's not because I gave it a system prompt inside of the agent. It's not that long. I mean, but who really cares about the exact degree? We can, you can mess around with the system prompt, maybe, but let's, let's just, from, from there, let's just, let's just add, let's just add an AI agent. You don't need it. We, we can literally put. Post the response to the, to the webhook. In fact, let's just do that. Let's go webhook. Respond. Respond to webhook. Now, we want it to be the first. We don't want it to be the first incoming item. We want it to, to be the. Is it the first incoming item? It is the, it is the first incoming item. So I think that we would probably get away with using first incoming item. Here, you can change this based off of what, you know, what you have. So like, you know, if you wanted to like, have, you know, respond with a custom JSON JSON body and then, you know, copy this and paste it in there and have the JSON body as the message. You could do that. But I think from a perspective of what we're actually trying to do, I think, I think the first incoming item is going to give us what we need. So let's come out of here, command S and let's see, shall we? I'm going to open this data, I'm going to execute the workflow and I'm going to go in the chat. Hello, how can I help you today? Hi, Ryan, can you tell me what the weather's like in Manchester today? I am sorry, I'm having trouble getting the weather for Manchester today. Please try again in a few minutes. In the meantime, is there anything Else I can help you with? Yeah. Okay. What would you like help with? Can you. Can you tell me. That was smooth. Can you tell me what the. It doesn't matter. Ryan. Thanks. Cheers. And the call webhook node respond parameter using webhook node. There we go. Right, so exactly what's happened here? This has been a very, very good video for learning. If you're learning alongside me, because this webhook, right, this failed. This would have worked and it will work in a moment. Same thing again. But what I forgot to do, which I actually referenced earlier in the video, is we have a. We have a response to webhook. Okay. But in the middle of that we have a perplexity search which is actually going out there to find the information. And even though it's quick, it's relatively quick for the basic questions we're asking by having the webhook set to respond immediately. If we use the respond to webhook node, it will wait until it has all of the individual information before it does that. So change that to respond to webhook node command S. Let's save it now. Let's execute the workflow again and let's go and try and start. Why are you listening, brother? Let's refresh. Hello, how can I help you today? Hello, mate. What's the weather like in Texas today? The weather in Texas varies by region. Today, Southeast Texas, including Houston will be mostly dry after storms, with more expected on Monday. North Texas around Dallas Fort Worth may see thunderstorms and near normal temperatures. Central Texas, including Austin and San Antonio can expect scattered thunderstorms in the late afternoon and evening. Is there anything else I can help you with? No, that's great. Thank you very much. So you can see now how we've just created a two way AI agent. Now if we wanted to add something in the middle of here, we could add another AI agent which, because that was very long. Okay, it was very long. So what we want to do is like, we want to. We want to base it off of the message again and we want to add a system message. You're a helpful assistant that is designed to provide clear and concise responses to clearing. Clear and concise. Can I not spell concise responses to. I can't spell concise responses to. To. To search queries. Your job is to strip the search query down to a maximum of three sentences, never exceed this. We want the answers to be factual and sharp. Okay? So let's just come out of there, give it a brain, because it does need a brain. So I'm just going to use open AI for this. 4.1 mini. That's. That should be. Yeah, let's just leave that like now I'm going to hit command S again and we're now going to execute the workflow again and I'm going to start it again. Hello, how can I help you today? Hey Ryan mate, what's the weather like in Skegness today? The weather in Skegness today will be warm and humid starting with sunny intervals before light rain develops. Late morning, the maximum temperature will be around 29 degrees Celsius with easing winds. Later the night will be mostly dry and warm with clear spells. Okay, is there anything else I can help you with today? No. Do you know that. Do you know the time in London now? I am sorry, I am unable to retrieve the current time in London at the moment. Is there anything else I can help you with? No, Ryan, that was perfect. Thank you. Happy I could help. Have a wonderful day. Amazing. So you can see how we have just created a two way AI agent that is a conversational elevenlabs voice agent that you can tailor to your own individual voice. That is incredibly powerful. Take this away, do whatever you will with it. And if you really, really, really want to dig deep into this, drop me a message in the comments and I'll build out one which is specific for a business so that you could see how it would actually work. I will make the phone call live. It depends on your interest on this. So you guys need to hit me up and let me know. Cheers, thanks for watching.