Everything You Need To Know About AI Voice Agents in 2025

Hey everyone. Welcome to today's show. Today we are talking about the rise of AI voice agents and the reason is we are about to have a watershed moment in AI voice. Amazon just released the new version of Amazon Alexa that is much smarter and kind of brings Alexa into this post AI world and it's going to open up a whole set of use cases around AI voice that people in society just weren't comfortable with before. And we're lucky today to be joined by Flo Crivello, who is the CEO and founder of Lindy AI and he's going to break down all of the world of voice AI. How to actually build. He's going to walk through and show us how to build an AI agent who can book appointment, take inbound phone calls for your company and make you real money quickly. And this is going to be transformational to any business who's looking to scale and might not have the time to just pick up the phone all the time. You all are building a platform for people to make AI agents, but voice agents are a huge part of the work that you're doing and you probably know more about AI and voice than 99.9% of the humans on the planet. So I guess probably the right place to start is like, how are we going to be using these AI voice use cases and what are you seeing companies do already?

We all seen an explosion of use cases and usage around AI voice agents because it is truly a new capability. Like having autoresponders to like emails and having like chatbots embedded on your website that could respond to messages. Like that's been around for a while. It wasn't good, but it's been around for a while. Like having automated phone calls, that's not really been possible until now. Like the only capabilities have been like press 1 to do X and like that was just like so bad that like companies didn't even do it. Whereas now all of a sudden there's been a quantum leap where you went from basically having nothing because especially small businesses did not do this whole phone tree thing to having something actually really good. And we're going to demo today. The kind of agents that you can create in five minutes is just mind blowing. So I just think companies are going to stop being on the phone. I think that's the other reason why AI voice agents are having such a moment is because every minute that an AI voice agent is spending on the phone is a minute that a human is not spending on the phone. So there is a one to one ROI here They're basically blue collar workers. Right. We're having restaurants as customers, we're having plumbers as customers. And you're a plumber, you're busy plumbing all day. Yeah.

You don't want to answer your phone. Right.

And so every phone call that you're missing is business that you're leaving on the table. Right. And so for them, the ROI is instance. Like, look, I'm plumbing. This thing is picking up the phone for me. It's making appointments. And it's just like it needs to book one or two appointments a month for me to realize an ROI on this. I just think like, the funny thing is that businesses like receiving phone calls no more than people like making them.

And I think there's a lot of people watching this show probably don't yet understand what is possible and what you can really do with the technology where it is today. Could you maybe walk us through some use cases, like an education session for us of like, if you're a business out there today, how can you actually use voice agents today?

I'll do a live demo because I think that's just the easiest way to really write.

Yeah, I love, I love a good live demo.

Lindy is the name that we give to our agents, AI agents. And I'm going to go ahead here and create a new DND from scratch. So I'm going to call it the Marketing against the Grain Lindy. And let's suppose that this Lindy, her job is you have ingested a new lead. So a lead has asked to receive a demo and to book some time. So pretty common B2B SaaS use case and we are going to make her react to that request. So for now I'm just going to make it so that I can chat with this Lindy and tell her about this lead. But you could hook her up to Google Sheets, for example, or you know, we've got Typeform and Slack and Stripe. We've got all sorts of integrations. I'm just going to make it possible to chat with this thingy for now and I am going to plug this thingy to my calendar. So I have a Google Calendar action here and I have an action that is find available times and I'm just going to make a find time for a 15 minutes meeting. And now I'm going to make a phone call. So I'm just going to pick one phone number that I have provisioned on my account and I'll pick the language here and here you can pick the voice. I like Archer. Hey, There.

My name is Archer.

Oh, I like his accent. We got a little Aussie, don't we?

Yeah, yeah, I like that. And when the call begins, I'm going to make it an AI agent. So we'll model agnostics. So this is where you select which model you want to power.

Oh that's sweet.

Models have trade offs in terms of intelligence and speed and speed method. A great deal in phone gold. I like using GPT4O. These names drive me crazy. It's just insane. Like so like the November 2024 I guess is the one that I like using better.

4.5 will be good once it gets cheaper. But 4.0 is a great example of a model for this use case.

4.5 is excellent but it is incredibly expensive.

So expensive.

4.0 mini and Gemini flash are also good. Gemini 2 flash is really good. Not quite good enough for fun gongs though. It's kind of dumb sometimes. I'm going to prompt my agent, I'm going to transcribe my voice here and I'm going to tell it. You are a phone voice agent which job is to book a demo with a prospect which information you just received keep your adherencies extremely short. People hate when you ramble on the phone and offer one availability at a time, only offering the next one if the one that you already offered doesn't work for the prospect. Once you have collected an availability that works for the prospect, also ask them if there is anything in particular that they would like to cover on the demo.

It's a basic scheduling agent, right? Somebody's going to call, they're going to try to find a time and they're going to give you a little bit of context as to why the heck that call's happening.

That's exactly right. That's exactly right. So you can see here and I think this is the thing that people underestimate. Like I've basically created my AI voice agent right here. It's full steps, it's like two minutes. And the prompt people always ask me like, oh, do you have any advice for like how to prompt engineer? These things I think like speak naturally like I just did with my disgusting French accent and it just works like Phil Shop.

Prompt engineering is really just like a proxy term for like do you understand the problem you're trying to solve and can you articulate it clearly? Right. And now the models are so good you don't have to format it in specific ways and be super detailed. But you, you do have to know the problem you're Trying to solve and like the specifics around the problem you're trying to solve. Like your prompt was a great example. It's like I'm trying to get people to book a meeting with me. I only want those meetings to be 15 minutes and I want to know what that call is about.

Right.

But if you don't go in knowing that it doesn't matter if it's a perfectly engineered prompt, it's not going to work.

That's exactly right. That's exactly right. I tell people to think of these agents as like an intern. You are onboarding a very eager Internet. Not a ton of experience, not the smartest guy ever, but very eager. And if you explain to him exactly what you want step by step in simple terms, he'll get it and he'll do it. I'll add one last thing here, which is at the end of the call, I'm going to make the Lindy actually create the event when I allow her to schedule conflicts and then I'm going to make her send me a message back with the information you see here. You will notice as I'm creating these actions, I'm actually not even configuring them. I'm not telling her anything about the message to send me the title to give to the meeting. None of that. I can if I want. If I want I could like hard code the name of the meeting to be like demo called. But I'm not doing that. I'm just going to let it to auto. And that's the beauty of AI agents is like they can figure it out. Right. So now let's run a test call flow at. This is my phone number. I'm wondering if I should just ah, whatever. This is my phone number and his email is flowindy AI. So I'm going to receive the phone call. Hello?

Hi, I'm calling to schedule a demo. Are you available today?

No, I can't meet today.

How about Monday morning?

Yeah. What time do you have in mind?

9:00Am does that work?

Yeah, 9:00am works.

Is there anything specific you'd like to cover during the demo?

I'd like to talk about voice agents and how they can help my business.

Got it.

I'll make sure we cover voice agents and their business impact. See you Monday at 9:00am thank you. You're welcome. Have a great day.

That's. It took two minutes. This takes about like one more minute to execute. So after call ends branch, we're waiting for a callback from Twilio but you'll see after the Call ends. It's going to create the calendar event that we just agreed upon and then it's going to send me a message back to be like, hey, we did it.

Hey everyone, I want to talk about prompting with you today. Being great at prompting is like, gives you superpowers. It changes how you can build and work and do everything with AI. And because of that, we wanted to make a special guide just for you around how to get better at prompting for how you actually build prompts. And so we've got this baseline framework that you can then go and build and customize any prompts that you want to. You can check it out in the link of the description below. And right now we're gonna get back to today's show. Yeah, what you're really walking through here is like this is kind of a universal problem most small, medium sized businesses have, right? Once you get to an enterprise, yes, this problem exists, but it's way more complex. But the thing that actually matters here is, is the, is the coverage. You know, there's still a massive amount of these phone calls are just getting unanswered, right? Yeah, I guess my, I guess my question, because I'm going to play a little bit of like the, the listener, viewer here. What if it's not as straightforward? What if it's not just like a demo call, right? Like, what if there could, what if it's a more general purpose phone number where it's like maybe they have a customer question, maybe it's a demo, maybe it's something else. You know, I'm thinking the analog of like, you know, you call a phone number and there's like five or six options when you, when you hit the buttons, right? Like what's the modern post AI equivalent of that?

Now you could just have, I mean I could give a demo right now. Like you could just have this Lindy receive a phone call. This time I'm going to make her in English, I'm going to make her Eric's voice and I'm going to make her enter her voice agent here, which is again, I like 4o November. And I'm going to be like, suppose I'm a restaurant, right? So you are the receptionist of a restaurant. You are fielding phone calls. You help people book reservations and you help people inquire about business hours. About business hours or 12:00pm to 9:00pm, Monday to Friday, just making something up. Obviously if you want to make reservations, you would need to connect it to your reservation system and so forth. But right here, now I can make I can make this phone call. I'm using this example, by the way. You would be shocked. Full restaurants receive an enormous amount of phone calls at peak hour. Right. And those companies, like, that's the problem.

Right. These restaurants are getting.

That's right.

Called when they don't have any time to answer these calls.

That's right. And half of these calls are just, are you open? It's like, yes, we'll open. So. Hello. Hello. Thank you for calling.

How can I assist you today?

Yeah, I'm just calling to check if you're open.

We're open Monday to Friday from 12pm to 9pm Let me know if there's anything else I can help you with.

No, that's it. Thank you. So in actuality here, what I would do is I would prompt it to be like, use the current time to tell them if you're currently open and so forth.

Right.

This is a simple example, but I think that gives you a taste of. It's just prompting. Just prompted to do it.

Yeah, it's just prompting. And then it's a little back about going back and looking at the interactions that you've had and adding new prompts and new instructions. Right. Like that's. You're just kind of building more logic as you determine that your customers need different stuff.

That's right. You can also make it also just like transfer the goal. Right. You could just transfer the call is like if the. If the person seems pissed on the line or if you're lost and apparently you're not helping, just transfer them to me.

I love that. So you obviously have people using this, and it's a. It's a live product. People can go and view. What have you learned about people's willingness to interact with an agent on the phone? How do people. Do you have any issues? Consumers like it. What's. What have you. What have you taken away? Like, what's the data say?

It's definitely working. The ROI is tremendous. But I will say, like, this demo skill that I'm giving, I'm speaking to the agent in a way that I know it will understand. And in particular, one thing we are finding people have to adjust to a little bit, at least in these early stages, is you've got to be mindful of the latency. Latency right now is the name of the game for these AI voice agents. We're doing an insane amount of work to reduce this latency to a minimum. So, for example, we do this thing we call speculative generation. So behind the scenes, we are continuously pretending as if the person stopped speaking and we're generating the response. Right. And if we detect that the person actually stopped speaking, then we had started generating the response 300 milliseconds ago. So right here we just shaved 300 milliseconds. Right. So there's a lot of tricks like that that you can do to shave some latency, but there is always latency that remains much more than for humans. Another thing that these AI agents are still not as good as humans for is interruption. Like when it's in the middle of talking, like if you try to interrupt, it's not going to be as smooth as a human. So, you know, we are finding there is a little bit of an adjustment here to be made. But by and large, I mean, you're seeing the experience, it's quite good. It's really almost as good as a human.

Yeah. What we're saying here is the contextualization is pretty good. It's the social norms like the interrupting and the latency that's the challenge. And as somebody who's used voice agents, who's used video avatars, latency is the issue. It seems like that's a solvable issue. How far are we away from that not being the primary issue here, you think?

I would say the current latency is like 7 out of 10. I think a year from now is going to be 8 out of 10, 8 and a half, I think. If you want 10 out of 10 latency, I have a hypothesis that it's going to require a re architecture of the models. It's going to require full, what we call full duplex audio in and out models. So today many of these voice agents, they're actually not audio models. They're still text based models. And you're adding on top of that the transcription model to turn the audio waves into text and then the text back into audio waves. Some of these models actually do accept audio tokens, not all of them, and even those are quite immature. But even those that do don't accept full duplex audio tokens. So they can't receive tokens at the same time as they can send tokens like a human can. Like you have. You can use your ears at the same time as you can use your mouse. Even the native audio models can't do that. Yet there are some labs that are working on full duplex audio models, but I think it's at least a year away maybe.

Okay, so that was honestly just a bunch of personal curiosity for me because I see so much potential and I'm So excited. One of the reasons we're doing this show is because I think this type of automation and support for businesses, especially small and medium sized businesses, is so transformative. But we there is some like uncanny valley, I do think at like 8 out of 10, like you said, where you thought we'd potentially be in a year, that does work for the vast majority of use cases. Not for everything, but probably for like 80 plus percent, right?

100%. 100%.

Okay. So that to me is exciting.

Yeah, yeah. We never jump on a call or I would say almost never. Never. I think maybe it's happened with 1 or 2% of our deals. When we jump on a call with a prospect and they tell us about their need and we show them what agents can do, it works, they're happy. The reason why a deal may not go through actually the vast majority of the time has to do with internal reasons, not with the state of the.

Well. Yeah, I think the internal reasons is interesting. That's kind of why I was asking about how, how people are perceiving the technology from the consumer side. Because that's, I imagine if somebody's watching this and they're like, oh, this is pretty cool, I'd like to run this for my business. But like my co worker is gonna hate this or my boss is gonna hate this because they're gonna think it's a bad customer experience to not have them talk to a real human. Which I don't agree with personally and I think that in many ways it's a way better experience. Cause you're not going on hold. You're getting straight to the problem you're trying to solve very quick. I saw that one of the things you can do is like connect your knowledge base so you can get basic support questions answered, which is super valuable.

Right?

Like how do we, how do you think about like the consumer side of this and the objection, like those objections make any sense or is it like they're just kind of stuck in an older way of thinking?

Yeah. The bigger issue we're making is not even an objection. Like most of the time people are in, the bigger issue of meeting is technical. You need to integrate with your internal systems. And it's only once you try to integrate these things into your internal systems, especially big and large and old organizations, that's when they realize that their APIs are not that good. So we try to integrate within these internal APIs and we're like, it's not documented. It errors out half the time, the error codes are not clear. And so there's actually a lot of iteration model. Like, look, the platform works. It's just our API is not what we thought it was. So now we need to fix the API and expose more information or less fix the error codes and so forth before we can really get this thing to work.

So it's really the technology side. And if you're in technology at all and you're watching this, integrated with phone systems has not historically been an easy thing to do. And the back office systems and phone systems is a little challenging, but I know we'll get, I know we'll get through that. So if you were, as we kind of, we went through the demo, we kind of understand the use cases. If you're out there as somebody who is the top expert, like somebody who really knows a lot about this, what advice would you give to the businesses out there who are trying to scale with AI, with voice AI in terms of, like, how they're interacting with their prospects and customers? What would you tell them?

I think the biggest advice I give to people is start small and start fast. I love that we jump on calls all the time and people are so excited. And look, I feel the excitement. That's why I'm working on this now. Like, I want to do this, I want to do that. And even myself, I have to sometimes hold myself back. I'm like, okay, let's do all the things, like 100%. Let's start small, though. Like, what's the one problem that you want to do? And once they've defined that, even then very often we have to smaller, smaller, smaller. Right. Like, let's really, let's get a win on the board like tomorrow, right? And really start to build the muscle memory, start to build the expertise with these internal systems and build momentum like that, even internally. Right. So get to a point where you're like 24 hours later, I've got something that works and delivers value to my business and then iteratively add to it little by little by little by little by little.

Yeah. I think the takeaway for me here is like, is the technology perfect yet? No. But is it by far good enough for most use cases? Yes. So don't worry about that. Don't over complicate it and start getting some of that basic value, even if you're only using it for some very basic use cases, as we've talked about, it's a big free up of time and improvement of customer experience. Because the last thing I want to do is just sit on hold for 20 minutes waiting to talk to a human. I literally can't think of anything worse than my day. And we're trying to ease that up. Okay, so that's your advice? What are the biggest mistakes that people can make when they're trying to get into this part of AI?

The field of AI agents is guilty. A little bit of like hucksterism.

Little overhyping maybe.

It's ah, you know this is, this is the greatest thing ever idea is here. You can do everything. Look, the reality of it is you can't do everything yet. And I don't blame the customer for not understanding yet the limits of these systems because they will sold the moon. It's like ah, they can do everything right. And so sometimes you jump on these calls with people and it's like I want, I want like an assistant. I want like a cmo, an AI cmo. I want like an AI Director of demand gen. It's like look, I want that as well.

We all want it.

And so I think just like we tell people to start small, we also like part of our job when we jump on these calls with folks is to educate them on the current state of the technology. And so today these agents are really good at task level automation. Not job level automation. Not yet. There is a roadmap to get there but we're driving towards it. So I'd say that's, that's one big mistake we're seeing people make is having some realistic expectations about where the technology is.

I like that. Task level automation versus job level automation framework. Have there been any other like kind of frameworks that you've used around these agents that are just really helpful for us to all understand them better?

I really like the image of the Intel. Yes. I think the intern is good because it's like you wouldn't ask your intel to like own an entire dimension program. You asked your intern to go grab coffee. You asked your intern like a very, very discreet, simple thing.

I like your eager intern because it's also, it's like it's going to do it and it's going to do it fast and unabashedly right? This agent.

I'll also say the beauty of machines in general is that they are obviously much cheaper and obviously much more scalable than humans. So I always use the analogy of the dishwasher here. It's like these thresholds are actually slower than humans at washing disease. But who cares right? It's like idle most of the time anyway. So it doesn't matter that it takes a Long time. It's not my time. And so I think here it's the same thing. It's like, look, even if it was slower than you, it wouldn't matter. And sometimes it is slower than you. But most importantly, it is infinitely horizontally scalable. There is this Lindy that I love showing off. That's. I think it's called the Lead Researcher. Right here I have this list. I have the CSV here of unicorn founders, and I can drag and drop a CSV here in the Lindy, and it's basically going to create one task per row in the CSV. So this CSV contains a bunch of contact information of unicorn founders. And now you'll see the beauty of it. So this Lindy's job is to research each of these leads online. So I can open any of them here. Like, there's a ramp right here. There's a sana. So it's researching. It's researching the company and the founder, and then it's writing an outreach email. And so you can see here, in the span of literally what, 30 seconds, it wrote 50 outreach emails. This is like a day's worth of work for, like a rep. So to me, that's the other amazing thing. It's like, don't try to replace like a full human. Identify the task of a human that can be cleanly encapsulated like that in an AI agent and then just crank the wheel. Just like gets this thing to do it at scale for you.

I love that. And that was kind of the theme of today's show. This is like all about scale. And that AI continues to be a big unlock of scale. And the customer service part of this, the rep part of this, those are gonna be two ways. Those are two places where companies spend a lot of time and a lot of money, right? And to get more efficient. And the other thing I would say my observations are kind of. My last question for you before we go close out today is like, people often think about AI and agents as being way more efficient. I have found them to not just be way more efficient, but to lead to way better results as well. It's like you get both. Have you seen that same thing?

I would say they're like, way more consistent, for sure. So they may not get right first shot. But for example, customer support automation is another one big use case of ours. And so, you know, look, it's just like a human, right? You onboard it. I would say actually that is another big piece of advice that I give to people is like, be patient I'm always surprised the difference in expectations that people have when it comes to onboarding AI agents versus onboarding human coworkers. You onboard the human coworkers, you're fully prepared for them to be sort of like ramping up for the first couple of weeks. You onboard an AI agent, you expect it to work first, chat within the first hour. And so I'm like, be patient and be ready to iterate on your AI agents continuously. The good news is that once you've iterated on it and once you found the right prompt and the right instructions and so forth, and then you just know it's going to consistently follow the prompt and it's going to scale to infinity with it. So for sure, I think the consistency has been a huge plus.

Perfect. I love that. That is, I think, a great note to end on. I really appreciate you actually showing us the tools today and what is possible because I think we're all trying to learn where we should really put AI into our business. And the customer support phone use case is a really great example that works in the here and now. Flo, I thank you so much for joining us on Marketing against the Grain. We're. We'll see everybody on the next show.

Thank you so much, Gabe. This data is wrong every freaking time. Have you heard of HubSpot?

HubSpot is a CRM platform where everything is fully integrated.

Whoa. I can see the client's whole history. Calls, support tickets, emails. And here's a task from three days ago I totally missed. HubSpot Grow better.