The Future Of AI Video And Content Creation With Joshua Xu | INBOUND 2024

Foreign.

Wow.

Thanks for coming to Inbound and seeing us, everyone. We normally do this in our basements, so this is a little bit different than doing it in a little room. But we're super excited to be with you all today. I'm Kip Bodnar. This is Kieran Flanagan. We host a podcast called Marking against the Grain and we talk about marketing and AI and technology and basically practical things everyone can do to build and grow their business. And today, Kieran, we have an incredibly exciting topic and an amazing guest. Could you set that up for us today?

So we made some predict predictions in January and we said that this year was probably the year of video. AI Video. We thought it was going to be an incredible year for AI video. Now you could argue that it is the year of AI coding. We've seen a lot of evolution. There's. But there has been so much transformative things that have happened this year and we truly believe next year is going to be an incredible unlock for AI video. So we had to get one of the world's leading experts in this subject to come onto the podcast to come to Inbound and tell us all how that is going to happen. Welcome, Joshua Hu, the incredible co founder of Heygen.

Welcome, Josh. Sam.

Hi, I'm Joshua and I'm thrilled to be on this week's episode of Marketing against the Grain with Kip and Kieran. Hola. Soy Joshua. Y estoy encantado de estar en el episodio de esta semana. The marketing against the grain con keep I quiran.

All right, welcome.

Awesome. Thank you.

Thank you for having me.

So Joshua, one of the exciting things I get to do outside of my day to day role is a bunch of angel investing. And so I get to meet incredible founders who have incredible ideas. And one of the questions I love to ask is why this problem? Like there's a whole array of problems that you could dedicate your life to solving. Why was this the problem that you wanted to solve?

Sure.

Yeah. So thank you for having me. So we live at Video First World. There were a billion hours of video being watched on YouTube every day. And the average person watch 17 hours of video every week. So video, just like the most engaging format, you know, but. And every company wants to make more videos. But the problem is that video is extremely expensive to make. And you know, if you look at the traditional video production pipeline, it involves like scripting, scouting, production, post production, localization, or we shoot if we made some mistake. And the industry rule of thumb of every one minute video, it costs about $1,000, right. So I think, you know, we found that. All right, if we can remove the barrier of a camera, we would remove the barrier for visual storytelling for businesses. So that's why it comes down to like the AI generation solution. And you know, with AI, we can really generate a video, you know, 10x faster, 10x cheaper without using the camera. So that's why we built hey Gen is an AI video platform that help customer to grow by making AI video and making a video much faster. We help customer to create video from text, localize the video into more than 175 languages and personalize them at scale again, you know, in the video. First of all, our mission is to help the business grow by making visual storytelling accessible to all.

Awesome. I think that's one of the amazing things about AI. It allows you to go from idea to realizing that idea so much quicker with tools like. Hey Jin. One of the things we love to do, Joshua, in this podcast is we love to demystify this for everyone. We like to turn it into actionable takeaways that they can all go away from this talk and actually start to implement straight away, hopefully right here in the conference with the new Heygin and HubSpot integration. But we are going to go through three core use cases. How you can use hey Jin. How you can use AI video in your business right today. And Kip, you're going to kick us off with the first use case. Yeah.

So what we want to do today is say, hey AI video is not this thing you just see random clips of on the Internet. It is very practical and applicable to all the work we all do. And the number one use case that we've started to use at HubSpot is localization. So you saw in that intro video from Josh that he, he welcomed being here in both English and Spanish. Josh, I don't think you speak Spanish. He's like, no, actually not. And he just said, hey jin can do 175 different languages. What we found at HubSpot is we go to market in eight different languages. And it was that exact problem. We were spending five to $10,000 to take a video from one language and recreate it, make it work in a different language. And that would take us two to three weeks to kind of do that start to finish. Like your point, Josh? It's like takes a little bit of time. And then instead with Heygin, we can now do it in one to two days. It costs us about $500. Cause we have a human review and do some light editing to that process. But it's way, it's, it's 5x faster, 5x cheaper and the viewer engagement and performance of those videos is the same or better than the old way we were doing it. And I think we got a quick video to kind of show the localization. So let's, let's roll that. And then I have a couple questions for Josh.

Searching for your ideal hotel?

Do you have tons of tabs open.

Because every site has a different price.

For the same room? Well, Trivago does all the work for you.

So you have here a commercial for Trivago that started in English can have the exact same commercial because the same value prop. It's a, it's a trip booking service. It doesn't change in France versus the US it's pretty universal for them. And they could make that one ad multilingual extremely quickly. And so I'd love Josh, for you to break through how have you seen companies using multilingual AI dubbing, maybe explain the technology and talk a few examples for us?

Yeah, sure. So, so first of all, you know, the entire translation process is extremely simple. You know what we need to do is that you upload the video, select the target language and click submit and the video will be ready in just like one minute. So we have seen customer use it in two ways. One is that customer taking the existing content. Either they produce it in English and then they can translate that and localize that into different languages and use that to open up a new market and reach the new audience in different part of a region. And I think over time we are also seeing customers every time they're filming, even they go to production of filming new videos. They only need to film it in one language. But I mean after that, you know, a multiple version will be ready. So we have seen customer like, you know, Trivango is a great example, they use it for advertising. And we have seen customer like Sinclair Broadcast group usually for tennis.com and many, many small business and companies are using that to localize the content for their new audience.

Yeah, like at HubSpot we use a lot for HubSpot Academy where a lot of our like educational videos where we are trying to teach customers the same thing, but we're trying to do it across all these languages. And I think that's the use case that exists. Sure you're always going to have cultural nuances and you might make ad content or marketing content different for certain markets, but there's always going to be some that are universal across those markets. And it seems like that's the best place for everyone to start. And even maybe simple video ads are also a really good way to start. Is that how you're seeing most people use the product today?

Yeah, yeah. Like, like externally, like, you know, product screener advertising, you know, and even we have seen lots of customer, like just localizing the asset for local field marketing and as well as some of the product announcement videos.

Yeah, I think that's the other good point. Like Josh brought up field marketing, I think historically with video. One of the problems that I've always had with video is that because it's so expensive and takes so long, you only use it for your highest value things, right? You're like, okay, this is a big campaign or a big priority. I can have video in it now. And what we're saying is, like, now you can kind of have videos in everything. You can have a video that maybe you made for one market and you could localize it and have it for your field marketing and sales teams and maybe an emerging market where you don't spend a ton of marketing or sales dollars, but you still want that, like, local experience. Where in the past, candidly I would have been like, well, you either just have to use the English version or you just don't have a video like everybody else, right?

Yeah, exactly.

Yeah. It's pretty magical that we live in a world where you can build a company and you can go international from day one, like you can actually start to internationalize your business. I want to talk about a second use case that I find really magical. And so the second use case I had my first daughter in January. I decided at the same time I was trying to be a new parent, I would spend my nights releasing an AI course. And so I released an AI course where I had AI create the course and then me as an AI avatar using Heygen and HubSpot at that time, before the integration actually teach the course. And so I used Heygen to create an avatar of myself. And I'm just fascinated by how you can personalize video at scale. And, you know, one of the things where all of us who have kind of worked in B2B for a while, one of the more magical things is when we could use our CRM data to dynamically change an email to. To personalize it for a segment of people or even an individual. And it's incredible to think that AI is going to do the same thing for video, right? We're actually going to be able to use Our data, use our CRM data and personalize video for segments and groups of people and even individual people in HubSpot. We're going to use the integration to actually start to create personalized video in our outreach campaigns to be able to get more meetings booked, be able to reach out to people and in that kind of way. And so I would love to hear what are some of the best use cases you've seen in terms of company using personalized video or even hey Jim themselves and how you've used that technology to grow your business?

Yeah, sure, 100%. So let me start with an example at HeyGen first, you know, we have this tradition that we have been running, you know, a weekly webinar every Thursday morning. So every time a customer sign up on the webinar, they will get automatic email from us. And with me inside the video, we say, hey Kieran, thank you for signing up the webinar. We are looking forward to seeing you on Thursday. So that's one way for us to build that customer connection very, very early on. And you know, they would get a video within two minutes as long as they sign up. And we also, we've seen customer like, you know, sharebird, one of the largest product marketing community, they use the personalized video for their events. Similarly, when the customers register event, they would get an email from speaker and with a video sending a greeting message to them. And we are also seeing, you know, company like publicist group, they're leveraging this technology to build the avatar for their executive teams. And they're sending over 100,000 email and with the thank you video to their employees. So another example would be Ogilvy. You know, they're leveraging the technology to help catalog basically to spin up the, you know, personalized video campaign for their serial product line. Yeah. So I would say this would be the way how we are really looking at it is that, you know, every single piece of the personalized test content, mostly around the email format, can be turned into a video. And with the automatic, you know, API and pipeline integration, we can really turn that test content to video content that connect with the customers. And what's even more, I think is that, you know, we can also put that particular, you know, let's say I'm reaching out the customer, I can use my avatar. If my sales team, our customer success team, they can use their own avatar to really build up that connection early on.

Yeah, it's creating that course is when I, when your message or your mission kind of rang true. Make the camera Obsolete because It would be 11 o' clock at night. I should be in bed trying to get sleep. As a new parent, I want to create a new module in my course. I don't want to film myself because I look a mess. And so I have a transcript, and I can then just upload the transcript and create the perfect video of myself. The other examples you've showed is that when a customer is going through some sort of onboarding process or they're in your sales sequence, you can now gather data and make that a more human experience by creating that kind of personalized video for them very, very early on. One of the things we talked about backstage, I'm curious to get your take on. This is okay, we, the technology is there. The technology is getting better to integrate personal video into what we do. Where do you think we all are in terms of how we interact with that personalized video? Like in terms of customers, how do we feel about getting these kind of personalized videos? And are people freaked out by getting Avatar videos?

That's really what you're, you're like asking, come on, man.

I said backstage, I said backstage, are people freaked out? And I was going to like conference.

Speak it and then conference speak it. People are freaked. Do people get freaked out getting these super personalized videos?

Yeah, I mean, so the way how we look at it is that there's definitely an adoption curve for any sort of new technology. But by its nature, when we talk about video generation, it's really a way, a means for us to like, you know, deliver the content, deliver the visual storytelling. Right. So I think, you know, when we talk about the early days on, you know, we can really personalize in the automatic way, say, you know, put a name on the email. That's one way or one sort of way to auto automate the workflow. And we are just taking that workflow automation to the next level. On the video level, I would say, you know, this would become a new norm for the business communication.

Yeah, I want to jump in here. We've mentioned a couple times the analog to personalized email and email tokens. And I want everybody to understand that. I actually think that's a bad comparison. And the reason is with email personalization, you would write one email and then you'd have some tokens throughout that email that would call on data and personalize that specific email. For AI video, the video is just generated from text. So you can have a thousand videos that all of those videos are completely different. It's not like there's a master template to those videos. Right? Like, you can. You can basically have a prompt that generates a completely unique video per person, which I think is part of why you're asking the question, are people going to get freaked out? Because it's not just like, hey, Kieran, it's like, knows deep. You can have a deeply personal contextual video experience.

Here's a thought process I was going through in relation to this, where you could have a avatar today trained on every singular piece of data the company has. So it has all of the calls that your sales team has done, all of the support tickets, all of your product knowledge, and you can join a call and talk to that avatar, and it can perfectly answer all of your questions if you're a prospect and you're trying to decide if you want to buy that product. And under the opposite side, a human will answer those questions, but will be informed. Inferior in terms of knowledge, but they are purposely inferior because they are trying to, like, gravitate you towards the next stage of that sales cycle where the AI avatar is actually just trained to perfectly answer your question. And so there's a way to learn how to integrate these things, like, into your customer journey in a way that, that makes sense. But I do, I think I agree with what you said, is it's a new type of content format and a way to bring text to life in better and more interesting ways. I think there's a ton of creative ways that everyone here and all of us can actually start to use that within our own business.

Yeah, 100% AI video is just way.

More dynamic than text. Is the point that I was trying to get much more creative on this one. Okay, we want to. I want to talk about the next use case.

You ready?

You ready to move forward?

Let's do it.

Okay.

Let's do it. Okay, so use case number three is we've talked about these AI avatars in terms of a couple of things. Localizing your video as well as personalized video. But they can also act as kind of a reactive concierge, and you can have a personal concierge that can triage questions. Kind of how I think about this. When inbound marketing started, people would come to your website and they'd fill out a form. Then a little while later, people come to your website and they'd interact with you on chat or have. And then a little while later, you had chat automation where there would be a basic bot or AI that would provide a little text. Now we're talking about having an AR avatar that's audio and video on your website that can interact with you in real time like a person could. And I think we've got a video from our friend Ryan Hoover who can kind of demonstrate that for us. So let's play that and then we'll get into it. Hi, I'm Ryan Hoover.

Nice to meet you.

Hey, Joshua. Great to hear from you. At HubSpot inbound, I'm Ryan Hoover, the founder of Product Hunt, a platform dedicated, dedicated to discovering and sharing new technology products. I also co founded the Weekend Fund where we invest in early stage startups. I'm passionate about community building, innovation and exploring emerging technologies.

How's the event going so far? Event's going great.

So what you see there is Josh is able to interact with this avatar, this demo via text or audio. And it's not just that, Ryan, that this AR avatar has like set language. It is contextual to the questions.

Right.

It's using a large language model on the back end to say, oh, Josh is at an event, I need to reference that event in my response and ask him how it's going at Inbound. And to me this is like kind of mind blowing because it's a very different experience than we're all used to. We're all used to the voiceless, faceless chat bottom and waiting for 10 minutes to connect to an agent. And for us and our like laptop goes to sleep or our phone goes to sleep and then we have to reconnect all of those things. So it's a pretty big shift in how we're interacting. So Josh, I would love for you to walk us through how you're seeing companies use these avatars and again, kind of how consumers are feeling as part of that interaction.

Yeah, 100%. So I think, you know, the, we call this product and use case interactive avata. So it's not only just a demo, it's actually available for everybody trying out in the product today. So the use cases for interactive avatar has been very horizontal ranging from all sorts of customer interaction experience like sales, marketing, customer success and support especially. One example would be support that we are really can turn the chat bubble usually is a text interaction and turn that into like, you know, a video interaction.

Right.

So and we are also rolling out a new version of that interactive avatar where you can actually, we call that meeting avatar where you can have the avatar to attend a zoom meeting for you. So where you know, you can actually send the avatar to join some, you know, customer support or success calls. And so, and I think, I think we're still pretty early on in terms of like how our customer reacted to it. But I think what was like one thing we did at Heygen is that we actually use our own interactive avatar to help to answer the questions about this particular product. And I think so far it's been great. You know, there's probably, you know, many, many calls happening every day without us like having to like jump onto the call. I think customers get what they want, you know, get a question, answer asking about the pricing, asking about where's the API dog, you know, how does the product work? I think yeah, this kind of like, you know, how it works right now.

So, so it seems like these interactive avatars with an LLM behind it are better at answering more complex questions, having a more real time interaction, dealing, like handling and deflecting a lot more, especially in support and customer success. And then you, like Yamini said this morning in her keynote, leaving the real like high, subtlety, high detail questions for human interaction. And so it seems like the next, next step past like that chat agent, chat bot interaction is that, is like, is that how you all are viewing it? I guess is what I'm trying to understand.

Yeah, yeah, so definitely. So I think the one thing I've been personally feeling most excited about this, you know, interactive avatar is that, you know, traditionally when we talk about the CRM data, right? So the data has to come from a human input or it has to come from, you know, we connect some external data source to get to join that together. But all this like conversation happening on the interactive avatar, they are unstructured, but they are really, really important and valuable data about that customer because it represents what the question they ask, what the journey look like, you know, where they are in terms of the, you know, the customer journey. And I see a great potential. We can integrate this conversational data even though there's an unstructured, but it can be totally processed with the latest, you know, technology around LLM and put that and add to that to the existing cim. You know, I think that would be the most exciting area.

I couldn't agree. That is a lot of where the future of CRM is going. Andy Nyamini this morning talked about structured and unstructured data and that those two things are coming together. What you're saying is like, hey, you have all this unstructured data, just questions people are chatting in and LLMs are going to be able to organize that and make it clear in the CRM, make it more like structured data so that you can then do follow on engagement, follow on interactions with that which is going to incentivize all of us to use these interactive type of avatars to continue to have better customer data, prospect data for, for all of our go to markets.

Right?

Yeah, exactly. I think, you know, another thing is that I think the whole large language model is moving towards to a multi model model. Right. So I think the model will start very, very soon to understand not only the test content about the conversation, but also the emotion, you know, the tone of the voice of a customer. And I think that's also another level of the important information. Do they ask this question without or just like purely a question to curious to learn more about it? I think those will be important, you know, valuable information at CIM as well.

I mean there's two big things that changes for B2B companies like as part of this conversation is like the customer journey to date is mostly text driven. Yes. Like you go to chat, you get email and it's mostly personalized one to many. And I think there's two things that will drastically change with AI in that we'll have a multimodal customer journey where the AI will be able to switch between text, video and auditory based upon what your likes are, you can imagine in the chat, you'll be able to switch between text, audio and video. You'll be able to consistently carry that experience throughout the customer journey. And then the second one is you're going to be able to, as you said, learn how that person interacts with you throughout that customer journey and start to personalize that experience much, much word more. So it's truly like everyone gets the concierge to walk themselves through buying a product and to stay with them after they actually have bought that product and interact in the kind of modality that they want.

Yeah.

All right. We can't leave without talking about the future. Right? Like we got a couple minutes left. Those are all three things that you all can go do today. Like literally go out in the hall and you can do the localization, you can do personalized video, you can beta the interactive avatar. But Kieran, let's set us up for what we, what we are going to be talking about.

We covered. Yeah, you covered one of our big ones which is our people freaked out. We kind of got that out of the way. We believe, Joshua, that next year really is going to be the year for AI for video. Like we've seen a ton of improvements this year. Heygen have improved their technology incredibly. So what are the things that you are really excited about next year that will come in terms of AI video be unlocked for how we all use that technology.

Yeah, sure. You know, the way how we really break down the problem about video generation is that we take it into two components, right? The A roll and the B roll. And when we talk about Avatar replacing the camera, you know, localizing the content for the editor, for the speaker, it's more about a roll. I think there's a huge ton of, you know, application there alongside the B roll, like, you know, transition, adding element, you know, voiceover music and you know, layout of the video, stuff like that. I think, you know, one of the most exciting area that we are working on is actually integrating the B roll together with the avatar. So essentially what you're generating is not only an actor's footage, but it's actually end to end video. I think that's one. I think another thing is that just like, you know, if we look at Avatar today, it still have not, you know, passed a Turing test for some of the high end external marketing use cases. Especially, you know, if we talk about advertising, I think the application there is still not that many. And I think, you know, one of the exciting area that we are actively working on is enabling the gesture rendering of the avatar. Essentially if we look at, we, if we look at the problem about how do we make the avatar more natural, right? So usually when we speak, we don't speak only with the mouse. You speak together with your head movement, with your eyebrow movement, with your body and with your gesture as well. I think that level of the gesture rendering will really bring the avatar and bring the A roll of the content to be the next level in terms of engagement and natural mix. Yeah, those are the two things.

And we talked a lot about this. My original video, I'm all hands, hands, hands, hands. And the aesthetics on the hands has got much, much better. And so you can kind of see that come to life in the, in the new, in the kind of new models. What's your, my, my, my one is I didn't know that I could have an AI avatar go to Zoom meetings. So that is my number one use case that I am going to be signing up for. So you will never see me again.

I'm just, I'm just gonna talk to Avatar Kieran in my meetings now.

Yeah, you can talk to AI avatar meeting Kieran.

It'll make me feel it'll be similar. Can it do the podcast for you too?

It can do the podcast, it can take the notes. It's going to be a full thing.

No, in all seriousness, the use case that I am most excited about is that concierge use case, which is the most emerging use case. And I think we're just getting started at that point. Right. The products are beta for you all. We're, you know, there's a bunch of companies trying to figure that out. We're just. Companies like us are trying to get our data organized in a way that can handle that. So I think the first two are going to be the most necessary over the next six months. But I think six months from, from now, we're going to unlock the next one. Anything else, gentlemen? Before we, before we close out, I think we've covered. We have covered AI video.

It is AI video.

Next year, AI video is going to be way more dynamic. B2B. Go to markets are going to be video, audio and text, not just text. I think that was a great point. I would love for everybody to give a big round of applause to our guest, Joshua.

Thank you, thank you.

Thank you for being here. You can find marking against your grain wherever you get your podcast and on YouTube and we'll see you again sometime soon.