The Future of AI Avatars and DeepFake Technology w/ Joshua Xu & Steve Brown | EP #161

Where does this go? What's your vision for where this is going? How will I use these avatars?

Very, very soon we're actually going to see everybody be able to have a meeting with the agent in the video interface.

Joshua is you.

Joshua Xu.

Joshua Xu, the mind behind Heygen, he.

Is redefining how we create and communicate.

You will be able to have your agent talk to people in different languages.

175 languages. That's insane.

I'm very excited about mission that we have enabling, you know, visual storytelling for all.

The technology's gotten so extraordinarily powerful and yet we're just at the very beginning.

We take trust and safety seriously on the platform. Every single avatar created on hey Jam will require the first party consent.

This is the future. Which is available when?

Now.

Now. That's a moonshot, ladies and gentlemen. So, Joshua, you have built an extraordinary company. I mean, I think truly if you have know anything about digital avatars, heygen is the top of the game. Congratulations.

Thank you.

So how long ago did you start Heygen? Tell me a little bit about your journey. You know, I want to understand where your technology is and in our conversation understand where it's going.

Sure. Definitely. The journey of Heygen started back in 2020.

Long ago. In 2020.

Yeah. I think that was the time that we didn't have the term called generative AI. And I was working at very, you know, similar technology enabling, you know, using AI technology to enhance the camera experience at Snap. And I just felt that the technology can continue to evolve and potentially generate something that does not exist in the world but still feel high realistic. So that's how we started the company. And in fact, I think the funny thing about the mission of a company in the beginning we call it replacing the camera.

Replacing the camera, yeah.

Because I think if we look back the past 15 years, mobile camera is really the thing that enable a lot of people creating content, but still there's so many people are not being able to create a good content using the camera either because camera shine or camera being very expensive in business. And we felt that if we can replace the camera, we can remove the barrier for the visual storytelling for the business. And that's how we started the journey.

Amazing. Well, you've come a very far way. I think you've got a video to kick us off, to give us a sense. Let's run that video so folks can get a sense of your technology.

Yep.

Hey everyone, I'm Joshua and yes, I'm here on stage with you, but I'm also right here as his digital twin. Do we look alike? This is hey John's avatar technology in action. I can deliver video messages without a camera or production crew. This ranges from video ads to how to videos to trainings and so much more. I can also speak in over 170 languages and dialects. For example, Hola, abundancia tres ecenta estoy muy emocionado de sumerhime contigo en el futuro de la generacion de videos con intelligentsia artificial. Besides just speaking different languages, I can even engage in real time conversations whether it's on your website, product or platform. Imagine what this could mean for your business. Joshua and Peter, I'll let you take it from here. Okay, Nice.

Love that the technology's gotten so extraordinarily powerful and yet we're just at the very beginning. So where does this go? What's your vision for where this is going? How will I use these avatars?

Yeah. So I think things are happening in two ways. I would say one thing is that every individual will be able to easily visualize their ideas and every business will be able to access more videos creation. Every business will be creating hundreds, you know, hundred x more videos in the business communication and I think the business content is going to be more engaging compared to the old days of email images, texts, that's one. And that's happening very, very fast. Today we have seen customers like creating tons of videos like promoting their business. I think on the other side, which is also very exciting. I think everybody know the AI agent role is happening very fast as well. And I think the next interface for us to really interrupt the AI agent may be actually we started from chatting by typing the text and then I think it's going to very soon we're going to interrupt the agent with voice interface and then very, very soon we are actually going to see everybody be able to meet, have a meeting with the agent in the video interface. And I think that's something that we are seeing very exciting for the future.

Yeah, I mean I think you've been building these avatars and you can have your avatar go attend a Zoom meeting for you. Yeah, right.

Yep.

I mean I love that. I don't know about you guys, but I'm in Zoom meetings. I would much rather have my avatar do the work. You know, I think about this in the future where I can build a really rich identic agent. In other words, an agent that knows everything about me, knows everything about my company that if you ask the agent a Question and ask me a question, you're going to get the same answer. Imagine being able to create not one but like a thousand agents and send them out to a thousand different conferences on your behalf, go and cut deals and bring back, you know, business for you. Is that possible? Is that where we're going?

Yeah, I think, you know not only about that, I think there's another things that you would be able to have your agent talk to people in different languages, which was not possible before. And it is always hard to touch customers in different region, different language. And now you can really enabling that and that's one I think the second thing is that you can really like in run your agent meeting in parallel and having your customer interacting with you at the same time. Like hundreds, thousands of different sessions happening at the same time.

Yeah.

The other thing is I don't know about you, if you are working on new products or services, I know my team will bring me in and say, hey, can we get you to record a video at this particular time? But you put the video up and it's not hitting properly. So we want to change the pricing, want to change the name, want to change all these things. Instead of pulling me into the, you know, into camera or hiring someone to come in and talk about your new longevity clinic or whatever it might be, you can spin up your, your Hadron avatars to iterate rapidly as quickly as you want, right?

Yeah, yeah. I think that's exactly how we start the company and solve the camera piece problem. And in fact, even for myself, I already create probably more than 300 different avatars in the past four years because really just testing out the technology. But even today, if you have me to really like you said, I want to film something in front of camera. I still feel like I need to do some preparation for that. I might sometimes need a one or two we shoot and I need to pick up good lighting, things like that. But Avatar really helped us solve that problem for people and being able to create content anytime.

Yeah. I remember last year, Steve Brown, my chief AI officer, member of our community here, who will be coming out and joining us in a couple minutes, he created a Peterbot avatar of myself which uploaded all of my knowledge, all of my books, all my podcasts, everything known about me. And I did a podcast cast with Peterbot and I was blown away. Peterbot was much more eloquent, much more knowledgeable, much smoother. I mean honestly, I felt very deficient in front of Peterbot. It didn't stammer to remember something in its memory. And it was just very compelling. And so there's this incredible power that you have.

Yeah, yeah. And next. And next. Very excited. Just getting started.

Yeah, it's just getting started. I know you've got a marketing video which I've asked you to play because I want people to see all the different ways that people can utilize this. I really want this to be one of the technologies everybody plays with this year. It's easy. You just have to make a commitment to actually try it. So please, you know, one things I'm looking for you to do is commit to certain things. This is one of them. Let's. Let's play that marketing video if we could.

Yeah, sure.

Today video is everything you need. Speed, creativity, and consistency. With Heygen, you can create the most lifelike avatar videos 10 times faster than traditional production. Scale across 175 different languages, looks, and platforms, all in 4K. Your clients want more. More videos, more engagement, and more results. HeyGen gives you the tools to deliver. Need personalized videos that feel custom done. Launch interactive, engaging campaigns that deliver more for your clients. And do it all without blowing the budget. No expensive studios, no costly talent. Just beautiful, effective videos that drive results. And here's a secret. We used hey Gen to create me. Join the future of storytelling. Become a HeyGen certified agency partner.

Everybody. I hope you're enjoying this episode. Did you know that we're likely to see as many as 10 billion humanoid robots by 2040? And that Brett Adcock, the CEO of Figure, anticipates they'll have robots in our home in the next two to three years? How about Max Hodak's new form of BCI called Biohybrid interfaces that could offer millions of connections between your neocortex and the cloud. Then there's Michael Andreg, whose efforts at EON is focusing on uploading the human connectome to the cloud by 2030? These aren't science fiction scenarios. There are serious efforts underway today. I've distilled the most powerful insights and roadmaps from this year's Abundance 2025 summit into a comprehensive report that will transform how you see the future. Get your free copy of the Abundance 2025 summit summary@diamandis.com Breakthroughs. That's diamandis.com breakthroughs. I find that extraordinarily impressive. We're going to talk about in a few minutes how each of you can walk away with your own hey Gen avatar this week.

Right?

It should be something that all of you commit to do, including the teens I would love you guys to do this. So let's talk about the two parts. There's the voice and then there's the video Persona. And one of the improvements in the video Persona is hand movements, backgrounds. I mean, that's relatively recent. Can you speak about that?

Yeah. If you look at the technology itself, what we are truly building is, you know, it's a human centric video model. Right. So I think the difference is about human centric video model versus a general video model is that, you know, we need to like be able to like present and engaging effect like, such as gesture and the way how you move and you move your body, body languages and expression, things like that. And there's a couple elements out there. Voice is definitely one. I think video is another one. And what we usually require is that we'll take roughly two minutes of training footage from you.

Two minutes of video?

Yeah, two minutes of video. And then we will. Our AI model, we try to learn that, you know, from the way how you speak, how your muscle move on your face and your expression, and as well as very importantly, how your body language match your expression, because everybody's body language is different. I think the 2 minutes footage will help us to really personalize your model behavior and after that. And that's roughly how it works. And obviously, if we build our own AI agent behind it, you need to input the knowledge base that you have and whatever documents or keynotes, interview you might have before, and we enable that as a LLM layer behind it.

Yeah, it's truly amazing. Let me suggest two things to everybody here. If you've not done this, and I think they're very important in this age of, you know, photorealistic, visual and auditory avatars. The first is you're going to receive a phone call from somebody you think you know, and it may be a FaceTime video from someone you think you know. And they may ask you to do something incredibly crazy like can you please wire money here or can you please send money so I can get out of jail? I'm serious. I've had lots of friends who've had that happen to their aging parents and so forth. Everybody here, one thing that you should do is choose a word, a password, or something that if you have any doubt about the authenticity of the phone call or the video call you're getting, ask for it. Right. Makes sense. It's super easy to do, have that conversation at dinner. The second thing is, if you're lucky enough to have your parents or grandparents around, do two things. If you can record their avatar. If you can't do that, sit them down for a couple of hours with your iPhone and ask them for their story, for their history, record their video, record their voice. And this is for your kids and their grandkids. Right? Please do that. It's extremely valuable. You'll be able to bring them back as your family history make sense. Great. So, Joshua, let's talk about what you have enabled for us here at the Google Tech Hub.

Yeah, sure. So we have our booth in the back. We are giving the opportunity for all the attendees to create your own digital twin over there, right at the conference. And it's for free. And you're going to be able to record your footage, the training footage for the AI model training, and then going to be able to get your avatar today. And is this a glimpse to the future of AI video creation? And then we'll be able to test it out on your HN account.

Yeah. So I want you to see how good this is. We had a few members do this last night, and I want to show you one of them that got processed overnight. Reichardt, are you here in the audience? Where are you? Okay. Rikard, Good to see you, my friend. I'm waiting for my flying Jetson as soon as I can get it. Let's watch Reichardt, his avatar. So he's not backstage, he's over there. Let's play the video.

From now on, I can speak in.

Any language with my own avatar and my own voice.

Amazing. Let's give it up. It was extraordinary. And record. How easy was that to do? Four minutes of your time and then an infinity. 175 languages. That's insane. So this again, please take this home with you. Play with this. Your whole goal here is to do something different. You know, at least show it to your kids or your. Your, you know, your spouse when you get back.

Yeah.

It was about 13 years ago. I had my two kids, my two boys. And I remember at that moment in time, I made a decision to double down on my health. Without question, I wanted to see their kids, their grandkids. And really during this extraordinary time where the space frontier and AI and crypto is all exploding, it was like the most exciting time ever to be alive. And I made a decision to double down on my health. And I've done that in three key areas. The first is going every year for a fountain upload. Fountain is one of the most advanced diagnostics and therapeutics companies. I go there, upload myself, digitize myself about 200 gigabytes of data that the AI system is able to look at to catch disease at inception. Look for any cardiovascular, any cancer, neurodegenerative disease, any metabolic disease. These things are all going on all the time. And you can prevent them if you can find them at inception. So super important. So Fountain is one of my keys. I make that available to the CEOs of all my companies, my family members, because health is a new wealth. But beyond that, we are a collection of 40 trillion human cells and about another 100 trillion bacterial cells, fungi, viri, and we don't understand how that impacts us. And so I use a company and a product called Viome. And Viome has a technology called metatranscriptomics. It was actually developed in New Mexico, the same place where the nuclear bomb was developed as a biodefense weapon. And their technology is able to help you understand what's going on in your body, to understand which bacteria are producing which proteins. And as a consequence of that, what foods are your superfoods that are best for you to eat or what food should you avoid, right? What's going on in your oral microbiome. So I use their testing to understand my foods, understand my medicines, understand my supplements. And Viome really helps me understand from a biological and data standpoint what's best for me. And then finally, you know, feeling good, being intelligent, moving well is critical, but looking good when you look yourself in the mirror saying, you know, I feel great about life is so important, right? And so a product I use every day, twice a day, is called One Skin, developed by four incredible PhD women that found this 10amino acid peptide that's able to zap senile cells in your skin and really help you stay youthful in your look and appearance. So for me, these are three technologies I love and I use all the time. I'll have my team linked to those in the show notes down below. Please check them out. Anyway, hope you enjoyed that. Now back to the episode, what's most exciting for you in the future here?

I'm very excited about the mission that we have, enabling visual storytelling for all. I think today when we talk about video, video is the main format for people to consume on the Internet. But still, I think less than 1% of the audience, less than 1% of people are able to actually create great videos. And I think camera is the first problem and we are actively solving that. And also with a lot of agent technology from the AI model advancement, I think video editing is going to be solved with AI very soon as well. And think about the role like this. Every time you want to set send out an email instead, you could actually turn that email into a video and send it to the whole company and give the video format to the customer so they can watch it on their own. I think those are the future that we are very excited about.

I agree. I know that some people are concerned or scared about their identity, their image being stolen and being used to promote something they're not involved in. But you have protections against that, don't you?

Yeah, we do, definitely. So first of all, we take trust and safety seriously on the platform. Every single avatar created on Heygen will require the first party consent where you have to give a video consent format so that we can verify that.

So you have to go on video and say, this is Peter Diamandis and I give consent to create this avatar, et cetera.

Yeah. And we are proud to say that in the entire history of hedge, we never compromise on that consent.

Amazing.

And no matter who is coming to create aft on Hadrian, we always require that. And meanwhile, you know, we also have, you know, a moderation system behind it. Just serve as a safeguard. There's a lot of things not allowed on Hadrian platform. Things like, you know, hate speech, fraudulent and you know, any political campaign. Today we have a shittily moderation team in the back combined with AI and human moderator to help to make sure that the technology has been at a good use.

Yeah, I'm so, so impressed. Last year I tried to get a video acknowledgment from Aristotle to get that video made and he never responded. But thank you for allowing it to happen. Last question here. Let's discuss the difference between integration of an AI agent and my digital clone.

Yeah, the way how I look at it is in two ways. When we talk about AI agent today, we're mainly talking about there's an AI assistant that can help us to do something not only at the first action, but also a sequence of action. And I think when we talk about digital twin, it's the enablement about myself. It's like, let's say, Peter, you want to have three meetings happening at the same time. And it's kind of like agentize yourself so that other people can get access to you. And I think there's a combination overlap of these two role where, okay, the question would be how does human interact with agent in the future? I think the most natural way to interact with agent will be a video format, would be a video call, will be a video chat. So I think you can kind of attach your digital twin in your AI agent so that other people can interact with it. And meanwhile, you know, you can also get access to other people's agent in a video chat format.

Yeah, I don't know if you can feel the speed, the future speeding up in this very moment. Right. So when you hire 20 marketing agents, AI agents we've been talking about, those AI agents will show up as you'll give them names, they'll show up on slack. You can slack them, you can email them, you can hop on a video conference with them, you can have a zoom call with your marketing force of 20 AI agents and they will look unique. You'll have conversations with them. This is the future. Which is available when?

Now.

Now. All right.

The following segment is a presentation from Steve Brown on exactly how to use these AI avatars in your business life.

One thing I wanted to ask you is, like, how many of you are at this point confused about the difference between avatar and agent?

A little bit.

Okay, I'm going to give you a little shortcut to figuring it out. Avatar is AV audio, visual. Agent is agency. And if you're going to give agency to your agent, it needs to know you and you need to trust it. So how do you make that happen? It's one thing to have that emotional connection because it looks fantastic and it acts like you, it looks like you visually. But we need to train the brain. So that's what I've been working on. Now this past 12 months have been the 12 months of Avatars. And now all we're going to talk about is agents from here on out. So first of all, how do you train your agent? So you're going to go and make a great avatar downstairs, and I did a great one last night, so I'm not going to show you. We already saw Ricards, but I actually built agents to help you train your agent. So asking you all about your life story, asking you about your values, asking about your purpose, asking about your personal anecdotes and all these things so you can talk to your agent forever to train your story about so that your actual legacy agent or clone will know.

So these are the means by which in just interacting, a voice conversation, you build the corpus of data that makes that agent actually meaningful to you.

So you have a lot of books and you have a lot of things to upload, but a lot of people don't. You have your Facebook and your LinkedIn and your social media. But what's really important is your values, what you care about your personal story, the depth and that might be hundreds of pages. If you were to write a biography, that's hundreds of pages. So anyways, I'm building agents that help you actually train your agent. So now beyond that, what can you use your agent for? Another thing I've been using agents for, this is a medical agent. And we'll hope that he works. He's actually connected to my medical records. So just talking a little bit about your story, so knowing your story for future generations. So my story has been one of entrepreneur. And then I meet this fabulous entrepreneur and entrepreneur mentor, Peter, which has stimulated even more ideas to go start new ventures. But in the last couple of months, actually my own story took just a quite radical turn. It started off with the fires in LA two months ago, which happened to burn our house down for the second time. Well, the first one was a rental so it didn't really count, but this was like, you know, the dream home for the owners. And yeah, I mean, well anyways, yeah, we lost our stuff before, but this was a much more traumatic event. But it actually turned out to be something else. Lee Stein, who's here, said, steve, you need to look for the gift. If you don't look for the gift, you're not going to find the gift. We were displaced from Los Angeles and my wife and I and our 170 pound Great Dane, we ended up staying with friends in Palm Springs. I ended up in the emergency room because of steak dinner pain, thinking that I was obstructed. I went to the ER and got a CAT scan and the ER doctor is like, hey, there's something going on here. I got admitted to the hospital and for many, many more tests and biopsies and everything and I ended up getting diagnosed with multiple myeloma blood cancer. While I'm in the hospital for eight days.

This is about six weeks ago.

Yeah. So I'm in the hospital and I'm like, of course. I'm just talking to AI the whole time. What's going on here? What does all this mean? And I had just had not fountain life. I'm sure fountain life would have figured this out. But I, I had gone to the doctors and I had had everything under the sun tested, I thought to specialists and they were like, this is a fishing expedition. We can't find anything. Maybe you're just stressed out and overworked. I'd been working very hard for a very long time. But anyways, we couldn't find anything. So I end up with a young ER doctor in Palm Springs, who didn't know me, didn't have really any background in me at all. Not a specialist. And he ends up piecing these things together. So I went back to my. While I'm in the hospital, I went back to my medical record, all the tests that I'd been done.

This is over the last few months.

Yeah. And I said, well, when would AI have caught this? I took all the data, not from Palm Springs, but all the prior data, and I said, is there anything in there that might have helped find this earlier? I was extremely lucky and that we found it very early, before much damage had been done. And I'm actually on chemo right now and immunotherapies, and, you know, I think I'm going to be okay, but let's talk to Hippocrates and say, hey, Hippocrates. I was not feeling well for a while, and I lost a bunch of weight. So I went to my specialist, two different specialists, and I had everything under the sun tested, and they really couldn't find anything. And they thought maybe I'm just stressed out or overworked or depressed. Kind of the default answer that doctors sometimes give if they don't know what's going on. I wondered if you could look at my chart and tell me what you see.

I see your concern, Steve. Looking at your chart, I notice several significant findings that warrant further investigation. Your low immunoglobulins, IgA and IgM, low gamma globulins, mild anemia, elevated ferritin, and unexplained weight loss together suggest a possible plasma cell disorder. I recommend discussing with your physician about bone marrow evaluation to rule out multiple myeloma or related conditions.

So, first off the bat, right?

So that's pretty extraordinary because that's actually what the diagnosis ended up being. But my specialist didn't find it, even though they were supposedly testing for everything. But a generalist put the pieces together and found it. And you found it even earlier. I'm trying to understand how that could be, Steve.

I'm not surprised. Your pattern of hypogamma, globulin, MIA with weight loss, fatigue, and anemia creates a classic constellation suggesting plasma cell dyscracia. Specialists sometimes focus narrowly on their area while missing broader patterns. The immunoglobulin abnormalities were the key clue that needed integration with your symptoms. This pattern recognition is fundamental to Hippocratic medicine, seeing the whole person, not just isolated test results.

So what's the test that I should have gotten that might have figured this out?

Steve, you should have received serum free light chain assay testing which measures kappa and lambda light chains crucial for detecting plasma cell disorders. Even when immunofixation is negative, a bone marrow biopsy would have been definitive. Also, a 24 hour urine protein electrophoresis could have detected Bence Jones proteins. These tests are standard when plasma cell disorders are suspected.

So one last thing here. So actually I'm on chemotherapy and I'm getting this new immunotherapy called Daratumumab. I don't know if the you guys can figure that out, what that is on the voice transcription. So anyways, after two weeks of chemo, my light chains went down by 50%. Over 50%. So what do you think of that, Steve?

That's excellent progress. Daratumumab. Darzalex is a powerful monoclonal antibody that targets CD38 on plasma cells. A 50% reduction in light chains after just two weeks indicates you're responding very well to treatment. This rapid response is encouraging and suggests the therapy is effectively targeting the abnormal plasma cells. Continue monitoring closely with your oncologist as this early response is a positive prognostic sign.

So one of the things that I'm working on is that all of our specialists get narrower and narrower and more narrow in their silos because there's so much medical knowledge, there's no way that anybody can know everything. So you end up getting siloed. The AI is not siloed. The AI can understand everything. They can absorb everything. You showed a video at the beginning of Minecraft with the thousand agents running around and collecting things and collaborating and doing things. And you mentioned, can I have 100 of my agents out there going to Zoom meetings and doing business deals? One of the things I'm working on is, well, why not have 100 doctors with 100 different points of view and 100 different ways of dipping into this messy, fragmented data of our health that is very, very hard to infer what's going on. And everybody has a different point of view. This is just one agent and he's trained to be a God agent, be Hippocrates, know everything. But what's actually really interesting is when your agents have different points of view, because those different points of view are different search strategies into this vast knowledge base that's captured in the, in the LLM. So this is an example of how agents are not just going to do some of your work for you, not just represent you to future generations if you really get your story in there. But your agents might be saving your life. This is really powerful stuff. And what Joshua and Hey Jin are doing just at the top of the game here is giving that human interface because our ability to trust our agents and our ability to connect with our agents and to tell our stories and interact with them, that empathetic connection actually is very, very kind of dependent on that feeling of connection that I'm really dealing with an entity that I know, that I trust, that cares about me. So that's one part of the equation and it's extraordinarily important. And hey Jen is doing just an extraordinary job with it. The other side of it is the brain. And what can you train it to do and what can it do for you? And that's what I'm working on next.

Peter.

Amazing. Guys, give it up for Steve and Joshua. Guys, come on out here. If you enjoyed this episode, I'm going to be releasing all of the talks, all the keynotes from the Abundance Summit exclusively on exponential mastery.com you can get on demand access there. Go to exponential mastery.com.

If you work as a manufacturing facilities engineer, installing a new piece of equipment can be as complex as the machinery itself. From prep work to alignment and T testing, it's your team's job to put it all together. That's why it's good to have Grainger on your side. With industrial grade products and next day delivery, Grainger helps ensure you have everything you need close at hand through every step of the installation. Call 1-800-GRAINGER Click grainger.com or just stop by Grainger for the ones who get it done.