47: Virtual Beings are Here!

This is going to be a fun one. We're talking about virtual beings.

Oh, okay. I am Noreen. I'm the CEO and one of the co founders of Mesh Kapade and I've been in the background. My background has been in computer vision, computer research, and from there building machine learning models and now building a company that is using all of the machine learning models and making amazing stuff.

That's awesome. Well, I'm hot and bothered by virtual beings. I just got this new pair of digital glasses from Even Realities and they're not where we're going to be in three years, but it shows you that the computer people are able to shrink things down a pair of glasses. And soon we're going to have a pair of glasses on like this and we can have a virtual being next to us.

Exactly.

You're one of the first companies I've seen to really show me a virtual being that people developers can use and build and all that. So yeah, what's going on in virtual? Why is this field so important? And did I lay it out right that hey, soon we're going to have glasses? We're going to have virtual beings walking around with us, sort of like a robot that we can see and we can dress it up and we can talk to it thanks to AI? Did I lay it out right?

Exactly. So yeah, I'm right there with you with the virtual. Like these glasses. I keep looking at every new company that's coming up with the latest versions and I sometimes pre order like on the Kickstarters. I'm super excited about all of those.

But yes, me too. That's why I'm wearing them. And they're starting to look like glasses and not, of course, this doesn't have a camera, so I can't do a slam map, which is the 3D environment around you that a virtual being would walk around.

Right, Exactly. That's the thing. So until we get to the holodeck, the actual holodeck from Star Trek, until then, we have to rely on these headsets or AR VR glasses and different devices.

You can see your virtual being holding a phone. Right. That's not very immersive. Plus you got to hold a phone, not wear a pair of glasses to see it.

Glasses are perfect for that. I think it's a great first step. And the idea is before a virtual character can interact with you, it needs to learn about what it means to interact with you. So to be able to interact with each other. Like when we're growing up, we're learning by Observing the world like babies, they observe the world and they start understanding what objects are, what people are and, and how to interact. They start observing and then start replicating. And that's exactly what we need to train AI to be able to do. And that's what we've been building towards. Human behavior is complex. So to be able to train AI characters, virtual beings, and then eventually also physical systems, physical, like robots, to be able to interact with humans, just like humans interact with each other, we need to be able to keep. Capture the physical reality of the world and then train AI models that can replicate the behavior by understanding what they're seeing. And that's, that's at the crux of all of this.

Yeah, I had a prototype, an entrepreneur brought some virtual beings over, and that's a prototype. And as I was showing it my, my room on my phone, it could. The virtual being could and could talk about it. Oh, oh, nice lights on your wall, you know, and nice nano. Nanoleaf lights because it knows what those lights are. Right? The multimodal models can see like a coffee cans. Oh, you got your coffee, you're ready to go this morning, right? Something like that.

Exactly. And yeah, I actually also appreciate your lights. They're super cool, but yeah, exactly. Not. And it goes beyond just being able to understand the world that people are in. It's also about being able to immerse themselves into that world. So if you imagine a future where there are these virtual beings that are projected onto your glasses, if they can also see the world, representation of the world, that is what spatial intelligence. That's what. Where this space is going. Being able to recreate and represent the real world in 3D. And then so for a virtual being to be able to interact with you in your space, it also needs to be able to understand how to interact with objects. Like how you hold a phone is different from how you hold a glass of water and how much you press it and stuff like that, because then that understanding is part of our intelligence. This is just naturally comes to us because we interact with objects constantly as we're growing up. This is spatial and semantic intelligence. And combining that together with postural or behavioral movement intelligence, that's all that goes into creating a virtual virtual being.

Do we see what these things look like so that people have a context of how good they are and what they can do?

So, okay, yeah, I can show some, some things. I have some videos that I can show you guys or show you. And yeah, you might have to hit.

Share screen on Google. We're using Google Meet to do this video.

Let me share over here.

There we go. I can see it.

Yeah, yeah. The first step. So there's a problem of building embodied intelligence because humans are complex, complex creatures, and we interact with the world. We interact with objects in the world and with each other, and being able to train AI systems to. To do all of that, we need to solve this problem. And we're doing this in two phases. The first phase is what we call the human understanding machine. So what you see here, this is actually what is live on our platform right now. So on the surface, this might look like 3D motion capture, which it is, but actually it goes way beyond traditional motion capture, or no cap, as people usually call it. Traditional motion capture is restricted to a studio environment, but we are actually capturing human behavior out in the real world, where people are playing with footballs in the field, or walking on logs, or skating on skateboards or rollerblades. And beyond that, there's also a very important piece in addition to human movement and human body shape and movement that we're actually also capturing here, we're actually collecting all of the spatial and semantic information about where these people are. Because when you see any of these videos, there's information about where the light is coming from, where the ground plane is, where these people are, and why they're doing certain things. Because this person is in a, you know, in a fighting rink, so this person is boxing. So all of that spatial information, we take it for granted as humans because we know about, we've seen all of these scenes, but we're training computers to understand that. And then we're taking that information to build intelligence into characters. So initially we're starting with game characters, and this is the second phase where we're building the embodied intelligence. We're using this information about human movement, behavior, spatial and semantic information from real world scenes, and building it into virtual beings or AI characters. So, for example, I can show a motion to an AI character through a video, and I can tell it, hey, just do that kicking motion. Or I can say in text, hey, kick it harder. Or I can give it this iconic photo of Saint Bolt or pose and say, hey, hit that pose at the end. So this is very special to humans where you can tell a person, hey, you did that kick. No, do it a little bit harder. And then end with this specific pose. For a virtual being to be able to understand what comes first, what comes after, and how to do certain movements, that is embodied intelligence that only comes from interacting with the world and Humans, and that's what we're training computers to be able to do.

So you're using just a camera to take video of humans, to add on more behaviors or more understanding of the world, or more understanding of the humans so that you can replicate that in your virtual being.

Precisely so from just a single camera. That was the phase one, which is now something that we're making available for motion capture. Like, you know, game, game developers, as well as like any company that needs motion capture. It is in a. Essentially a motion capture system. But for us, we're actually capturing all of the semantic information to train AI models, large behavior models of how humans move.

Interesting. And how many, how many millions of videos have you ingested into your model?

We've invested quite a few million.

You're not Tesla. Tesla has millions of videos.

We have. We have been generating our own synthetic data as well. And that's how we're, that's how we're actually training this to be so intelligent and be able to understand real world. Because just from a video, you can't train the AI model about all of the information that exists. So for the training, we're actually generating our own synthetic data of people moving in different environments with different kinds of camera angles, different kind of movement behaviors. And with that, now we can go to any video that we're buying, count of videos from, you know, different sort of sources and just learning from people doing different kinds of actions, you know, holding things, showing things, and interacting with each other.

I was just at Stanford's Robotics Day where they had a bunch of researchers showing off their work with robots. It's almost exactly what you're showing me, actually.

Funny you should say that. Let me show you one more thing. This is something that we just demoed at the Game Developers Conference at gdc. This is a model of behavior that is reacting to input in real time. Basically on the left, we're poking this character in real time with this foam finger, and every time it reacts, it creates a new reaction. This is not hand animated, like not recorded by anyone by hand or animated by an animator. This is learned from behavior of real people, like tons and tons of data of real people and how they behave. And with that, we've basically trained different kinds of movement behaviors. Now for this virtual character, it's a virtual being that will always create, is generating its own specific behavior. And we've trained it to learn from all kinds of human data, all kinds of human movement and behavior. So it's now first version iteration already that we're now ready to start launching into games. And the future is, you know, this is just the beginning.

You can see a whole need for these in gaming and what we're going to call movies in the future. You know, interactive entertainment products.

Right now imagine like, exactly. So if you have. If you're in a virtual space with a. With this kind of a character where you can first, like right now, we're just reacting with physical actions, but also being able to interact with, you know, objects and two virtual characters interacting with each other with, you know, with no script. Imagine that. So, yeah, that's what's the next step towards this. And eventually this can go into robots as well.

Well, you know, you're basically programming a virtual robot. Right. A robot without a physical form, but the same thing. I mean, if I have a virtual being standing here, that's the same as a robot standing here. It's just. It doesn't have a physical form to it.

Precisely. That's precisely it. And going first from these virtual characters towards where, you know, we'll see how the human. Because the humanoid robots is now becoming bigger and bigger. When we started building these things, these models, it was many years ago when humanoid robots weren't the thing that was happening. But now we are already talking to companies who are building humanoid robots to start building this. I like to call it this behavior engine. It's the brain.

It would be smart for a Tesla to build a virtual Optimus long before we could buy an Optimus for our home homes to get people used to talking to an optimist and give you the data, or give Tesla the data to complete the Optimus and get it into people's homes, because homes are very chaotic and very changing environments. Right.

That would make perfect sense. Yeah. Basically. If you're talking to an optimist, what's the purpose of this humanoid robot? A lot of people are, you know, are training these humanoid robots to do laundry, which, you know, I completely understand is one of the most boring things to do in life. Yeah, the dishes and such.

They can't get the shirts inside out yet, though. At Stanford, they showed off the latest laundry robots and they can fold really well and fast and you can even mess with the robots and take, you know, move the shirt around or bunch it up and it'll redo it.

Exactly.

But they don't demonstrate inside out because that requires a lot of tactile, you know, understanding of clothing.

And we can do weird, complex things with all. We can manipulate objects in weird ways. The more we train AI systems to Watch humans, the better they can be become at manipulating those same like they have. If humanoid robots are being built with the same kinds of mechanism and, you know, motor functions as humans or even more, they can learn to adapt to that, to the human movements.

Yeah. On the more serious side, this is going to be the user interface for AI soon. I mean, we're not going to be using Chat GPT for long. With text already we're getting used to talking to it with our voice, but it doesn't have a 3D form on our screens, on our phones or on our desktop. With your work, you could be plug that in and become the AI, the user interface for a chat GPT or a Google Gemini or a Grock @Xai or, you know, anthropic cloud or perplexity.

Precisely. All of this really depends on use case. If you just want somebody to review a paper or do your homework, chatgpt without a representation is sufficient. But if you want to have an interactive conversation, for example, if you want to learn how to fix your engine for your car, if there is a virtual being that can actually show you the different pieces and take them out and then also guide you through different things, that is where these kinds of things will be more, more needed and more appropriate.

Oh great, I'm going to have a virtual being helping me rebuild a carburetor or something.

Exactly.

If we take those four screws out and then take.

Well, actually what I'm imagining, all of us will have our own space shuttles and those are what we will be fixing.

How do developers use this stuff today? You know, how do they build a virtual being? How do they pay for it or license it for their own apps? If they're building a game or working at OpenAI and building, wanting to build a visual representation, an AI friend or whatever we're going to call these things.

So we have an online platform for people to try things out so they can upload videos and they can get the 3D motion output from there. But for companies who want to then start integrating this into their workflows, we have this enterprise package. Basically they can just install our different methods. It's kind of like what Chat GPT has had, they have, or like what OpenAI has, they have different kinds of models and different models apply to different kinds of use cases. So similarly, so when it comes to motion capture, so for gaming companies, it's just the motion capture piece that they need and it's super helpful for them to just upload their videos either to the cloud to Our cloud service or on their on site installation and just get the output. For robotics and simulation companies, they want to be able to do both. Be able to upload a video and generate a human moving around in the street or something, or just output the. Do the motion capture or generate the movement by telling it, hey, okay, step a little to the left or the right and you know, stuff like that. So both of those. It really depends on the use case. But our primary target right now are gaming mostly and AI tech companies like.

Film simulation, which makes sense because they have the biggest needs at the moment. Let's say I was working at OpenAI and I wanted to make a visual representation of chat GPT, right? A virtual being. That's going to be the new chat GPT interface. Can I change everything about that? Can I change how the eyes look? How the. Well, the Optimus doesn't even have eyes or nose. It has sort of a plate in the front of it, right? Can I change the character completely and how do I do that?

So there are two ways to do that. So right now, so our model is built on real human data. So we had scanned thousands, hundreds of thousands of real people and created a human, a very human representative model. So if you go to our website meshcapi Me, so there's that model there and you can create different body shapes for that model. It will always recreate a human for you. So you can create chubby people, skinny people, tall, short, all kinds of variations. But beyond that, for gaming, for example, all of our system, all of our output is fully compatible with every game engine for gaming. People can actually now already just stream directly from our API service to inside of Unreal Engine. So some of the demos that I was showing you, especially the generative model demo that was streaming directly into Unreal. So there are two different ways. If you want more stylized characters, you can use a game engine. If you want a humanoid, an actual human, not humanoid, but humans, and you want to create variations. We have our body like model representation of humans.

On that you can change the clothing and the shoes.

I will add one thing. So one thing that we actually just also demoed at GDC was appearance of hair. So you can change clothing and shoes, but those are just representatives, representations in terms of a layer of clothing. So it's a typical representation that you use in gaming systems. But when it comes to appearance and different kinds of pieces of appearance, one of the most important things, most interesting things that people are interested in in is facial shape and facial animation, but also hair. So this is something that we actually just, just developed from a single photo. You can recreate the hair in 3D and then bring it into unreal for your character. So, so you can even, you know, recreate your own hair. It still doesn't work with mine. Yeah, but you can recreate your own hair or you can create variations of, of your hair just within seconds with the, with the models that we've created. So all kinds of representations of human exist with our system. And of course, if you want to go into gaming systems and create stylized representations, that's, that's fairly easy to connect.

To in the VR world. We talk about when we start getting toward human interaction or it's something that looks like a human, we call it. And often the real human goes, oh, that's fake. Or there's something wrong with it. There's an uncanny valley, we call it. Where are we in that uncanny valley with this? Because you're trying to recreate human forms that are digital, right. That are AI driven. How close are we to actually really fooling everybody? Where. Oh my God, I can't tell that's not a human.

I think in terms of the body shape, because with our models, people have literally told us like, the movement is so fluid because you can. When humans, when we move, when we, like when I bend on my elbow, this area, this bulges out and all of that, all of that is very well represented. With our model. The only thing that's missing is ice. And that's the uncanniest of uncanny valley pieces. Right. So that I think if there is a very good model, there are many artist based systems used in games and movies that do eyes really well. As long as the eyes have the full reflectance, then you're good. Because otherwise our facial expressions and the body movement and response is all the way there for removing that uncanny valley effect.

Got it. And reflection is so important with, with eyes, we, we can see it and we're so. We evolved to really understand eyes very deeply. This is why we always had arguments about Google Glass and stuff. Even these, right. They're a little bit of a, it introduces a little bit of a noise in between our eyes. Right.

We don't like. Yeah, with the, with the avp, I think that was it, the EVP that had the eyes being appearing in the front.

Yeah, yeah.

I think they did a pretty good job at that. Because it wasn't your actual eyes that the person was seeing. Yeah, there you go.

On the front of it. And when I turn it on, you would see the eyes.

Yeah, but eyes, eyes are the most important for the, for this uncanny valley piece.

Yeah. So we're still going to be able to tell, oh, that's a digital human being a little bit, but it's going to get close. Right. And that's why I'm talking to you because starting to get really pretty amazing.

Exactly. And when it comes to eyes, you know, you want the reflectance. Right. And that means that, that all of the lighting has to be very well modeled and eyes have multiple layers of reflectance. So that, let's say we're going to leave that part as work homework for the reader, for the viewer.

Yeah, we're, we're going to take these virtual beings to, on vacation, like to Disneyland. Right. And, and where I'm going is Disneyland. The multimodal models, like Chat, GPT or Grok or anything. You take a picture of, like if you're in Disneyland on Main street, you take a picture of the firehouse. It tells you all about the firehouse. It tells you, oh, this is where Walt Disney stayed while he was in the park. Right. And it gives you all sorts of interesting stuff. Well, your virtual being is going to have those virtual, those multimodal models behind it, running it. Right.

I, I, I find it very interesting that you brought in, brought up Disney because animatronics, that's where another, like, really big piece happens with like, there's, we've been having a lot of these conversations with people because if you have this, like at these theme parks, there are these animatronic models, like a Lincoln model, and it only can do a few different, you know, things very robotically.

Yeah.

Imagine if the Lincoln could just actually see you and step towards you. Well, maybe that would get too creepy, but, but actually see you and then be like, oh, yeah, hi.

If we had a virtual Abraham Lincoln, we could walk up to it. Right?

Exactly. If it's, if it's, Can I shake your hand? You know, and there's, there's so many, so such exciting possibilities. If you have these virtual characters that come to life and an Einstein model, you know, an Einstein acting just like, you know, the real person.

Or even a rubber Scoville. I've given my data, I mean, I've been scanned a lot of times so people can take my Image and my 3D form and in fact, hold on a second.

You have a 3D print.

I was in one, I was in one of these, these cameras, you know, and scanned me all around and made a 3D printed model of me, you.

Know, yeah, it just unblurred on it. It Good. That's a. That's a very good model.

Yeah, yeah, yeah. Color and everything, Right.

You know, and then. Then that. That model, that virtual being having your behavior, you know, you have a certain kind of a laugh, a certain kind of, like, way of interacting. Those are very personal to you. And anybody who even, like, even if the model doesn't have your face, people can recognize that. About. Yeah, about people that are popular. For example, for, you know, anybody knows this is Arnold Schwarzenegger or this is. I can't think of any other famous people. So, Robert.

We want Darth Vader.

Darth Vader. Yeah. Yoda.

Yeah. Something like. Of course, if you try to do that, Disney's gonna come after you for intellectual property, so you're gonna have to license that. But, you know, that's coming, right. I mean, 10 years from now, we're going to have Darth Vader in our house, interacting with us, maybe even playing lightsaber with us, right? And we'll have a lightsaber in our hand. One of those toy things we spent 40 bucks on at Disney World in Disneyland. And you're going to be playing with a virtual Darth Vader who's playing with you, right?

Absolutely. I just recently started playing Beat Saber on.

There we go.

Yes. 100. I would love that because. Yeah, those. See, there's. Entertainment is something that will always exist, and that's something that people will always, always want more of. And then these new innovations start. Start bringing new kinds of ways of bringing entertainment to people. I think this is. This is inevitable, first of all, but the possibilities and the potential not just to do. To have more kinds of entertainment, but more kinds of learning. This is like the carburetor, like this virtual character teaching you about your car or your shuttle, space shuttle.

The virtual beings that you're building, they have a model that runs the being and gives it a voice and gives it a face and gives it a body and gives it behaviors in the world. It's. It's not giving it the smarts about the world. Like what. What's. What is on Main street at Disneyland. That. That would come from a. Another model, like a large language model, like a chat GPT or a grok or something.

Both of them, actually. So. So for the voice. Actually, the voice part, we're. We're going to. We. We don't need to do that ourselves because there's many voice models that exist.

Like 11 labs kind of stuff.

Exactly.

There's a whole group of them. I have a list.

Yeah, exactly. But when it comes to understanding of the world, yes, we are combining with existing LLMs, but we're building that into a 3D understanding because that's the piece that doesn't exist with existing LLMs because it understands in 2D and being able to regenerate that same visual in 2D. Current models, large models know how to do that, but the 3D recreation, that's the harder part. And when humans have to interact with the world in 3D, or virtual beings have to interact with the world in 3D, it has to model the 3D correctly. So our language, so our models are being trained on the 3D understanding of the real world and training our virtual humans or virtual beings to have an understanding of that.

I just, you know, in 10 years, when everybody has glasses on or 3D TVs, because, you know, if you don't want to wear glasses, there's going to be new types of TVs that are coming in 2027 that have depth, that have 3Dness. You could create a virtual being for that TV and then everybody could be talking to the 3D thing on that 3D TV in, in that world. It seems to me there's new kind of job coming, which is creating a new virtual character. You know, coming up with the behaviors and the clothing and, you know, the voice and the.

And that's what always happens with new innovations. People feel like this is taking away jobs, but it, what it always, always does is actually creates completely new kinds of. Of possibilities and. Yes, exactly. This will bring a new era of new kind of storytelling, really. There will be new kinds of entertainment. Of course, there are new kind of characters that you can create. But then imagine those characters will actually be interacting with a 3D world or maybe even with your 3D world. So being able to understand in your virtual space, your virtual game space that you've created, where the tables are, where the chairs are, being able to go and sit down in a chair. Maybe that chair is a representative of a chair that I have in my real living room.

Yeah.

With my glasses, I can just see a virtual character sitting on the chair in front of me.

I don't know if you saw magic leaps. Micah was about eight years ago now, you put on a magically pair of glasses and Micah walked around a wall partition, walked over to a table you were sitting at, and sat in the chair right across from you and then played a little game with you. It was quite, I think it's quite innovative, you know, and and really was a leader in this field, but it. The company just hasn't done anything with it. You're taking this to. To reality.

Exactly. Because that's precisely the behavior of like, okay, also different people based on their size sit on chairs in different ways. So just being able to create behavior based on. On the person that. That. That virtual being is representing. That's. That's a whole level, like, different level of understanding and understanding about the world and navigating to a chair, knowing and understanding where the chair is in this 3D scene and being able to talk to people about, oh, hey, can you pass me the drink? The virtual character can potentially also pass you your drink. It just won't be real.

Yeah, people don't understand that. Right. Because now the computer can see a coffee cup or something like that, segment it out, and then change it. So it could move my coffee cup around visually, not in reality. Right, Reality that if you want real coffee, you gotta have your real cut. But it could actually move that coffee cup over to my chair behind me. Right?

Exactly, exactly.

And make the original coffee cup disappear. Right. Even though it's still there. It gets messy in mixed reality. Right.

Because exactly. The. The world can change.

Real. See the way it looks, but you can't change the atoms actually on the table.

Definitely. Mixed. Mixed reality in multiple ways.

It's going to be a fun, fun world. What is the challenges for you now? You know, this is the early days of virtual beings. They're just getting launched. People don't know why they need one.

Yeah.

Gamers, you know, game designers are starting to build them into games because they're going to GDC and seeing what you're doing. But normal, everyday people don't realize they need one of these. Or. Or 50.

Exactly. So that's one of the challenges. Like, because this is a new world that a new, new kinds of technologies are coming up. How do you use them? Where do you use them? So that's why, you know, in the AI and tech space, we have a lot of people who have been using our technologies or our products. So that makes sense. But the other challenge is also we're building all of this understanding of humans and human behavior from 3D. And there isn't a lot of 3D information or data out there in the world. So that's why we have to generate our own. We have, we're creating, we're synthesizing a lot of our own data, or we're going and collecting data in motion capture systems or 3D scanning systems so that's why that's where these, this is a little bit slower, but it is essential for this virtual, these virtual spaces, these AR and VR spaces, and then eventually physical systems like robotics or even animatronics and, you know, any other kinds of physical systems that will have to interact with humans. They really can't just operate on a 2D understanding of the world. They have to have a 3D understanding. So that's why this is important. But also data. Well, data is a problem for every space, actually. So what am I complaining about?

Well, I could see a robot company buying you or wanting your data because you're going to give them the data to complete their robot. Right. And because if we're using a lot of virtual beings in homes, we're going to have a lot of virtual beings long before we have a lot of humanoid robots in our homes. Right. And so you're going to have a lot more data than the robot companies are going to have about how the home works, how many dogs are in the house, you know, what are those stupid lights, you know, what's, you know, all the chaos in the house.

We actually. So last year we trained the first human 3D foundation model, which exactly did what you're just describing. It's just an, it's a, it's like a ChatGPT model, but it understands everything that exists in a 3D space. And then it can recreate a human in that 3D space, doing an activity, but at the same time understanding what, how to navigate that 3D space. So those are exactly the things. But yeah, a lot of robotics companies are using our products like they are training on, like they're uploading videos to train robots on the human movement of videos. So, yeah, it makes sense.

I've interviewed a lot of legendary video game people or like the guy built flight Simulator. Even the skills they needed were very technical, that they needed to know how to write assembler code in the early days to make lines appear on the screen fast. And that was really the core skill of. It seems like in this new world we're heading into, that doesn't matter much at all. It's, it's the storyteller that matters. It's the ability to create a character or create, you know, clothing for a virtual being or, or a story that the virtual being would be a part of. Like a Darth Vader in Star Wars.

Right, I, I hear you and I agree with you, but I also feel sad admitting this because, yeah, I, I remember learning assembly way, way long ago. And it was Just so amazing and fascinating knowing people who could do a lot with assembly because it's so hard. But yeah, I, you. I agree. I think in this future, things will be built very differently and some of the core understanding of how code works or like, core skills about coding might not survive, but new things will come up. So we'll see how this goes.

Yeah. What do people need to know about your company or what kind of questions should I ask? Because I've had a little experience with virtual being, but I'm not building the reinforcement learning systems and the simulators to build models like what you're doing, what kind of questions should I be asking?

Or, you know, so, so for us, you know, the biggest thing that we. We're training is this behavior model and how people interact with the world and training that intelligence, this behavioral intelligence into AI characters or virtual beings. So for me, it really is a question of what comes next, whether. So, for example, if you're talking about a game, it can be a virtual character, a main character that has completely realistic behaviors, or a lot of the times, you know, in games they have these NPCs that are super dumb. And there's like, lots of videos about games that have very dumb NPCs that are doing stupid things. But imagine a new kind of a scenario where the NPC has intelligence about the world and about each player that comes into the world and other NPCs in the world. So much so that the NPC has its own story. The NPC becomes a whole game in itself. So for me, this is really fascinating, but in some ways, it also goes towards this whole idea of a digital west world. It could eventually become a real, like.

Oh boy.

Yeah, yeah. But that's. For me, that's more too, too dystopian. So that's why I always like your. Your take on it. Like the Holodeck, which is Star Trek.

Yeah.

Very, very optimistic. For me, the. The question here only becomes about, like, okay, what kind of future do we want to build? For me, it's always I. That's why I also like your take on things because more about optimism and the possibilities that it can create rather than being worried.

We can always do naughty things with technology, but there's always a good to it too.

Exactly. And there's. There's so many people who are, who are willing, who are interested in building good things out of technology. So it's just a matter of building the safeguards that you need for the world to be not destructive and dystopian, but I. When it comes to questions, I honestly don't know. But for me, the most interesting thing is really just that the possibilities that can happen, whether it's virtual entertainment, games, virtual storytelling, or physical systems, I think the possibilities of the future are really.

Really great and then really crazy. You just remind me, I talked to Dylan over at Unity Labs all the time about consciousness. Digital, both biological consciousness. Like me and you hopefully have a consciousness and a digital consciousness which you're building a virtual being. These virtual beings could evolve on their own by their having their own lives in their own game world or their own, or they walking around my house. They could have their own memory and their own experience, and they could evolve over time.

I'll tell you something, so this.

I know people who are building digital souls. It's really crazy.

So this demo that we showed at GDC where you could poke this character, I kid you not, sometimes it just seems like this character is sort of like just fanning that pokey finger away. Like it's just learned because that behavior, that reaction behavior we've trained on that character, it actually is just from real world data of how humans behave. And if you're poking somebody on the shoulder, how they behave, or on the, on the thigh or something like that, in the end, so what, what that made me think about was human behavior. Everything that we do, it comes from our brain. And our brains are just electrical signals. Right? So this can get very philosophical. But in the end, what even is human behavior? What even is this sort of soul that we talk about? I. Maybe virtual being.

You mean when, when I get a neural link in 10 years, if I'm still alive, the virtual being could go into my head?

It. Well then would you have two intelligences inside of your head?

Yes. And isn't that the singularity where they merge?

Okay, now that's gotten even more philosophical. Oh my gosh, Yes. I don't know. It could be.

Yeah, yeah, that's a weird future to think about. But it's common. There's three people with neuralinks on now. Their neuralinks can only read the brain, can't inject thinking into the brain yet. But. But that's common.

And, and honestly, because we don't have.

A few brain computer interfaces around here, you know.

Oh, wow. Okay, that's.

That's common.

Yeah. And to me, it's really not about like, you know, it's. Everything is chemicals and electrical signals. Even this world. We don't know, maybe this world itself is a simulation.

It's a pretty weird One.

That's why it might really be.

Oh, man. It's fun. I. I could talk to you for a long time, and. And I want to get your virtual beings in my house, so I got to figure that out.

Yes, absolutely. Absolutely.

I want. I want it. I want an AI assistant that looks like a human or looks like an optimist on my phone. You know, hey, Optimus, what's going on?

Exactly. And then, you know, the more you talk to it, the more you tell it about things, the more it understands how you behave. And then it. It starts learning from observing people doing different things.

Yeah. Learning from observing people. Yeah. Did I tell you about the microphones in my Chinese lights?

There's microphones in your Chinese light?

There are actually. There's a computer and microphones in the light, so I can listen to the room and react. Right. So if I put music on, I can make it strobe and do all sorts of fun stuff. Yeah. But the. You know, we could take it a little further.

You could. Definitely, definitely.

Well, I mean, these. These glasses have microphones and have a digital display in them. It's still a black and white. Well, a green and black display. It's a rudimentary. But next year, there's glasses coming that have full color display and cameras in them, maybe just a little bit bigger than these. And so I. I dream about the world you're building, so thank you so much.

Well, so do I. That's why we're building it.

Yeah, it's coming. Yeah. You can. You can use your SDKs right now and build your own virtual.

Art online. And. And we give on premise installation versions of our APIs to people as well, who want to not upload data to the cloud, so. Exactly. Those are all accessible to our website@mashcapade.com or meshcapit.me is the platform itself. You can even play around with the platform, and you can type in different kinds of movements. The character will do that. We're just building more and more intelligence into it.

They're gonna get smarter.

Yeah.

They're gonna be fun to play with. Next year is gonna be a big deal, I think, for this whole industry. So thanks for giving me a little taste of what's coming.

Well, thanks for having me. I'm. I to tell you, I'm very, very excited about talking to Robert Scoville.

I'm excited about talking to you because you're building the future. I just talk about mutual excitement.

Awesome.

Yeah.

Yes.

Thank you so much.

Thank you. Yeah.